Hi Kyeong, Thank you very much for the response. Yes, I will try the duty cycling and averaging method.
Thanks! On Fri, Jan 20, 2017 at 4:22 PM, Kyeong Su Shin <[email protected]> wrote: > Hello Mallesham, > > That will convert I-Q data (two numbers per a point) to a real number data > (one number per a point), so you do save some storage when you do that. > However, not enough to lift the storage concern. (Or maybe, if you have > sufficiently large storage and you re-tune your receiver to different > center frequencies frequently). > > If you are collecting PSD data, then I guess it has to do something with > spectrum sensing. If that is the case, I suggest try duty-cycling averaging > or max-holding the PSD data to reduce number of PSD blocks that you have to > store per a unit time. That is often sufficient. Or, maybe you can look at > some compressive sensing techniques, if the signal is sparse (although I am > not sure how easy would they be to implement with GNURadio + COTS sensors, > as I never tried it out). > > Regards, > Kyeong Su Shin > > > On Fri, Jan 20, 2017 at 1:02 PM, Mallesham Dasari < > [email protected]> wrote: > >> Hi Marcus, >> >> I am recording only power values(PSD), as you described in one of the >> previous emails. Not raw IQ or FFT samples. Sorry for the confusion. >> >> On Fri, Jan 20, 2017 at 3:21 PM, Marcus Müller <[email protected]> >> wrote: >> >>> Mallesham, >>> >>> you should __really__ look into what kind of data you need to store. Are >>> you sure what you want to do in the end is analyze 24 h * 3600 s/h * 32·10⁶ >>> S/s * 8B/s = 20 Terabyte of data? That is what you get when you store the >>> FFTs of one day worth of samples. I know that you're probably tuning in >>> between, but again, as we all stressed: the FFT does *not* reduce the >>> amount of data! It is just a transform into a different base system. >>> >>> Maybe you can take a step back and describe your *application*. What is >>> it that you need to *do*? I think your system requirements are not fully >>> developed, yet. >>> >>> Best regards, >>> >>> Marcus >>> On 01/20/2017 09:15 PM, Mallesham Dasari wrote: >>> >>> Hi All, >>> >>> Sorry, it was a typo. The sample rate is 32MHz. Thank you very much for >>> the information. I will look for the larger storage or some other mechanism. >>> >>> Thanks! >>> >>> On Fri, Jan 20, 2017 at 3:01 PM, Kyeong Su Shin <[email protected]> wrote: >>> >>>> To whom it may concern: >>>> >>>> As like other people have explained, amount of information carried by >>>> the FFT data equals that of the inputted raw I-Q data. At 32GS/s, that >>>> would be 64Gbit per a second even if the ADCs are 1-bit. (I am actually not >>>> aware of such fast SDR transceivers.. Is it possible?) >>>> >>>> However, do you really need all that data? What are you trying to >>>> achieve? What I do (or some FFT-related blocks do) is duty-cycling the data >>>> collection. For an example, you can do something like storing only 1 FFT >>>> vector per every 1000 generated and still get a decent information about >>>> the channel. If you really need to have all that data, then no, I am not >>>> aware of any feasible implementation. >>>> >>>> Regards, >>>> Kyeong Su Shin >>>> >>>> On Fri, Jan 20, 2017 at 11:42 AM, Mallesham Dasari < >>>> [email protected]> wrote: >>>> >>>>> Hi Marcus, >>>>> >>>>> Thanks for the quick response. I am recording the FFT samples >>>>> continuously. But, I am getting overflow after some time when the file >>>>> size >>>>> has become huge. My sample rate is high (32GHz) and hence writing to the >>>>> file takes so long and hence the usrp_spectrum_sense getting overflow. >>>>> >>>>> On Fri, Jan 20, 2017 at 2:33 PM, Marcus Müller < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello Mallesham, >>>>>> >>>>>> I'm afraid not, since I'm afraid that to my current understanding, >>>>>> what you want is mathematically impossible. Either you want much data – >>>>>> and >>>>>> that seems to be the case, since you want to record 24h of raw IQ data – >>>>>> or >>>>>> you can store it in what comparably little RAM modern computers have. >>>>>> >>>>>> Maybe, however, we haven't fully understood the problem. Can you, >>>>>> mathematically, define what you want to observe and record? >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Marcus >>>>>> >>>>>> >>>>>> >>>>>> On 01/20/2017 08:28 PM, Mallesham Dasari wrote: >>>>>> >>>>>> Hello everyone, >>>>>> >>>>>> Can anyone give some solution for this? Even writing to the ramdisk >>>>>> is not enough for running the flow graph for so long. I am facing the >>>>>> same >>>>>> issue. >>>>>> >>>>>> Thank you! >>>>>> >>>>>> On Thu, Jan 12, 2017 at 5:41 PM, Hasini Abeywickrama < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Thank you very much for the informative responses. >>>>>>> >>>>>>> My requirement is to run the flowgraph for a long time (ideally 24 >>>>>>> hours) and store the FFT data in the memory (ramdisk) to they can be >>>>>>> processed later or in chunks, not everything at the same time. >>>>>>> >>>>>>> So far, I have increased the size of the ramdisk and it works fine >>>>>>> for a few hours. But it still is not the solution I'm looking for. >>>>>>> >>>>>>> Regards, >>>>>>> Hasini >>>>>>> >>>>>>> On Thu, Jan 12, 2017 at 8:30 PM, Marcus Müller < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> But if you do a single 1024-FFT, you'd only operate on 1024 of the >>>>>>>> input samples! >>>>>>>> >>>>>>>> And: the FFT doesn't just give you power values, but complex >>>>>>>> values; mathematically, the FFT is a DFT, and the DFT is an invertible >>>>>>>> linear operator [image: $\mathcal F$]: >>>>>>>> >>>>>>>> [image: $\mathcal F: \mathbb C^{N} \mapsto \mathbb C^{N}$] >>>>>>>> >>>>>>>> which maps complex vectors to complex vectors of size [image: >>>>>>>> $N,\quad 0<N\in\mathbb N$]. It is, in fact, representable as >>>>>>>> square matrix with column (and row) vectors being samples of the >>>>>>>> orthogonal >>>>>>>> complex sinusoids $e^{j\frac{2\pi}N nk},\, k=0,\ldots,N-1$; that is, >>>>>>>> it can >>>>>>>> also be understood as a *base change matrix*, that just represents >>>>>>>> the "input vector" according to a different base, orthogonal base. >>>>>>>> In the physical sense: the input vector base was represented by the >>>>>>>> standard basis $\mathbf e_N$, meaning that each base vector represents >>>>>>>> a >>>>>>>> single point in time – the sample time of the respective entry; the >>>>>>>> "output" of the transform is represented on a base of orthogonal >>>>>>>> frequencies. This is an invertible operation – really just another way >>>>>>>> to >>>>>>>> look at *the same signal*. I think this is really important to >>>>>>>> keep in mind: >>>>>>>> >>>>>>>> The Fourier transforms are *not* magical by any means. What they >>>>>>>> do is represent *the same signal* from a different point of view. >>>>>>>> It can be *interpreted* as transform between time and frequency >>>>>>>> domain (or space and impulse, or...). The DFT is still just a boring, >>>>>>>> old, >>>>>>>> square, orthogonal, invertible matrix that produces output of the same >>>>>>>> dimensionality as it takes input. >>>>>>>> >>>>>>>> As you can see, the DFT/FFT itself never reduces the amount of >>>>>>>> data. >>>>>>>> >>>>>>>> What you might be referring to is some kind PSD estimate done by >>>>>>>> first |·|² a lot of DFTed vectors and then averaging them. The data >>>>>>>> reduction here lies in the magnitude square operation and the average, >>>>>>>> not >>>>>>>> in the DFT. >>>>>>>> The point here is that you're throwing away a whole lot of >>>>>>>> information, and I'm not convinced that's what Hasini needs! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Marcus >>>>>>>> >>>>>>>> On 12.01.2017 05:54, Mallesham Dasari wrote: >>>>>>>> >>>>>>>> Hi Marcus, >>>>>>>> >>>>>>>> Raw IQ samples take lots of memory because each sample will be >>>>>>>> around 8Bytes. Suppose, if we 1Msps sample rate, just for 10 minutes of >>>>>>>> data, we get 10*60*1M*8B = 4.8GB data. On the other hand, if you store >>>>>>>> just >>>>>>>> FFT with 1024 bin, we get 4.8GB/1024 power values right (which has very >>>>>>>> less size)? >>>>>>>> >>>>>>>> Please correct me if I am wrong. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> On Wed, Jan 11, 2017 at 7:32 AM, Marcus Müller < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Mallesham, >>>>>>>>> >>>>>>>>> I don't understand – the raw IQ samples and their FFT have the >>>>>>>>> same size, and data type. >>>>>>>>> Maybe you've understood something that I (and Martin) didn't – >>>>>>>>> could you elaborate? >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Marcus >>>>>>>>> >>>>>>>>> >>>>>>>>> On 01/11/2017 12:56 AM, Mallesham Dasari wrote: >>>>>>>>> >>>>>>>>> Hi Hasini, >>>>>>>>> >>>>>>>>> If you are trying to print just the FFT, it should not be an >>>>>>>>> issue. If you print raw iq samples, then you will run out of memory. >>>>>>>>> By >>>>>>>>> long, you mean how long? Days? >>>>>>>>> >>>>>>>>> On Tue, Jan 10, 2017 at 3:16 PM, Martin Braun < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hasini, >>>>>>>>>> >>>>>>>>>> can you please re-state what you're trying to do? That might help >>>>>>>>>> you >>>>>>>>>> getting some answers. It is not quite clear from this email. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 01/02/2017 09:16 PM, Hasini Abeywickrama wrote: >>>>>>>>>> > Hi all, >>>>>>>>>> > >>>>>>>>>> > I have a flowgraph that reads a signal and writes its FFT >>>>>>>>>> samples to a >>>>>>>>>> > file. I need to run this continuously (for a long time), >>>>>>>>>> without running >>>>>>>>>> > out of memory. >>>>>>>>>> > >>>>>>>>>> > I tired deleting the earlier FFT samples from the file but that >>>>>>>>>> messes >>>>>>>>>> > up with reading the data. I also tried starting writing to a >>>>>>>>>> different >>>>>>>>>> > file after some time so the initial file can be completely >>>>>>>>>> deleted. But >>>>>>>>>> > it did not work as well. >>>>>>>>>> > >>>>>>>>>> > What would be the best approach for this? Any thought would be >>>>>>>>>> very much >>>>>>>>>> > appreciated. >>>>>>>>>> > >>>>>>>>>> > Regards, >>>>>>>>>> > Hasini >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > _______________________________________________ >>>>>>>>>> > Discuss-gnuradio mailing list >>>>>>>>>> > [email protected] >>>>>>>>>> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Discuss-gnuradio mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Best Regards, >>>>>>>>> *Mallesham Dasari* >>>>>>>>> Department of Computer Science >>>>>>>>> Stony Brook University >>>>>>>>> USA - 11794 >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Discuss-gnuradio mailing >>>>>>>>> [email protected]https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>>> >>>>>>>>> _______________________________________________ Discuss-gnuradio >>>>>>>>> mailing list [email protected] >>>>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>> >>>>>>>> -- >>>>>>>> Best Regards, >>>>>>>> *Mallesham Dasari* >>>>>>>> Department of Computer Science >>>>>>>> Stony Brook University >>>>>>>> USA - 11794 >>>>>>>> >>>>>>>> _______________________________________________ Discuss-gnuradio >>>>>>>> mailing list [email protected] >>>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>> >>>>>>> -- >>>>>> Best Regards, >>>>>> *Mallesham Dasari* >>>>>> Department of Computer Science >>>>>> Stony Brook University >>>>>> USA - 11794 >>>>>> >>>>>> -- >>>>> Best Regards, >>>>> *Mallesham Dasari* >>>>> Department of Computer Science >>>>> Stony Brook University >>>>> USA - 11794 >>>>> _______________________________________________ Discuss-gnuradio >>>>> mailing list [email protected] https://lists.gnu.org/mailman/ >>>>> listinfo/discuss-gnuradio >>>> >>>> _______________________________________________ Discuss-gnuradio >>>> mailing list [email protected] https://lists.gnu.org/mailman/ >>>> listinfo/discuss-gnuradio >>> >>> -- >>> Best Regards, >>> *Mallesham Dasari* >>> Department of Computer Science >>> Stony Brook University >>> USA - 11794 >>> >>> >> >> >> -- >> Best Regards, >> *Mallesham Dasari* >> Department of Computer Science >> Stony Brook University >> USA - 11794 >> > > > _______________________________________________ > Discuss-gnuradio mailing list > [email protected] > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > > -- Best Regards, *Mallesham Dasari* Department of Computer Science Stony Brook University USA - 11794
_______________________________________________ Discuss-gnuradio mailing list [email protected] https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
