Hi Andy, Added a few comments inline (marked with "==" in lieu of a better email client)..
But overall corr_est works very consistently. I did have a few observations: - changed feed-forward AGC num_samples = 128 (from 2048, to allow faster recovery with fading ch sim). The value has some effect on whether the first packet correlates or gets scrambled. - added FLL for course freq alignment, which you alluded to below. this works well only on a uniform spectrum, so repeating 4B 4B symbols was causing issues.. initially setting eb=0.5 and increasing the filter size helped a bit. - with tx_sps=10, setting *output* sps values for the polyphase block to anything other than 2, 5, or 10 results in a periodic phase rotation. my guess is the tagged packet was getting dropped at the polyphase output when tx_sps % osps != 0, i.e. tx_sps/osps should be an integer. I was half-expecting your answer on alternate way for generating the samples via a separate flowgraph :) nevertheless thank you for the level of detail, as always. My plan is to change tx_sps to something other than 10, and try to repeat your calibration method (i.e. the symbol trimming). I hope you don't mind another question or two as I go along. Best Regards, Arik ________________________________________ From: Andy Walls [a...@silverblocksystems.net] Sent: Monday, April 04, 2016 8:59 AM To: Landsman, Arik Cc: discuss-gnuradio@gnu.org Subject: Re: [Discuss-gnuradio] costas ambiguity and correlate-and-sync block in qpsk On Sun, 2016-04-03 at 19:33 -0400, Andy Walls wrote: > On Sun, 2016-04-03 at 20:24 +0000, Landsman, Arik wrote: > > > btw, played with tag delay manually, this has an immense effect on > > costas rotation once fading model is added to channel. > > Well there are a number of things going on here, so let me state how > things are set up: > > 1. The reference sample sequence given to the corr_est block is your 8 > symbol preamble, modulated at 10 samples/symbol, minus the first 5 > samples of the first symbol, and minus the last 5 samples of the last > symbol. So 150 samples total, spanning from preamble symbol 0.5 to > preamble symbol 7.5. Argh... ... 16 symbol preamble ... .. to preamble symbol 15.5. Counting bits is too hard for me apparently. :) == :) yap, preamble is "FC 0C 30 00" -Andy ________________________________________ From: Andy Walls [a...@silverblocksystems.net] Sent: Sunday, April 03, 2016 7:33 PM To: Landsman, Arik Cc: discuss-gnuradio@gnu.org Subject: Re: [Discuss-gnuradio] costas ambiguity and correlate-and-sync block in qpsk On Sun, 2016-04-03 at 20:24 +0000, Landsman, Arik wrote: > Hi Andy, > > So now that the Thrift issue is resolved (separate thread, thank you! > ), had a few follow up questions on the actual corr_est implementation > and placement of tags: > > >> ./Rx_syncd.py | grep -v volk | sed -e 's/(//g' -e 's/)//g' > > foo.txt > > this doesn't want to port the samples; it does however add a header > with something to the likes of "Volk machine: sse4_1_64". Tried as > sudo as well, still no go.. After you have GRC generate the Rx_syncd_(n).py script, you have to hand edit it near the bottom to print out the samples of the preamble generated my the modulate_vector block. The edit looks something like this: ... tb.start() tb.show() # Hand added statement: print tb.get_modulated_preamble_untrimmed() def quitting(): tb.stop() tb.wait() ... So when you run that edited script, it will emit, to stdout, the sample vector created by the modulate_vector block in one of the flowgraphs that I gave you. (In the above specific example, the Rx_syncd_2.grc has a modulate vector block that generates "modulate_preamble_untrimmed".) == of course, how did I miss that.. was working off of a GRC-generate .py and left it out. How could the interpreter possibly NOT know what to output into foo.txt... :) > > I also couldn't find setup_env.sh, searched the whole folder > structure. so didn't run >>source <etc>. maybe that's part of the > problem? Nope. If you're running gnuradio-companion just fine without that script and you don't know what it is, then you don't need that environment script. Here are the contents of mine, just so you feel better about ignoring it: $ cat ../target/setup_env.sh # WARNING: This file is auto-generated by pybombs, any manual changes to it may be overwritten! export PATH="/home/andy/work/target/bin/:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/andy/.local/bin:/home/andy/bin:/usr/lib64/qt4/bin/" export PYTHONPATH="/home/andy/work/target/python/:/home/andy/work/target/lib/python2.6/site-packages/:/home/andy/work/target/lib64/python2.6/site-packages/:/home/andy/work/target/lib/python2.6/dist-packages/:/home/andy/work/target/lib64/python2.6/dist-packages/:/home/andy/work/target/lib/python2.7/site-packages/:/home/andy/work/target/lib64/python2.7/site-packages/:/home/andy/work/target/lib/python2.7/dist-packages/:/home/andy/work/target/lib64/python2.7/dist-packages/:/home/andy/work/target/python/:/home/andy/work/target/lib/python2.6/site-packages/:/home/andy/work/target/lib64/python2.6/site-packages/:/home/andy/work/target/lib/python2.6/dist-packages/:/home/andy/work/target/lib64/python2.6/dist-packages/:/home/andy/work/target/lib/python2.7/site-packages/:/home/andy/work/target/lib64/python2.7/site-packages/:/home/andy/work/target/lib/python2.7/dist-packages/:/home/andy/work/target/lib64/python2.7/dist-packages/" export LD_LIBRARY_PATH="/home/andy/work/target/lib/:/home/andy/work/target/lib64/:" export PKG_CONFIG_PATH="/home/andy/work/target/lib/pkgconfig/:/home/andy/work/target/lib64/pkgconfig/:" Super boring stuff that is probably already set for you, since you can run GNURadio applications. My GNURadio custom build is in /home/andy/work/target/ . > Any other way to send the raw samples into a file from GRC? otherwise > going the python route.. ) Sure, just build a simple flowgraph with the modulator modulating six consecutive instances of your preamble, streaming the complex I&Q floats to a file sink. You can then use MatLab, octave, Python, whatever to parse the 32 bit float, native-endian, I,Q,I,Q,I,Q,... samples from the file. == I was half-expecting your answer, that was the one I gave myself as well :) thank you for the formatting comments though > btw, played with tag delay manually, this has an immense effect on > costas rotation once fading model is added to channel. Well there are a number of things going on here, so let me state how things are set up: 1. The reference sample sequence given to the corr_est block is your 8 symbol preamble, modulated at 10 samples/symbol, minus the first 5 samples of the first symbol, and minus the last 5 samples of the last symbol. So 150 samples total, spanning from preamble symbol 0.5 to preamble symbol 7.5. 2. The corr_est block is performing a complex cross-correlation of this reference sample sequence with the input signal. The corr_est block outputs that cross correlation, undelayed, on its second output. The corr_est block outputs the original input signal, *delayed* by the length of your reference sample sequence, on its primary output. 3. This paper goes over some properties of a particular complex cross-correlation that one can generalize somewhat: Marple, Jr., S. L., "Estimating Group Delay and Phase Delay via Discrete-Time 'Analytic' Cross-Correlation, _IEEE_Transcations_ _on_Signal_Processing_, Volume 47, No. 9, September 1999 But my comment in the corr_est code says it all: // The analytic cross-correlation is: // // 2A*e_bb(t-t_d)*exp(-j*2*pi*f*(t-t_d) - j*phi_bb(t-t_d) - j*theta_c) // // The analytic auto-correlation's envelope, e_bb(), has its // peak at the "group delay" time, t = t_d. The analytic // cross-correlation's center frequency phase shift, theta_c, // is determined from the argument of the analytic // cross-correlation at the "group delay" time, t = t_d. // // Taking the argument of the analytic cross-correlation at // any other time will include the baseband auto-correlation's // phase term, phi_bb(t-t_d), and a frequency dependent term // of the cross-correlation, which I don't believe maps simply // to expected symbol phase differences. So the peak of the cross-correlation happens exactly at the delay time when the reference sequence and the signal line up, which is at the end of the preamble (or preamble symbol 7.5 as I have chosen things). This is the *only* instant, when we can learn learn the phase of the incoming signal preamble's relative to the phase of the reference preamble at that final sample. The time dependent phase and frequency terms obscure the value everywhere else, but that's OK, *if* we are frequency locked and the phase offset introduced by the channel is relatively constant. 4. The polyphase clock sync block looks at the "time_est" tag emitted by the corr_est block and expects that tag to be on the sample at the center of a symbol - the optimal sampling time for clock recovery. This means, given the 150 sample reference sample sequence I used and the tx_sps value of 10 and me starting and ending the reference sequence mid-symbol, the sensible tag marking delays are either: 149, 139, 129, ...., 19, 9, or 0. Other delay values will not resync clock recovery properly, and make clock recovery take longer. If you a not locked in frequency, then really the best place to mark is at the correlation peak, using a corr_est tag delay of 149, in our working example. 5. The costas loop block looks for the "phase_est" tag emitted by the corr_est block, and expects the tag at the place where the phase tracking should be reset. In PSK, it makes sense to mark this in the middle of a symbol where the phase should take on a well defined value very near a constellation point. So again, the sensible tag marking delays are either: 149, 139, 129, ...., 19, 9, or 0 in our working example. If you a not locked in frequency, then really the best place to mark is at the correlation peak, the only place that phase offset estimate is really valid, using a corr_est tag delay of 149, in our working example. If you have it noticed it so far, frequency lock to the carrier is important if dealing with absolute phase values. The costas loop will do that fine frequency locking for you, as long as you are close with your coarse frequency lock. Your initial phase offset estimate, determined by cross-correlation with a preamble, is happening before fine frequency locking and relies on the matched filtering equivalence of the cross correlation to remove noise. Without doing anything differently, you are probably stuck marking the phase_est at the peak of the correlation (end of the preamble) and nowhere else. You ought to experiment with channel impairments one at a time to see which one has the most effect, while still using a _realistic_ channel model. (Anyone can break a system with a drastically severe, unrealistic channel.): - Carrier frequency offset: constant, slow time-varying, fast time-varying, etc. - Carrier phase offset: constant, slow time-varying, fast time-varying, etc. - Rayleigh fading channel (no dominant LOS signal) - Rician fading channel (one strong LOS) - Narrow frequency selective fades in your signal bandwidth. I suppose for extracting absolute phase offset, the Rayleigh fading, fast varying CFO, and fast varying Phase Offset will be the worst. == very much appreciated, lots of good information here. Going to spend a bit more time on it in the next couple of days. Regards, Andy > Thanks, > Arik > > > ______________________________________________________________________ > From: Andy Walls [a...@silverblocksystems.net] > Sent: Saturday, April 02, 2016 8:11 PM > To: Landsman, Arik > Cc: discuss-gnuradio@gnu.org > Subject: Re: [Discuss-gnuradio] costas ambiguity and > correlate-and-sync block in qpsk > > > > Hi Arik: > > On Sat, Apr 2, 2016 at 7:41 PM, Landsman, Arik > <arik.lands...@tufts.edu> wrote: > Hi Andy, > > Still looking it over, but one thing jumps out right away - > for some reason I couldn't execute either Rx_syncd_2.grc or > the Rx_syncd.py. getting the same error in both cases: > > "" > Generating: > '/home/ubuntu/Desktop/MSProject/alternateVersions/Rx_syncd_2.py' > > Executing: '/usr/bin/python2 > -u /home/ubuntu/Desktop/MSProject/alternateVersions/Rx_syncd_2.py' > > Using Volk machine: sse4_1_64 > Thrift: Sat Apr 2 19:05:54 2016 TServerSocket::listen() BIND > 9090 > terminate called after throwing an instance of > 'apache::thrift::transport::TTransportException' > what(): Could not bind: Transport endpoint is not connected > "" > > does this look familiar by any chance? > > > It looks like a GNURadio ControlPort / Apache Thrift error. > > > I took your flowgraph and modified it slightly, only adding a few > blocks: > > -modulate_vector > > -skip_head > > -keep_1_in_n > > -qt_time_sink > > -feedforward_agc > > -multiply const > > -qt gui range > > > > One of those might be doing it. Disable skip_head, keep_1_in_n, and > the constellation sink they connect to. Hopefully it's one of them. > > Otherwise, you didn't drop a control port related block on the > flowgraph did you? > > > > FWIW, I also modified the Polyphase clock sync filter taps, the > Corr_Est symbols, and changed a few 3.14's to math.pi. > > > > Also could you elaborate a bit on what you did with > > "" > $ source <path_to_target>/setup_env.sh > $ ./Rx_syncd.py | grep -v volk | sed -e 's/(//g' -e > 's/)//g' > foo.txt > $ octave > octave:1> x = csvread('foo.txt'); > octave:2> t = [1:length(x)]; > octave:3> plot(t,abs(x)) > octave:4> plot(t,arg(x)/pi) > "" > > at the very least where can I find setup_env.sh?. > > > If you build GNURadio with the PyBOMBS build system and install in a > target directory somewhere, PyBOMBS generates this environment file > which is essential to run GNURadio Companion and gnuradio scripts > properly. If you are using the GNURadio supplied by your > distribution, then sourcing a setup_env.sh file is not necessary. > > > > > . wondering what <path_to_target> may be. > I suppose you then pipe a portion of the stream in Rx_syncd.py > into foo.txt to parse with octave? > > > Yes, I just have it emit the complex samples vector that the > modulate_vector block creates, and then inspect the samples in octave. > You can do it in MatLab or Python or Excel, or whatever. > > > going to try it if I find setup_env.sh on my end. and once I > can run Rx_syncd.py... ) > > Thanks, > Arik > > > And another, "Oh by the way" for when things are working. In the > flowgraph I sent, I mark the phase correction on the last symbol of > the preamble - where the phase correction was measured. This means, > that the previous symbols in the preamble, if they suffered a phase > rotation, may not get corrected by the costas loop. You can "fix" > that by setting the corr_est tag marking delay to 0, to mark the first > symbol in the preamble with the phase correction. This is OK as long > as you are very close to the correct center frequency. The measured > phase rotation will become less correct the farther you move from the > measurement point at the peak of the correlation, if you don't have > good frequency lock when performing the correlation. > > > > > Regards, > > Andy > > > > > > ________________________________________ > From: Andy Walls [a...@silverblocksystems.net] > Sent: Friday, April 01, 2016 9:18 AM > To: Landsman, Arik > Cc: discuss-gnuradio@gnu.org > Subject: Re: [Discuss-gnuradio] costas ambiguity and > correlate-and-sync block in qpsk > > Hi Arik, > > I have attached a slightly improved flowgraph and a hand > edited .py > file. > > The only modifications to the flowgraph from before, is that I > set the > reference samples for the corr_est block to start at the > _middle_ of the > first symbol of the preamble and stop at the _middle_ of the > last symbol > in the preamble; I also ensured the phase_est and time_est > tags stayed > marked at the center of the last symbol of the preamble. This > has two > benefits: > > 1. The phase_est value will not be thrown off by the end of > symbol > transition to a different unknown symbol after the preamble. > 2. The correlation won't be sensitive to the unknown symbols > before and > after the preamble. > > In other words, it was a mistake to have the correlation > reference > sequence start and end at the symbol transition times. :P > > In the generated *.py script I added one line near the end: > > print tb.get_modulated_preamble_untrimmed() > > So you can so this: > > $ source <path_to_target>/setup_env.sh > $ ./Rx_syncd.py | grep -v volk | sed -e 's/(//g' -e > 's/)//g' > foo.txt > $ octave > octave:1> x = csvread('foo.txt'); > octave:2> t = [1:length(x)]; > octave:3> plot(t,abs(x)) > octave:4> plot(t,arg(x)/pi) > > and see why I picked the values I did to trim the modulated > preamble to > the correct set of samples. > > The modulator that generates the reference sequence has a > pretty long > delay, and the first copy of the preamble it outputs is a > little > distorted at the first symbol. That's why I snipped out the > second copy > of the preamble. > > Regards, > Andy > > > On Thu, 2016-03-31 at 22:04 -0400, Andy Walls wrote: > > Hi Arik, > > > > Try the attached flowgraph. I didn't get to put everything > back in, but > > it demonstrates the principal. > > > > 1. Observe the corr_est block correlates to your preamble > bytes. > > 2. Slide the phase offset around: > > a. observe the phase changing/wrapping in the time plot of > the output of > > the correlator > > b. observe the phase_est tag value approximately matches the > value of > > the phase offset slider. > > c. Observe the constellation rotate as you change the phase > offset. > > d. Observe your decoded bits stay the same. > > > > Regards, > > Andy > > > > > > > > _______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio