Hi Joe - Thanks for the new results! Getting ready for class here - so just a quick note. Your results, though more extensive than mine, point in the same direction. As it stands, sfrsd seems to be around 0.5 dB shy of kvasd. I have a long list of ideas to try to do better - but it will take time to sort through them.
To answer your question on optimizing metrics - the short answer is that they haven’t been optimized at all. Feel free to play with them. Intuitively, I like the one that I was using - but last night I played with yours and see that it’s not bad either. One task that I have on my list is to collect statistics on the probability of the second-best-symbol estimate being the correct symbol, as a function of the two probability metrics. I expect there to be a sweet spot where p2/p1 is near 0.5 and p1 is not too small and not too large where mr2sym is going to be useful. Collecting these statistics would help us decide which symbol metrics (yours or mine) are “better” for this purpose. This would also point the way toward a true Chase-type algorithm where we actually use the second-most-reliable symbols. At the moment, the only metric that is used is the probability associated with mrsym - and that is used in a crude way, as you pointed out. The idea would be to have some finite probability of replacing a symbol with mr2sym instead of marking it as an erasure - but only for symbols where mr2sym is reasonably likely to be the correct symbol. If/when we replace a symbol with mr2sym instead of marking it as an erasure, we’ll need to tell BM to re-calculate the syndromes - which is done using the last argument that I added to KA9Q’s routine. Last night, I found a potentially powerful metric to use as a stopping criterion. Note that the algorithm runs all ntrials before stopping at present. Hence, it is dog-slow, as you found out. I found that a good stopping criterion can be created by calculating nhard-n2ndbest where nhard is the number of symbols where the received symbol vector differs from the decoded codeword (the number of “hard” errors) and n2ndbest is the number of places where the second most reliable symbol is the *same* as the symbol in the decoded codeword. I found that a bad codeword rarely has n2ndbest larger than 2. I also found that a threshold like nhard-n2ndbest<45 (I’m not sure if 45 is the best - but you get the idea) works well. With this in mind - the average running time can be dramatically reduced if we simply stop (break out of the ntrials loop) when we find a codeword that meets this type of criterion. BTW, as currently configured, the ntrial loop breaks out after it finds 5000 codewords. This limits the effectiveness of increasing ntrials to large number (like 100000). To get the full benefit of large ntrials, you would probably need to increase that threshold to 20k or more - or just eliminate the threshold and implement something like the stopping criterion that I mentioned above. It is instructive to watch the results in sfrsd.log scroll by - you can see how sensitive the number of “found” codewords is to the selected erasure probabilities and the metrics… If you have time to play, feel free to mess around with sfrsd - or create an jtrsd and we can merge them later. Steve k9an > On Sep 25, 2015, at 8:11 AM, Joe Taylor <j...@princeton.edu> wrote: > > Hi Steve and all, > > I've added more lines to the table summarizing my tests of decoding > weak, isolated JT65A signals. As before, the final number on each line > is the number of valid decodes from a thousand files at > S/N=-24 dB. > > 1. WSJT-X (BM only) 2 > 2. WSJT (BM only) 5 > 3. WSJT-X + kvasd 189 > 4. WSJT-X + kvasd (thresh0=1, ntest>0) 207 > 5. WSJT-X + sfrsd (Linux) 302 > 6. WSJT-X + sfrsd (Win32) 309 > 7. WSJT-X + sfrsd (Linux, thresh0=1) 348 > 8. WSJT-X + sfrsd (Win32, thresh0=1) 350 > 9. WSJT + kvasd (Linux) 809 > 10.WSJT + kvasd (Win32) 809 > > 11.WSJT + sfrsd (10000) 464 > 12.WSJT + sfrsd (SFM no ntest 10000) 519 > 13.WSJT + sfrsd (SFM no ntest 20000) 543 > 14.WSJT + sfrsd (SFM no ntest 1000) 342 > 15.WSJT + sfrsd (SFM no ntest 100000) 706 > 16.WSJT + sfrsd (SFM no ntest 1000000) 786 (took 11 hours!) > > 17.WSJT + kvasd (SFM no ntest) 897 > > Test 11 simply replaced kvasd with sfrsd, with no other changes. Tests > 12-16 used Steve's metrics for the symbol probabilities in demod64a.f90 > and commented out the following lines in extract.F90: > > ! if(ntest.lt.50 .or. nlow.gt.20) then > ! ncount=-999 !Flag bad data > ! go to 900 > ! endif > > The number of random erasure vectors is specified for each of these > runs. Test 17 is a final run using kvasd. > > With "reasonable" numbers of random erasure trials, sfrsd seems to be > something like 0.5 dB shy of the sensitivity of kvasd. Steve, I know > you have already done some parameter-tuning for sfrsd, but possibly > there's still room for improvement? How did you choose the values 0.5 > 0.6 0.6 0.6 0.8 and the step locations 32 128 196 256, lines 197-207 in > sfrsd.c ? Have you thought about other possible ways to speed up the > search by eliminating some candidate erasure vectors? > > -- Joe, K1JT > > ------------------------------------------------------------------------------ > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel ------------------------------------------------------------------------------ _______________________________________________ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
