Hi Joe - Welcome back! Your results are consistent with mine. I got 488 decodes for the -24dB files using the current r5970.
After mulling over the results for a few days, and after doing one more experiment (results to follow), I’ve concluded that the new candidate-identifying-scheme is just not presenting as many good candidates as the old one. The latest experiments that I did were two more runs on my batch of hf files: (i) version 1.5 (stock, no modifications) and (ii) r5970 with thresh0 lowered to 1.5. Results are: 1. v1.5 BM only: 2551 decodes 2. v1.5 BM+kvasd: 3128 decodes (577 soft, 18.4% of total) 3. v1.6.1 r5970 threshold=1.5 BM only: 2387 decodes 4. v1.6.1 r5970 threshold=1.5 BM+sfrsd2: 3020 decodes (633 soft, 21.0% of total) The two most significant aspects of these results seem to be (a) 164 more BM only decodes from v1.5 and (b) increased percentage of soft-symbol decodes in v1.6.1. I take (a) to mean that we were better off with the original candidate selection scheme and I have been taking (b) as indication that the soft-decoding is working well, perhaps better than kvasd with the settings used in v1.5, even with ntrials=2000 as is currently the default in r5970. I decided to wait to see if you reached the same conclusions before taking any next steps… Steve k9an > On Oct 15, 2015, at 1:35 PM, Joe Taylor <j...@princeton.edu> wrote: > > Hi Steve and all, > > In coming days I hope to catch up with your work on sfrsd. I haven't > yet tested r5970 under crowded-band, HF-style conditions. > > I did make a quick test on my group of single-signal 1000 files > generated by SimJT, with S/N=-24 dB. The program ran well and was fast, > but the results were only fair: 476/1000 decodes. Revision 5942 > produced 837/1000 good decodes (and NO bad decodes) with ntrials=10000. > Of course we can increase ntrials, though my decodes-vs-ntrials graph > http://physics.princeton.edu/pulsar/K1JT/decodes_vs_ntrials3.pdf > suggests that this is not the issue. Do you think we may need to use > different erasure probabilities for HF and EME-like conditions? > > -- Joe > > On 10/11/2015 5:11 PM, Steven Franke wrote: >> Hi Joe and all, >> >> This message summarizes my recent work on jt65 decoding in 1.6.1. >> >> I’ve added 3 new entries to the table summarizing JT-65a decoding results on >> my batch of 333 20m .wav files. >> >> Program Good Bad Soft Decoder >> -------------------------------------------------------------------------------------------------------- >> 1. WSJT-X r5922 3125 0 574 BM+kvasd (18.4% kvasd) >> 2. WSJT-X r5922 3123 2 572 BM+sfrsd (18.4% sfrsd, >> ntrials=10000) >> 3. WSJT-X r5922 2551 0 0 BM only >> 4. WSJT-X r5955 2704 0 482 BM+sfrsd2 (17.8% sfrsd, >> ntrials=5000) >> 5. WSJT-X r5955 2222 0 BM only >> 6. WSJT-X r5970' 2352 0 BM only (ntrials set equal >> to 0, thresh0=2.5) >> 7. WSJT-X r5970 2930 ? 579 BM+sfrsd2 (19.7% sfrsd, >> thresh0=2.5, ntrials=2000) >> 8. WSJT-X r5970’ 3013 ? ? BM+sfrsd2 (thresh0=1.5, >> ntrials=2000) >> >> The r5970 version includes a number of tweaks: >> - threshold is now set to 2.5, ntrials is 2000 >> - a couple of fixes for out-of-bounds errors related to the switch to the >> sync65 routing that I experienced on my data set >> - new erasure probabilities for use with the simple “sf” symbol metrics >> - simplification of the codeword acceptance criteria in sfrsd2 and extract >> - all acceptance tests moved into sfrsd2. >> >> I did many runs (my notes from this weekend include 25 runs with various >> tweaks and changes) and concluded that, for my data, I got better results >> using matched sf symbol metrics and erasure probabilities — ymmv. >> >> While performance is significantly better in r5970 than it was in r5955, the >> overall number of decodes is still less than what it was back in r5922. On >> the other hand r5970 is running much faster than r5922 did, because r5922 >> used ntrials=10000. >> >> Case 8 shows that we can obtain 83 more decodes by lowering the threshold to >> 1.5, but execution time increases significantly because of the larger number >> of spurious syncs. >> >> Also, comparison of cases 6 and 7 shows that the percentage of total decodes >> obtained using sfrsd2 is almost 20% in r5970, and is higher than previously >> seen with kvasd or sfrsd in r5922. >> >> Steve k9an >> ------------------------------------------------------------------------------ >> _______________________________________________ >> wsjt-devel mailing list >> wsjt-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > ------------------------------------------------------------------------------ > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel ------------------------------------------------------------------------------ _______________________________________________ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
