Very interesting Joe. Is there any reason to think that JT-65 and WSJT-X use the same symbol metrics?
Here, I am working on writing out an s3.bin file for my 20m .wav files so that I can use rsdtest to generate the erasure probabilities. Steve k9an > On Oct 2, 2015, at 3:25 PM, Joe Taylor <j...@princeton.edu> wrote: > > Hi Steve and all, > > On 10/2/2015 9:50 AM, Steven Franke wrote: > >> Just re-read what I sent last night - I should have said “… 8 or more >> no-decodes and only 2 decodes”. It is likely that many of those >> no-decodes will eventually turn into decodes when the algorithm is >> properly tuned. > > Thanks for the update. I was wondering where your false decodes had come > from, since I haven't seen them. > > > I've been doing something a little different today, motivated by a desire to > understand differences reported by Richard, M0CLZ, in the decoding > performance of JT65-HF-HB9HQX-Edition and WSJT-X. > > A week or so ago I ran JT65-HF in parallel with WSJT-X (v1.6.1 r5912) for > about 15 hours, receiving the JT65 sub-band on 20 meters. (This was an > overnight run, and the band was mostly dead for 5 of those hours.) WSJT-X > saved all the data so I could play with it afterwards. > > The real-time decoding statistics are shown in the first two lines of the > table below. Line 3 shows results on the same data with r5955, which uses > sfrsd rather than kvasd; line 4 used r5955 modified to use BM (hard-decision) > decoding only. > > Correct False > Program Decodes Decodes Decoder > ------------------------------------------ > JT65-HF 2329 24 BM + kvasd > WSJT-X r5912 2249 0 BM + kvasd > WSJT-X r5955 2114 0 BM + sfrsd > WSJT-X r5955 1816 0 BM only > > JT65-HF yielded 104 more decodes that r5912, but 24 of them were false. None > of the runs with WSJT-X produced any false decodes. It's clear that in the > necessary trade-off between maximum number of good decodes and minimum number > of false decodes, these versions of WSJT-X have been tuned less aggressively > than JT65-HF. > > Using the output from the first two runs, I extracted the UTC and message > content from from each decoded text line, then did an "sdiff" of the > resulting two files to produce output shown in the attached file. Even > setting the false decodes aside, I was surprised at how many differences > there are. If I've counted things correctly JT65-HF produced 263 correct > decodes that were missed by WSJT-X, but using the same data WSJT-X produced > 203 decodes that were missed by JT65-HF. > > Sometimes one program seems the clear winner, for a while. For example, > between 2034 and 2055 UTC WSJT-X yielded 204 decodes and JT65-HF only 182. > Then the tables are turned ... > > These were crowded band conditions, most of the time. There are many > overlapping signals, some of them strong. It's not surprising that small > differences in candidate selection criteria, details of time/frequency > synchronization, etc., will cause some differences in output. But it's clear > that in principle both programs could be improved by at least 10%, if maximum > number of decodes is the criterion. > > Note that up to now, fine-tuning of the sdrsd decoder parameters has been > attempted only for weak, idealized daata with no fading. We need to see > whether fine-tuning for real-world HF band conditions makes any significant > difference. > > -- Joe, K1JT > <diff.txt>------------------------------------------------------------------------------ > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel ------------------------------------------------------------------------------ _______________________________________________ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
