Hi Joe - I will likely not make much progress on the decoder stuff til next week, so I wanted to pass along the following information in case that you get around to doing some tests — I’ve tested the decoder on 2 separate batches of HF .wav files and also on 3 batches of SimJT weak-signal gaussian-noise no-fading files at SNR’s -23, -24, -25.
For the SimJT weak-signal cases, I find that the using the most reliable symbol probability (p1) as the metric for deciding on erasure probabilities works better than using the difference (p1-p2), where p2 is the second-best symbol probability. In fact, using the difference is no better than no metric (no metric means setting all erasure frequency probabilities equal to 0.6). On the other hand, the situation is reversed on the HF files - the difference (p1-p2) works best for both batches of HF files. The next step for me will be to see if I can figure out to better utilize the symbol quality information, and to devise a metric that is “best” in both cases. In the meantime, you may want to just use p1 or p2-p1 as appropriate. Here’s a brief summary of some more results. Using SimJT, 1000 JT65A files, and sfrsd with 10000 trials and p1 as metric: -25 dB SNR - kvasd 3/1000, sfrsd 2/1000 -24 dB SNR: kvasd decodes 198/1000, sfrsd 182/1000 -23 dB SNR: kvasd decodes 835/1000, sfrsd 840/1000 Steve k9an > On Sep 20, 2015, at 2:50 PM, Joe Taylor <j...@princeton.edu> wrote: > > Hi Steve, > > Congratulations on a really first-rate piece of work!! > > I hope to find some time next week (or possibly the week after) to do > some independent tests -- presumably, to confirm your results. > > I believe your test used simulated data on an AWGN channel. It might be > useful to try also > > - simulated, near threshold signals with Rayleigh fading > - real EME signals > - a standard set of files each containing many on-the-air JT65 signals > > Yes, we certainly should do a parallelized version for multi-core > machines. As Bill already pointed out, there's no need for this > processing step to remain a separate, stand-alone program; it should > simply be called as a C (or possibly C++) function. > > -- Joe, K1JT > > On 9/20/2015 11:11 AM, Steven Franke wrote: >> Joe - >> For the record, here are the results of analyzing 1000 SimJT JT65A >> files with sync level set to 0 (a setting of -1 gave the same >> results) and with Rx frequency set equal to the simulated signal’s frequency: >> >> kvasd (this result should be for the high-effort “on frequency” setting): >> 198/1000 (20%) >> sfrsd 5000 trials, p1 metric, erasure probabilities 0.5, 0.6, 0.6, 0.6, 0.8: >> 157/1000 (16%) >> sfrsd 10000 trials, p1 metric, erasure probabilities 0.5, 0.6, 0.6, 0.6, >> 0.8: 182/1000 (18%) >> sfrsd 20000 trials, p1 metric, erasure probabilities 0.5, 0.6, 0.6, 0.6, >> 0.8: 201/1000 (20%) >> >> - Overall yield roughly doubled when the sync threshold was lowered from the >> wideband setting to the on-frequency setting. >> >> - The results confirm that sfrsd is capable of achieving the same decode >> probability as kvasd on weak simulated signals. >> >> You have probably already noticed the fact that the sfrsd should be easy to >> parallelize. It should be possible to test a bunch of candidate vectors at >> once. Perhaps that is something worth investigating for sfrsd. In an earlier >> thread, Bill had suggested that using Qt’s threading facility is the way to >> go, in general, for WSJT-X — but it’s not clear to me if that applies for >> this stand-alone C program. >> >> Steve k9an >> >>> On Sep 19, 2015, at 5:03 PM, Steven Franke<s.j.fra...@icloud.com> wrote: >>> >>> I have just now realized that the sync threshold is higher for “off >>> frequency” signals. Since I had set Rx freq to 3000, the results reported >>> below were obtained using the higher wideband sync threshold. That probably >>> explains the lower than expected yield. I’m re-running with Rx frequency >>> set equal to the simulated signal’s frequency… >>> Steve k9an >>> >>>> On Sep 19, 2015, at 2:57 PM, Steven Franke<s.j.fra...@icloud.com> wrote: >>>> >>>> Thanks Joe! This has been a fun project, and I’m learning a lot from it. >>>> >>>> I used your SimJT program to generate 1000 JT65A files at -24 dB SNR. >>>> Here’s what I get: >>>> >>>> kvasd (with Rx frequency set to 3000, so this should be the low-effort >>>> setting): 106 decodes out of 1000 (10.6%) >>>> sfrsd 5000 trials: 94 decodes (9.4%) >>>> sfrsd with 10000 trials: 112 decodes (11.2%) >>>> >>>> I expected higher percentages overall at this SNR but, in any case, the >>>> results seem to confirm that sfrsd is a viable decoder. I noticed that >>>> only a fraction of the files result in a vector being submitted to the >>>> decoder - maybe around 50% - I didn’t count carefully. Maybe this is >>>> another reason to take a look at the upstream processing. I think that >>>> your earlier results showed something closer to 40% at this SNR. >>>> >>>> Also, I am surprised at how little guidance the soft information is >>>> providing with the settings that we are using. Almost none, as it turns >>>> out. Note that if I change just the two probabilities that I use to set >>>> the erasure frequency of the “best” and “worst” symbols, such that all of >>>> the probabilities are the same, then we wouldn’t even need to sort the >>>> data according to probability (for the stochastic patterns), i.e. we would >>>> be using no soft information whatsoever! The results would change - but >>>> we’d be in the same ballpark. This makes me feel like one or both of the >>>> following might be true: >>>> >>>> 1. it may be possible to achieve the same results with fewer trials if the >>>> soft-symbol information was utilized in a smarter way >>>> >>>> and/or >>>> >>>> 2. the quality of the presently used soft-symbol information is so bad >>>> that it’s not worth using. >>>> >>>> In 2, I do not mean to say that there is something wrong with the signal >>>> processing. Instead, I’m wondering if information only on 2 of 63 symbols >>>> is just too watered down to be useful. >>>> >>>> Steve k9an >>>> >>>> >>>>> On Sep 19, 2015, at 2:11 PM, Joe Taylor<j...@princeton.edu> wrote: >>>>> >>>>> Hi Steve, >>>>> >>>>> I've looked again at the innards of sfrsd. I'm *much* impressed by what >>>>> you have done. >>>>> >>>>> Soon, it may be time to look again at the upstream decisions made in the >>>>> JT65 decoding chain -- decisions that determine what symbol vectors are >>>>> passed on to the actual decoder. Among other things to consider: >>>>> various parameters affecting those decisions were optimized (a long time >>>>> ago) for the EME situation with very weak signals, little if any QRM, >>>>> and only one or two signals in the Rx passband at a time. For HF use we >>>>> might want to change some of these parameters. >>>>> >>>>> -- Joe, K1JT >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> _______________________________________________ >>>>> wsjt-devel mailing list >>>>> wsjt-devel@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> _______________________________________________ >>>> wsjt-devel mailing list >>>> wsjt-devel@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> wsjt-devel mailing list >>> wsjt-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> wsjt-devel mailing list >> wsjt-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > ------------------------------------------------------------------------------ > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel ------------------------------------------------------------------------------ _______________________________________________ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
