Hi Steve,
My tests with the 1000 SimJT files were done using a narrow frequency
window, 1270 +/- 20 Hz. (The sync tone of the generated signal is at
1270.5 Hz.) The un-modified r5970 passed 992/1000 beyond the "thresh0"
filter; 990 of these produced just one synchronized candidate, and two
produced two candidates. Of the 994 total candidates, 293 were rejected
by this test in extract.f90:
if(ntest.lt.1300) then ...
The remaining 701 candidates were submitted to sfrsd, and 476 produced
good decodes.
For these test files the "ntest" criterion is too stringent, so I
removed the test commenting it out. Then all 994 candidates are
submitted for decoding, and 662 produced valid decodes.
I conclude that for these files the candidate selection is OK
(preferably with a somewhat higher threshold for ntest), but sfrsd is
not decoding as many as it "should". I suspect that for marginal
signals either different metrics or different values in the probability
matrix will yield better results.
-- Joe
On 10/15/2015 3:06 PM, Steven Franke wrote:
> Hi Joe -
> Welcome back!
>
> Your results are consistent with mine. I got 488 decodes for the -24dB files
> using the current r5970.
>
> After mulling over the results for a few days, and after doing one more
> experiment (results to follow), I’ve concluded that the new
> candidate-identifying-scheme is just not presenting as many good candidates
> as the old one. The latest experiments that I did were two more runs on my
> batch of hf files: (i) version 1.5 (stock, no modifications) and (ii) r5970
> with thresh0 lowered to 1.5. Results are:
>
> 1. v1.5 BM only: 2551 decodes
> 2. v1.5 BM+kvasd: 3128 decodes (577
> soft, 18.4% of total)
> 3. v1.6.1 r5970 threshold=1.5 BM only: 2387 decodes
> 4. v1.6.1 r5970 threshold=1.5 BM+sfrsd2: 3020 decodes (633 soft, 21.0% of
> total)
>
> The two most significant aspects of these results seem to be (a) 164 more BM
> only decodes from v1.5 and (b) increased percentage of soft-symbol decodes in
> v1.6.1. I take (a) to mean that we were better off with the original
> candidate selection scheme and I have been taking (b) as indication that the
> soft-decoding is working well, perhaps better than kvasd with the settings
> used in v1.5, even with ntrials=2000 as is currently the default in r5970.
>
> I decided to wait to see if you reached the same conclusions before taking
> any next steps…
>
> Steve k9an
>
>> On Oct 15, 2015, at 1:35 PM, Joe Taylor<j...@princeton.edu> wrote:
>>
>> Hi Steve and all,
>>
>> In coming days I hope to catch up with your work on sfrsd. I haven't
>> yet tested r5970 under crowded-band, HF-style conditions.
>>
>> I did make a quick test on my group of single-signal 1000 files
>> generated by SimJT, with S/N=-24 dB. The program ran well and was fast,
>> but the results were only fair: 476/1000 decodes. Revision 5942
>> produced 837/1000 good decodes (and NO bad decodes) with ntrials=10000.
>> Of course we can increase ntrials, though my decodes-vs-ntrials graph
>> http://physics.princeton.edu/pulsar/K1JT/decodes_vs_ntrials3.pdf
>> suggests that this is not the issue. Do you think we may need to use
>> different erasure probabilities for HF and EME-like conditions?
>>
>> -- Joe
>>
>> On 10/11/2015 5:11 PM, Steven Franke wrote:
>>> Hi Joe and all,
>>>
>>> This message summarizes my recent work on jt65 decoding in 1.6.1.
>>>
>>> I’ve added 3 new entries to the table summarizing JT-65a decoding results
>>> on my batch of 333 20m .wav files.
>>>
>>> Program Good Bad Soft Decoder
>>> --------------------------------------------------------------------------------------------------------
>>> 1. WSJT-X r5922 3125 0 574 BM+kvasd (18.4% kvasd)
>>> 2. WSJT-X r5922 3123 2 572 BM+sfrsd (18.4% sfrsd,
>>> ntrials=10000)
>>> 3. WSJT-X r5922 2551 0 0 BM only
>>> 4. WSJT-X r5955 2704 0 482 BM+sfrsd2 (17.8% sfrsd,
>>> ntrials=5000)
>>> 5. WSJT-X r5955 2222 0 BM only
>>> 6. WSJT-X r5970' 2352 0 BM only (ntrials set equal
>>> to 0, thresh0=2.5)
>>> 7. WSJT-X r5970 2930 ? 579 BM+sfrsd2 (19.7% sfrsd,
>>> thresh0=2.5, ntrials=2000)
>>> 8. WSJT-X r5970’ 3013 ? ? BM+sfrsd2 (thresh0=1.5,
>>> ntrials=2000)
>>>
>>> The r5970 version includes a number of tweaks:
>>> - threshold is now set to 2.5, ntrials is 2000
>>> - a couple of fixes for out-of-bounds errors related to the switch to the
>>> sync65 routing that I experienced on my data set
>>> - new erasure probabilities for use with the simple “sf” symbol metrics
>>> - simplification of the codeword acceptance criteria in sfrsd2 and
>>> extract - all acceptance tests moved into sfrsd2.
>>>
>>> I did many runs (my notes from this weekend include 25 runs with various
>>> tweaks and changes) and concluded that, for my data, I got better results
>>> using matched sf symbol metrics and erasure probabilities — ymmv.
>>>
>>> While performance is significantly better in r5970 than it was in r5955,
>>> the overall number of decodes is still less than what it was back in r5922.
>>> On the other hand r5970 is running much faster than r5922 did, because
>>> r5922 used ntrials=10000.
>>>
>>> Case 8 shows that we can obtain 83 more decodes by lowering the threshold
>>> to 1.5, but execution time increases significantly because of the larger
>>> number of spurious syncs.
>>>
>>> Also, comparison of cases 6 and 7 shows that the percentage of total
>>> decodes obtained using sfrsd2 is almost 20% in r5970, and is higher than
>>> previously seen with kvasd or sfrsd in r5922.
>>>
>>> Steve k9an
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> wsjt-devel mailing list
>>> wsjt-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> wsjt-devel mailing list
>> wsjt-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> wsjt-devel mailing list
> wsjt-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
------------------------------------------------------------------------------
_______________________________________________
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
