JT9 is multithreaded.  Printf() is thread-safe but it is not atomic.  So you 
need to use a mutex to access it safely from multiple threads.
Mike W9MDB

-----Original Message-----
From: Steven Franke [mailto:s.j.fra...@icloud.com] 
Sent: Saturday, September 12, 2015 11:59 PM
To: WSJT software development
Subject: Re: [wsjt-devel] update on sfrsd

Joe and Bill,

I am running wsjt-x (wsjtx_exp branch) using my decoder program (sfrsd) renamed 
to kvasd. It seemed to work OK at first, although the decodes that wsjt-x 
displays are shown in a different order than they are when I use the “real” 
kvasd.  

Now, in addition to the results coming out in a different order, I have started 
to get segmentation faults and strange errors. This started after I 
significantly increased the amount of data that I am writing to a log file 
(/tmp/sfrsd.log) using fprintf statements from within sfrsd. I notice that the 
contents of the log file are garbled - as if more than one instance of the 
program was trying to write to it at the same time. Are you spawning multiple 
instances of kvasd? Is there anything that I need to do to my program to make 
it safe to run when called from wsjt-x?

Steve k9an

> On Sep 12, 2015, at 4:02 PM, Steven Franke <s.j.fra...@icloud.com> wrote:
> 
> Joe,
> 
> FYI, examining the results a bit more, I notice that sfrsd decoded cases 
> containing as many as 38 bad symbols (out of 63). I haven’t done the same 
> study for kvasd - it doesn’t return the total number of errors, unfortunately.
> 
> More important, I also notice that the vast majority of the successful 
> decodes take less than 100 trials (that’s the number of erasure locator 
> vectors that we have to try before we get lucky and hit one that will decode) 
> - whereas the cutoff is set to 10000 at the moment. This points the way 
> toward achieving a very significant speedup. Note the following:
> 
> Total number of calls to sfrsd: 1408
> Total number of successful decodes: 104
> 
> Of the 1304 unsuccessful decodes, a large fraction (probably 90%) consist of 
> long strings of nearly identical rx vectors. I am using your suggestion to 
> rename sfrsd to kvasd for the purpose of running these tests, so your birdie 
> removal should be operative. I have seen as many as 15 un-decodable and 
> identical rx vectors from one file. Since most of the successful decodes 
> finish after fewer than 100 trials, whereas every single un-decodable one 
> will run out the full 10000 trials, much less than 1% of the execution time 
> is actually spent on decodable vectors. So there’s a lot to gain if we can 
> figure out a way to reduce the number of un-necessary calls to the decoder. 
> 
> As I noted in the last message, I have a long list of things that I can do to 
> speed up the execution time, but none of them promise the huge improvement 
> available if we can cut out a bunch of those bad rx vectors. I can certainly 
> look at this --- I don’t mean to throw it in your lap --- but I’m guessing 
> that you may already have some insight into what’s going on.
> 
> Steve k9an
> 
>> On Sep 12, 2015, at 1:04 PM, Steven Franke <s.j.fra...@icloud.com> wrote:
>> 
>> Hi Joe -
>> Since you asked about where the sfrsd routine stands with respect to kvasd, 
>> I did a quick test this morning. Before I ran the test, I found and fixed a 
>> fundamental flaw in the sfrsd routine that significantly improved its 
>> performance. Then I did a run with the number of trials set to 10000, which 
>> makes it very slow (it takes a long time to time out) but I wanted to see 
>> where we stand in terms of decoding probability, irrespective of decoding 
>> time. The results are encouraging. Here’s a brief summary of the results 
>> from my set of 212 .wav files:
>> 
>> baseline (BM HDD only): 618 decodes
>> kvasd: 104 decodes (this is above and beyond the 618 BM decodes)
>> sfrsd: 104 decodes (2 bad decodes)
>> 
>> kvasd decoded 14 cases that were not decoded by sfrsd.
>> sfrsd decoded 12 cases (not counting 2 bad ones) that were not 
>> decoded by kvasd
>> 
>> This result is obtained using a very simple-minded algorithm, which can be 
>> described in just a few lines:
>> 
>> 1. sort the symbol probabilities
>> 2. among the 51 smallest symbol probabilities, randomly select a subset and 
>> set them as “erasures”
>> 3. try to decode with BM
>> 4. if decode succeeds, quit. Otherwise go to 2.
>> 
>> This algorithm only makes use of the symbol probabilities to identify the 51 
>> symbols that are erasure candidates. Obviously, one can do a better job by 
>> using the symbol quality to determine the probability of setting that symbol 
>> as an erasure. That’s the next step.
>> 
>> I am not too worried about the bad decodes at this stage. The algorithm 
>> should be enhanced such that it saves a list of all of the codewords 
>> produced by the BM algorithm. It should then select the codeword with the 
>> best metric (e.g. some measure of distance from the received symbol vector). 
>> It should also apply a threshold to reject codewords that are too “far” away 
>> from the received codeword. I have not implemented any of that yet. 
>> Steve k9an
>> 
>>> On Sep 11, 2015, at 12:11 PM, Steven Franke <s.j.fra...@icloud.com> wrote:
>>> 
>>> Hi Joe,
>>> 
>>> Re the sfrsd program. Please regard this as merely a framework for future 
>>> development. Nothing in the current code is optimized and I am almost 
>>> certain that I can do better. I think that it is premature to do any 
>>> quantitative performance comparisons with KV.  I will say that it was very 
>>> encouraging to see how easy it is to come up with something that provides 
>>> useful gains via very simple use of soft-symbol information.
>>> 
>>> Regarding the prospects for improving on KV - long story short - I am 
>>> convinced that the class of algorithms that I am playing with can be tuned 
>>> to be competitive with KV if we are willing to wait long enough for the 
>>> algorithm to run to completion. What I don’t know yet is if it will be 
>>> possible to come up with something that runs fast enough to be useful. 
>>> 
>>> My next step will be to pick apart the existing KA9Q Berlekamp Massey code 
>>> so that we can eliminate some redundant calculations. In particular, I 
>>> recently realized that the syndromes only have to be calculated once for 
>>> each received symbol vector, whereas I am now re-calculating them thousands 
>>> of times in the main loop.  I plan to implement the current sfrsd algorithm 
>>> with this improvement and then move on to improving algorithm. I’ll let you 
>>> know when I’ve got it to a point where I think that it’s worth comparing to 
>>> KV. 
>>> 
>>> By the way, one thing that I’ve already noticed is that some files produce 
>>> lots of identical un-decodable received symbol vectors. Since I have to try 
>>> a whole bunch of virtual received signal vectors before I can decide that 
>>> these are un-decodable, my decoder bogs down on these whereas KV seems to 
>>> be able to blow right past them. I’m guessing that these events result from 
>>> interference, or birdies - I can’t think of anything else that would 
>>> produce a slew of identical vectors. It may eventually be necessary to 
>>> figure out a way to identify these events before they are sent to the 
>>> decoder. 
>>> 
>>> Re JTMSK - I am very interested in it, but I haven’t had time to look at 
>>> what you are doing there. If you have carefully read the relevant sections 
>>> of Proakis, then you certainly know more about CPFSK techniques than I do. 
>>> I am aware of the fact that it should be possible, in principle, to do some 
>>> form of block demodulation, perhaps using the Viterbi algorithm, as opposed 
>>> to symbol-by-symbol demodulation. While this could produce significant 
>>> gains - it would seem that this approach will require the channel to be 
>>> reasonably well behaved (i.e. no significant frequency offset or random 
>>> phase walks), no?  Off-list I’ll send you a manuscript that does some 
>>> comparisons of different demodulation schemes. I haven’t yet had time to 
>>> distill it down to any simple wisdom - maybe you’ll get something useful 
>>> out of it.
>>> 
>>> 73, Steve k9an
>>> 
>>> 
>>> 
>>> On Sep 11, 2015, at 8:54 AM, Joe Taylor <j...@princeton.edu> wrote:
>>>> 
>>>> Hi Steve,
>>>> 
>>>> I finally found time for a quick look at "sfrsd", your drop-in 
>>>> replacement for kvasd that uses the stochastic Chase algorith.
>>>> 
>>>> I compiled and ran it on a few test cases this morning, and (as 
>>>> will be no surprise to you) found that it works, and it succeeds in 
>>>> some cases where the hard-decision Berlekamp-Massey algorithm 
>>>> fails.  Just like kvasd... and no patents, non-disclosure 
>>>> agreements, or non-open source code to worry about!  Very impressive!!
>>>> 
>>>> I have not yet done any thorough tests of sensitivity relative to 
>>>> the Koetter-Vardy algorithm.  Have you?  Are there any theoretical 
>>>> reasons to expect that it should be (or not be) as good as K-V?
>>>> 
>>>> I will do further tests, as time permits.
>>>> 
>>>> 
>>>> On another matter, somewhat along the same lines:
>>>> 
>>>> Have you looked at all at the way I am presently decoding the MSK 
>>>> signals in JTMSK?  Especially since about r5848, I think the 
>>>> decoder is pretty good; but since the demodulation process does not 
>>>> yet take full advantage of signal coherency or the inherent 
>>>> symbol-to-symbol "memory"
>>>> in MSK modulation, I think a still better decoder is possible.
>>>> 
>>>> From reading Proakis I'm aware that (quite apart from the Viterbi 
>>>> algorithm used to decode the K=13, r=1/2 convolutional code of 
>>>> JTMSK) a Viterbi algorithm could also be used as an optimum 
>>>> receiver for these continuous-phase signals.  I also understand 
>>>> that using modulation index
>>>> h=0.715 (rather than the h=0.5 of MSK) might have advantages while 
>>>> still requiring bandwith no more than 2 kHz.
>>>> 
>>>> Do you have any opinions here?  Any interest in looking into these 
>>>> questions further?
>>>> 
>>>>    -- 73, Joe, K1JT
>>>> 
>>>> -------------------------------------------------------------------
>>>> ----------- _______________________________________________
>>>> wsjt-devel mailing list
>>>> wsjt-devel@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
>>> 
>>> 
>>> --------------------------------------------------------------------
>>> ---------- _______________________________________________
>>> wsjt-devel mailing list
>>> wsjt-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
>> 
>> 
>> ---------------------------------------------------------------------
>> --------- _______________________________________________
>> wsjt-devel mailing list
>> wsjt-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
> 
> 
> ----------------------------------------------------------------------
> -------- _______________________________________________
> wsjt-devel mailing list
> wsjt-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wsjt-devel


------------------------------------------------------------------------------
_______________________________________________
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel


------------------------------------------------------------------------------
_______________________________________________
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel

Reply via email to