Hi Steve, Joe, Can you post the invocations and options you used for the three cases below?
Case 1. wsprd Case 2. wsprd_exp Fano Case 3. wsprd_exp Jelink I realize the source file <-Input locations will be different. I want to play with this a bit and try to come up with a way to use GnuPlot to display results with various input parameters. Thanks 73's Greg, KI7MT On 07/30/2015 09:25 PM, Steven Franke wrote: > Joe, > Here are my results for your data set. I ran 3 cases. The execution times are > the average of two runs. > > Cases > 1. wsprd > 2. wsprd_exp (Fano, 10000 cycles) > 3. wsprd_exp (Jelinek, 5000 cycles) > > Results > 1. 2657 (2) decodes in 359s > 2. 2760 (13) decodes in 359s > 3. 2749 (3) decodes in 346s > > The interesting part is the number in parentheses. This time, I paid > attention to the number of obviously bad decodes. It’s not easy to find the > bad decodes that show up as type 1 callsigns - but it is easy to find and > count the ones that show up as type 2 or 3 callsigns with a forward slash > “/“. The number in parentheses is the number of bad decodes with a slash in > the callsign. It needs to be said that we see only the bad decodes that > aren’t trapped by a sanity check in the unpacking routines. > > There is something funny going on with the Fano decoder in case 2. Here is > the result of doing a grep for “/“ in the ALL_WSPR results from the three > cases: > > Case 1. wsprd > $ grep / Results_wsprd > 0342 -28 0.5 0.001523 0 PH6/OK1SCE 10 > 0630 -14 -0.8 0.001518 0 M0N/BX6IJG 30 > > Case 2. wsprd_exp Fano > $ grep / Results_Fano.txt > 0114 -16 -0.3 0.001524 0 88Y/9E3XMR 33 > 0148 -22 -0.9 0.001523 0 C4S/U23 27 > 0512 -12 -1.4 0.001544 -1 EYJ/BD3OWF 43 > 0526 -5 -1.1 0.001515 -1 J28/JH9VOA 10 > 0530 -10 -1.3 0.001527 -1 XIR/L12IRI 57 > 0534 -21 0.1 0.001451 0 5EY/588TIB 53 > 0540 -9 -1.3 0.001498 -1 286/CI7RCI 13 > 0614 -8 -1.2 0.001523 -1 W64/CZ9IYO 13 > 0616 -31 -1.0 0.001491 0 M2Q/ZG4VPX 13 > 0704 -10 -1.3 0.001550 -1 KN4OHP/44 53 > 0746 -8 -1.4 0.001525 -1 ATD/012KCR 27 > 0856 -19 -0.8 0.001523 0 I02/VK3PNP 20 > 1400 -17 -0.5 0.001459 0 P2INE/2 53 > > Case 3. wsprd_exp Jelinek > $ grep / Results_Jelinek5000.txt > 0114 -16 -0.3 0.001524 0 88Y/9E3XMR 33 > 0616 -31 -1.0 0.001491 0 M2Q/ZG4VPX 13 > 0654 -12 -1.3 0.001523 -1 1LY/GH4 40 > > Note the large number of bad decodes coming out of the Fano decoder in case > 2. There is only one bad decode that is common to cases 2 and 3. If you look > at the times, it appears that the bad decodes in case 2 are coming in bursts. > I have to wonder if this corresponds to special noise conditions, e.g. > lightning storm. > > It’s hard to reconcile the large difference in bad decodes between pairs 1-2 > and 2-3. In 1-2 the decoding algorithm is the same and in 2-3 the candidates > are the same. Strange, eh? > > I’ve just gone back and looked at bad decodes using the same “forward-slash” > criterion on two groups of my own wav files and in each case I see either 2 > or 3 bad decodes out of about 2000 for Fano and Jelinek. There are no big > differences between the number of bad decodes in cases 1-3 for my test data. > Still strange. > > Steve k9an > >> On Jul 30, 2015, at 8:01 AM, Joe Taylor <j...@princeton.edu> wrote: >> >> Hi Greg, >> >> I have updated Makefile.win32 in ^/branches/wsjtx so that it builds >> Steve's new wsprd_exp.exe correctly. >> >> I have made a tarfile with the set of WSPR *.wav files I used most >> recently. It is now posted at >> >> http://physics.princeton.edu/pulsar/K1JT/wspr_data.tgz >> >> It's about 1 GB in size. >> >> -- Joe, K1JT >> >> On 7/29/2015 11:52 PM, KI7MT wrote: >>> Hi Steve, Joe, >>> >>> I hit another show stopper error also. I'll look at what Joe is using in >>> ^/branches/wsjtx_exp and see what the diff's are from the main devel >>> branch. >>> >>> By chance, do you all have a standard set of WSPR .wav files that your >>> using to compare with? Would be nice to be able to use a standard set >>> for testing. >>> >>> 73's >>> Greg, KI7MT >>> >>> >>> >>> On 07/29/2015 07:36 PM, Steven Franke wrote: >>>> Greg - >>>> The windows Makefile has not been updated for the stack decoder. I’ve only >>>> tested it on linux and osx. It looks like the Makefile in Joe’s >>>> experimental branch is close to what would be needed - though it shouldn’t >>>> need the extended stacksize anymore since we moved the big arrays to heap >>>> storage. >>>> Steve k9an >>>> >>>>> On Jul 29, 2015, at 1:22 AM, Greg Beam<ki7m...@gmail.com> wrote: >>>>> >>>>> Hi Joe, Steve, >>>>> >>>>> This is off list. >>>>> >>>>> I tried to build wsprd_exp on Windows (using Qt 5.2.1 Tool-Chain, not the >>>>> 5.5 Tool-Chain) and ran into an error. I'm using >>>>> ^/branches/wsjtx/lib/wsprd folder, and Makefile.win32, is that the >>>>> correct location and Makefile file? >>>>> >>>>> Here's the error I'm getting: >>>>> ----- >>>>> wsprd_exp.o:wsprd_exp.c:(.text.startup+0x244c): undefined reference to >>>>> `jelinek' >>>>> collect2.exe: error: ld returned 1 exit status >>>>> Makefile.win32:34: recipe for target 'wsprd_exp' failed >>>>> mingw32-make: *** [wsprd_exp] Error 1 >>>>> ---- >>>>> >>>>> Any Ideas? >>>>> >>>>> 73's >>>>> Greg, KI7MT >>>>> >>>>> >>>>> >>>>> On 7/28/2015 1:49 PM, Steven Franke wrote: >>>>>> Hi Greg and Joe, >>>>>> >>>>>> As Joe said, the stack decoder is only in wsprd_exp and it requires a >>>>>> command-line argument (-J) to activate it, as the default algorithm in >>>>>> wsprd_exp is the Fano algorithm. >>>>>> >>>>>> The tests conducted by me and Joe show that my implementation of >>>>>> Jelinek’s stack-bucket algorithm doesn’t seem to provide any significant >>>>>> advantage over the Fano decoder in the wspr application. I am still >>>>>> inclined to replace the current wsprd with the Fano version of >>>>>> wsprd_exp. All of my tests indicate that the default configuration of >>>>>> wsprd_exp produces more decodes in less time than wsprd does. This is >>>>>> due to improvements in the sync algorithm and not due to anything >>>>>> related to the sequential decoder. However --- when Joe compared wsprd >>>>>> to wsprd_exp (Fano) he didn’t find any significant difference between >>>>>> the two. If you decide to do some tests, Greg, it’d be interesting to >>>>>> see if you see any difference between wsprd and wsprd_exp (with the >>>>>> default settings). >>>>>> >>>>>> Steve >>>>>> >>>>>>> On Jul 28, 2015, at 2:22 PM, Joe Taylor<j...@princeton.edu> wrote: >>>>>>> >>>>>>> Hi Greg, >>>>>>> >>>>>>> The experimental ("Jelinek") decoder is currently being built only as >>>>>>> wsprd_exp. >>>>>>> -- Joe >>>>>>> >>>>>>> On 7/28/2015 3:08 PM, Greg Beam wrote: >>>>>>>> Hi Joe, Steve, >>>>>>>> >>>>>>>> Are these updates in the main wsprd binary, or do we still need to >>>>>>>> build >>>>>>>> the wsprd_exp binary and copy it over to wsprd to test the changes? >>>>>>>> >>>>>>>> 73's >>>>>>>> Greg, KI7MT >>>>>>>> >>>>>>>> On 7/28/2015 12:22 PM, Joe Taylor wrote: >>>>>>>>> Hi Steve, >>>>>>>>> >>>>>>>>> Nice job with implementation of a sequential decoder using the Jelinek >>>>>>>>> Stack Algorithm! >>>>>>>>> >>>>>>>>> I have now run some reasonably thorough tests of it, comparing results >>>>>>>>> with the default Fano decoder in wsprd. I confirm essentially all of >>>>>>>>> your basic conclusions. Jelinek with maxcycles=5000 produces nearly >>>>>>>>> the >>>>>>>>> same results, in the same execution time, as Fano with >>>>>>>>> maxcycles=10000. >>>>>>>>> >>>>>>>>> In my tests the command-line "-d" option produced about 7-8% more >>>>>>>>> decodes, at the cost of roughly 5 x longer execution time. Again, >>>>>>>>> this >>>>>>>>> was true for both Fano and Jelinek. >>>>>>>>> >>>>>>>>> Now we know... which is good! >>>>>>>>> >>>>>>>>> -- Joe, K1JT >>>>>>>>> >>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>> _______________________________________________ >>>>>>>>> wsjt-devel mailing list >>>>>>>>> wsjt-devel@lists.sourceforge.net >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>>>> >>>>>>>> ------------------------------------------------------------------------------ >>>>>>>> _______________________________________________ >>>>>>>> wsjt-devel mailing list >>>>>>>> wsjt-devel@lists.sourceforge.net >>>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>>> ------------------------------------------------------------------------------ >>>>>>> _______________________________________________ >>>>>>> wsjt-devel mailing list >>>>>>> wsjt-devel@lists.sourceforge.net >>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> _______________________________________________ >>>>>> wsjt-devel mailing list >>>>>> wsjt-devel@lists.sourceforge.net >>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>> >>>> >>> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> wsjt-devel mailing list >> wsjt-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > ------------------------------------------------------------------------------ > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- ---------------------------------------------------------------- Launchpad....: https://launchpad.net/~ki7mt Ubuntu Hams..: https://launchpad.net/~ubuntu-hams-devel Debian Hams..: https://alioth.debian.org/projects/pkg-hamradio/ JTSDK........: https://sourceforge.net/projects/jtsdk/ OpenPGP......: C177 6630 7115 78FE 9A2B 9F7F 18C0 F6B7 0DA2 F991 ------------------------------------------------------------------------------ _______________________________________________ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
