Re: [Therion] Wrong average calculation from compass/backcompass data
The correct way to average bearings is to the treat them as vectors, then average the x and y components separately before converting the resulting vector back to a bearing. It is likely this is what the trig functions mentioned in this thread are doing. Footleg On Fri, 18 May 2018 at 12:57, Bruce Mutton via Therionwrote: > I agree to some extent with both sentiments below, however the software > should not abort just because of backsight discrepancies. > > What about adding one more line to the Therion and or Survex log file; > either > > “There are no backsights in the centreline”, or > > “Maximum backsight discrepancy is xx [degrees|grads]”, as approprite > > That way the user can be fully informed and make their own decision about > whether to investigate further if the information is not what they expect. > > Bruce > > > > 17. 5. 2018 v 9:25, Evaristo Quiroga via Therion : > > > > In this case is not a problem with my formula, is a serious magnetic > anomaly (100 degrees difference) and the program should to stop and to > send a warning. > > > > Evaristo, > > > > Therion is a program to interprete your data, not to solve problems with > your data. > > > > It is your responsibility what the data you import into Therion. > > > > Martin S. > ___ > Therion mailing list > Therion@speleo.sk > https://mailman.speleo.sk/listinfo/therion > ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
[Therion] Wrong average calculation from compass/backcompass data
I agree to some extent with both sentiments below, however the software should not abort just because of backsight discrepancies. What about adding one more line to the Therion and or Survex log file; either "There are no backsights in the centreline", or "Maximum backsight discrepancy is xx [degrees|grads]", as approprite That way the user can be fully informed and make their own decision about whether to investigate further if the information is not what they expect. Bruce 17. 5. 2018 v 9:25, Evaristo Quiroga via Therion>: In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning. Evaristo, Therion is a program to interprete your data, not to solve problems with your data. It is your responsibility what the data you import into Therion. Martin S. ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
On Thu, May 17, 2018 at 09:25:23AM +0200, Evaristo Quiroga wrote: > I attach the original file with the fore/back sight (SCAGUA.th) and a > manually averaged (SCAGUA_avg.th) to compare if Survex and Therion work > well. The error loop is similar between the two files (2,9m) but I have to > check if the station are positioned at the same coordinates. I feel I should point out that Survex's handling of backsights isn't coming in to play here at all - Therion flattens away the backsights so they don't appear in what it asks Survex to process. Out of interest I did a crude conversion of the two attached files to Survex format and processed them natively, and with scagua.279.0 fixed in each case the maximum discrepancy between the two results is: Moved by (-0.11,0.13,0.03): scagua.a.10 Which seems entirely plausible - I wouldn't expect exact agreement in this case as some of the data has backsights and some doesn't. In the data from SCAGUA.th, Survex will treat the data with backsights as slightly more accurate (because the average of two essentially independent readings is expected to be more accurate than a single reading) and that affects the distribution of loop misclosures. Cheers, Olly ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
Hi, Ok OIly and Wookey, you are right, my formula has not been very lucky. It was an attempt to understand what error he was experiencing when compiling data with fore/back bearings. I found a large difference in loop closure error between Auriga (2.85 m in 1967m loop), Compass (2.9m) and Therion (14m). Debugging the compilation I have found that the data transferred to Survex (data.svx), don't have sense to me. I have searched in the code source and found the complicated formula. I made my own formula in a Excell, and calculate the average. With this I have detected a strong magnetic anomaly in one leg (Auriga has this bearing manually discarded, and the translation to Compass was OK, but to Therion the fore-bearing was translated with a 0, a 56 degrees difference with the inverse back-bearing). After to correct the magnetic anomaly and to calculate the fore-bearing with the inverse back-bearing, the loop error it's OK too in Therion (2.9m). Only that it was necessary to understand what numbers Therion was going through to Survex, and that some degrees were deviating from those calculated manually. With the two files attached in the previous message: -a file with magnetic declination to 0 and fore/back bearings -a file with DM=0 and my average bearings. I have compared the results, that are the same. After looking carefully at the files and the compilation log, I found the difference. The data transferred to Survex are: bearing - magnetic declination - meridian convergence. These two components are what made me see different data between those calculated by Therion and those calculated manually. After some more checks, I have verified that Therion is correctly calculating the average between fore/back bearings. MY MISTAKE. The issue is closed. Best regards and thanksfor the patience. Evaristo. El 17/05/2018 a las 15:02, Wookey via Therion escribió: On 2018-05-17 09:25 +0200, Evaristo Quiroga via Therion wrote: El 17/05/2018 a las 0:36, Olly Betts escribió: I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2 Your proposed formula gives wrong answers in some cases - consider: bearing = 80, backbearing = 0 These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180). In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning. Yes a warning should probably be issued about poor data, but it _is_ a problem with your formula. It's just an example showing that all cases have to be dealt with correctly, including the wrap-around at 0/360 (or 0/400 for grads) and your simplified formula doesn't. An example with a much smaller difference between back and foresight could still be constructed to show the issue: Fore: 175, Back: 0 AverageBearing= (175+0-180)/2 = -2.5. That's wrong. It should be 177.5 in this case. Wookey ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
On 2018-05-17 09:25 +0200, Evaristo Quiroga via Therion wrote: > El 17/05/2018 a las 0:36, Olly Betts escribió: > I think the formula is too complicated. I purpose a simpler formula, > like: > If bearing <=180 > AverageBearing = (bearing + (backbearing > -180))/2 > else > AverageBearing = (bearing + (backbearing > +180))/2 > > Your proposed formula gives wrong answers in some cases - consider: > > bearing = 80, backbearing = 0 > > These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to > 310), but this should be 130 (average of 80 and 180). > > > In this case is not a problem with my formula, is a serious magnetic anomaly > (100 degrees difference) and the program should to stop and to send a > warning. Yes a warning should probably be issued about poor data, but it _is_ a problem with your formula. It's just an example showing that all cases have to be dealt with correctly, including the wrap-around at 0/360 (or 0/400 for grads) and your simplified formula doesn't. An example with a much smaller difference between back and foresight could still be constructed to show the issue: Fore: 175, Back: 0 AverageBearing= (175+0-180)/2 = -2.5. That's wrong. It should be 177.5 in this case. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
> 17. 5. 2018 v 9:25, Evaristo Quiroga via Therion: > > In this case is not a problem with my formula, is a serious magnetic anomaly > (100 degrees difference) and the program should to stop and to send a > warning. Evaristo, Therion is a program to interprete your data, not to solve problems with your data. It is your responsibility what the data you import into Therion. Martin S.___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
Hi Olly, El 17/05/2018 a las 0:36, Olly Betts escribió: I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2 Your proposed formula gives wrong answers in some cases - consider: bearing = 80, backbearing = 0 These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180). In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning. The therion formula is attempting to average the angles by trigonometry, which seems a reasonable approach (though probably slower than trying to average more directly like you're suggesting). It looks essentially correct to me, though it's a bit oddly written since cos(90 - x) is sin(x), sin(90 - x) is -cos(x), 90 - atan2(y,x) is atan2(x,y) (possibly +/- a multiple of 360). But if I work it out with a calculator for 236.8 and 56.8 then I do get 236.8. In the afternoon I have detected a strong anomaly problem in one station and corrected manually the data. Now the loop error is similar to Auriga and Compass (2.9m for a 1967m loop). But I have to check if Therion do the correct average calculation. If it's still not working with the declination set to zero, perhaps you should show us a complete small example we can process to see what is going on? I attach the original file with the fore/back sight (SCAGUA.th) and a manually averaged (SCAGUA_avg.th) to compare if Survex and Therion work well. The error loop is similar between the two files (2,9m) but I have to check if the station are positioned at the same coordinates. Regards, Evaristo. encoding utf-8 survey SCAGUA # Auriga 2.42, 2018-05-16T06:54:45Z centerline # 09.10.10 date 2010.9.4 units tape meters units compass degrees units clino degrees units backcompass degrees units backclino degrees #declination 0.00 degrees extend right station 279.0 "SO-279" entrance data normal from to length compass backcompass clino backclino left right up down 279.0 - 0.000.000.00 0.000.000.00 0.000.000.00 279.0279.0A 2.33 236.856.8-11.411.40.00 0.000.000.00 # .superficie 279.0A 279.1 4.51 163.2 343.2-32.232.20.00 0.000.000.00 # .superficie hasta anclaje desviador 279.1 279.2 12.41 1.5 181.5-77.777.70.00 0.000.000.00 279.2 279.3 1.95 212.233.1-17.716.60.00 0.000.000.00 279.3 279.4 3.58 340.6 160.6-32.932.90.00 0.000.000.00 279.4 279.5 1.96 317.9 138.4-30.430.50.00 0.000.000.00 279.5 279.6 1.27 250.968.3-31.831.10.00 0.000.000.00 # Fisura 279.6 279.7 0.71 339.5 159.5-12.412.40.00 0.000.000.00 # Fisura 279.7 279.8 2.31 215.635.6-10.410.40.00 0.000.000.00 # Lago bifurcacion 279.8 279.9 5.57 354.6 174.6 -4.1 4.10.00 0.000.000.00 # Lago derecho 279.8279.20 3.98 158.7 339.6 -9.910.50.00 0.000.000.00 # Seguimos el.Lago a izkierda. Bifuracion lago a derecha 279.9279.10 11.24 329.5 149.5 -4.0 4.00.00 0.000.000.00 # Lago 279.10279.11 9.81 3.5 183.5 3.3-3.30.00 0.000.000.00 # Lago sifonado a12 metros 279.20 279.20A 3.44 221.941.9-11.811.80.00 0.000.000.00 # Inicio otra galeria lago a derecha 279.21279.22 7.00 176.7 357.2 -2.8 2.50.00 0.000.000.00 279.22279.23 6.49 109.6 290.5 0.6-0.60.00 0.000.000.00 279.23279.24 16.34 134.5 315.0 1.6-2.00.00 0.000.000.00 # Lago. En la estacion 2 pequeñas galerias bajas a derecha station 279.24 continuation 279.23279.44 2.19 166.0 345.9 -7.0 7.30.00 0.000.000.00 # Lago 279.24279.25 3.6623.0 202.6 2.5-2.90.00 0.000.000.00 # Lago 279.25279.26 6.6787.2 267.0 -0.3 0.00.00 0.000.000.00 # Lago. En estacion fisura a derecha station 279.26 continuation 279.26279.27 8.0689.7 270.0 -0.7 0.70.00 0.000.000.00 # Lago. EN la estacion galeria
Re: [Therion] Wrong average calculation from compass/backcompass data
On Wed, May 16, 2018 at 04:45:54PM +0200, Evaristo Quiroga via Therion wrote: > I think the problem is the average calculation formula in "thdb1d.cxx" > > >// check backwards compass reading > > if ((lei->data_type == TT_DATATYPE_NORMAL) || > > (lei->data_type == TT_DATATYPE_DIVING) || > > (lei->data_type == TT_DATATYPE_CYLPOLAR)) { > > if (!thisnan(lei->backbearing)) { > > if (thisnan(lei->bearing)) { > > lei->backbearing -= 180.0; > > if (lei->backbearing < 0) > > lei->backbearing += 360.0; > > lei->bearing = lei->backbearing; > > } > > else { > > lei->backbearing -= 180.0; > > if (lei->backbearing < 0) > > lei->backbearing += 360.0; > > // calculate average of two angles > > //lei->bearing += lei->backbearing; > > //lei->bearing = lei->bearing / 2.0; > > double sumx, sumy; > > sumx = cos((90.0 - > >lei->bearing)/180.0*THPI) + cos((90.0 - lei->backbearing)/180.0*THPI); > > sumy = sin((90.0 - > >lei->bearing)/180.0*THPI) + sin((90.0 - lei->backbearing)/180.0*THPI); > > lei->bearing = 90.0 - (atan2(sumy, > >sumx) / THPI * 180.0); > > if (lei->bearing < 0.0) > > lei->bearing += 360.0; > > } > > } > > } > > I think the formula is too complicated. I purpose a simpler formula, like: > If bearing <=180 > AverageBearing = (bearing + (backbearing -180))/2 > else > AverageBearing = (bearing + (backbearing +180))/2 Your proposed formula gives wrong answers in some cases - consider: bearing = 80, backbearing = 0 These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180). The therion formula is attempting to average the angles by trigonometry, which seems a reasonable approach (though probably slower than trying to average more directly like you're suggesting). It looks essentially correct to me, though it's a bit oddly written since cos(90 - x) is sin(x), sin(90 - x) is -cos(x), 90 - atan2(y,x) is atan2(x,y) (possibly +/- a multiple of 360). But if I work it out with a calculator for 236.8 and 56.8 then I do get 236.8. If it's still not working with the declination set to zero, perhaps you should show us a complete small example we can process to see what is going on? Cheers, Olly ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
I think Therion subtract the magnetic declination before to transfer data Survex. Now to compare better I have put magnetic declination to 0. declination 0.00 degrees Data transferred to Survex by the "data.svx" file *data normal from to tape compass clino 1 2 0.00 269.423853 0.00 1 3 2.33 236.223853 -11.40 3 4 4.51 162.623853 -32.20 4 5 12.41 0.923853 -77.70 5 6 1.95 212.073853 -17.15 You can see Therion do a wrong fore/back sight compass average calculation. The good data average are: 1 3 236.8 -11.4 3 4 163.2 -32 4 5 1.5 -77.7 5 6 212.65 -17.15 I think the problem is the average calculation formula in "thdb1d.cxx" // check backwards compass reading if ((lei->data_type == TT_DATATYPE_NORMAL) || (lei->data_type == TT_DATATYPE_DIVING) || (lei->data_type == TT_DATATYPE_CYLPOLAR)) { if (!thisnan(lei->backbearing)) { if (thisnan(lei->bearing)) { lei->backbearing -= 180.0; if (lei->backbearing < 0) lei->backbearing += 360.0; lei->bearing = lei->backbearing; } else { lei->backbearing -= 180.0; if (lei->backbearing < 0) lei->backbearing += 360.0; // calculate average of two angles //lei->bearing += lei->backbearing; //lei->bearing = lei->bearing / 2.0; double sumx, sumy; sumx = cos((90.0 - lei->bearing)/180.0*THPI) + cos((90.0 - lei->backbearing)/180.0*THPI); sumy = sin((90.0 - lei->bearing)/180.0*THPI) + sin((90.0 - lei->backbearing)/180.0*THPI); lei->bearing = 90.0 - (atan2(sumy, sumx) / THPI * 180.0); if (lei->bearing < 0.0) lei->bearing += 360.0; } } } I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2 Regards, Evaristo. ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Wrong average calculation from compass/backcompass data
Hi, I found that the error had been fixed in version 5.4.1 Therion 5.4 (5.4.1 now 2017-04-18) * bugs and issues fixed: 27. invalid compass/backcompass averaging formula But I am using the "Therion 5.4.1+6884381 (2018-04-12)", were I have found the actual bug. Evaristo. El 16/05/2018 a las 14:17, Evaristo Quiroga via Therion escribió: Hi, I have data from a survey with fore/back sight compass and clino. I have detected a Therion bug calculating the average data, comparing with the error loops with Auriga, Visualtopo, Compass and Therion. See the next compare. Example from the data in .th file: data normal from to length compass backcompass clino backclino left right up down 279.0 - 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 279.0 279.0A 2.33 236.8 56.8 -11.4 11.4 0.00 0.00 0.00 0.00 # .superficie 279.0A 279.1 4.51 163.2 343.2 -32.2 32.2 0.00 0.00 0.00 0.00 # .superficie hasta anclaje desviador 279.1 279.2 12.41 1.5 181.5 -77.7 77.7 0.00 0.00 0.00 0.00 279.2 279.3 1.95 212.2 33.1 -17.7 16.6 0.00 0.00 0.00 0.00 Data transferred to Survex by the "data.svx" file *data normal from to tape compass clino 1 2 0.00 268.348831 0.00 1 3 2.33 235.148831 -11.40 3 4 4.51 161.548831 -32.20 4 5 12.41 359.848831 -77.70 5 6 1.95 210.998831 -17.15 You can see Therion do a wrong fore/back sight compass average calculation. The good data average are: 1 3 236.8 -11.4 3 4 163.2 -32 4 5 1.5 -77.7 5 6 212.65 -17.15 The clino average seems OK. Searching in the mail-list I have found the bug was reported before. I think is not corrected since then. https://www.mail-archive.com/therion@speleo.sk/msg04660.html Regards, Evaristo. ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
[Therion] Wrong average calculation from compass/backcompass data
Hi, I have data from a survey with fore/back sight compass and clino. I have detected a Therion bug calculating the average data, comparing with the error loops with Auriga, Visualtopo, Compass and Therion. See the next compare. Example from the data in .th file: data normal from to length compass backcompass clino backclino left right up down 279.0 - 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 279.0 279.0A 2.33 236.8 56.8 -11.4 11.4 0.00 0.00 0.00 0.00 # .superficie 279.0A 279.1 4.51 163.2 343.2 -32.2 32.2 0.00 0.00 0.00 0.00 # .superficie hasta anclaje desviador 279.1 279.2 12.41 1.5 181.5 -77.7 77.7 0.00 0.00 0.00 0.00 279.2 279.3 1.95 212.2 33.1 -17.7 16.6 0.00 0.00 0.00 0.00 Data transferred to Survex by the "data.svx" file *data normal from to tape compass clino 1 2 0.00 268.348831 0.00 1 3 2.33 235.148831 -11.40 3 4 4.51 161.548831 -32.20 4 5 12.41 359.848831 -77.70 5 6 1.95 210.998831 -17.15 You can see Therion do a wrong fore/back sight compass average calculation. The good data average are: 1 3 236.8 -11.4 3 4 163.2 -32 4 5 1.5 -77.7 5 6 212.65 -17.15 The clino average seems OK. Searching in the mail-list I have found the bug was reported before. I think is not corrected since then. https://www.mail-archive.com/therion@speleo.sk/msg04660.html Regards, Evaristo. ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion