Re: [Therion] Handling of tape and backtape in survey data
On Mon, Dec 27, 2021 at 05:04:20PM -0600, Bill Gee wrote: > My question about significant digits is really related to how much > precision can and should be carried through a calculation. If compass > readings are taken to four significant digits, then the mean of two > such numbers is really only good to about three and a half significant > digits. That seems wrong to me - we expect the mean to have *better* accuracy than the individual readings (otherwise one would want to avoid averaging readings!) Assuming independence (which may not an entirely valid assumption here admittedly) and that both readings have the same standard deviation x, then the s.d. of the mean of 2 readings is x/sqrt(2) - or about 0.707*x. I think that means you actually would expect to gain log(1/sqrt(2))/log(1/10) or about 0.15 decimal significant digits from taking the mean of two readings. (And it's 1/sqrt(n) for n readings, so if you average 100 independent readings then you'd expect to gain a whole decimal significant digit.) > Displaying more significant digits implies more precision > that actually exists. Right, but that argues for suitably presenting reported values, not for rounding intermediate values during the calculations. > A compass reading even from a DistoX2 is > probably accurate to only +/- 2 least significant digits. > > https://en.wikipedia.org/wiki/Significant_figures I'm not quite sure what "2 least significant digits" means, and that wikipedia page doesn't seem to help. But I think I get the gist, and indeed any instrument will have a limit to its accuracy, especially so for anything you'd be likely to consider taking underground. Cheers, Olly ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
Hi Olly - Many thanks for the answers. I think I will give BACKTAPE a try the next time I do some survey, if only to see what happens. My question about significant digits is really related to how much precision can and should be carried through a calculation. If compass readings are taken to four significant digits, then the mean of two such numbers is really only good to about three and a half significant digits. Displaying more significant digits implies more precision that actually exists. A compass reading even from a DistoX2 is probably accurate to only +/- 2 least significant digits. https://en.wikipedia.org/wiki/Significant_figures[1] However, you have an excellent point about easting and northing data. Those number can be measure accurately to enough significant digits that a double-precision data type is needed. Speed is not an issue. Survex and Therion both compile my biggest maps in seconds. Shaving a few seconds is of little benefit. And to your last comment about furlongs ... LOL! I remember a huge argument some years ago about what the speed of light is in furlongs per fortnight. It was a hoot! Thanks! Bill Gee On Monday, December 27, 2021 3:56:04 PM CST Olly Betts wrote: > On Sat, Dec 25, 2021 at 08:50:26AM -0600, Bill Gee wrote: > > 1) Does Therion also recognize the BACKTAPE data type? > > It seems not - there's no match in the source code repo for: > > git grep -i BACKTAPE > > > 2) What does Survex do if both TAPE and BACKTAPE are given? Does it > > average the two readings? Does Therion do the same thing? > > Survex warns if they differ by more than 3 standard deviations, and then > takes the mean of the readings. > > > I understand that Therion will use Survex to reduce the centerline > > data - if Survex is installed. In case Survex is NOT installed, then > > Therion reduces the centerline itself. As a result they might handle > > this situation differently. > > Also, Therion generates a .svx file behind the scenes, but not > necessarily the .svx file you'd write yourself given the same data. > For example, I think Therion collapses backsights to a single reading > itself before processing with Survex. > > > As I write this, another related question occurred to me. When either > > Therion or Survex averages a forward and backward reading, how many > > significant digits does it carry in the calculation? Can the > > significant digits be changed? > > Survex uses double precision for this: > > https://en.wikipedia.org/wiki/Double-precision_floating-point_format > > That has 15-17 significant decimal digits - i.e. many more than matter > here. > > Using more precision seems overkill. > > If you really want to use fewer it should be possible to build cavern > to use single precision, but I don't think anyone actually does this. > We made it an option as decades ago it seemed you might want to do this > to reduce memory usage (4 bytes per floating point number instead of 8) > but cavern's memory use is very modest by modern standards, and single > precision only gives 6-9 significant decimal digits, which isn't even > enough to reliably store a UTM Northing to the nearest metre so you'd be > limited in what coordinate systems you could use in such a build. > > I don't see why you'd want to be able to dynamically force rounding to > fewer significant figures - what's the benefit? > > Downsides of rounding are it would slow things down a bit, and it risks > introducing systematic biases - a dumb example to make the issue > clearer: if all readings are to the nearest inch and you round the > average to the nearest inch and always round 0.5 up, then then assuming > an even distribution there's an average bias of +0.25 inches on each > tape reading. The problem with systematic biases is they accumulate > rather than tending to cancel. > > There are other rounding schemes which try to address this sort of > issue (like rounding 0.5 to the nearest even number) but it's hard to be > certain they might not introduce a more subtle bias (some cave systems > have dominant direction passage develops along so the distribution is > may not uniform across odd and even) and there doesn't seem a compelling > reason to be rounding in the first place. > > > And the same question applies to loop closure calculations. How many > > significant digits are carried through the calculations? > > The same. > > The final results are then stored in the .3d file to the nearest cm > (that's 0.49710 furlongs for our US readers). That precision was > chosen so that coordinate values will fit in a 32 bit integer, and seems > adequate for the final results. > > Cheers, > Olly > [1] https://en.wikipedia.org/wiki/Significant_figures ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
On Sun, Dec 26, 2021 at 08:00:49AM +, Ben Cooper wrote: > What does Survex actually do for all these loops? It deal with any backsights up front and just adds a single link to the network for each survey leg. It's harder to properly account for partial backsights if we created a separate link for the backsight (e.g. backcompass and backclino but no backtape is common). But even just from a network solving perspective, if we generated a trivial loop for each backsight the network optimisations would later rediscover each trivial loop and collapse it to a single leg, but that would be less efficient than collapsing it when we have the knowledge it's a trivial loop as we read in that leg. Cheers, Olly ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
On Sat, Dec 25, 2021 at 08:50:26AM -0600, Bill Gee wrote: > 1) Does Therion also recognize the BACKTAPE data type? It seems not - there's no match in the source code repo for: git grep -i BACKTAPE > 2) What does Survex do if both TAPE and BACKTAPE are given? Does it > average the two readings? Does Therion do the same thing? Survex warns if they differ by more than 3 standard deviations, and then takes the mean of the readings. > I understand that Therion will use Survex to reduce the centerline > data - if Survex is installed. In case Survex is NOT installed, then > Therion reduces the centerline itself. As a result they might handle > this situation differently. Also, Therion generates a .svx file behind the scenes, but not necessarily the .svx file you'd write yourself given the same data. For example, I think Therion collapses backsights to a single reading itself before processing with Survex. > As I write this, another related question occurred to me. When either > Therion or Survex averages a forward and backward reading, how many > significant digits does it carry in the calculation? Can the > significant digits be changed? Survex uses double precision for this: https://en.wikipedia.org/wiki/Double-precision_floating-point_format That has 15-17 significant decimal digits - i.e. many more than matter here. Using more precision seems overkill. If you really want to use fewer it should be possible to build cavern to use single precision, but I don't think anyone actually does this. We made it an option as decades ago it seemed you might want to do this to reduce memory usage (4 bytes per floating point number instead of 8) but cavern's memory use is very modest by modern standards, and single precision only gives 6-9 significant decimal digits, which isn't even enough to reliably store a UTM Northing to the nearest metre so you'd be limited in what coordinate systems you could use in such a build. I don't see why you'd want to be able to dynamically force rounding to fewer significant figures - what's the benefit? Downsides of rounding are it would slow things down a bit, and it risks introducing systematic biases - a dumb example to make the issue clearer: if all readings are to the nearest inch and you round the average to the nearest inch and always round 0.5 up, then then assuming an even distribution there's an average bias of +0.25 inches on each tape reading. The problem with systematic biases is they accumulate rather than tending to cancel. There are other rounding schemes which try to address this sort of issue (like rounding 0.5 to the nearest even number) but it's hard to be certain they might not introduce a more subtle bias (some cave systems have dominant direction passage develops along so the distribution is may not uniform across odd and even) and there doesn't seem a compelling reason to be rounding in the first place. > And the same question applies to loop closure calculations. How many > significant digits are carried through the calculations? The same. The final results are then stored in the .3d file to the nearest cm (that's 0.49710 furlongs for our US readers). That precision was chosen so that coordinate values will fit in a 32 bit integer, and seems adequate for the final results. Cheers, Olly ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
On 27/12/2021 19:41, Tarquin Wilton-Jones via Therion wrote: But because DistoX is an electronic device it is only way to be sure it works properly. The same as check, if three (four) readings for surveying leg with rotation around the longitudinal axe of DistoX are in tolerance interval. Oh very true. This is one of the checks I recommend performing occasionally to test calibration (or every leg when working around scaffolding or magnetic rocks). Create legs with one reading flat, one rotated 90 degrees longitudinally, another reading flat or in any other orientation. Then validate with a splay back to the previous station. I think, but have no proof, that if you rotate the distoX it is probably better than a manual backsight. pockettopo records the rotation, and I have this in the therion file. People surveying with me know I check that they rotate at least every 5 stations and i relity they rotate most station. All the times I've been with people doing backsights (mainly US cavers) all it seems to stop is blunders, if they don't get a reading that is within their personal tolerance they do it all again. The real question is how much are they doing expectation bias? I cannot really comment further as I have too little experience of this. On 27/12/2021 19:36, Bill Gee wrote: I remember seeing a presentation at a recent NSS Convention (2020, maybe?) where someone took an uncalibrated DistoX2 and ran it around a survey course on the surface. By averaging the forward and backward shots he got loop closures around 1% over 400 feet and a dozen stations. The data using Suunto/tape and a calibrated DistoX2 was not significantly different in the loop closure. A compass that is 10° (well any degrees off will give a 0% loop error, the longer the loop the lower the percentage error, however bad the survey. (bit of over simplification, but relatively true) High percent loop closure is a poor survey, the opposite is not true. This could turn into a long interesting debate, please feel free to argue with me, let's see how robust our ideas are :-) Andrew ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
On 27/12/2021 19:15, Martin Sluka via Therion wrote: > > >> 26. 12. 2021 v 19:32, Tarquin Wilton-Jones via Therion >> mailto:therion@speleo.sk>>: >> >> However, for Disto >> surveying, it feels largely unnecessary, as the accuracy is already a >> lot more than most surveys need when just using forward sightings. > > But because DistoX is an electronic device it is only way to be sure it > works properly. The same as check, if three (four) readings for > surveying leg with rotation around the longitudinal axe of DistoX are in > tolerance interval. Oh very true. This is one of the checks I recommend performing occasionally to test calibration (or every leg when working around scaffolding or magnetic rocks). Create legs with one reading flat, one rotated 90 degrees longitudinally, another reading flat or in any other orientation. Then validate with a splay back to the previous station. But personally I consider the leg forward, then duplicate backsighted *leg* for every single leg, to be largely overkill. There are already 3 readings for every leg, and a rotational test (with optional backsight splay) already serves the purpose of testing calibration and magnetic effects. And standard leapfrogging can already counteract clinometer calibration faults. But I am sure someone will have their reason to make each leg from 6 readings. :) ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
Ben's comment about backshots being a form of loop is interesting. I had not though of it that way, but he is right! I do not use electronic sketching in a cave. All the data from the DistoX2 goes on paper just like Suunto/tape data. I have looked at Qave and TopoDroid, and for now I think they are not quite ready for prime time. There are people out there who swear by electronic sketching. I am just an old curmudgeon! But not quite old enough to go back to Bruntons. I am also not convinced that a DistoX is more accurate than Suunto/tape methods. It is certainly easier to use, especially for newbies. It is much less bulky since the tape reel can be left behind, and it really helps with LRUD measurements. But -- Do not assume that "electronic" is the same as "accurate". In a recent update to one of my project caves, I noticed the loop closure errors from Survex were impressive. Most were under 2%, and several under 1%. Even better - those loops used readings taken with both Suunto/tape and DistoX2 by several teams over a period of three years. I think the critical factor is getting good back shots to verify the data in the field, and then having the computer average the forward and back shots. I remember seeing a presentation at a recent NSS Convention (2020, maybe?) where someone took an uncalibrated DistoX2 and ran it around a survey course on the surface. By averaging the forward and backward shots he got loop closures around 1% over 400 feet and a dozen stations. The data using Suunto/tape and a calibrated DistoX2 was not significantly different in the loop closure. Bill Gee On Monday, December 27, 2021 1:15:27 PM CST Martin Sluka via Therion wrote: > > > 26. 12. 2021 v 19:32, Tarquin Wilton-Jones via Therion : > > > > However, for Disto > > surveying, it feels largely unnecessary, as the accuracy is already a > > lot more than most surveys need when just using forward sightings. > > But because DistoX is an electronic device it is only way to be sure it works > properly. The same as check, if three (four) readings for surveying leg with > rotation around the longitudinal axe of DistoX are in tolerance interval. > > Martin S. > ___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Re: [Therion] Handling of tape and backtape in survey data
> 26. 12. 2021 v 19:32, Tarquin Wilton-Jones via Therion : > > However, for Disto > surveying, it feels largely unnecessary, as the accuracy is already a > lot more than most surveys need when just using forward sightings. But because DistoX is an electronic device it is only way to be sure it works properly. The same as check, if three (four) readings for surveying leg with rotation around the longitudinal axe of DistoX are in tolerance interval. Martin S.___ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
