> On 27 Aug 2018, at 09:57, Ethan Gardener <eeke...@fastmail.fm > <mailto:eeke...@fastmail.fm>> wrote: > > On Mon, Aug 27, 2018, at 8:06 AM, Haravikk wrote: >> >>> On 26 Aug 2018, at 15:05, Ethan Gardener <eeke...@fastmail.fm >>> <mailto:eeke...@fastmail.fm>> wrote: >>> >>> However, everything I've ever learned about databases says that it's a bad >>> idea to just look at the cost of the disks. The cost of *accessing* the >>> data (with acceptable performance) goes up exponentially with the size of >>> the dataset. >> >> This shouldn't be the case; any properly indexed database should have >> roughly constant, or at worst logarithmic, access times for data. > > *sigh* "Logarithmic" is another way of saying "exponential'. It is not a > trivial matter unless the data set is comparatively small. In asking about > these things, I'm particularly thinking of InWorldz, where asset data was > growing at a terabyte a month. Not small.
Logarithmic complexity isn't the same as exponential complexity, in fact it's the opposite; logarithmic complexity plateaus as the search space increases, meaning it effectively becomes constant time, an algorithm with exponential complexity would grow by a power of the search space size (e.g- the search time might quadruple each time the search space doubles in size). See here: https://en.wikipedia.org/wiki/Logarithmic_growth <https://en.wikipedia.org/wiki/Logarithmic_growth> https://en.wikipedia.org/wiki/Exponential_growth <https://en.wikipedia.org/wiki/Exponential_growth> A logarithmic algorithm is usually one that partitions the search space, for example a binary search which divides the search space in two with each iteration until only the target result (or nothing) remains. Some databases do use such algorithms but usually for intermediate operations that asset lookups shouldn't normally require. For the vast majority of asset lookups (should be all of them, but it may depend on database used) a lookup will be constant time, e.g- using a form of hash-table to immediately find (or not) a request GUID. The amount of data stored shouldn't have a big impact on such lookups, the only case where you might notice a difference is if previously the entire index fit within RAM, but after growing no longer does (so has to come from disk), but most database engines are optimised on the assumption that you'll never have enough RAM to rely on keeping a whole index in memory anyway. >> Most (all?) external asset requests are now served to the viewer via >> HTTP, so you could use Squid proxy or similar to cache those requests >> before they touch your asset server, it might even be possible to use a >> reverse proxy such as Cloudflare to do this for you, though I've never >> tried it (you might need to use page rules to force it to cache asset >> URLs). > > I'm sure this would help, but wouldn't it miss the case of large vehicles > making sim crossings? A vehicle may be non-physical and not phantom, so the > sim needs to know what shape it is and the shape may be complex. In any > case, sim crossings seem to be the first thing to go as a grid grows. I'm not overly familiar with how OpenSim handles inter-region asset exchanges; but with regions belonging to the same grid there should be no need to send the asset to the asset server, it should either be sent directly to the destination region, or be retained by the origin region on failure. An increase in failures/laggy crossings would normally be an issue of a slow or overloaded connection between regions, or the destination region itself being overloaded. In cases where a region does send an asset to the asset server a cache wouldn't help anyway, as you'd be storing a new asset (or overwriting an existing one, in the case of worn attachments). But this again wouldn't be an issue with the size of the asset server, but the amount of traffic being sent to it. _______________________________________________ Opensim-users mailing list Opensim-users@opensimulator.org http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
