Hi Denis I personaly think, that this is not a discussion about propabilities but about fail-safe and robust implementation.
I understand that you do not want to implement a logic to the id generation which can mitigate collisions (e.g. collision-bit), because the probability that this logic ever solves a problem is "low" and the code must be maintained. I agree that this is a question about probabilities if this feature is useful and necessary. Yet, I think if anybody hits a collision, which still may happen with enough bad luck, than this person wants the system to behave robust and fail-safe and that it never ever corrupts stored data. IMHO this is therefore not a question of probabilites but a question of fail-safe implementation and thus a question of software quality. Only my 5cents. Regards, Fabian 2018-02-08 9:24 GMT+01:00 Denis Gervalle <[email protected]>: > Hi Marc and Thomas, > > I followed your discussion with great interest. I agree that Thomas very > light proposal is good to put in place, since it has almost no negative > impact and only benefit. I think there is also a possibility to mitigate > the object issue with something close (check integrity of what we get, to > at least detect an issue), but that's not perfect of course. > > That’s said, I would like to point you to this interesting question on > StackOverflow (https://stackoverflow.com/questions/22029012/ > probability-of-64bit-hash-code-collisions) and remind you that base on > the Birthday Paradox, with the released of 4.x, we have raised our worrying > threshold of documents/objects from 65535, to more than 4 billion… and it > took a while (4 versions of XWiki) before we had the strong feeling we need > to raise. So, while before 4.x, the worrying threshold was really low, the > effective happening of a collision was already low. > > My own experience was the risk before 4.x was really high with generated > names, much hight than with names use by real user. When I was it by that > issue, I remember being really bad about it. This is also probably why you > have raised this thread. The previous hash was too small and had also a > discutable distribution. > > The MD5 algorithm like many crypto hashes is particularly well suited for > providing a good distribution (http://michiel.buddingh.eu/ > distribution-of-hash-values), the cutting at 64 bits may lower this, but > I doubt it would be significant for us. So, personally, I feel really > comfortable with the current implementation, and I think you can sleep in > peace as well. > > Just my thought about not raising fears when it’s no more really justified. > Regards, > > -- > Denis Gervalle > SOFTEC sa - CEO > > On 7 Feb 2018, 16:10 +0100, Denis Gervalle <[email protected]>, > wrote: > > > > Hi Marc and Thomas, > > > > I followed your discussion with great interest. I agree that Thomas very > light proposal is good to put in place, since it has almost no negative > impact and only benefit. I think there is also a possibility to mitigate > the object issue with something close (check integrity of what we get, to > at least detect an issue), but that's not perfect of course. > > > > That’s said, I would like to point you to this interesting question on > StackOverflow (https://stackoverflow.com/questions/22029012/ > probability-of-64bit-hash-code-collisions) and remind you that base on > the Birthday Paradox, with the released of 4.x, we have raised our worrying > threshold of documents/objects from 65535, to more than 4 billion… and it > took a while (4 versions of XWiki) before we had the strong feeling we need > to raise. So, while before 4.x, the worrying threshold was really low, the > effective happening of a collision was already low. > > > > My own experience was the risk before 4.x was really high with generated > names, much hight than with names use by real user. When I was it by that > issue, I remember being really bad about it. This is also probably why you > have raised this thread. The previous hash was too small and had also a > discutable distribution. > > > > The MD5 algorithm like many crypto hashes is particularly well suited > for providing a good distribution (http://michiel.buddingh.eu/ > distribution-of-hash-values), the cutting at 64 bits may lower this, but > I doubt it would be significant for us. So, personally, I feel really > comfortable with the current implementation, and I think you can sleep in > peace as well. > > > > Just my thought about not raising fears when it’s no more really > justified. > > Regards, >
