Re: CRC was: Re: [HACKERS] beta testing version

2000-12-12 Thread Nathan Myers
On Thu, Dec 07, 2000 at 07:36:33PM -0500, Tom Lane wrote: > [EMAIL PROTECTED] (Nathan Myers) writes: > > 2. I disagree with way the above statistics were computed. That eleven > >million-year figure gets whittled down pretty quickly when you > >factor in all the sources of corruption, e

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 03:38:09PM -0500, Tom Lane wrote: > Bruce Guenter <[EMAIL PROTECTED]> writes: > > MD5 is a cryptographic hash, which means (AFAIK) that ideally it is > > impossible to produce a collision using any other method than brute > > force attempts. > True but irrelevant. What we

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: > MD5 is a cryptographic hash, which means (AFAIK) that ideally it is > impossible to produce a collision using any other method than brute > force attempts. True but irrelevant. What we need to worry about is the probability that a random error will be

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: > ... Taking an > arbitrary 32 bits of a MD5 would likely be less collision prone than > using a 32-bit CRC, and it appears faster as well. ... but that would be an algorithm that you know NOTHING about the properties of. What is your basis for asserting

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 10:36:39AM -0800, Ian Lance Taylor wrote: >Incidentally, I benchmarked the previously mentioned 64-bit fingerprint, >the standard 32-bit CRC, MD5 and SHA, and the fastest algorithm on my >Celeron and on a PIII was MD5. The 64-bit fingerprint was only a hair >

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 01:58:12PM -0500, Tom Lane wrote: > Bruce Guenter <[EMAIL PROTECTED]> writes: > > ... Taking an > > arbitrary 32 bits of a MD5 would likely be less collision prone than > > using a 32-bit CRC, and it appears faster as well. > > ... but that would be an algorithm that you k

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Ian Lance Taylor
Date: Fri, 8 Dec 2000 12:19:39 -0600 From: Bruce Guenter <[EMAIL PROTECTED]> Incidentally, I benchmarked the previously mentioned 64-bit fingerprint, the standard 32-bit CRC, MD5 and SHA, and the fastest algorithm on my Celeron and on a PIII was MD5. The 64-bit fingerprint was onl

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-08 Thread Bruce Guenter
On Thu, Dec 07, 2000 at 04:01:23PM -0800, Nathan Myers wrote: > 1. Computing a CRC-64 takes only about twice as long as a CRC-32, for >2^32 times the confidence. That's pretty cheap confidence. Incidentally, I benchmarked the previously mentioned 64-bit fingerprint, the standard 32-bit CRC,

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-07 Thread Tom Lane
[EMAIL PROTECTED] (Nathan Myers) writes: > 2. I disagree with way the above statistics were computed. That eleven >million-year figure gets whittled down pretty quickly when you >factor in all the sources of corruption, even without crashes. >(Power failures are only one of many s

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-07 Thread Nathan Myers
On Thu, Dec 07, 2000 at 04:35:00PM -0500, Tom Lane wrote: > Remember that we are already sitting atop hardware that's really > pretty reliable, despite the carping that's been going on in this > thread. All that we have to do is detect the infrequent case where a > block of data didn't get written

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-07 Thread Tom Lane
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes: >> I would strongly suggest to use strong hashes like RIPEMD or >> MD5 instead of CRC-32 and the like. > Other opinions? Also, we shouldn't forget licence issues. I agree with whoever commented that crypto hashes are silly for this application. A 64-

RE: CRC was: Re: [HACKERS] beta testing version

2000-12-07 Thread Mikheev, Vadim
> > This may be implemented very fast (if someone points me where > > I can find CRC func). And I could implement "physical log" > > till next monday. > > I have been experimenting with CRCs for the past 6 month in > our database for internal logging purposes. Downloaded a lot of > hash librarie

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-07 Thread Nathan Myers
On Thu, Dec 07, 2000 at 06:40:49PM +1100, Horst Herb wrote: > > This may be implemented very fast (if someone points me where > > I can find CRC func). And I could implement "physical log" > > till next monday. > > As the logging might include large data blocks, especially now that > we can TOAST

Re: CRC was: Re: [HACKERS] beta testing version

2000-12-07 Thread Hannu Krosing
Horst Herb wrote: > > > This may be implemented very fast (if someone points me where > > I can find CRC func). And I could implement "physical log" > > till next monday. > > I have been experimenting with CRCs for the past 6 month in our database for > internal logging purposes. Downloaded a lo

CRC was: Re: [HACKERS] beta testing version

2000-12-06 Thread Horst Herb
> This may be implemented very fast (if someone points me where > I can find CRC func). And I could implement "physical log" > till next monday. I have been experimenting with CRCs for the past 6 month in our database for internal logging purposes. Downloaded a lot of hash libraries, tried differ