Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Mark Kirkwood
Tom Lane wrote: Mark Kirkwood [EMAIL PROTECTED] writes: Interesting that the slice-8 algorithm seems to work noticeably better on FreeBSD than Linux - Are you running similar gcc versions on both? I realize I forgot to document what I was using: HPPA: gcc version 2.95.3 20010315 (release)

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Simon Riggs
On Sun, 2006-10-22 at 17:18 +0200, Martijn van Oosterhout wrote: On Sun, Oct 22, 2006 at 08:10:56PM +0530, Gurjeet Singh wrote: Hi all, Michael Kounavis has given a green signal (please refer the forwarded message). I don't think that helps. The publishing date of this article was

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Anders Steinlein
On Mon, 23 Oct 2006 07:35:08 +0200, Tom Lane [EMAIL PROTECTED] wrote: I realize I forgot to document what I was using: HPPA: gcc version 2.95.3 20010315 (release) PPC: gcc version 4.0.1 (Apple Computer, Inc. build 5363) both Intels: gcc version 4.1.1 20060525 (Red Hat 4.1.1-1) Interesting

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Heikki Linnakangas
Tom Lane wrote: Are you running similar gcc versions on both? I realize I forgot to document what I was using: HPPA: gcc version 2.95.3 20010315 (release) PPC: gcc version 4.0.1 (Apple Computer, Inc. build 5363) both Intels: gcc version 4.1.1 20060525 (Red Hat 4.1.1-1) Interesting that it's

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Jeremy Drake
On Mon, 23 Oct 2006, Mark Kirkwood wrote: Tom Lane wrote: Yah, I checked. Several times... but if anyone else wants to repeat the experiment, please do. Or look for bugs in either my test case or Gurjeet's. Just for fun, I tried it out with both GCC and with Intel's C compiler

Re: [SPAM?] Re: [HACKERS] Asynchronous I/O Support

2006-10-23 Thread Zeugswetter Andreas ADI SD
So far I've seen no evidence that async I/O would help us, only a lot of wishful thinking. is this thread moot? while researching this thread I came across this article: http://kerneltrap.org/node/6642 describing claims of 30% performance boost when using posix_fadvise to ask the

Re: [SPAM?] Re: [HACKERS] Asynchronous I/O Support

2006-10-23 Thread Zeugswetter Andreas ADI SD
So far I've seen no evidence that async I/O would help us, only a lot of wishful thinking. is this thread moot? while researching this thread I came across this article: http://kerneltrap.org/node/6642 describing claims of 30% performance boost when using posix_fadvise to ask the o/s

Re: [SPAM?] Re: [HACKERS] Asynchronous I/O Support

2006-10-23 Thread Zeugswetter Andreas ADI SD
Yup, that would be the scenario where it helps (provided that you have a smart disk or a disk array and an intelligent OS aio implementation). It would be used to fetch the data pages pointed at from an index leaf, or the next level index pages. We measured the IO bandwidth difference

Re: [HACKERS] bug or feature, || -operator and NULLs

2006-10-23 Thread Albe Laurenz
Mario Weilguni wrote: This has been been discussed before, but Oracle behaves differently, and IMHO in a more correct way. The following query returns NULL in PG: SELECT NULL || 'fisk'; But in Oracle, it returns 'fisk': SELECT NULL || 'fisk' FROM DUAL; The latter seems more logical...

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Gregory Stark
Simon Riggs [EMAIL PROTECTED] writes: Slice-By-8 was first mentioned here: Are you sure? US patent 7,047,479 filed in 2002 sounds like it may be relevant: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1Sect2=HITOFFd=PALLp=1u=%2Fnetahtml%2FPTO%2Fsrchnum.htmr=1f=Gl=50s1=7047479.PN. --

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Mark Kirkwood
Mark Kirkwood wrote: Tom Lane wrote: Are you running similar gcc versions on both? I realize I forgot to document what I was using: Ah - good point, FreeBSD is using an older compiler: FreeBSD: gcc (GCC) 3.4.6 [FreeBSD] 20060305 Linux: gcc (GCC) 4.1.1 (Gentoo 4.1.1) Hmm - there is a

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Simon Riggs
On Mon, 2006-10-23 at 05:22 -0400, Gregory Stark wrote: Simon Riggs [EMAIL PROTECTED] writes: Slice-By-8 was first mentioned here: Are you sure? US patent 7,047,479 filed in 2002 sounds like it may be relevant:

[HACKERS] MVCC question

2006-10-23 Thread Gevik Babakhani
Folks, How long are we supporting MVCC? It is from the beginning or is it added later to PG -- Regards, Gevik Babakhani www.postgresql.nl www.truesoftware.nl ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Benny Amorsen
MK == Mark Kirkwood [EMAIL PROTECTED] writes: MK Here are the results after building gcc 4.1.2 (repeating results MK for gcc 3.4.6 for comparison). I suspect that performance is MK probably impacted because gcc 4.1.2 (and also the rest of the MK tool-chain) is built with gcc 3.4.6 - but it

Re: [HACKERS] MVCC question

2006-10-23 Thread imad
Release 6.5 brings MVCC to PostgreSQL. Check out the following doc for details http://www.postgresql.org/docs/7.1/static/release-6-5.html --Imad EnterpriseDB (http://www.enterprisedb.com) On 10/23/06, Gevik Babakhani [EMAIL PROTECTED] wrote: Folks, How long are we supporting MVCC? It is from

Re: [HACKERS] MVCC question

2006-10-23 Thread Peter Eisentraut
Am Montag, 23. Oktober 2006 13:50 schrieb Gevik Babakhani: How long are we supporting MVCC? 6.5 -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives?

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Tom Lane
Jeremy Drake [EMAIL PROTECTED] writes: So at this point I realize that intel's compiler is optimizing the loop away, at least for the std crc and probably for both. So I make mycrc an array of 2, and substript mycrc[j1] in the loop. That's not a good workaround, because making mycrc expensive

Re: [HACKERS] PgSQL users quota

2006-10-23 Thread Jim C. Nasby
On Fri, Oct 20, 2006 at 03:30:40AM +0300, Tux P wrote: Hi .* Is there any chance to see the quota implementation described in this post in any next releases? http://archives.postgresql.org/pgsql-hackers/2004-07/msg00392.php Since Jonah hasn't done anything with it he's presumably lost

Re: [HACKERS] PgSQL users quota

2006-10-23 Thread Jonah H. Harris
On 10/23/06, Jim C. Nasby [EMAIL PROTECTED] wrote: Since Jonah hasn't done anything with it he's presumably lost interest, so you'd need to find someone else looking for an itch to scratch. And it appears the original patch was against 7.4, so it'd probably need a decent amount of work to make

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Simon Riggs
On Sun, 2006-10-22 at 18:06 -0400, Tom Lane wrote: These numbers are um, not impressive. Considering that a large fraction of our WAL records are pretty short, the fact that slice8 consistently loses at short buffer lengths is especially discouraging. Much of that ground could be made up

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Instead, I'd like to include a parameter to turn off CRC altogether, for heavily CPU bound operations and the WAL drive on trustworthy hardware. No can do --- we rely on the checksums to be able to tell when we've hit the end of WAL during replay. You may

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Dawid Kuroczko
On 10/23/06, Tom Lane [EMAIL PROTECTED] wrote: It's not so much that I don't trust Intel as that a CRC algorithm is exactly the sort of nice little self-contained thing that people love to try to patent these days. What I am really afraid of is that someone else has already invented this same

[HACKERS] Problems starting Template1...

2006-10-23 Thread luis garcia
Hello, I'm from Venezuela, and I've been making some modifications to Postgre's Catalog, but it seems to be a problem creating the Template1 Database. When the creation of the database is starting this is what happens: [EMAIL PROTECTED]:~ /home/luis/pgsql/bin/initdb -D /home/luis/pgsql/data/The

Re: [HACKERS] [PATCHES] smartvacuum() instead of autovacuum

2006-10-23 Thread Jim C. Nasby
If the decision to vacuum based on autovacuum criteria is good enough for you then I think you should just focus on getting autovac to do what you want it to do. Perhaps you just need to decrease the sleep time to a few seconds, so that autovac will quickly detect when something needs to be

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Josh Berkus
Bruce, Tom, All: Given the numbers I posted earlier today, the proposal is dead in the water anyway, quite aside from any legal considerations. Agreed.  I just wanted to point out we have other sharks in the water. *IF* Slice-by-8 turned out to be a winner, I could get the legal issues

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Simon Riggs
On Mon, 2006-10-23 at 13:52 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Instead, I'd like to include a parameter to turn off CRC altogether, for heavily CPU bound operations and the WAL drive on trustworthy hardware. No can do --- we rely on the checksums to be able to

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Joshua D. Drake
In most cases, it would be foolish to avoid: but there are cases where the data is CRC checked by the hardware/system already, so I'd like to make an option to turn this off, defaulting to on, for safety. How would we know? What are those cases? Sounds like a foot gun to me. Sincerely,

Re: [HACKERS] [PATCHES] smartvacuum() instead of autovacuum

2006-10-23 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: The only case I can think of where autovac might not work as well as smartvacuum would be if you had a lot of databases in the cluster, since autovacuum will only vacuum one database at a time. It's conceivable that it'd make sense to allow multiple

[HACKERS] Tsearch2 index size

2006-10-23 Thread richard-pgodbc
I am running versions 8.1.2 and I installed 8.2B last week. I dumped the data from the old version into the new version. The db consists of several million records. Total disk usage is approximately 114GB. My two observations are as follows... Also, keep in mind these are truly just

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2006-10-23 at 13:52 -0400, Tom Lane wrote: No can do --- we rely on the checksums to be able to tell when we've hit the end of WAL during replay. No we don't: Zero length records are the trigger for EOF. Only if the file happens to be all-zero

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Jeremy Drake
On Mon, 23 Oct 2006, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: So at this point I realize that intel's compiler is optimizing the loop away, at least for the std crc and probably for both. So I make mycrc an array of 2, and substript mycrc[j1] in the loop. That's not a good

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Tom Lane
Jeremy Drake [EMAIL PROTECTED] writes: On Mon, 23 Oct 2006, Tom Lane wrote: That's not a good workaround, because making mycrc expensive to access means your inner loop timing isn't credible at all. Instead try making the buffer array nonlocal --- malloc it, perhaps. That did not make any

Re: [HACKERS] Tsearch2 index size

2006-10-23 Thread Tom Lane
[EMAIL PROTECTED] writes: 1. The release notes indicate more efficient vacuuming. However, both vacuums seems to take about the same amount of time, ie. approx: 9 hours. I think the improvements were only in btree index vacuuming, which it sounds like isn't your big problem. 2. I have a

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Jeremy Drake
On Mon, 23 Oct 2006, Tom Lane wrote: Hmm. Maybe store the CRCs into a global array somewhere? uint32 results[NTESTS]; for ... { INIT/COMP/FIN_CRC32... results[j] = mycrc; } This still adds a bit of overhead to the outer loop, but not

[HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Zdenek Kotala
I tried to use COPY command to export and import tables from catalog, but COPY command has problem with data type regproc. See example create table test (like pg_aggregate); copy pg_aggregate to '/tmp/pg_agg.out'; copy test from '/tmp/pg_agg.out'; ERROR: more than one function named

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Andrew Dunstan
Zdenek Kotala wrote: I tried to use COPY command to export and import tables from catalog Is it just me or does this seem like a strange thing to want to do? I am trying to think of a good use case, so far without much success. cheers andrew ---(end of

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Alvaro Herrera
Zdenek Kotala wrote: I tried to use COPY command to export and import tables from catalog, but COPY command has problem with data type regproc. See example create table test (like pg_aggregate); copy pg_aggregate to '/tmp/pg_agg.out'; copy test from '/tmp/pg_agg.out'; ERROR: more

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Zdenek Kotala
Andrew Dunstan wrote: Zdenek Kotala wrote: I tried to use COPY command to export and import tables from catalog Is it just me or does this seem like a strange thing to want to do? I am trying to think of a good use case, so far without much success. I'm playing with catalog upgrade. The

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Hmm, maybe it should be using regprocedure instead? Not unless you want to break initdb. The only reason regproc still exists, really, is to accommodate loading of pg_type during initdb. Guess what: we can't do type lookup at that point.

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Hmm, maybe it should be using regprocedure instead? Not unless you want to break initdb. The only reason regproc still exists, really, is to accommodate loading of pg_type during initdb. Guess what: we can't do type lookup at that

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Andrew Dunstan
Zdenek Kotala wrote: Andrew Dunstan wrote: Zdenek Kotala wrote: I tried to use COPY command to export and import tables from catalog Is it just me or does this seem like a strange thing to want to do? I am trying to think of a good use case, so far without much success. I'm playing

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Zdenek Kotala
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Hmm, maybe it should be using regprocedure instead? Not unless you want to break initdb. The only reason regproc still exists, really, is to accommodate loading of pg_type during initdb. Guess what: we can't do type lookup at that

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: I'm playing with catalog upgrade. The very basic idea of my experiment is export data from catalog and import it back to the new initialized/fresh catalog. That is never going to work, at least not for any interesting catalogs. A system with a fresh (I

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: if( donot_resolve_procname == TRUE) { result = (char *) palloc(NAMEDATALEN); snprintf(result, NAMEDATALEN, %u, proid); } What for? If you want numeric OIDs you can have that today by casting the column to OID. More to the point,

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Zdenek Kotala
Tom Lane wrote: Zdenek Kotala [EMAIL PROTECTED] writes: I'm playing with catalog upgrade. The very basic idea of my experiment is export data from catalog and import it back to the new initialized/fresh catalog. That is never going to work, at least not for any interesting catalogs. A system

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Andrew Dunstan
Zdenek Kotala wrote: Tom Lane wrote: The right way to implement pg_upgrade is to transfer the catalog data at the SQL-command level of abstraction, ie, pg_dump -s and reload. I'm not sure if it is important, but I think that preserve OID is important and SQL level does not allow set OID.

Re: [HACKERS] Tsearch2 index size

2006-10-23 Thread richard-pgodbc
2. I have a tsearch2 index which is 756MB in size in 8.1.2 but balloons to 910MB in 8.2! FILLFACTOR? Tom, Of course! I had it in my head that fillfactor had to be explicitely set. But then, after RTFM, it looks like there are defaults! Thank you! One more inane question, though.

Re: [HACKERS] COPY does not work with regproc and aclitem

2006-10-23 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Zdenek Kotala wrote: I'm not sure if it is important, but I think that preserve OID is important and SQL level does not allow set OID. Does it matter in any case other than where it refers to an on-disk object? And does that need anything other than

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Mark Kirkwood
Benny Amorsen wrote: MK == Mark Kirkwood [EMAIL PROTECTED] writes: MK Here are the results after building gcc 4.1.2 (repeating results MK for gcc 3.4.6 for comparison). I suspect that performance is MK probably impacted because gcc 4.1.2 (and also the rest of the MK tool-chain) is built with

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Tom Lane
Mark Kirkwood [EMAIL PROTECTED] writes: Right - I think the regression is caused by libc and kernel being built with gcc 3.4.6 and the test program being built with gcc 4.1.2. Why do you think that? The performance of the CRC loop shouldn't depend at all on either libc or the kernel, because

Re: [HACKERS] [PATCHES] smartvacuum() instead of autovacuum

2006-10-23 Thread Matthew O'Connor
Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The only case I can think of where autovac might not work as well as smartvacuum would be if you had a lot of databases in the cluster, since autovacuum will only vacuum one database at a time. It's conceivable that it'd make sense to

Re: [HACKERS] New CRC algorithm: Slicing by 8

2006-10-23 Thread Mark Kirkwood
Tom Lane wrote: Mark Kirkwood [EMAIL PROTECTED] writes: Right - I think the regression is caused by libc and kernel being built with gcc 3.4.6 and the test program being built with gcc 4.1.2. Why do you think that? The performance of the CRC loop shouldn't depend at all on either libc or

Re: [HACKERS] [PATCHES] smartvacuum() instead of autovacuum

2006-10-23 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 03:08:03PM -0400, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The only case I can think of where autovac might not work as well as smartvacuum would be if you had a lot of databases in the cluster, since autovacuum will only vacuum one database at a time.

Re: [HACKERS] [PATCHES] smartvacuum() instead of autovacuum

2006-10-23 Thread Alvaro Herrera
Jim C. Nasby wrote: On Mon, Oct 23, 2006 at 03:08:03PM -0400, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The only case I can think of where autovac might not work as well as smartvacuum would be if you had a lot of databases in the cluster, since autovacuum will only vacuum

[HACKERS] Bug related to out of memory condition

2006-10-23 Thread Jeff Davis
This behavior exists in 8.1.4 and CVS HEAD. I list below my preexisting schema, a set of commands that behave as I expect (and result in an ERROR), and a similar set of commands that do not behave as I expect (and result in a PANIC). Note the position of BEGIN in each. This is quite finicky

[HACKERS] Replication documentation addition

2006-10-23 Thread Bruce Momjian
Here is my first draft of a new replication section for our documentation. I am looking for any comments. --- Replication === Database replication allows multiple computers to work together, making them appear as

Re: [HACKERS] [DOCS] Replication documentation addition

2006-10-23 Thread Bruce Momjian
Please disregard. I am redoing it and will post a URL with the most recent version. --- Bruce Momjian wrote: Here is my first draft of a new replication section for our documentation. I am looking for any comments.

[HACKERS] Replication documentation addition

2006-10-23 Thread Bruce Momjian
Here is a new replication documentation section I want to add for 8.2: ftp://momjian.us/pub/postgresql/mypatches/replication Comments welcomed. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your