Re: [GENERAL] gmake check problem

2010-04-27 Thread Kenichiro Tanaka
Hi Jim To resolve this problem, we have to indicate which SQL is hanging. stats test executes postgresql-8.4.3/src/test/regress/sql/stats.sql. we can see what the test does. And we can see the log at postgresql-8.4.3/src/test/regress/results/stats.out So I suppose we can indicate which SQL

[GENERAL] PostgreSQL Performance issue

2010-04-27 Thread A.Bhattacharya
Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. However, I have a batch program written in Java which processes the data and populates them into tables in Postgres database. I have 622,000 number of records but it is taking almost 4 and half hours to load

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Thom Brown
On 27 April 2010 09:11, a.bhattacha...@sungard.com wrote: Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. However, I have a batch program written in Java which processes the data and populates them into tables in Postgres database. I have *622,000

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Nikhil G. Daddikar
Most likely you are inserting one per transaction. Set autocommit to false and commit only after all the inserts are done. -n. On 27-04-2010 13:41, a.bhattacha...@sungard.com wrote: Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. However, I have a batch

[GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Rod
Hello, I have a web application where users upload/share files. After file is uploaded it is copied to S3 and all subsequent downloads are done from there. So in a file's lifetime it's accessed only twice- when created and when copied to S3. Files are documents, of different size from few

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Magnus Hagander
On Tue, Apr 27, 2010 at 10:11, a.bhattacha...@sungard.com wrote: Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. You really need to upgrade. At least to 8.3.10. It has many important bugfixes. However, I have a batch program written in Java which

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread John R Pierce
Rod wrote: Hello, I have a web application where users upload/share files. After file is uploaded it is copied to S3 and all subsequent downloads are done from there. So in a file's lifetime it's accessed only twice- when created and when copied to S3. Files are documents, of different size

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread John R Pierce
a.bhattacha...@sungard.com wrote: Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. However, I have a batch program written in Java which processes the data and populates them into tables in Postgres database. I have *622,000 number of records *but it is

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Rod
No, I'm not storing RDBMS in S3. I didn't write that in my post. S3 is used as CDN, only for downloading files. On Tue, Apr 27, 2010 at 6:54 PM, John R Pierce pie...@hogranch.com wrote: Rod wrote: Hello, I have a web application where users upload/share files. After file is uploaded it is

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Massa, Harald Armin
No, I'm not storing RDBMS in S3. I didn't write that in my post. S3 is used as CDN, only for downloading files. So you are storing your files on S3 ? Why should you store those files additionally in a PostgreSQL database? If you want to keep track of them / remember metadata, hashes will do

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Alban Hertroys
On 27 Apr 2010, at 10:11, a.bhattacha...@sungard.com a.bhattacha...@sungard.com wrote: Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. However, I have a batch program written in Java which processes the data and populates them into tables in Postgres

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Cédric Villemain
2010/4/27 Rod cckra...@gmail.com: Hello, I have a web application where users upload/share files. After file is uploaded it is copied to S3 and all subsequent downloads are done from there. So in a file's lifetime it's accessed only twice- when created and when copied to S3. Files are

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Rod
S3 is not primary storage for the files, it's a distribution system. We want to be able to switch form S3 to other CDN if required. So, Master copies of files is kept on private server. Question is should it be database of filesystem. On Tue, Apr 27, 2010 at 7:03 PM, Massa, Harald Armin

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread A.Bhattacharya
Thanks a lot for your help. However I am new to Postgres database therefore it would be nice if you can let me know how to set autocommit off. I know from psql client issuing \set Autocommit Off would set it off but unfortunately it doesn't set it off. Many thanks for your help.

Re: [GENERAL] Efficient intersect operation (Full Text Search)

2010-04-27 Thread Allan Kamau
On Wed, Apr 14, 2010 at 11:39 AM, Allan Kamau kamaual...@gmail.com wrote: I have a brief question - I can provide more information if it is not clear. I would like to perform pairwise intersect operations between several pairs of sets (where a set is a list or vector of labels), I have many

Re: [GENERAL] Unable to run createlang (or psql for that matter)

2010-04-27 Thread John Gage
Do you know of any guides to ritual suicide? On Apr 27, 2010, at 3:02 AM, Scott Mead wrote: Your path has 'PostgresPlus' Locate shows 'PostgreSQL' -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

[GENERAL] pg_hba.conf

2010-04-27 Thread jkunkel
I am putting up a new server on version 8.4.3. I copied pg_hba.conf from a running 8.3.6 system, changing only the public IP address for the local machine. I get the error: FATAL: no pg_hba.conf entry for host 209.159.145.248, user postgres, database arc pg_hba.conf contains the line: host

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Adrian von Bidder
On Tuesday 27 April 2010 11.17:42 Cédric Villemain wrote: Anyone had this kind of design problem and how did you solve it? store your files in a filesystem, and keep the path to the file (plus metadata, acl, etc...) in database. ... and be careful that db and file storage do not go out of

Re: [GENERAL] pg_hba.conf

2010-04-27 Thread Raymond O'Donnell
On 27/04/2010 11:42, jkun...@laurcat.com wrote: I am putting up a new server on version 8.4.3. I copied pg_hba.conf from a running 8.3.6 system, changing only the public IP address for the local machine. I get the error: FATAL: no pg_hba.conf entry for host 209.159.145.248, user postgres,

[GENERAL] invalid abstime to timestamp error only on specific date range

2010-04-27 Thread zhong ming wu
I have a postgres table with a column type abstime. (all data in this database are loaded from an ancient propriety database) When I filter data from this table with a specific date range on this column it works. When I filter with a different date range it gives me an error like invalid

Re: [GENERAL] pg_hba.conf

2010-04-27 Thread Kenichiro Tanaka
Hi Would there be a line earlier in the file that matches and is preventing a connection? At first, I think so too. But if there is a line earlier in the file ,we get following error. psql: could not connect to server:

Re: [GENERAL] pg_hba.conf

2010-04-27 Thread Chris Barnes
I've had problems before with the listen_addresses and had to set it accordingly. Wouldn't accept connections locally. listen_addresses = '*' # what IP address(es) to listen on; # comma-separated list of addresses;

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Rob Richardson
I am curious to know how much of your delay is due to PostgreSQL and how much to your Java batch program. If you comment out the call to the database function, so that you are reading your input file but not doing anything with the data, how long does your batch program take to run? RobR

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Alban Hertroys
On 27 Apr 2010, at 11:15, a.bhattacha...@sungard.com a.bhattacha...@sungard.com wrote: Thanks a lot for your help. However I am new to Postgres database therefore it would be nice if you can let me know how to set autocommit off. I know from psql client issuing “\set Autocommit Off” would

[GENERAL] lc_ctype does not work on windows ?

2010-04-27 Thread Filip Rembiałkowski
Hi I have a problem with locale on windows. postgresql 8.4. this does not work as expected: filip=# select 'A' ~ '\w', 'Ą' ~ '\w'; ?column? | ?column? --+-- t| f (1 row) -- investigating postgres settings: filip=# select name, setting, context, source, boot_val,

Re: [GENERAL] lc_ctype does not work on windows ?

2010-04-27 Thread Tom Lane
=?UTF-8?Q?Filip_Rembia=C5=82kowski?= plk.zu...@gmail.com writes: I have a problem with locale on windows. postgresql 8.4. this does not work as expected: filip=# select 'A' ~ '\w', 'Ą' ~ '\w'; The locale-aware regex operators don't really work in multibyte encodings. There's a (partial)

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Merlin Moncure
On Tue, Apr 27, 2010 at 5:17 AM, Alban Hertroys dal...@solfertje.student.utwente.nl wrote: On 27 Apr 2010, at 10:11, a.bhattacha...@sungard.com a.bhattacha...@sungard.com wrote: Dear All Experts, I am using in PostgreSQL 8.3.5 database on windows 64 bit OS. However, I have a batch program

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Bayless Kirtley
Thanks a lot for your help. However I am new to Postgres database therefore it would be nice if you can let me know how to set autocommit off. I know from psql client issuing “\set Autocommit Off” would set it off but unfortunately it doesn’t set it off. It's a client-side setting, not a

Re: [GENERAL] invalid abstime to timestamp error only on specific date range

2010-04-27 Thread Tom Lane
zhong ming wu mr.z.m...@gmail.com writes: I have a postgres table with a column type abstime. (all data in this database are loaded from an ancient propriety database) When I filter data from this table with a specific date range on this column it works. When I filter with a different date

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Greg Smith
a.bhattacha...@sungard.com wrote: I have *622,000 number of records *but it is taking almost *4 and half hours* to load these data into the tables. I have a simple function in db which is being called from Java batch program to populate the records into tables from flat files. Four likely

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread Vincenzo Romano
2010/4/27 Greg Smith g...@2ndquadrant.com: a.bhattacha...@sungard.com wrote: I have *622,000 number of records *but it is taking almost *4 and half hours* to load these data into the tables. Without the schema and the queries, all you can get is guessing. -- Vincenzo Romano NotOrAnd

Re: [GENERAL] Postgresql on EC2/EBS in production?

2010-04-27 Thread Greg Smith
Nikhil G. Daddikar wrote: I was wondering if any of you are using (or tried to use) PG+EC2/EBS on a production system. Are any best-practices. Googling didn't help much. A few articles I came across scared me a bit. There have been a couple of reports of happy users:

[GENERAL] Open Source BI Tool

2010-04-27 Thread akp geek
Hi all - I would like to know, if there is a open source BI tool for creating reports against Postgres database ? appreciate your help Regards

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Raymond O'Donnell
On 27/04/2010 17:33, akp geek wrote: Hi all - I would like to know, if there is a open source BI tool for creating reports against Postgres database ? appreciate your help Pardon my ignorance - what's a BI tool? Thanks. :-) Ray. -- Raymond O'Donnell :: Galway :: Ireland

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread akp geek
Business Intelligence tool ( crystal reports for example ) Regards On Tue, Apr 27, 2010 at 12:38 PM, Raymond O'Donnell r...@iol.ie wrote: On 27/04/2010 17:33, akp geek wrote: Hi all - I would like to know, if there is a open source BI tool for creating reports against

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Cédric Villemain
2010/4/27 akp geek akpg...@gmail.com: Hi all - I would like to know, if there is a open source BI tool for creating reports against Postgres database ? appreciate your help Pentaho have some good tools http://www.pentaho.com/ Regards -- Cédric Villemain -- Sent via

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Raymond O'Donnell
On 27/04/2010 17:40, akp geek wrote: Business Intelligence tool ( crystal reports for example ) Ah - grand - thanks! Ray. -- Raymond O'Donnell :: Galway :: Ireland r...@iol.ie -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Steve Atkins
On Apr 27, 2010, at 9:33 AM, akp geek wrote: Hi all - I would like to know, if there is a open source BI tool for creating reports against Postgres database ? appreciate your help Pentaho, BIRT, JasperReports (and associated tools like iReport), JFreeReport, DataVision,

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Joshua D. Drake
On Tue, 2010-04-27 at 12:33 -0400, akp geek wrote: Hi all - I would like to know, if there is a open source BI tool for creating reports against Postgres database ? appreciate your help Pentaho Regards -- PostgreSQL.org Major Contributor Command Prompt, Inc:

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Adrian von Bidder
On Tuesday 27 April 2010 19.12:31 Steve Atkins wrote: [...] BIRT [...] FWIW, my cow-orkers are quite happy with BIRT (especially with the designer environment in Eclipse) after having used Crystal and Actuate previously. I've no idea how BIRT compares with the other OSS tools, but compared

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread akp geek
Thank you all . regards On Tue, Apr 27, 2010 at 2:42 PM, Adrian von Bidder avbid...@fortytwo.chwrote: On Tuesday 27 April 2010 19.12:31 Steve Atkins wrote: [...] BIRT [...] FWIW, my cow-orkers are quite happy with BIRT (especially with the designer environment in Eclipse) after

[GENERAL] Errors starting postgres after initializing new database

2010-04-27 Thread Chris Barnes
I have just initialized a database with no errors, used the postgresql.conf file from another system running 8.4.2. Attempted to start and got the fatal error below. I than ran pgtune and got the same error. I am not sure what the problem is? Is there more detailed logging than below, or can

Re: [GENERAL] Errors starting postgres after initializing new database

2010-04-27 Thread Thom Brown
On 27 April 2010 21:30, Chris Barnes compuguruchrisbar...@hotmail.comwrote: I have just initialized a database with no errors, used the postgresql.conf file from another system running 8.4.2. Attempted to start and got the fatal error below. I than ran pgtune and got the same error. I am

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread John R Pierce
akp geek wrote: Business Intelligence isn't that an oxymoron? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

[GENERAL] Server Panic when trying to stop point in time recovery

2010-04-27 Thread Chris Copeland
I am running 8.2.4 on Solaris 10 x86. I have setup WAL file shipping from a primary server to a warm standby. I am able to start the standby server using a backup from the primary and get it to apply the log files as they arrive. My problem comes when I want to trigger the standby server to

Re: [GENERAL] Open Source BI Tool

2010-04-27 Thread Joshua D. Drake
On Tue, 2010-04-27 at 13:40 -0700, John R Pierce wrote: akp geek wrote: Business Intelligence isn't that an oxymoron? Depends, you want your paycheck? -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training,

[GENERAL] VACUUM FULL vs backup/restore

2010-04-27 Thread David Wall
We are copying a production database and then pairing it down dramatically (perhaps removing 99% of records, most of which are large objects). It seems my options are 1) VACUUM FULL with a reindex_db; or 2) backup, then restore. Is there anything one does better than the other? Our

Re: [GENERAL] Server Panic when trying to stop point in time recovery

2010-04-27 Thread Joshua D. Drake
On Tue, 2010-04-27 at 15:39 -0500, Chris Copeland wrote: Any help is greatly appreciated. Please let me know if I can provide any more information that will be helpful. Well the problem is that the files are missing. How are you copying them? Joshua D. Drake -Chris --

Re: [GENERAL] Server Panic when trying to stop point in time recovery

2010-04-27 Thread Chris Copeland
On Tue, Apr 27, 2010 at 4:02 PM, Joshua D. Drake j...@commandprompt.com wrote: Well the problem is that the files are missing. How are you copying them? Joshua D. Drake I am using rsync to copy the files from the master into a directory /opt/wal/archwalremote on the standby. My restore

Re: [GENERAL] VACUUM FULL vs backup/restore

2010-04-27 Thread Vick Khera
On Tue, Apr 27, 2010 at 4:50 PM, David Wall d.w...@computer.org wrote: It seems my options are 1) VACUUM FULL with a reindex_db; or 2) backup, then restore. I'd go with the backup+restore. The other option is to to a trivial ALTER to one of your fields which causes the table to be rewritten.

Re: [GENERAL] VACUUM FULL vs backup/restore

2010-04-27 Thread Scott Marlowe
On Tue, Apr 27, 2010 at 2:50 PM, David Wall d.w...@computer.org wrote: We are copying a production database and then pairing it down dramatically (perhaps removing 99% of records, most of which are large objects). It seems my options are 1) VACUUM FULL with a reindex_db; or 2) backup, then

Re: [GENERAL] invalid abstime to timestamp error only on specific date range

2010-04-27 Thread zhong ming wu
On Tue, Apr 27, 2010 at 11:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: zhong ming wu mr.z.m...@gmail.com writes: The closest thing I can find to that in the code is        cannot convert abstime invalid to timestamp Yes this is the message. There were invalid values in that table at first.

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Anthony
On Tue, Apr 27, 2010 at 5:17 AM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: store your files in a filesystem, and keep the path to the file (plus metadata, acl, etc...) in database. What type of filesystem is good for this? A filesystem with support for storing tens of

Re: [GENERAL] Postgresql on EC2/EBS in production?

2010-04-27 Thread Bryan Murphy
On Tue, Apr 27, 2010 at 11:31 AM, Greg Smith g...@2ndquadrant.com wrote: Nikhil G. Daddikar wrote: I was wondering if any of you are using (or tried to use) PG+EC2/EBS on a production system. Are any best-practices. Googling didn't help much. A few articles I came across scared me a bit.

Re: [GENERAL] Storing many big files in database- should I do it?

2010-04-27 Thread Adrian Klaver
On Tuesday 27 April 2010 5:45:43 pm Anthony wrote: On Tue, Apr 27, 2010 at 5:17 AM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: store your files in a filesystem, and keep the path to the file (plus metadata, acl, etc...) in database. What type of filesystem is good for

[GENERAL] Is the wiki down?

2010-04-27 Thread Chris Velevitch
I was just checking http://wiki.postgresql.org/wiki/PostgreSQL_Release_Support_Policy and I get a connection time out. Is the site down? Chris -- Chris Velevitch Manager - Adobe Platform Users Group, Sydney m: 0415 469 095 www.apugs.org.au Adobe Platform Users Group, Sydney May 2010: TBD Date:

Re: [GENERAL] Errors starting postgres after initializing new database

2010-04-27 Thread Greg Smith
Chris Barnes wrote: 2010-04-27 16:19:17 EDTFATAL: requested shared memory size overflows size_t This is what happens if you try and use a postgresql.conf with parameters set for a 64-bit installation on a 32-bit operating system. You're trying to do this: shared_buffers = 7680MB But

Re: [GENERAL] VACUUM FULL vs backup/restore

2010-04-27 Thread Greg Smith
David Wall wrote: It seems my options are 1) VACUUM FULL with a reindex_db; or 2) backup, then restore. Is there anything one does better than the other? Our impression is that the backup and restore will run faster (perhaps 2 hours), whereas we have a currently running VACUUM FULL that's

Re: [GENERAL] Postgresql on EC2/EBS in production?

2010-04-27 Thread Greg Smith
Bryan Murphy wrote: We had to fail over to one of our spares twice in the last 1.5 years. Not fun. Both times were due to instance failure. What do you mean by an instance failure here? The actual EC2 image getting corrupted so that it won't boot anymore, or just the instance going down

Re: [GENERAL] Is the wiki down?

2010-04-27 Thread Greg Smith
Chris Velevitch wrote: I was just checking http://wiki.postgresql.org/wiki/PostgreSQL_Release_Support_Policy and I get a connection time out. Is the site down? I'm not sure if it's down or not, but I haven't been able to reach it from here in some time either (home Verizon FIOS connection)

Re: [GENERAL] PostgreSQL Performance issue

2010-04-27 Thread A.Bhattacharya
I am curious to know how much of your delay is due to PostgreSQL and how much to your Java batch program. If you comment out the call to the database function, so that you are reading your input file but not doing anything with the data, how long does your batch program take to run? RobR