Re: [GENERAL] How to get normalized data from tekst column

2011-11-24 Thread Andrus
David, Thank you. Whether you can write a sufficient function with 8.1 features I do not know. You main issue is you need to be able to output multiple records from a single input record and doing so before 8.4 seems problematic since functions like substring cannot do that. comment field c

Re: [GENERAL] How to get normalized data from tekst column

2011-11-24 Thread David Johnston
On Nov 24, 2011, at 15:40, "Andrus Moor" wrote: > Harald, > > Thank you. > >> The query > >> SELECT id, a[1] AS name, a[2] AS percent >> FROM ( SELECT id, regexp_split_to_array(regexp_split_to_table(comment, ', >> *'), '\W+') AS a >> FROM project ) AS dummy > >> should work un every halfway

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Cédric Villemain
Le 24 novembre 2011 20:38, Gaëtan Allart a écrit : > Finally, it "crashed" againŠ :-( > > Here's the output of iotop while databased was inaccessible : > > 32361 be/4 postgres    0.00 B/s    0.00 B/s  0.00 % 99.99 % postgres: mydb > mydb host(34847) idle > 32244 be/4 postgres  163.48 K/s    0.00 B

Re: [GENERAL] How to get normalized data from tekst column

2011-11-24 Thread Andrus Moor
Harald, Thank you. The query SELECT id, a[1] AS name, a[2] AS percent FROM ( SELECT id, regexp_split_to_array(regexp_split_to_table(comment, ', *'), '\W+') AS a FROM project ) AS dummy should work un every halfway recent PostgreSQL version - dunno about 8.1. I tried it but got error

Re: [GENERAL] Convert string to UNICODE & migration FROM 7.4 to 9.1

2011-11-24 Thread Tom Lane
Philippe Lang writes: > I have a function under PG 7.4 that returns an xml structure, with the > content encoded in UNICODE. Basically, the only reason you got away with that in 7.4 is that 7.4 is so lax about encodings. In general, in modern releases, all text strings inside the backend are in

Re: [GENERAL] Compiler does not detect support for 64 bit integers

2011-11-24 Thread Antonio Franzoso
Il 23/11/2011 22:44, Tom Lane ha scritto: Antonio Franzoso writes: I'm using Code::block with MinGW on a Windows Seven x64 and I get these errors: ..\..\..\Program Files\PostgreSQL\9.0\include\server\c.h|284|error: #error must have a working 64-bit integer datatype| Um ... did you run the conf

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Gaëtan Allart
Finally, it "crashed" againŠ :-( Here's the output of iotop while databased was inaccessible : 32361 be/4 postgres0.00 B/s0.00 B/s 0.00 % 99.99 % postgres: mydb mydb host(34847) idle 32244 be/4 postgres 163.48 K/s0.00 B/s 0.00 % 99.99 % postgres: mydb mydb host(34660) SELECT 32045

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Cédric Villemain
Le 24 novembre 2011 17:02, Tomas Vondra a écrit : > On 24 Listopad 2011, 16:39, Robert Treat wrote: >> On Thu, Nov 24, 2011 at 9:09 AM, Tomas Vondra wrote: >>> On 24 Listopad 2011, 14:51, Gaëtan Allart wrote: Postgresql.conf : max_connections = 50 shared_buffers = 12G tem

Re: [GENERAL] "CREATE TABLE table_name AS EXECUTE name WITH DATA" becomes syntax error.

2011-11-24 Thread Tom Lane
Naoya Anzai writes: > Accroding to > http://www.postgresql.org/docs/9.1/interactive/sql-createtableas.html , > "CREATE TABLE table_name AS EXECUTE name WITH DATA" seems a right syntax, > but,this statement becomes a SYNTAX ERROR. Hmm ... it looks like WITH [NO] DATA is actually only implemented

[GENERAL] Convert string to UNICODE & migration FROM 7.4 to 9.1

2011-11-24 Thread Philippe Lang
Hi, I'm migrating a PG 7.4 database (encoded in LATIN1) to PG 9.1. The upgrade is just fine, except a problem with a conversion to UNICODE for which I was not able to find a solution yet: I have a function under PG 7.4 that returns an xml structure, with the content encoded in UNICODE. The fu

Re: [GENERAL] Reassign value of IN parameter in 9.1.1

2011-11-24 Thread Tom Lane
Gavin Casey writes: > It was actually the reassignment of an IN parameter that I was questioning, That was changed in 9.0, per the release notes: * Allow input parameters to be assigned values within PL/pgSQL functions (Steve Prentice) Formerly, input parameters were tre

Re: [GENERAL] "CREATE TABLE table_name AS EXECUTE name WITH DATA" becomes syntax error.

2011-11-24 Thread Adrian Klaver
On Thursday, November 24, 2011 1:55:53 am Naoya Anzai wrote: > Hi, > > Accroding to > http://www.postgresql.org/docs/9.1/interactive/sql-createtableas.html , > "CREATE TABLE table_name AS EXECUTE name WITH DATA" seems a right syntax, > but,this statement becomes a SYNTAX ERROR. > Is this a specifi

Re: [GENERAL] How to get normalized data from tekst column

2011-11-24 Thread Harald Fuchs
"Andrus" writes: > David, >>Regular Expressions are your friend here. If you do not know them you > should learn them; though if you ask nicely someone may just provide you > the solution you need. >>Split-to-array and unnest may work as well. > > Thank you very much. I dona**t know regexps. >

Re: [GENERAL] Incremental backup with RSYNC or something?

2011-11-24 Thread Phoenix Kiula
On Thu, Nov 24, 2011 at 11:53 PM, Benjamin Henrion wrote: > On Thu, Nov 24, 2011 at 4:49 PM, Phoenix Kiula > wrote: >> On Wed, Nov 23, 2011 at 6:13 AM, Alex Thurlow >> wrote: >>> On 11/22/2011 3:28 PM, Merlin Moncure wrote: >> .. >>> How long is this backup taking?  I have a ~100GB databas

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Gaëtan Allart
Tomas, I've enabled logging of checkpoints. I'm waiting for the next i/o crisisŠ Gaëtan Le 24/11/11 17:02, « Tomas Vondra » a écrit : >On 24 Listopad 2011, 16:39, Robert Treat wrote: >> On Thu, Nov 24, 2011 at 9:09 AM, Tomas Vondra wrote: >>> On 24 Listopad 2011, 14:51, Gaëtan Allart wrot

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Gaëtan Allart
Hi Robert, Thanks for your help as well. You're right about checkpoints, it's running pretty good at start then encounter heavy i/os. I've changed theses settings and also reduced work_mem a little and reduced effective_cache_size btw. LOG: parameter "work_mem" changed to "96MB" LOG: parameter

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Tomas Vondra
On 24 Listopad 2011, 16:39, Robert Treat wrote: > On Thu, Nov 24, 2011 at 9:09 AM, Tomas Vondra wrote: >> On 24 Listopad 2011, 14:51, Gaëtan Allart wrote: >>> Postgresql.conf : >>> >>> max_connections = 50 >>> shared_buffers = 12G >>> temp_buffers = 40MB >>> work_mem = 128MB >>> maintenance_work_m

Re: [GENERAL] Incremental backup with RSYNC or something?

2011-11-24 Thread Benjamin Henrion
On Thu, Nov 24, 2011 at 4:49 PM, Phoenix Kiula wrote: > On Wed, Nov 23, 2011 at 6:13 AM, Alex Thurlow > wrote: >> On 11/22/2011 3:28 PM, Merlin Moncure wrote: > .. >> How long is this backup taking?  I have a ~100GB database that I back up >> with pg_dump (which compresses as it dumps if you

Re: [GENERAL] Incremental backup with RSYNC or something?

2011-11-24 Thread Phoenix Kiula
On Wed, Nov 23, 2011 at 6:13 AM, Alex Thurlow wrote: > On 11/22/2011 3:28 PM, Merlin Moncure wrote: .. > How long is this backup taking?  I have a ~100GB database that I back up > with pg_dump (which compresses as it dumps if you want it to) and that only > takes 35 minutes.  Granted, I have i

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Robert Treat
On Thu, Nov 24, 2011 at 9:09 AM, Tomas Vondra wrote: > On 24 Listopad 2011, 14:51, Gaëtan Allart wrote: >> Hello everyone, >> >> I'm having some troubles with a Postgresql server. >> We're using PG has a database backend for a very big website (lots of data >> and much traffic). >> >> The issue :

Re: [GENERAL] Reassign value of IN parameter in 9.1.1

2011-11-24 Thread Pavel Stehule
Hello 2011/11/24 Gavin Casey : > This works in 9.1.1 but seems like a bug to me: > > create function xout(_x INTEGER) > returns integer > as $$ > begin >    _x = _x * 2; >    return _x; > end; > $$ LANGUAGE plpgsql; > > select xout(4); > > It would not have compiled in version 8. > > I came across

Re: [GENERAL] How to get normalized data from tekst column

2011-11-24 Thread Andrus
David, >Regular Expressions are your friend here. If you do not know them you should >learn them; though if you ask nicely someone may just provide you the solution >you need. >Split-to-array and unnest may work as well. Thank you very much. I don’t know regexps. Can you provide example, please

Re: [GENERAL] How to get normalized data from tekst column

2011-11-24 Thread David Johnston
On Nov 24, 2011, at 8:47, "Andrus" wrote: > Project table contains salesman names and percents as shown > below. Single comment column contains 1-2 salesman names and commissions. > How select normalized data from this table ? > > Andrus. > > CREATE TABLE project ( > id char(10) primary key,

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Tomas Vondra
On 24 Listopad 2011, 15:27, Gaëtan Allart wrote: > Hi Thomas, > > I will be using iotop ;) > Right now, most i/o come from "postgres: wal writer process". What do you mean by "most I/O" - how much data is it writing? Is there a vacuum running at the same time? What other processes are doing I/O?

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Gaëtan Allart
Hi Thomas, I will be using iotop ;) Right now, most i/o come from "postgres: wal writer process". -> effective_cache_size Okay, I'll rise it to 32Gb. -> fsync : changed to on ;) -> seq_pages : i'll run tests. Thanks. -> dirty : cat /proc/sys/vm/dirty_ratio 20 cat /proc/sys/vm/dirty_backgrou

Re: [GENERAL] Reassign value of IN parameter in 9.1.1

2011-11-24 Thread Gavin Casey
On 24 November 2011 14:12, Alban Hertroys wrote: > On 24 November 2011 14:52, Gavin Casey wrote: > > This works in 9.1.1 but seems like a bug to me: > > > > create function xout(_x INTEGER) > > returns integer > > as $$ > > begin > >_x = _x * 2; > > I would expect an error here, as having an

Re: [GENERAL] Reassign value of IN parameter in 9.1.1

2011-11-24 Thread Alban Hertroys
On 24 November 2011 14:52, Gavin Casey wrote: > This works in 9.1.1 but seems like a bug to me: > > create function xout(_x INTEGER) > returns integer > as $$ > begin >    _x = _x * 2; I would expect an error here, as having an expression without a context (an if-statement, for example) should be

Re: [GENERAL] General performance/load issue

2011-11-24 Thread Tomas Vondra
On 24 Listopad 2011, 14:51, Gaëtan Allart wrote: > Hello everyone, > > I'm having some troubles with a Postgresql server. > We're using PG has a database backend for a very big website (lots of data > and much traffic). > > The issue : server suddenly (1H after restart) becomes slow (queries not >

[GENERAL] Reassign value of IN parameter in 9.1.1

2011-11-24 Thread Gavin Casey
This works in 9.1.1 but seems like a bug to me: create function xout(_x INTEGER) returns integer as $$ begin _x = _x * 2; return _x; end; $$ LANGUAGE plpgsql; select xout(4); It would not have compiled in version 8. I came across such a reassignement doing a code review and was surprised

[GENERAL] General performance/load issue

2011-11-24 Thread Gaëtan Allart
Hello everyone, I'm having some troubles with a Postgresql server. We're using PG has a database backend for a very big website (lots of data and much traffic). The issue : server suddenly (1H after restart) becomes slow (queries not responding), load rises (>20 instead of 1), iowait rises (20 to

[GENERAL] How to get normalized data from tekst column

2011-11-24 Thread Andrus
Project table contains salesman names and percents as shown below. Single comment column contains 1-2 salesman names and commissions. How select normalized data from this table ? Andrus. CREATE TABLE project ( id char(10) primary key, comment char(254) ); insert into test values ('2010-12', 'Aa

[GENERAL] How to display the progress of query

2011-11-24 Thread pasman pasmański
Hi. I try to monitor a progress of the insert statement: insert into table1 (id,other fields) select id+0*nextval('public.progress'),other fields >From second session i run: select nextval('public.progress'); but sequence 'progress' looks unchanged. How to display number of processed rows

Re: [GENERAL] Recommendations for SSDs in production?

2011-11-24 Thread Yeb Havinga
On 2011-11-04 16:24, David Boreham wrote: On 11/4/2011 8:26 AM, Yeb Havinga wrote: First, if your'e interested in doing a test like this yourself, I'm testing on ubuntu 11.10, but even though this is a brand new distribution, the smart database was a few months old. 'update-smart-drivedb' ha

Re: [GENERAL] Table Design question for gurus (without going to "NoSQL")...

2011-11-24 Thread Phoenix Kiula
On Mon, Nov 21, 2011 at 6:14 PM, Tomas Vondra wrote: . >> >> An index on (a, b) can be used for queries involving only a but not for >> those involving only b. > > That is not true since 8.2 - a multi-column index may be used even for > queries without conditions on leading columns. It won't b

Re: [GENERAL] Installed. Now what?

2011-11-24 Thread Phoenix Kiula
On Thu, Nov 24, 2011 at 10:42 AM, Adrian Klaver wrote: >> >> Also, how can I tell the pgbouncer log not to log proper connections >> and their closing. Right now it's filling up with nonsense. I only >> want it to log when there's a warning or error. > > http://pgbouncer.projects.postgresql.org/do

[GENERAL] "CREATE TABLE table_name AS EXECUTE name WITH DATA" becomes syntax error.

2011-11-24 Thread Naoya Anzai
Hi, Accroding to http://www.postgresql.org/docs/9.1/interactive/sql-createtableas.html , "CREATE TABLE table_name AS EXECUTE name WITH DATA" seems a right syntax, but,this statement becomes a SYNTAX ERROR. Is this a specification? --- naoya=# SELECT VERSION();