Re: [GENERAL] SSD Drives

2014-04-04 Thread David Rees
On Fri, Apr 4, 2014 at 5:20 PM, Scott Marlowe wrote: > The real danger with consumer drives is they don't have supercaps and > can and will therefore corrupt your data on power failure. The actual > write cycles aren't a big deal for many uses, as now even consumer > drives have very long write cy

Re: [GENERAL] Remote troubleshooting session connection?

2014-04-04 Thread David Johnston
Jim Garrison wrote > Assuming I can pause the driving Java code between queries in a job, is > there any way to connect from PGAdmin (or another tool) and view the state > of tables in the in-progress transaction? If this is not currently > possible, how difficult would it be to do? What you want

Re: [GENERAL] SSD Drives

2014-04-04 Thread Scott Marlowe
On Fri, Apr 4, 2014 at 5:29 PM, Lists wrote: > On 04/02/2014 02:55 PM, Bret Stern wrote: >> >> Care to share the SSD hardware you're using? >> >> I've used none to date, and have some critical data I would like >> to put on a development server to test with. >> >> Regards, >> >> Bret Stern > > > S

Re: [GENERAL] SSD Drives

2014-04-04 Thread James Harper
> > It might be tempting to use a consumer-grade SSD due to the significant > cost savings, but the money saved is vapor. They may be OK for a dev > environment, but you *will* pay in downtime in a production environment. > Unlike regular hard drives where the difference between consumer and > ent

Re: [GENERAL] SSD Drives

2014-04-04 Thread Lists
On 04/02/2014 02:55 PM, Bret Stern wrote: Care to share the SSD hardware you're using? I've used none to date, and have some critical data I would like to put on a development server to test with. Regards, Bret Stern SSDs are ridiculously cheap when you consider the performance difference.

Re: [GENERAL] Remote troubleshooting session connection?

2014-04-04 Thread John R Pierce
On 4/4/2014 3:41 PM, Jim Garrison wrote: Assuming I can pause the driving Java code between queries in a job, is there any way to connect from PGAdmin (or another tool) and view the state of tables in the in-progress transaction? If this is not currently possible, how difficult would it be to

Re: [GENERAL] Remote troubleshooting session connection?

2014-04-04 Thread Adrian Klaver
On 04/04/2014 03:41 PM, Jim Garrison wrote: I'm working on an ETL system that is driven from Java/JDBC but is implemented mostly as SQL queries against a PostgreSQL database. An ETL "job" runs inside its own transaction and consists of a series of queries that transform the data from staging t

[GENERAL] Remote troubleshooting session connection?

2014-04-04 Thread Jim Garrison
I'm working on an ETL system that is driven from Java/JDBC but is implemented mostly as SQL queries against a PostgreSQL database. An ETL "job" runs inside its own transaction and consists of a series of queries that transform the data from staging tables to the destination tables. If a fail

Re: [GENERAL] SSD Drives

2014-04-04 Thread David Boreham
On 4/4/2014 3:57 PM, Steve Crawford wrote: Judicious archiving allows us to keep our total OS+data storage requirements under 100GB. Usually. So we should be able to easily stay in the $500/drive price range (200GB S3700) and still have plenty of headroom for wear-leveling. One option I'm co

Re: [GENERAL] SSD Drives

2014-04-04 Thread Steve Crawford
On 04/04/2014 10:15 AM, Merlin Moncure wrote: 2. Do I need both BBU on the RAID *and* capacitor on the SSD or just on one? Which one? I'm suspecting capacitor on the SSD and write-through on the RAID. You need both. The capacitor protects the drive, the BBU protects the raid controller. ?? In wr

Re: [GENERAL] SSD Drives

2014-04-04 Thread Merlin Moncure
On Friday, April 4, 2014, Scott Marlowe wrote: > On Fri, Apr 4, 2014 at 1:18 PM, John R Pierce > > > wrote: > > On 4/4/2014 12:08 PM, Scott Marlowe wrote: > >> > >> You don't technically need the BBU / flashback memory IF the > >> controller is in write through. > > > > > > if you HAVE the BBU/f

Re: [GENERAL] Indices and Foreign Tables

2014-04-04 Thread Torsten Förtsch
On 04/04/14 17:14, Kohler Manuel (ID SIS) wrote: > Here is the same query with fdw: > > db=# EXPLAIN VERBOSE select ds.code, count(*), sum(dsf.size_in_bytes) as > "raw_size",pg_size_pretty(sum(dsf.size_in_bytes)) as "size" from > data_set_files_fdw dsf, data_sets_fdw ds where dsf.parent_id is nu

Re: [GENERAL] Understanding setof

2014-04-04 Thread Jeff Janes
On Fri, Apr 4, 2014 at 12:42 PM, Robert James wrote: > I'm having trouble with setof returning functions > > Let's say I have function x() returning setof integers. I want to do > SELECT x(), but only keep the values which meet a criteria. Something > like: SELECT x() as xval WHERE xval = 10. >

Re: [GENERAL] Understanding setof

2014-04-04 Thread Raymond O'Donnell
On 04/04/2014 20:42, Robert James wrote: > I'm having trouble with setof returning functions > > Let's say I have function x() returning setof integers. I want to do > SELECT x(), but only keep the values which meet a criteria. Something > like: SELECT x() as xval WHERE xval = 10. > > How can I

Re: [GENERAL] SSD Drives

2014-04-04 Thread Scott Marlowe
On Fri, Apr 4, 2014 at 1:18 PM, John R Pierce wrote: > On 4/4/2014 12:08 PM, Scott Marlowe wrote: >> >> You don't technically need the BBU / flashback memory IF the >> controller is in write through. > > > if you HAVE the BBU/flash why would you put the controller in write > through?? the whole P

[GENERAL] Understanding setof

2014-04-04 Thread Robert James
I'm having trouble with setof returning functions Let's say I have function x() returning setof integers. I want to do SELECT x(), but only keep the values which meet a criteria. Something like: SELECT x() as xval WHERE xval = 10. How can I do that? In general, is there a way to "unroll" a set

Re: [GENERAL] SSD Drives

2014-04-04 Thread David Rees
On Fri, Apr 4, 2014 at 10:15 AM, Merlin Moncure wrote: > For all around performance, the > S3700 (2.5$/gb) IMO held the crown for most of 2013 and I think is > still the one to buy. The s3500 (1.25$/gb) came out and also looks > like a pretty good deal The S3500 can be had for $1.00/GB now these

Re: [GENERAL] SSD Drives

2014-04-04 Thread John R Pierce
On 4/4/2014 12:08 PM, Scott Marlowe wrote: You don't technically need the BBU / flashback memory IF the controller is in write through. if you HAVE the BBU/flash why would you put the controller in write through?? the whole POINT of bbu/flashback is that you can safely enable writeback cachi

Re: [GENERAL] SSD Drives

2014-04-04 Thread Scott Marlowe
On Fri, Apr 4, 2014 at 11:15 AM, Merlin Moncure wrote: > On Fri, Apr 4, 2014 at 11:04 AM, Steve Crawford > wrote: >> On 04/03/2014 12:44 PM, Brent Wood wrote: >> 2. Do I need both BBU on the RAID *and* capacitor on the SSD or just on one? >> Which one? I'm suspecting capacitor on the SSD and wri

Re: [GENERAL] Indices and Foreign Tables

2014-04-04 Thread Tom Lane
"Kohler Manuel (ID SIS)" writes: > here are the Query plans. The first plan is on the source database directly. > So no fdw involved: Do you have use_remote_estimate enabled? The rowcount estimates for the foreign tables seem pretty far from reality; use_remote_estimates should help.

Re: [GENERAL] Indices and Foreign Tables

2014-04-04 Thread Kohler Manuel (ID SIS)
Hi, here are the Query plans. The first plan is on the source database directly. So no fdw involved: source_db=# EXPLAIN ANALYZE select ds.code, count(*), sum(dsf.size_in_bytes) as "raw_size",pg_size_pretty(sum(dsf.size_in_bytes)) as "size" from data_set_files dsf, data_sets ds where dsf.parent

Re: [GENERAL] Why is pg_restore trying to create tables in pg_catalog?

2014-04-04 Thread Tom Lane
Seamus Abshere writes: > On 4/4/14, 12:58 PM, Tom Lane wrote: >> Hm ... does myschema actually exist in the target database? [...] if >> myschema doesn't exist, the creation target devolves to pg_catalog. This is >> not terribly elegant but I'm not sure if there's consensus to change it. > You

Re: [GENERAL] SSD Drives

2014-04-04 Thread John R Pierce
On 4/4/2014 10:15 AM, Merlin Moncure wrote: 2. Do I need both BBU on the RAID*and* capacitor on the SSD or just on one? >Which one? I'm suspecting capacitor on the SSD and write-through on the >RAID. You need both. The capacitor protects the drive, the BBU protects the raid controller. note

Re: [GENERAL] Why is pg_restore trying to create tables in pg_catalog?

2014-04-04 Thread Seamus Abshere
On 4/4/14, 12:58 PM, Tom Lane wrote: Seamus Abshere writes: Why is pg_restore trying to put stuff into the pg_catalog schema of all places? Hm ... does myschema actually exist in the target database? [...] if myschema doesn't exist, the creation target devolves to pg_catalog. This is not t

Re: [GENERAL] SSD Drives

2014-04-04 Thread Merlin Moncure
On Fri, Apr 4, 2014 at 11:04 AM, Steve Crawford wrote: > On 04/03/2014 12:44 PM, Brent Wood wrote: > > Hi David, My take: > Does the RAID 1 array give any performance benefits over a single drive? I'd > guess that writes may be slower, reads may be faster (if balanced) but data > security is imp

Re: [GENERAL] UDF calls and FDW

2014-04-04 Thread Tom Lane
Emmanuel Medernach writes: > So the problem is that is calls the UDF on the FDW node (Foreign > Scan on master_object_000 + Filter) and not on the remote server. Right. >- How to call the UDF on the remote server ? postgres_fdw intentionally refuses to do this, because it has no way to kno

Re: [GENERAL] SSD Drives

2014-04-04 Thread David Boreham
It would be useful to know more details -- how much storage space you need for example. fwiw I considered all of these issues when we first deployed SSDs and decided to not use RAID controllers. There have not been any reasons to re-think that decision since. However, it depends on your specif

Re: [GENERAL] SSD Drives

2014-04-04 Thread Steve Crawford
On 04/03/2014 12:44 PM, Brent Wood wrote: Hi David, Does the RAID 1 array give any performance benefits over a single drive? I'd guess that writes may be slower, reads may be faster (if balanced) but data security is improved. I've been looking into upgrading to SSD and wondering about RAID

Re: [GENERAL] Why is pg_restore trying to create tables in pg_catalog?

2014-04-04 Thread Tom Lane
Seamus Abshere writes: > Why is pg_restore trying to put stuff into the pg_catalog schema of all > places? Hm ... does myschema actually exist in the target database? If it doesn't, and the dump file doesn't include a command to create it, this is what you'll get. That's because what pg_dump e

[GENERAL] UDF calls and FDW

2014-04-04 Thread Emmanuel Medernach
Hello, I am using FDW to transparently access a remote Postgres database containing a table indexed with the Q3C library. On the remote server : postgres=# SELECT * FROM object_000 WHERE q3c_radial_query(ra_PS, decl_PS, 1.3, 3.4, .2) ; ... (2416 rows) Time: 130.300 ms But on the FDW

[GENERAL] Why is pg_restore trying to create tables in pg_catalog?

2014-04-04 Thread Seamus Abshere
hi, Why is pg_restore trying to put stuff into the pg_catalog schema of all places? It's ignoring the schema specified in the pg_dump itself (`myschema`) and even my search_path (`public`). $ psql stuff_development --command "show search_path" search_path - public (1 row) $ pg

Re: [GENERAL] hstore - jsonb

2014-04-04 Thread Oleg Bartunov
On Fri, Apr 4, 2014 at 7:17 PM, Thom Brown wrote: > On 4 April 2014 16:15, Oleg Bartunov wrote: >> We'll work on contrib/jsonxtra with all operators ported from hstore >> and release it after 9.4 as separate extension. > > That would be useful. :) > > Would there be an aim of getting that in-core

Re: [GENERAL] hstore - jsonb

2014-04-04 Thread Thom Brown
On 4 April 2014 16:15, Oleg Bartunov wrote: > We'll work on contrib/jsonxtra with all operators ported from hstore > and release it after 9.4 as separate extension. That would be useful. :) Would there be an aim of getting that in-core for 9.5? -- Thom -- Sent via pgsql-general mailing list

Re: [GENERAL] hstore - jsonb

2014-04-04 Thread Oleg Bartunov
We'll work on contrib/jsonxtra with all operators ported from hstore and release it after 9.4 as separate extension. On Fri, Apr 4, 2014 at 4:32 PM, Thom Brown wrote: > On 4 April 2014 13:04, Oleg Bartunov wrote: >> On Fri, Apr 4, 2014 at 12:20 PM, Armand Turpel >> wrote: >>> Hi, >>> >>> A few

Re: [GENERAL] Indices and Foreign Tables

2014-04-04 Thread Albe Laurenz
Kohler Manuel wrote: > I have a question regarding the use of indices when querying foreign data > tables (using postgres_fdw > of 9.3 to another postgres DB). > Everything works fine beside the fact that the indices which are defined in > the foreign DB are not > used at all when I do query it t

Re: [GENERAL] Postgres connection failures

2014-04-04 Thread Mark van Cuijk
On 04 Apr 2014, at 16:09 , Tom Lane wrote: > Your log extract looks like the server side thought it got a connection > closure command. In particular, the server process seems to have exited > normally, and it did not write "LOG: unexpected EOF on client connection", > which it would have done

Re: [GENERAL] Postgres connection failures

2014-04-04 Thread Tom Lane
Mark van Cuijk writes: > I’m developing an application using Postgres and when doing a transaction > with a bunch of INSERT statements, the connection to the server (on > localhost) often suddenly dies. Trying to diagnose the problem, I’ve > increased log levels and I’ve run a tcpdump session t

[GENERAL] Postgres connection failures

2014-04-04 Thread Mark van Cuijk
Hi, I’m developing an application using Postgres and when doing a transaction with a bunch of INSERT statements, the connection to the server (on localhost) often suddenly dies. Trying to diagnose the problem, I’ve increased log levels and I’ve run a tcpdump session to find out what’s going on,

[GENERAL] Indices and Foreign Tables

2014-04-04 Thread Kohler Manuel (ID SIS)
Hi, I have a question regarding the use of indices when querying foreign data tables (using postgres_fdw of 9.3 to another postgres DB). Everything works fine beside the fact that the indices which are defined in the foreign DB are not used at all when I do query it through the foreign data wrap

Re: [GENERAL] Linux vs FreeBSD

2014-04-04 Thread Achilleas Mantzios
As a side note, when we migrated the exact same pgsql 8.3 system from linux kernel 2.6 to 3.6, we experienced an almost dramatic slowdown by 6 times. Linux Kernel's were known to have issues around those dates, i recall. We had to set synchronous_commit to off, this gave a huge boost , but this w

Re: [GENERAL] Linux vs FreeBSD

2014-04-04 Thread François Beausoleil
Le 2014-04-04 à 08:11, Ray Stell a écrit : > > On Apr 4, 2014, at 12:03 AM, François Beausoleil wrote: > >> I have some performance issues on a Ubuntu 12.04 which I'd like to resolve. >> iowait varies a lot, between 5 and 50%. > > Is the SAN dedicated to this app? I wonder if the i/o, if n

Re: [GENERAL] create temp view from function inside plpgsql function.

2014-04-04 Thread Pavel Stehule
2014-04-04 14:16 GMT+02:00 Tjibbe : > Thanks that works! > > Little bit confusing ERROR. > yes, it could be - but hard to fix it, because it is based on cooperation two worlds - plpgsql and SQL - and it is sometimes not simple. When you understand how plpgsql interpret use variables in SQL queri

Re: [GENERAL] hstore - jsonb

2014-04-04 Thread Thom Brown
On 4 April 2014 13:04, Oleg Bartunov wrote: > On Fri, Apr 4, 2014 at 12:20 PM, Armand Turpel > wrote: >> Hi, >> >> A few questions about jsonb and hstore: >> >> 1. does jsonb replace hstore? > > no, it's different data type > >> 2. compatibility of jsonb <> hstore? > > hstore is a simple key-valu

Re: [GENERAL] create temp view from function inside plpgsql function.

2014-04-04 Thread Tjibbe
Thanks that works! Little bit confusing ERROR. Regards Tjibbe -- +31 6 29401726 tji...@rijpma.org Jacobusstraat 185 3012 JM Rotterdam On 4 April 2014 11:43, Tjibbe wrote: > I have a plpgsql function with: > > > PERFORM * FROM answers(_h);--works fine. > CREATE TEMP VIEW answers AS SELEC

Re: [GENERAL] Linux vs FreeBSD

2014-04-04 Thread Ray Stell
On Apr 4, 2014, at 12:03 AM, François Beausoleil wrote: > I have some performance issues on a Ubuntu 12.04 which I'd like to resolve. > iowait varies a lot, between 5 and 50%. Is the SAN dedicated to this app? I wonder if the i/o, if not related to your app, is being pressed by some other s

Re: [GENERAL] hstore - jsonb

2014-04-04 Thread Oleg Bartunov
On Fri, Apr 4, 2014 at 12:20 PM, Armand Turpel wrote: > Hi, > > A few questions about jsonb and hstore: > > 1. does jsonb replace hstore? no, it's different data type > 2. compatibility of jsonb <> hstore? hstore is a simple key-value model, while jsonb - richer document-based model. > 3. wil

Re: [GENERAL] create temp view from function inside plpgsql function.

2014-04-04 Thread Pavel Stehule
Hello 2014-04-04 11:43 GMT+02:00 Tjibbe : > I have a plpgsql function with: > > > PERFORM * FROM answers(_h);--works fine. > CREATE TEMP VIEW answers AS SELECT * FROM answers(_h); --gives error... > Inside view definition should not be plpgsql variable - this statement has no plan - CREATE

[GENERAL] create temp view from function inside plpgsql function.

2014-04-04 Thread Tjibbe
I have a plpgsql function with: PERFORM * FROM answers(_h);--works fine. CREATE TEMP VIEW answers AS SELECT * FROM answers(_h); --gives error... Why I get this error: ERROR: column \"_h\" does not exist\nLINE 1: ...TEMP VIEW answers AS SELECT * FROM antwoorden_view(_h) -- +31 6 29401726 tj

[GENERAL] json_populate_recordset and nested object, how to solve?

2014-04-04 Thread Raphael Bauduin
Hi, here is an example of a problem I encounter with json_populate_recordset because it does not support nested object. Actually, I would prefer that it ignores nested object rather than raise an error, as it fails on a key I don't even want to use! Here's the query: select e.timestamp::date, e.

[GENERAL] hstore - jsonb

2014-04-04 Thread Armand Turpel
Hi, A few questions about jsonb and hstore: 1. does jsonb replace hstore? 2. compatibility of jsonb <> hstore? 3. will the development of hstore continue? 4. is it recommended to use jsonb when planning new projects? Thanks Armand -- Sent via pgsql-general mailing list (pgsql-general@postgres

Re: [GENERAL] table insert/primary key question

2014-04-04 Thread Albe Laurenz
Scot Kreienkamp wrote: > I have a table like so: > > Receiptlimitid: BIGINT (Primary Key) > Profitcenterid: BIGINT > Receiptnumber: INTEGER > > All are set to Not Null also. > > > My question is, if I have an insert that goes idle in transaction for a while > before it commits, will > it stop

Re: [GENERAL] Linux vs FreeBSD

2014-04-04 Thread Achilleas Mantzios
FreeBSD is OK if you are experienced. As a system it requires much more maturity by the admin than lets say Ubuntu which is targeted at a larger user base. I'd say, explore your other Linux options first, since you already have experience with Linux. FreeBSD requires a much bigger learning curve