Re: [GENERAL] any way for a transaction to see inserts done earlier in the transaction?

2014-04-16 Thread Steven Schlansker
On Apr 16, 2014, at 4:27 PM, Susan Cassidy susan.cass...@decisionsciencescorp.com wrote: Is there any way to let a transaction see the inserts that were done earlier in the transaction? I want to insert a row, then later use it within the same transaction. If not, I will have to commit

Re: [GENERAL] any way for a transaction to see inserts done earlier in the transaction?

2014-04-16 Thread Steven Schlansker
On Wed, Apr 16, 2014 at 4:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Susan Cassidy susan.cass...@decisionsciencescorp.com writes: Is there any way to let a transaction see the inserts that were done earlier in the transaction? It works that way automatically, as long as you're talking

Re: [GENERAL] Is it safe to stop postgres in between pg_start_backup and pg_stop_backup?

2014-04-03 Thread Steven Schlansker
On Apr 2, 2014, at 3:08 PM, Jacob Scott jacob.sc...@gmail.com wrote: Hi, Hello there ;) Does upgrading a a disk being used by postgres (9.1, on Ubuntu) with the following process sound safe? • pg_start_backup • Take a filesystem snapshot (of a volume containing postgres

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-27 Thread Steven Schlansker
On Mar 27, 2014, at 5:29 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Mar 27, 2014 at 1:42 AM, Steven Schlansker ste...@likeness.com wrote: On Mar 25, 2014, at 7:58 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: Yea, vacuum just marks space as available for reuse

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-26 Thread Steven Schlansker
On Mar 25, 2014, at 7:58 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 03/25/2014 04:52 PM, Steven Schlansker wrote: Some more questions, what happens when things begin to dawn on me:) You said the disk filled up entirely with log files yet currently the number(size) of logs

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-26 Thread Steven Schlansker
On Mar 26, 2014, at 9:04 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Mar 25, 2014 at 6:33 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tuesday, March 25, 2014, Steven Schlansker ste...@likeness.com wrote: Hi everyone, I have a Postgres 9.3.3 database machine. Due to some

[GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
Hi everyone, I have a Postgres 9.3.3 database machine. Due to some intelligent work on the part of someone who shall remain nameless, the WAL archive command included a ‘ /dev/null 21’ which masked archive failures until the disk entirely filled with 400GB of pg_xlog entries. I have fixed

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
On Mar 25, 2014, at 3:52 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 03/25/2014 01:56 PM, Steven Schlansker wrote: Hi everyone, I have a Postgres 9.3.3 database machine. Due to some intelligent work on the part of someone who shall remain nameless, the WAL archive command

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
On Mar 25, 2014, at 4:02 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 03/25/2014 03:54 PM, Steven Schlansker wrote: On Mar 25, 2014, at 3:52 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 03/25/2014 01:56 PM, Steven Schlansker wrote: Hi everyone, I have a Postgres

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
On Mar 25, 2014, at 4:45 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 03/25/2014 04:17 PM, Steven Schlansker wrote: On Mar 25, 2014, at 4:02 PM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 03/25/2014 03:54 PM, Steven Schlansker wrote: On Mar 25, 2014, at 3:52 PM, Adrian

Re: [GENERAL] [JDBC] Cannot insert to 'path' field using EclipseLink

2014-03-12 Thread Steven Schlansker
On Mar 12, 2014, at 10:12 AM, Daryl Foster daryl.fos...@oncenter.com wrote: java.lang.ClassCastException: org.postgresql.geometric.PGpath cannot be cast to org.postgresql.geometric.PGpath That's a sure sign of ClassLoader confusion. Make sure there is only one copy of the driver jar in

[GENERAL] Timing for release with fix for Nov2013ReplicationIssue

2013-12-03 Thread Steven Schlansker
Hi everyone, I’ve seen murmuring on the list regarding https://wiki.postgresql.org/wiki/Nov2013ReplicationIssue Is there an ETA on a release with the bug fix for this? I’m putting off building from source because I prefer to use the pgdg RPM packages, but if we don’t get a release soon it

Re: [GENERAL] Timing for release with fix for Nov2013ReplicationIssue

2013-12-03 Thread Steven Schlansker
On Dec 3, 2013, at 2:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Steven Schlansker stevenschlans...@gmail.com writes: I’ve seen murmuring on the list regarding https://wiki.postgresql.org/wiki/Nov2013ReplicationIssue Is there an ETA on a release with the bug fix for this? I’m putting off

Re: [GENERAL] Blowfish Encrypted String

2013-09-26 Thread Steven Schlansker
On Sep 26, 2013, at 10:55 AM, Craig Boyd craigbo...@gmail.com wrote: Dmitriy, Thank you very much for the reply! Right...got the the data type...but how do I actually insert a binary string into the column? What does the SQL look like? For the moment assume I have the following bit of

Re: [GENERAL] JDBC driver for Postgres 9.3

2013-09-26 Thread Steven Schlansker
On Sep 26, 2013, at 6:35 AM, Kohler Manuel manuel.koh...@bsse.ethz.ch wrote: Hi, we are developing a Java based software with Postgres as a DB. Could someone tell me if there will be a JDBC driver for 9.3 out soon or is it safe and recommended to use the latest JDBC driver available?

Re: [GENERAL] Trying to create DB / user to import some data

2013-09-26 Thread Steven Schlansker
On Sep 26, 2013, at 10:28 PM, mdr monosij.for...@gmail.com wrote: create user import_dbms_user with password 'import_dbms'; create database import_dbms_db; grant all privileg However when I try to run psql from the command line: psql -h localhost -U import_dbms_user -WI enter password

Re: [GENERAL] Deduplication and transaction isolation level

2013-09-25 Thread Steven Schlansker
On Sep 25, 2013, at 6:04 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Sep 24, 2013 at 10:19 PM, François Beausoleil franc...@teksol.info wrote: Hi all! I import many, many rows of data into a table, from three or more computers, 4 times per hour. I have a primary key, and the

Re: [GENERAL] ZFS snapshots - to use pg_start_backup() and pg_stop_backup() or no?

2013-09-11 Thread Steven Schlansker
On Sep 11, 2013, at 4:29 PM, Gregory Haase haa...@onefreevoice.com wrote: I was trying to figure out how to get the following syntax to work: echo select pg_start_backup('zfs_snapshot'); \\! zfs snapshot zroot/zpgsql@test; \\ select pg_stop_backup(); | psql postgres I do: psql -c select

Re: [GENERAL] Best Postgresql books

2013-07-18 Thread Steven Schlansker
PostgreSQL 9.0 High Performance: http://www.amazon.com/books/dp/184951030X On Jul 18, 2013, at 3:11 PM, Pedro Costa pedrocostaa...@sapo.pt wrote: Hi guys, Can anyone tell me the best books about postgresql? Specialy about tunning and querys performances Thanks Enviado do meu

Re: [GENERAL] backend hangs at sendto() and can't be terminated

2013-07-11 Thread Steven Schlansker
On Jul 8, 2013, at 6:48 AM, Jov am...@amutu.com wrote: netstat show nothing about the socket of the process,so I think the TCP timeout took effect.so it is really wired. Jov blog: http:amutu.com/blog 2013/7/8 Tom Lane t...@sss.pgh.pa.us Merlin Moncure mmonc...@gmail.com writes: On

Re: [GENERAL] backend hangs at sendto() and can't be terminated

2013-07-11 Thread Steven Schlansker
On Jul 8, 2013, at 6:48 AM, Jov am...@amutu.com wrote: netstat show nothing about the socket of the process,so I think the TCP timeout took effect.so it is really wired. Jov blog: http:amutu.com/blog 2013/7/8 Tom Lane t...@sss.pgh.pa.us Merlin Moncure mmonc...@gmail.com writes: On

Re: [GENERAL] unique constraint violations

2013-06-26 Thread Steven Schlansker
On Jun 26, 2013, at 11:04 AM, pg noob pgn...@gmail.com wrote: Hi all, There are some places in our application where unique constraint violations are difficult to avoid due to multithreading. What we've done in most places to handle this is to retry in a loop. Generally it starts by

[GENERAL] Index over only uncommon values in table

2013-06-18 Thread Steven Schlansker
Hi everyone, I assume this is not easy with standard PG but I wanted to double check. I have a column that has a very uneven distribution of values. ~95% of the values will be the same, with some long tail of another few dozens of values. I want to have an index over this value. Queries that

Re: [GENERAL] Index over only uncommon values in table

2013-06-18 Thread Steven Schlansker
On Jun 18, 2013, at 12:23 PM, John R Pierce pie...@hogranch.com wrote: On 6/18/2013 12:17 PM, Steven Schlansker wrote: 1) The common value is not known at schema definition time, and may change (very slowly) over time. how could a value thats constant in 95% of the rows change, unless you

Re: [GENERAL] Index over only uncommon values in table

2013-06-18 Thread Steven Schlansker
On Jun 18, 2013, at 1:49 PM, David Johnston pol...@yahoo.com wrote: Steven Schlansker-3 wrote At some point, the code changes, and CURRENT_VERSION gets incremented. Rows then slowly (over a period of days / weeks) get upgraded to the new current version, in batches of thousands

Re: [GENERAL] Index over only uncommon values in table

2013-06-18 Thread Steven Schlansker
On Jun 18, 2013, at 2:29 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Jun 18, 2013 at 12:17 PM, Steven Schlansker ste...@likeness.com wrote: Hi everyone, I assume this is not easy with standard PG but I wanted to double check. I have a column that has a very uneven distribution

Re: [GENERAL] upsert functionality

2013-05-16 Thread Steven Schlansker
On May 15, 2013, at 11:52 PM, Thomas Kellerer spam_ea...@gmx.net wrote: Sajeev Mayandi, 16.05.2013 07:01: Hi, Our company is planning to move to postreSQL. We were initially using sybase where upsert functionality was available using insert on existing update clause. I know there

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steven Schlansker
On May 10, 2013, at 7:14 AM, Matt Brock m...@mattbrock.co.uk wrote: Hello. We're intending to deploy PostgreSQL on Linux with SSD drives which would be in a RAID 1 configuration with Hardware RAID. My first question is essentially: are there any issues we need to be aware of when

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steven Schlansker
On May 10, 2013, at 11:38 AM, Merlin Moncure mmonc...@gmail.com wrote: PostgreSQL configuration changes: synchronous_commit = off that's good info, but it should be noted that synchronous_commit trades a risk of some data loss (but not nearly as much risk as volatile storage) for a big

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steven Schlansker
On May 10, 2013, at 11:35 AM, Lonni J Friedman netll...@gmail.com wrote: I am not sure that these numbers will end up being anywhere near what works for you, but these are my notes from tuning a 4xMLC SSD RAID-10. I haven't proven that this is optimal, but it was way better than the

Re: [GENERAL] Simple SQL INSERT to avoid duplication failed: why?

2013-05-01 Thread Steven Schlansker
On May 1, 2013, at 9:36 AM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: I have to ask myself, is it more likely that I have discovered some PG anomaly in 9.0 that no one has ever noticed, or that the client has accidentally launched the process twice and doesn't know it? Given my

Re: [GENERAL] Simple SQL INSERT to avoid duplication failed: why?

2013-04-30 Thread Steven Schlansker
On Apr 30, 2013, at 4:00 PM, Carlo Stonebanks stonec.regis...@sympatico.ca wrote: Hi Tom, There's nothing obviously wrong with that, which means the issue is in something you didn't show us. Care to assemble a self-contained example? Unfortunately, it happens erratically and very,

[GENERAL] Using an index for IS DISTINCT FROM queries

2013-04-22 Thread Steven Schlansker
Hi everyone, I have a large table (~150M rows) that keeps a version field. At any given time, it is expected that the vast majority of the rows are on the current version, but some may be missing. To figure out all the missing our outdated values, I run a query along the lines of SELECT id

Re: [GENERAL] pg_restore from split files

2013-04-22 Thread Steven Schlansker
On Apr 22, 2013, at 12:47 PM, akp geek akpg...@gmail.com wrote: pg_dump dbname -n schemaname -t table_name -Fc | split -b 500m -t table.dump Since you split the files outside of the Postgres world, you have to combine them again. Roughly, cat table.dump.* table.dump.combined pg_restore

Re: [GENERAL] Queries seldomly take 4s while normally take 1ms?

2013-04-09 Thread Steven Schlansker
On Apr 9, 2013, at 11:25 AM, Scott Marlowe scott.marl...@gmail.com wrote: One of the most common causes I've seen for this is linux's vm.*dirty* settings to get in the way. Like so many linux kernel optimizations this one looks good on paper but gives at best middling improvements with

Re: [GENERAL] Testing Technique when using a DB

2013-03-13 Thread Steven Schlansker
On Mar 12, 2013, at 8:09 PM, Joe Van Dyk j...@tanga.com wrote: On Mar 12, 2013, at 8:42 AM, Perry Smith pedz...@gmail.com wrote: The other thought is perhaps there is a snap shot type concept. I don't see it in the list of SQL commands. A snap shot would do exactly what it sounds

Re: [GENERAL] best config

2013-02-06 Thread Steven Schlansker
On Feb 6, 2013, at 8:14 AM, Roberto Scattini roberto.scatt...@gmail.com wrote: hi list, we have two new dell poweredge r720. based on recommendations from this list we have configued the five disks in raid10 + 1 hot spare. You might mention a bit more about how your drives are configured.

Re: [GENERAL] best config

2013-02-06 Thread Steven Schlansker
On Feb 6, 2013, at 9:55 AM, Roberto Scattini roberto.scatt...@gmail.com wrote: hi steven, we have two new dell poweredge r720. based on recommendations from this list we have configued the five disks in raid10 + 1 hot spare. You might mention a bit more about how your drives are

[GENERAL] Using partial index in combination with prepared statement parameters

2013-02-03 Thread Steven Schlansker
we use prepared statements everywhere and this makes it very hard to use partial indices, which would offer us significant performance gains. Does anyone know of any acceptable workaround? Is there continued interest in maybe improving the PostgreSQL behavior in this case? Thanks! Steven

Re: [GENERAL] Combine Date and Time Columns to Timestamp

2013-01-18 Thread Steven Schlansker
On Jan 18, 2013, at 4:26 PM, Rich Shepard rshep...@appl-ecosys.com wrote: On Fri, 18 Jan 2013, Adrian Klaver wrote: How are they stored, as date and time type, strings, other? Adrian, ISO date and time. A sample of the data would help also. Example: 2012-10-29 | 10:19 |

Re: [GENERAL] Noticed something odd with pgbench

2012-11-16 Thread Steven Schlansker
On Nov 16, 2012, at 11:59 AM, Richard Huxton d...@archonet.com wrote: On 16/11/12 19:35, Shaun Thomas wrote: Hey guys, So, we have a pretty beefy system that runs dual X5675's with 72GB of RAM. After our recent upgrade to 9.1, things have been... odd. I managed to track it down to one

Re: [GENERAL] Confirming \timing output

2012-08-23 Thread Steven Schlansker
On Aug 23, 2012, at 11:13 AM, Gauthier, Dave dave.gauth...@intel.com wrote: With \timing set on, I run an update statement and it reports Time: 0.524 ms Is that really 0.524 ms? As in 524 nanoseconds? 0.524ms = 524000ns Perhaps you meant microseconds? 0.524ms = 524us If all

Re: [GENERAL] Ignore hash indices on replicas

2012-08-20 Thread Steven Schlansker
On Aug 19, 2012, at 8:01 PM, Scott Marlowe scott.marl...@gmail.com wrote: On Tue, Jul 10, 2012 at 1:09 AM, Steven Schlansker ste...@likeness.com wrote: I'm using Postgres hash indices on a streaming replica master. As is documented, hash indices are not logged, so the replica does not have

Re: [GENERAL] Ignore hash indices on replicas

2012-08-20 Thread Steven Schlansker
On Aug 19, 2012, at 2:37 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2012-07-10 at 00:09 -0700, Steven Schlansker wrote: I understand that the current wisdom is don't use hash indices, but (unfortunately?) I have benchmarks that show that our particular application is faster by quite

[GENERAL] Postgres will not compile on Mac 10.8 with contrib/uuid-ossp

2012-07-30 Thread Steven Schlansker
It is not possible to compile Postgres contrib/uuid-ossp on the newest release of Mac OS X, 10.8 The specific compile error: make -C uuid-ossp install /bin/sh ../../config/install-sh -c -d '/usr/local/Cellar/postgresql/9.1.3/lib' /usr/bin/clang -Os -w -pipe -march=native -Qunused-arguments

Re: [GENERAL] Postgres will not compile on Mac 10.8 with contrib/uuid-ossp

2012-07-30 Thread Steven Schlansker
On Jul 30, 2012, at 7:35 PM, Tom Lane t...@sss.pgh.pa.us wrote: Steven Schlansker ste...@likeness.com writes: It is not possible to compile Postgres contrib/uuid-ossp on the newest release of Mac OS X, 10.8 This looks like some variant of the same issue that OSSP's uuid package has had

Re: [GENERAL] Replication/cloning: rsync vs modification dates?

2012-07-16 Thread Steven Schlansker
I think it's pretty easy to show that timestamp+size isn't good enough to do this 100% reliably. Imagine that your timestamps have a millisecond resolution. I assume this will vary based on OS / filesystem, but the point remains the same no matter what size it is. You can have multiple

[GENERAL] Ignore hash indices on replicas

2012-07-10 Thread Steven Schlansker
I'm using Postgres hash indices on a streaming replica master. As is documented, hash indices are not logged, so the replica does not have access to them. I understand that the current wisdom is don't use hash indices, but (unfortunately?) I have benchmarks that show that our particular

Re: [GENERAL] Suboptimal query plan fixed by replacing OR with UNION

2012-07-09 Thread Steven Schlansker
On Jul 6, 2012, at 9:24 PM, Gurjeet Singh wrote: On Thu, Jul 5, 2012 at 7:16 PM, Steven Schlansker ste...@likeness.com wrote: On Jul 5, 2012, at 3:51 PM, Tom Lane wrote: Steven Schlansker ste...@likeness.com writes: Why is using an OR so awful here? Because the OR stops it from

Re: [GENERAL] Suboptimal query plan fixed by replacing OR with UNION

2012-07-06 Thread Steven Schlansker
On Jul 5, 2012, at 6:35 PM, Jasen Betts wrote: I note you've decided to rewrite this query as a union SELECT * FROM account WHERE user_id in (SELECT user_id FROM account WHERE id =

[GENERAL] Suboptimal query plan fixed by replacing OR with UNION

2012-07-05 Thread Steven Schlansker
does it pick a sequential scan? Is this an optimizer bug or have I missed something in my queries? Thanks much for any advice, Steven Schlansker -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [GENERAL] Suboptimal query plan fixed by replacing OR with UNION

2012-07-05 Thread Steven Schlansker
On Jul 5, 2012, at 3:51 PM, Tom Lane wrote: Steven Schlansker ste...@likeness.com writes: Why is using an OR so awful here? Because the OR stops it from being a join (it possibly needs to return some rows that are not in the semijoin of the two tables). Why does it pick a sequential