Re: [GENERAL] window function ordering not working as expected

2015-02-17 Thread Lonni J Friedman
On Tue, Feb 17, 2015 at 4:18 PM, Tom Lane wrote: > Lonni J Friedman writes: >> I'm interested in seeing: >> * the date for the most recent result >> * test name (identifier) >> * most recent result (decimal value) >> * the worst (lowest decimal value) test

[GENERAL] window function ordering not working as expected

2015-02-17 Thread Lonni J Friedman
Greetings, I have a postgresql-9.3.x database with a table with a variety of date stamped test results, some of which are stored in json format (natively in the database). I'm attempting to use some window functions to pull out specific data from the test results over a a time window, but part of t

[GENERAL] pg_basebackup: ERROR: could not find any WAL files (9.3)

2013-09-26 Thread Lonni J Friedman
Greetings, I've recently pushed a new postgres-9.3 (Linux-x86_64/RHEL6) cluster into production, with one master, and two hot standby streaming replication slaves. Everything seems to be working ok, however roughly half of my pg_basebackup attempts are failing at the very end with the error: pg_b

Re: [GENERAL] partitioned table + postgres_FDW not working in 9.3

2013-09-26 Thread Lonni J Friedman
On Thu, Sep 26, 2013 at 8:52 AM, Tom Lane wrote: > Lonni J Friedman writes: >> Thanks for your reply. This sounds like a relatively simple >> workaround, so I'll give it a try. Is the search_path of the remote >> session that postgres_fdw forces considered to b

Re: [GENERAL] partitioned table + postgres_FDW not working in 9.3

2013-09-26 Thread Lonni J Friedman
anada wrote: > Hi Lonni, > > 2013/9/25 Lonni J Friedman : >> The problem that I'm experiencing is if I attempt to perform an INSERT >> on the foreign nppsmoke table on cluster a, it fails claiming that the >> table partition which should hold the data in the

Re: [GENERAL] postgres FDW doesn't support sequences?

2013-09-25 Thread Lonni J Friedman
On Wed, Sep 25, 2013 at 2:47 PM, Tom Lane wrote: > Lonni J Friedman writes: >> If I INSERT a new row into the local table (not the foreign table >> version), without specifying the 'id' column explicitly, it >> automatically is assigned the nextval in the seq

[GENERAL] postgres FDW doesn't support sequences?

2013-09-25 Thread Lonni J Friedman
I've got two 9.3 clusters, with a postgres foreign data wrapper (FDW) setup to point from one cluster to the other. One of the (foreign) tables associated with the foreign server has a bigint sequence for its primary key, defined as: id | bigint | not null default

[GENERAL] partitioned table + postgres_FDW not working in 9.3

2013-09-24 Thread Lonni J Friedman
Greetings, I've got two different 9.3 clusters setup, a & b (on Linux if that matters). On cluster b, I have a table (nppsmoke) that is partitioned by date (month), which uses a function which is called by a trigger to manage INSERTS (exactly as documented in the official documentation for partiti

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-18 Thread Lonni J Friedman
On Wed, Sep 18, 2013 at 2:02 AM, Kevin Grittner wrote: > Lonni J Friedman wrote: > >> top shows over 90% of the load is in sys space. vmstat output >> seems to suggest that its CPU bound (or bouncing back & forth): > > Can you run `perf top` during an episode and see

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-18 Thread Lonni J Friedman
On Wed, Sep 18, 2013 at 2:02 AM, Kevin Grittner wrote: > Lonni J Friedman wrote: > >> top shows over 90% of the load is in sys space. vmstat output >> seems to suggest that its CPU bound (or bouncing back & forth): > > Can you run `perf top` during an episode and see

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
On Tue, Sep 17, 2013 at 3:47 PM, Andres Freund wrote: > Hi, > > On 2013-09-17 09:19:29 -0700, Lonni J Friedman wrote: >> I'm running a PostgreSQL 9.3.0 cluster (1 master with two streaming >> replication hot standby slaves) on RHEL6-x86_64. Yesterday I upgraded >>

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
Thanks for your reply. Comments/answers inline below On Tue, Sep 17, 2013 at 11:28 AM, Jeff Janes wrote: > On Tue, Sep 17, 2013 at 11:22 AM, Lonni J Friedman > wrote: >> >> >> > c) What does logs say? >> >> The postgres server logs look perfectly no

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
On Tue, Sep 17, 2013 at 9:54 AM, Eduardo Morras wrote: > On Tue, 17 Sep 2013 09:19:29 -0700 > Lonni J Friedman wrote: > >> Greetings, >> I'm running a PostgreSQL 9.3.0 cluster (1 master with two streaming >> replication hot standby slaves) on RHEL6-x86_64. Yeste

[GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
Greetings, I'm running a PostgreSQL 9.3.0 cluster (1 master with two streaming replication hot standby slaves) on RHEL6-x86_64. Yesterday I upgraded from 9.2.4 to 9.3.0, and since the upgrade I'm seeing a significant performance degradation. PostgreSQL simply feels slower. Nothing other than the

Re: [GENERAL] WAL Replication Working but Not Working

2013-08-21 Thread Lonni J Friedman
The first thing to do is look at your server logs around the time when it stopped working. On Wed, Aug 21, 2013 at 7:08 AM, Joseph Marlin wrote: > We're having an issue with our warm standby server. About 9:30 last night, it > stopped applying changes it received in WAL files that are shipped ov

Re: [GENERAL] Streaming Replication Randomly Locking Up

2013-08-15 Thread Lonni J Friedman
gt; #log_min_messages = warning > #log_min_error_statement = error > #log_min_duration_statement = -1 > #log_checkpoints = off > #log_connections = off > #log_disconnections = off > #log_error_verbosity = default > > I'm going to have a look at the NICs to make s

Re: [GENERAL] Streaming Replication Randomly Locking Up

2013-08-15 Thread Lonni J Friedman
syncs right > back up and all if working again so if it is a network issue, the > replication is just stopping after some hiccup instead of retrying and > resuming when things are back up. > > Thanks! > > > > On Thu, Aug 15, 2013 at 11:32 AM, Lonni J Friedman > wrote:

Re: [GENERAL] Streaming Replication Randomly Locking Up

2013-08-15 Thread Lonni J Friedman
I've never seen this happen. Looks like you might be using 9.1? Are you up to date on all the 9.1.x releases? Do you have just 1 slave syncing from the master? Which OS are you using? Did you verify that there aren't any network problems between the slave & master? Or hardware problems (like the

Re: [GENERAL] postgres FDW cost estimation options unrecognized in 9.3-beta1

2013-07-26 Thread Lonni J Friedman
On Fri, Jul 26, 2013 at 3:28 PM, Tom Lane wrote: > Lonni J Friedman writes: >> nightly=# ALTER SERVER cuda_db10 OPTIONS (SET use_remote_estimate 'true') ; >> ERROR: option "use_remote_estimate" not found > >> Am I doing something wrong, or is this a

[GENERAL] postgres FDW cost estimation options unrecognized in 9.3-beta1

2013-07-26 Thread Lonni J Friedman
Greetings, I have a postgresql-9.3-beta1 cluster setup (from the yum.postgresql.org RPMs), where I'm experimenting with the postgres FDW extension. The documentation ( http://www.postgresql.org/docs/9.3/static/postgres-fdw.html ) references three Cost Estimation Options which can be set for a fore

Re: [GENERAL] upgrading from 9.3-beta1 to 9.3-beta2 requires dump & reload?

2013-07-24 Thread Lonni J Friedman
On Wed, Jul 24, 2013 at 2:05 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Lonni J Friedman escribió: >>> I'm using the RPMs from yum.postgresql.org on RHEL6. Is this >>> expected, intentional behavior? Do I really need to dump & reload to >>>

[GENERAL] upgrading from 9.3-beta1 to 9.3-beta2 requires dump & reload?

2013-07-24 Thread Lonni J Friedman
Greetings, I just got around to upgrading from 9.3-beta1 to 9.3-beta2, and was surprised to see that the server was refusing to start. In the log, I'm seeing: 2013-07-24 13:41:47 PDT [7083]: [1-1] db=,user= FATAL: database files are incompatible with server 2013-07-24 13:41:47 PDT [7083]: [2-1] d

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Lonni J Friedman
, caught up and has been working for about 2 hours. > > The file in the error message was an index. We rebuilt it just in case. > Is there any way to debug the issue at this point? > > > > -----Original Message- > From: Lonni J Friedman [mailto:netll...@gmail.com] > S

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Lonni J Friedman
Looks like some kind of data corruption. Question is whether it came from the master, or was created by the standby. If you re-seed the standby with a full (base) backup, does the problem go away? On Sat, Jun 22, 2013 at 12:43 PM, Dan Kogan wrote: > Hello, > > > > Today our standby instance sto

Re: [GENERAL] 9.3-beta postgres-fdw COPY error

2013-06-21 Thread Lonni J Friedman
I was afraid someone would say that. Is this a limitation that might be removed in the future (like 9.4), or is there a technical reason why its not possible to do a COPY against a foreign table? On Fri, Jun 21, 2013 at 10:52 AM, Adrian Klaver wrote: > On 06/21/2013 10:39 AM, Lonni J Fried

[GENERAL] 9.3-beta postgres-fdw COPY error

2013-06-21 Thread Lonni J Friedman
Greetings, I'm trying to test out the new postgres-fdw support in postgresql-9.3 (beta) in preparation for an upgrade from 9.2 later this year. So far, everything is working ok, however one problem I'm encountering is with the COPY command. When I run it against a foreign table (which is also in a

[GENERAL] how to reference variables in pgbench custom scripts?

2013-06-18 Thread Lonni J Friedman
I'm attempting to write a custom pgbench script (called via the -f option), with a variable set at the top with: \setrandom aid 100 50875000 However, I can't quite figure out how to reference the new aid variable. The documentation simply states that a variable is referenced with a colon in front

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Lonni J Friedman
On Thu, May 16, 2013 at 11:03 AM, Igor Neyman wrote: > > From: pgsql-general-ow...@postgresql.org > [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of AI Rumman > Sent: Thursday, May 16, 2013 1:56 PM > To: Fabio Rueda Carrascosa > Cc: pgsql-general > Subject: Re: [GENERAL] pg_upgrade link m

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Lonni J Friedman
On Fri, May 10, 2013 at 11:23 AM, Steven Schlansker wrote: > > On May 10, 2013, at 7:14 AM, Matt Brock wrote: > >> Hello. >> >> We're intending to deploy PostgreSQL on Linux with SSD drives which would be >> in a RAID 1 configuration with Hardware RAID. >> >> My first question is essentially: ar

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Lonni J Friedman
On Fri, May 10, 2013 at 10:20 AM, Merlin Moncure wrote: > On Fri, May 10, 2013 at 12:03 PM, David Boreham > wrote: >> On 5/10/2013 10:21 AM, Merlin Moncure wrote: >>> >>> As it turns out the list of flash drives are suitable for database use is >>> surprisingly small. The s3700 I noted upthread

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Lonni J Friedman
Sergey Koposov wrote: > > On Fri, 10 May 2013, Lonni J Friedman wrote: > >> Its definitely not a bug. You need to set/increase wal_keep_segments >> to a value that ensures that they aren't recycled faster than the time >> required to complete the base back

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Lonni J Friedman
Its definitely not a bug. You need to set/increase wal_keep_segments to a value that ensures that they aren't recycled faster than the time required to complete the base backup (plus some buffer). On Fri, May 10, 2013 at 9:48 AM, Sergey Koposov wrote: > Hi, > > I've recently started to use pg_ba

Re: [GENERAL] Replication terminated due to PANIC

2013-04-25 Thread Lonni J Friedman
If its really index corruption, then you should be able to fix it by reindexing. However, that doesn't explain what caused the corruption. Perhaps your hardware is bad in some way? On Wed, Apr 24, 2013 at 10:46 PM, Adarsh Sharma wrote: > Thanks Sergey for such a quick response, but i dont think

Re: [GENERAL] corrupted item pointer in streaming based replication

2013-04-03 Thread Lonni J Friedman
You should figure out what base/16384/114846.39 corresponds to inside the database. If you're super lucky its something unimportant and/or something that can be recreated easily (like an index). If its something important, then you're only option is to try to drop the object and restore it from t

Re: [GENERAL] Streaming replication slave crash

2013-03-29 Thread Lonni J Friedman
Looks like you've got some form of coruption: page 1441792 of relation base/63229/63370 does not exist The question is whether it was corrupted on the master and then replicated to the slave, or if it was corrupted on the slave. I'd guess that the pg_dump tried to read from that page and barfed.

Re: [GENERAL] UNLOGGED TEMPORARY tables?

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 4:49 PM, Michael Paquier wrote: > > > On Tue, Mar 26, 2013 at 8:26 AM, Lonni J Friedman > wrote: >> >> I'm pretty sure that unlogged tables and temp tables are two separate >> & distinct features, with no overlap in functionality.

Re: [GENERAL] UNLOGGED TEMPORARY tables?

2013-03-25 Thread Lonni J Friedman
I'm pretty sure that unlogged tables and temp tables are two separate & distinct features, with no overlap in functionality. It would be nice if it was possible to create an unlogged temp table. On Sun, Mar 24, 2013 at 1:32 PM, aasat wrote: > I was tested write speed to temporary and unlogged ta

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 1:23 PM, AI Rumman wrote: > > > On Mon, Mar 25, 2013 at 4:03 PM, AI Rumman wrote: >> >> >> >> On Mon, Mar 25, 2013 at 4:00 PM, Lonni J Friedman >> wrote: >>> >>> On Mon, Mar 25, 2013 at 12:55 PM, AI Rumman wrote:

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 12:55 PM, AI Rumman wrote: > > > On Mon, Mar 25, 2013 at 3:52 PM, Lonni J Friedman > wrote: >> >> On Mon, Mar 25, 2013 at 12:43 PM, AI Rumman wrote: >> > >> > >> > On Mon, Mar 25, 2013 at 3:40 PM, Lonni J Friedman >&

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 12:43 PM, AI Rumman wrote: > > > On Mon, Mar 25, 2013 at 3:40 PM, Lonni J Friedman > wrote: >> >> On Mon, Mar 25, 2013 at 12:37 PM, AI Rumman wrote: >> > Hi, >> > >> > I have two 9.2 databases running with hot_standby

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 12:37 PM, AI Rumman wrote: > Hi, > > I have two 9.2 databases running with hot_standby replication. Today when I > was checking, I found that replication has not been working since Mar 1st. > There was a large database restored in master on that day and I believe > after th

Re: [GENERAL] postgres 9.0.2 replicated database is crashing

2013-03-09 Thread Lonni J Friedman
ing for the most straightforward path I'd recommend going to 9.0.12. Also be sure to read the release notes first. > > We use GIST indexes quite a bit. and we gis also > > I recently compiled postgres 9.2 .. > > Regards > > > > On Sat, Mar 9, 2013 at 5:09 PM, Lonn

Re: [GENERAL] postgres 9.0.2 replicated database is crashing

2013-03-09 Thread Lonni J Friedman
On Sat, Mar 9, 2013 at 1:51 PM, akp geek wrote: > thank you. As you mentioned, I understood that I am starting the streaming > scratch which is not what I wanted to do. > > Here is what I am planning to . > > Our replication process was down since March5th. > > 1. Is it Ok to get all wals from Mar

Re: [GENERAL] postgres 9.0.2 replicated database is crashing

2013-03-09 Thread Lonni J Friedman
That process merely sets up a new server, it doesn't start streaming, unless the server has been configured correctly. You state that the slave crashed after two hours. How did you make this determination? All you seem to be doing is setting up the slave from scratch repeatedly, and assuming tha

Re: [GENERAL] Replication stopped on 9.0.2 after making change to conf file

2013-03-09 Thread Lonni J Friedman
It sounds like all you did was setup the slave from scratch with a fresh base backup, without understanding or debugging what caused everything to break. Clearly whatever was wrong on March 5 is still wrong, and nothing has been fixed. The first step in debugging this problem is to look at and/or

Re: [GENERAL] broke postgres, how to fix??

2013-02-26 Thread Lonni J Friedman
What is "read only style", and how does postgres know about this? http://www.postgresql.org/docs/9.2/static/backup-file.html > > Thanks, > -JD > > On Tue, Feb 26, 2013 at 7:04 PM, Lonni J Friedman > wrote: >> >> On Tue, Feb 26, 2013 at 4:02 PM, JD Wong wrot

Re: [GENERAL] broke postgres, how to fix??

2013-02-26 Thread Lonni J Friedman
On Tue, Feb 26, 2013 at 4:02 PM, JD Wong wrote: > Hi Adrian, yes I completely copied the config-file and data directories > over. > > Lonnie, I don't remember. I might not have shut down the "old" postgres, > yes I set PGDATA accordingly. That's guaranteed to break everything badly. -- Sent v

Re: [GENERAL] broke postgres, how to fix??

2013-02-26 Thread Lonni J Friedman
Did you shut down the 'old' postgres before copying these files? Did you (re)configure the 'new' postgres to set its $PGDATA directory to the location of the 'new' files? On Fri, Feb 22, 2013 at 3:46 PM, JD Wong wrote: > I tried copying postgres over to a new directory. it was working until I >

[GENERAL] special procedure required when running pg_basebackup from a standby?

2013-01-13 Thread Lonni J Friedman
Greetings, I'm running postgres-9.2.2 in a Linux-x86_64 cluster with 1 master and several hot standby servers. Since upgrading to 9.2.2 from 9.1.x a few months ago, I switched from generating a base backup on the master, to generating it on a dedicated slave/standby (to reduce the load on the mast

[GENERAL] data corruption when using base backups generated from hot standby

2013-01-10 Thread Lonni J Friedman
Greetings, I'm running postgres-9.2.2 in a Linux-x86_64 cluster with 1 master and several hot standby servers. Since upgrading to 9.2.2 from 9.1.x a few months ago, I switched from generating a base backup on the master, to generating it on a dedicated slave/standby (to reduce the load on the mast

Re: [GENERAL] pgpool2 load balancing not working

2013-01-04 Thread Lonni J Friedman
On Fri, Jan 4, 2013 at 3:42 PM, Greg Donald wrote: > Sorry if this is the wrong list, but I've been stuck for a couple days > now. I tried pgpool-general but that list appears to not like me. > I'm not getting any posts and my post hasn't shown up in the archives. Specifically which address are

Re: [GENERAL] Table with million rows - and PostgreSQL 9.1 is not using the index

2012-12-04 Thread Lonni J Friedman
I'm no expert on this, but it will likely be more helpful to others if you include the table description with all the indices. On Tue, Dec 4, 2012 at 8:44 PM, Edson Richter wrote: > I've a table with >110 rows, with streets. > I'm making a partial search using zip code, and PostgreSQL is igno

Re: [GENERAL] Postgresql logfilename and times in GMT - not EST

2012-12-04 Thread Lonni J Friedman
On Tue, Dec 4, 2012 at 2:42 PM, Tom Lane wrote: > Lonni J Friedman writes: >> On Tue, Dec 4, 2012 at 1:59 PM, Bryan Montgomery wrote: >>> I changed postgres.conf to have timezone = 'EST' and restarted postgres. >>> However the log file is still 5 hours

Re: [GENERAL] Postgresql logfilename and times in GMT - not EST

2012-12-04 Thread Lonni J Friedman
On Tue, Dec 4, 2012 at 1:59 PM, Bryan Montgomery wrote: > We have a test 9.2.0 db running on openSuse 12.2. When I select now() I get > the correct timezone and date back (-5 hours). > When I do date at the os prompt, I get the right timezone back. > > I changed postgres.conf to have timezone = 'E

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 8:31 PM, Ian Harding wrote: > > > > On Mon, Nov 5, 2012 at 8:15 PM, Lonni J Friedman wrote: >> >> On Mon, Nov 5, 2012 at 8:13 PM, Ian Harding wrote: >> > >> > >> > >> > On Mon, Nov 5, 2012 at 7:57 PM, Lonni J F

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 8:13 PM, Ian Harding wrote: > > > > On Mon, Nov 5, 2012 at 7:57 PM, Lonni J Friedman wrote: >> >> On Mon, Nov 5, 2012 at 7:49 PM, Ian Harding wrote: >> > >> > >> > On Mon, Nov 5, 2012 at 7:46 PM, Lonni J Friedman >

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 7:49 PM, Ian Harding wrote: > > > On Mon, Nov 5, 2012 at 7:46 PM, Lonni J Friedman wrote: >> >> On Mon, Nov 5, 2012 at 7:40 PM, Ian Harding wrote: >> > I had a 9.0.8 hot standby setup, one master, two slaves, working great. >> > The

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 7:40 PM, Ian Harding wrote: > I had a 9.0.8 hot standby setup, one master, two slaves, working great. > Then, I tried to re-initialize by making a base backup, the way I've done it > many times before, but for some reason I can't get the standby to accept > connections. I c

Re: [GENERAL] Quick estimate of num of rows & table size

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 3:56 PM, Thalis Kalfigkopoulos wrote: > > On Mon, Nov 5, 2012 at 7:14 PM, Lonni J Friedman wrote: >> >> On Mon, Nov 5, 2012 at 2:02 PM, Thalis Kalfigkopoulos >> wrote: >> > Hi all, >> > >> > I read somewhere that the foll

Re: [GENERAL] Quick estimate of num of rows & table size

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 2:02 PM, Thalis Kalfigkopoulos wrote: > Hi all, > > I read somewhere that the following query gives a quick estimate of the # of > rows in a table regardless of the table's size (which would matter in a > simple SELECT count(*)?): > > SELECT (CASE WHEN reltuples > 0 THEN > p

Re: [GENERAL] dropdb breaks replication?

2012-10-31 Thread Lonni J Friedman
On Wed, Oct 31, 2012 at 11:01 AM, Edson Richter wrote: > Em 31/10/2012 15:39, Lonni J Friedman escreveu: >> >> On Wed, Oct 31, 2012 at 10:32 AM, Edson Richter >> wrote: >>> >>> I've two PostgreSQL 9.1.6 running on Linux CentOS 5.8 64bit. >>>

Re: [GENERAL] dropdb breaks replication?

2012-10-31 Thread Lonni J Friedman
On Wed, Oct 31, 2012 at 10:32 AM, Edson Richter wrote: > I've two PostgreSQL 9.1.6 running on Linux CentOS 5.8 64bit. > They are replicated asynchronously. > > Yesterday, I've dropped a database of 20Gb, and then replication has broken, > requiring me to manually synchronize both servers again. >

Re: [GENERAL] 9.1 to 9.2 requires a dump/reload?

2012-10-22 Thread Lonni J Friedman
pg_upgrade has worked fine for several releases. I believe that the only time when pg_upgrade isn't a viable option is for some types of GIST indices. On Mon, Oct 22, 2012 at 2:55 PM, Nikolas Everett wrote: > I was just looking at > http://www.postgresql.org/docs/devel/static/release-9-2.html an

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Lonni J Friedman
On Fri, Oct 12, 2012 at 7:44 AM, Chitra Creta wrote: > Hi, > > I currently have a table that is growing very quickly - i.e 7 million > records in 5 days. This table acts as a placeholder for statistics, and > hence the records are merely inserted and never updated or deleted. > > Many queries are

Re: [GENERAL] strange hot_standby behaviour

2012-10-01 Thread Lonni J Friedman
On Mon, Oct 1, 2012 at 7:28 AM, pfote wrote: > Hi, > > I had a very strange effect on the weekend that smells like a bug, so i'd > like so share it. > > Setup: > machine A: 16 CPU Cores (modern), 128GB RAM, nice 6-drive SAS Raid-10 > machines B, C: 8 Cores (substantially older than A), 48GB Ram, s

Re: [GENERAL] initial sync of multiple streaming slaves simultaneously

2012-09-19 Thread Lonni J Friedman
Just curious, is there a reason why you can't use pg_basebackup ? On Wed, Sep 19, 2012 at 12:27 PM, Mike Roest wrote: > >> Is there any hidden issue with this that we haven't seen. Or does anyone >> have suggestions as to an alternate procedure that will allow 2 slaves to >> sync concurrently. >

Re: [GENERAL] initial sync of multiple streaming slaves simultaneously

2012-09-19 Thread Lonni J Friedman
On Wed, Sep 19, 2012 at 8:59 AM, Mike Roest wrote: > Hey Everyone, > We currently have a 9.1.5 postgres cluster running using streaming > replication. We have 3 nodes right now > > 2 - local that are setup with pacemaker for a HA master/slave set failover > cluster > 1 - remote as a DR. > > C

Re: [GENERAL] File system level backup

2012-07-26 Thread Lonni J Friedman
On Thu, Jul 26, 2012 at 3:39 AM, Manoj Agarwal wrote: > Hi, > > > > I have two virtual machines with two different versions of Postgresql. One > machine contains Postgres 7.4.19 and another has Postgres 8.4.3. I also > have other instances of these two virtual machines. I need to transfer the >

Re: [GENERAL] insert binary data into a table column with psql

2012-07-24 Thread Lonni J Friedman
On Tue, Jul 24, 2012 at 7:16 PM, Tom Lane wrote: > jtkells writes: >> Thanks much for your reply, that does the trick quite nicely. But, I just >> came to the realization that this only works if your are running the >> client and the file both resides on the database server. I thought that >> I

Re: [GENERAL] insert binary data into a table column with psql

2012-07-24 Thread Lonni J Friedman
On Tue, Jul 24, 2012 at 2:22 PM, John R Pierce wrote: > On 07/24/12 1:28 PM, jkells wrote: >> >> from psql >> I have tried several ways including creating a function to read a file >> without any success but basically I want to do something like the >> following from a bash shell >> >> psql -c "i

Re: [GENERAL] big database resulting in small dump

2012-07-20 Thread Lonni J Friedman
On Fri, Jul 20, 2012 at 11:23 AM, Tom Lane wrote: > Lonni J Friedman writes: >> On Fri, Jul 20, 2012 at 11:05 AM, Ilya Ivanov wrote: >>> I have a 8.4 database (installed on ubuntu 10.04 x86_64). It holds Zabbix >>> database. The database on disk takes 10Gb. SQL dump t

Re: [GENERAL] big database resulting in small dump

2012-07-20 Thread Lonni J Friedman
On Fri, Jul 20, 2012 at 11:05 AM, Ilya Ivanov wrote: > I have a 8.4 database (installed on ubuntu 10.04 x86_64). It holds Zabbix > database. The database on disk takes 10Gb. SQL dump takes only 2Gb. I've > gone through > http://archives.postgresql.org/pgsql-general/2008-08/msg00316.php and got > s

Re: [GENERAL] efficiency of wildcards at both ends

2012-06-20 Thread Lonni J Friedman
On Wed, Jun 20, 2012 at 10:10 AM, Sam Z J wrote: > Hi all > > I'm curious how is wildcards at both ends implemented, e.g. LIKE '%str%' > How efficient is it if that's the only search criteria against a large > table? how much does indexing the column help and roughly how much more > space is neede

Re: [GENERAL] pg_basebackup blocking all queries

2012-06-06 Thread Lonni J Friedman
ce on any query (read or write) being horrible (seconds to minutes). As soon as the basebackup completes, perf returns to normal (and the load drops back down to 1.00 or less). How can I debug what's wrong? On Tue, May 22, 2012 at 3:20 PM, Lonni J Friedman wrote: > Thanks for your reply.  

Re: [GENERAL] autovacuum running for a long time on a new table with 1 row

2012-06-01 Thread Lonni J Friedman
On Fri, Jun 1, 2012 at 10:54 AM, Tom Lane wrote: > Lonni J Friedman writes: >> On Fri, Jun 1, 2012 at 10:34 AM, Tom Lane wrote: >>> This seems to have been noticed and fixed in HEAD: >>> http://git.postgresql.org/gitweb/?p=postgresql.git&a=commitdiff&h=b4e07

Re: [GENERAL] autovacuum running for a long time on a new table with 1 row

2012-06-01 Thread Lonni J Friedman
On Fri, Jun 1, 2012 at 10:34 AM, Tom Lane wrote: > Lonni J Friedman writes: >> Running 9.1.3 on Linux-x86_64.  I'm seeing autovacuum running for the >> past 6 hours on a newly created table that only has 1 row of data in >> it.  This table did exist previously, b

[GENERAL] autovacuum running for a long time on a new table with 1 row

2012-05-31 Thread Lonni J Friedman
Running 9.1.3 on Linux-x86_64. I'm seeing autovacuum running for the past 6 hours on a newly created table that only has 1 row of data in it. This table did exist previously, but was dropped & recreated. I'm not sure if that might explain this behavior. When I strace the autovacuum process, I se

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-25 Thread Lonni J Friedman
On Thu, May 24, 2012 at 12:57 PM, Tom Lane wrote: > Lonni J Friedman writes: >> On Thu, May 24, 2012 at 12:34 PM, Tom Lane wrote: >>> Can you correlate the performance hit with any specific part of >>> autovacuum? In particular, I'm wondering if it matters wheth

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-24 Thread Lonni J Friedman
On Thu, May 24, 2012 at 12:34 PM, Tom Lane wrote: > Lonni J Friedman writes: >> No, not lots of subqueries or ORDERing, and most queries only touch a >> single table.  However, I'm honestly not sure that I'm following where >> you're going with this.   The

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-24 Thread Lonni J Friedman
On Wed, May 23, 2012 at 2:45 PM, Gavin Flower wrote: > On 24/05/12 08:18, Lonni J Friedman wrote: > > On Wed, May 23, 2012 at 12:36 PM, Gavin Flower > wrote: > > On 24/05/12 05:09, Lonni J Friedman wrote: > > On Wed, May 23, 2012 at 9:37 AM, Tom Lane wrote: >

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-23 Thread Lonni J Friedman
On Wed, May 23, 2012 at 3:33 PM, Tom Lane wrote: > Gavin Flower writes: >>> 16 core Xeon X5550 2.67GHz >>> 128GB RAM >>> $PGDATA sits on a RAID5 array comprised of 3 SATA disks.  Its Linux's >>> md software RAID. > >> How does this compare to your other machines running the same, or >> similar, d

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-23 Thread Lonni J Friedman
On Wed, May 23, 2012 at 12:36 PM, Gavin Flower wrote: > On 24/05/12 05:09, Lonni J Friedman wrote: > > On Wed, May 23, 2012 at 9:37 AM, Tom Lane wrote: > > Lonni J Friedman writes: > > After banging my head on the wall for  a long time, I happened to > notice that khug

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-23 Thread Lonni J Friedman
On Wed, May 23, 2012 at 9:37 AM, Tom Lane wrote: > Lonni J Friedman writes: >> After banging my head on the wall for  a long time, I happened to >> notice that khugepaged was consuming 100% CPU every time autovacuum >> was running.  I did: >> echo "madvise"

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-23 Thread Lonni J Friedman
Thanks for your reply. On Tue, May 22, 2012 at 7:19 PM, Andy Colson wrote: >  On Mon, May 21, 2012 at 2:05 PM, Lonni J Friedman >  wrote: >>> >>> Greetings, >>> >>> When I got in this morning, I found >>> an autovacuum process that had

Re: [GENERAL] pg_basebackup blocking all queries

2012-05-22 Thread Lonni J Friedman
rlowe wrote: > Do the queries here help? > > http://wiki.postgresql.org/wiki/Lock_Monitoring > > On Tue, May 22, 2012 at 12:42 PM, Lonni J Friedman wrote: >> Greetings, >> I have a 4 server postgresql-9.1.3 cluster (one master doing streaming >> replication to 3 hot stan

[GENERAL] pg_basebackup blocking all queries

2012-05-22 Thread Lonni J Friedman
Greetings, I have a 4 server postgresql-9.1.3 cluster (one master doing streaming replication to 3 hot standby servers). All of them are running Fedora-16-x86_64. Last Friday I upgraded the entire cluster from Fedora-15 with postgresql-9.0.6 to Fedora-16 with postgresql-9.1.3. I'm finding that I

[GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-22 Thread Lonni J Friedman
No one has any ideas or suggestions, or even questions? If someone needs more information, I'd be happy to provide it. This problem is absolutely killing me. On Mon, May 21, 2012 at 2:05 PM, Lonni J Friedman wrote: > Greetings, > I have a 4 server postgresql-9.1.3 cluster (one m

[GENERAL] significant performance hit whenever autovacuum runs after upgrading from 9.0 -> 9.1

2012-05-21 Thread Lonni J Friedman
Greetings, I have a 4 server postgresql-9.1.3 cluster (one master doing streaming replication to 3 hot standby servers). All of them are running Fedora-16-x86_64. Last Friday I upgraded the entire cluster from Fedora-15 with postgresql-9.0.6 to Fedora-16 with postgresql-9.1.3. I made no changes

[GENERAL] problems after restoring from a pg_basebackup

2012-04-27 Thread Lonni J Friedman
Greetings, I'm running postgresql-9.1.3 on a Linux-x86_64 (Fedora16, if it matters) system. I noticed the existence of pg_basebackup starting in 9.1, and figured I'd try it out and see if it would simplify our backup & management processes. $ pg_basebackup -P -v -D /tmp/backup -x -Ft -z -U postgr

Re: [GENERAL] pg_basebackup issues

2012-04-24 Thread Lonni J Friedman
On Fri, Apr 20, 2012 at 12:31 PM, Magnus Hagander wrote: > On Fri, Apr 20, 2012 at 19:51, Lonni J Friedman wrote: >> Anyway, lesson learned, I need to either invoke pg_basebackup as the >> same user that runs the database (or is specified with the -U >> parameter ?)

[GENERAL] pg_basebackup issues

2012-04-20 Thread Lonni J Friedman
Greetings, I'm running postgresql-9.1.3 on a Linux-x86_64 (Fedora16, if it matters) system. I noticed the existence of pg_basebackup starting in 9.1, and figured I'd try it out and see if it would simplify our backup & management processes. I setup a test system (same OS & postgresql version as p

Re: [GENERAL] pg_upgrade + streaming replication ?

2012-03-20 Thread Lonni J Friedman
On Tue, Mar 20, 2012 at 11:46 AM, Bruce Momjian wrote: > On Mon, Mar 19, 2012 at 03:07:02PM -0700, Jeff Davis wrote: >> On Mon, 2012-03-19 at 15:30 -0400, Bruce Momjian wrote: >> > On Thu, Mar 01, 2012 at 02:01:31PM -0800, Lonni J Friedman wrote: >> > > I'

Re: [GENERAL] pg_upgrade + streaming replication ?

2012-03-19 Thread Lonni J Friedman
On Mon, Mar 19, 2012 at 12:30 PM, Bruce Momjian wrote: > On Thu, Mar 01, 2012 at 02:01:31PM -0800, Lonni J Friedman wrote: >> I've got a 3 node cluster (1 master/2 slaves) running 9.0.x with >> streaming replication.  I'm in the planning stages of upgrading to >>

Re: [GENERAL] how to measure wal_buffer usage

2012-03-16 Thread Lonni J Friedman
On Fri, Mar 16, 2012 at 2:45 AM, Albe Laurenz wrote: > Lonni J Friedman wrote: >> After reading this interesting article on shared_buffers and wal_buffers: >> http://rhaas.blogspot.com/2012/03/tuning-sharedbuffers-and-walbuffers.html >> >> it got me wondering if my set

[GENERAL] how to measure wal_buffer usage

2012-03-15 Thread Lonni J Friedman
After reading this interesting article on shared_buffers and wal_buffers: http://rhaas.blogspot.com/2012/03/tuning-sharedbuffers-and-walbuffers.html it got me wondering if my settings were ideal. Is there some way to measure wal_buffer usage in real time, so that I could simply monitor it for som

[GENERAL] pg_upgrade + streaming replication ?

2012-03-01 Thread Lonni J Friedman
I've got a 3 node cluster (1 master/2 slaves) running 9.0.x with streaming replication. I'm in the planning stages of upgrading to 9.1.x, and am looking into the most efficient way to do the upgrade with the goal of minimizing downtime & risk. After googling, the only discussion that I've found o

Re: [GENERAL] returning rows from an implicit JOIN where results either exist in both tables OR only one table

2011-12-01 Thread Lonni J Friedman
On Thu, Dec 1, 2011 at 1:57 PM, David Johnston wrote: > -Original Message- > From: pgsql-general-ow...@postgresql.org > [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of Lonni J Friedman > Sent: Thursday, December 01, 2011 4:13 PM > To: pgsql-general > Subject: [

[GENERAL] returning rows from an implicit JOIN where results either exist in both tables OR only one table

2011-12-01 Thread Lonni J Friedman
Greetings, I've got a PostgreSQL-9.0.x database that manages an automated testing environment. There are a bunch of tables that contain assorted static data (OS versions, test names, etc) named 'buildlist' & 'osversmap'. However, there are also two tables which contain data which changes often. T

Re: [GENERAL] autovacuum stuck on a table for 18+ hours, consuming lots of CPU time

2011-11-22 Thread Lonni J Friedman
On Tue, Nov 22, 2011 at 7:49 PM, Tom Lane wrote: > Lonni J Friedman writes: >> I suspect you're right.  I just ran strace against that PID again, and >> now all the lseek & read FD's are referrring to a different number >> (115), so that means its moved onto

Re: [GENERAL] autovacuum stuck on a table for 18+ hours, consuming lots of CPU time

2011-11-22 Thread Lonni J Friedman
On Tue, Nov 22, 2011 at 7:19 PM, Tom Lane wrote: > Lonni J Friedman writes: >> Thanks for your prompt reply.  I was pretty sure that I was using the >> default, but just to confirm, I just ran: >> 'SHOW vacuum_cost_delay;' > > What about autovacuum_vacuu

  1   2   3   >