Re: [GENERAL] authentication/privileges

2013-05-10 Thread Jasen Betts
On 2013-05-10, Alvaro Herrera wrote: > Tom Lane escribió: > >> It's fairly common for distro-supplied packages to create a postgres >> OS user but not assign it any password. In that state, the only way to >> become postgres is to "su" to it from root, or perhaps from a sudoer >> account with roo

Re: [GENERAL] PG in cash till machines

2013-05-10 Thread John R Pierce
On 5/10/2013 2:11 PM, Bexley Hall wrote: Having designed (regulated) gaming and "grey area" devices (each handling hard currency), I can tell you that you have to have already performed a pretty exhaustive threat analysis (e.g., red team, blue\ team) *before* you start the product's design. If y

Re: [GENERAL] PG in cash till machines

2013-05-10 Thread Bexley Hall
Hi Carlos, On 5/10/2013 6:43 AM, Carlos Henrique Reimer wrote: We are developing a solution which will run in thousands of small cash till machines running Linux and we would like to use PostgreSQL but there is a insecurity feeling regarding the solution basically because these boxes would be ex

Re: [GENERAL] PL/R etc.

2013-05-10 Thread Mark Morgan Lloyd
Merlin Moncure wrote: On Fri, May 10, 2013 at 2:32 PM, Mark Morgan Lloyd wrote: Merlin Moncure wrote: On Fri, May 10, 2013 at 4:41 AM, Mark Morgan Lloyd wrote: I don't know whether anybody active on the list has R (and in particular PL/R) experience, but just in case... :-) i) Something l

Re: [GENERAL] PL/R etc.

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 2:32 PM, Mark Morgan Lloyd wrote: > Merlin Moncure wrote: >> >> On Fri, May 10, 2013 at 4:41 AM, Mark Morgan Lloyd >> wrote: >>> >>> I don't know whether anybody active on the list has R (and in particular >>> PL/R) experience, but just in case... :-) >>> >>> i) Somethin

Re: [GENERAL] PL/R etc.

2013-05-10 Thread Mark Morgan Lloyd
Merlin Moncure wrote: On Fri, May 10, 2013 at 4:41 AM, Mark Morgan Lloyd wrote: I don't know whether anybody active on the list has R (and in particular PL/R) experience, but just in case... :-) i) Something like APL can operate on an array with minimal regard for index order, i.e. operation

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steven Schlansker
On May 10, 2013, at 11:35 AM, Lonni J Friedman wrote: >> >> I am not sure that these numbers will end up being anywhere near what works >> for you, but these are my notes from tuning a 4xMLC SSD RAID-10. I haven't >> proven that this is optimal, but it was way better than the defaults. We >

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steven Schlansker
On May 10, 2013, at 11:38 AM, Merlin Moncure wrote: >> >> PostgreSQL configuration changes: >> synchronous_commit = off >> > > that's good info, but it should be noted that synchronous_commit > trades a risk of some data loss (but not nearly as much risk as > volatile storage) for a big increa

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 1:23 PM, Steven Schlansker wrote: > > On May 10, 2013, at 7:14 AM, Matt Brock wrote: > >> Hello. >> >> We're intending to deploy PostgreSQL on Linux with SSD drives which would be >> in a RAID 1 configuration with Hardware RAID. >> >> My first question is essentially: are

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Lonni J Friedman
On Fri, May 10, 2013 at 11:23 AM, Steven Schlansker wrote: > > On May 10, 2013, at 7:14 AM, Matt Brock wrote: > >> Hello. >> >> We're intending to deploy PostgreSQL on Linux with SSD drives which would be >> in a RAID 1 configuration with Hardware RAID. >> >> My first question is essentially: ar

Re: [GENERAL] Large amount of serialization errors in transactions

2013-05-10 Thread Kevin Grittner
Vegard Bønes wrote: > I have a problem understanding how transactions with serializable > isolation level works in postgresql. What exactly may cause a > serialization error? A pattern of read-write dependencies among concurrent transactions which indicate that a serialization anomaly is possibl

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steven Schlansker
On May 10, 2013, at 7:14 AM, Matt Brock wrote: > Hello. > > We're intending to deploy PostgreSQL on Linux with SSD drives which would be > in a RAID 1 configuration with Hardware RAID. > > My first question is essentially: are there any issues we need to be aware of > when running PostgreSQL

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Alvaro Herrera
Steve Clark escribió: > Well we have dual redundant power supplies on separate UPS so could something > go wrong yes, but a tornado could > come along and destroy the building also. .. hence your standby server across the country? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ Po

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread David Boreham
On 5/10/2013 11:23 AM, Lonni J Friedman wrote: There's also the 520 series, which has better performance than the 320 series (which is EOL now). I wouldn't use the 520 series for production database storage -- it has the Sandforce controller and apparently no power failure protection. --

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread David Boreham
On 5/10/2013 11:20 AM, Merlin Moncure wrote: I find the s3700 to be superior to the 710 in just about every way (although you're right -- it is suitable for database use). merlin The s3700 series replaces the 710 so it should be superior :) -- Sent via pgsql-general mailing list (pgsql-gen

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Lonni J Friedman
On Fri, May 10, 2013 at 10:20 AM, Merlin Moncure wrote: > On Fri, May 10, 2013 at 12:03 PM, David Boreham > wrote: >> On 5/10/2013 10:21 AM, Merlin Moncure wrote: >>> >>> As it turns out the list of flash drives are suitable for database use is >>> surprisingly small. The s3700 I noted upthread

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Steve Clark
On 05/10/2013 12:46 PM, Merlin Moncure wrote: On Fri, May 10, 2013 at 11:34 AM, Evan D. Hoffman wrote: I'd expect to use a RAID controller with either BBU or NVRAM cache to handle that, and that the server itself would be on UPS for a production DB. That said, a standby replica DB on conventio

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 12:03 PM, David Boreham wrote: > On 5/10/2013 10:21 AM, Merlin Moncure wrote: >> >> As it turns out the list of flash drives are suitable for database use is >> surprisingly small. The s3700 I noted upthread seems to be specifically >> built with databases in mind and is li

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Magnus Hagander
On Fri, May 10, 2013 at 7:00 PM, Sergey Koposov wrote: > > On Fri, 10 May 2013, Lonni J Friedman wrote: > >> Its definitely not a bug. You need to set/increase wal_keep_segments >> to a value that ensures that they aren't recycled faster than the time >> required to complete the base backup (plus

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread David Boreham
On 5/10/2013 10:21 AM, Merlin Moncure wrote: As it turns out the list of flash drives are suitable for database use is surprisingly small. The s3700 I noted upthread seems to be specifically built with databases in mind and is likely the best choice for new deployments. The older Intel 320 is a

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Lonni J Friedman
That's a good point. Then i dunno, perhaps it is a bug, but I'd be surprised if this wasn't working, as its not really a corner case that could be missed in testing, as long as all the options were exercised. Hopefully someone else can weigh in. On Fri, May 10, 2013 at 10:00 AM, Sergey Koposov

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Sergey Koposov
On Fri, 10 May 2013, Lonni J Friedman wrote: Its definitely not a bug. You need to set/increase wal_keep_segments to a value that ensures that they aren't recycled faster than the time required to complete the base backup (plus some buffer). But I thought that wal_keep_segments is not needed

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Lonni J Friedman
Its definitely not a bug. You need to set/increase wal_keep_segments to a value that ensures that they aren't recycled faster than the time required to complete the base backup (plus some buffer). On Fri, May 10, 2013 at 9:48 AM, Sergey Koposov wrote: > Hi, > > I've recently started to use pg_ba

[GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Sergey Koposov
Hi, I've recently started to use pg_basebackup --xlog-method=stream to backup my multi-Tb database. Before I did the backup when there was not much activity in the DB and it went perfectly fine, but today, I've started the backup and it failed twice almost at the same time as the CREATE INDE

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 11:34 AM, Evan D. Hoffman wrote: > I'd expect to use a RAID controller with either BBU or NVRAM cache to handle > that, and that the server itself would be on UPS for a production DB. That > said, a standby replica DB on conventional disk is definitely a good idea in > any

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Evan D. Hoffman
I'd expect to use a RAID controller with either BBU or NVRAM cache to handle that, and that the server itself would be on UPS for a production DB. That said, a standby replica DB on conventional disk is definitely a good idea in any case. On Fri, May 10, 2013 at 12:25 PM, Merlin Moncure wrote:

Re: [GENERAL] authentication/privileges

2013-05-10 Thread Alvaro Herrera
Tom Lane escribió: > It's fairly common for distro-supplied packages to create a postgres > OS user but not assign it any password. In that state, the only way to > become postgres is to "su" to it from root, or perhaps from a sudoer > account with root-equivalent privileges. While that might be

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 11:11 AM, Evan D. Hoffman wrote: > Not sure of your space requirements, but I'd think a RAID 10 of 8x or more > Samsung 840 Pro 256/512 GB would be the best value. Using a simple mirror > won't get you the reliability that you want since heavy writing will burn > the drive

Re: [GENERAL] PL/R etc.

2013-05-10 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/10/2013 06:33 AM, Merlin Moncure wrote: > you have full access to sql within the pl/r function, so nothing > is keeping you from paging data into the frame via a cursor, but > that only helps so much. I have thought about implementing the paging

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 10:19 AM, Matt Brock wrote: > After googling this for a while, it seems that High Endurance MLC is only > starting to rival SLC for endurance and write performance in the very latest, > cutting-edge hardware. In general, though, it seems it would be fair to say > that SL

Re: [GENERAL] authentication/privileges

2013-05-10 Thread Tom Lane
"Sebastian P. Luque" writes: > With peer authentication, one can only login as postgres from a local > connection. I'm not sure what password the postgres user was set up in > the OS, however, I assigned one to it (the same as for the PostgreSQL > user). I've read somewhere that the postgres OS

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Evan D. Hoffman
Not sure of your space requirements, but I'd think a RAID 10 of 8x or more Samsung 840 Pro 256/512 GB would be the best value. Using a simple mirror won't get you the reliability that you want since heavy writing will burn the drives out over time, and if you're writing the exact same content to b

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-10 Thread Larry Rosenman
On 2013-05-10 10:57, Tom Lane wrote: Larry Rosenman writes: On 2013-05-10 09:14, Tom Lane wrote: ... and verify you get a cheap plan for each referencing table. We don't :( Ugh. I bet the problem is that in some of these tables, there are lots and lots of duplicate account ids, such that seqs

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-10 Thread Tom Lane
Larry Rosenman writes: > On 2013-05-10 09:14, Tom Lane wrote: >> ... and verify you get a cheap plan for each referencing table. > We don't :( Ugh. I bet the problem is that in some of these tables, there are lots and lots of duplicate account ids, such that seqscans look like a good bet when s

[GENERAL] authentication/privileges

2013-05-10 Thread Sebastian P. Luque
Hi, Although I'm quite happy with the way my system (Debian sid) has set up the server (PosgreSQL 9.1), I'm not sure I'm using the authentication/privilege mechanism properly. In particular, I'd like to understand how the administrative user (postgres) is set up. Here is what pg_hba contains: #

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread David Boreham
On 5/10/2013 9:19 AM, Matt Brock wrote: After googling this for a while, it seems that High Endurance MLC is only starting to rival SLC for endurance and write performance in the very latest, cutting-edge hardware. In general, though, it seems it would be fair to say that SLCs are still a bett

Re: [GENERAL] PG in cash till machines

2013-05-10 Thread Paul Jungwirth
>> Our major concern is related to the write-back issues we can face in this >> environment. I agree this is the OS's responsibility. Greg Smith's Postgres: High Performance book has a lot to say about this, but there's also a lot you could read online, since it's really a requirement for any Post

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Matt Brock
After googling this for a while, it seems that High Endurance MLC is only starting to rival SLC for endurance and write performance in the very latest, cutting-edge hardware. In general, though, it seems it would be fair to say that SLCs are still a better bet for databases than MLC? The number

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-10 Thread Larry Rosenman
On 2013-05-10 09:14, Tom Lane wrote: Larry Rosenman writes: Any ideas on how to figure out if we ARE getting seqscan check plans, and better fix it? Try an EXPLAIN ANALYZE VERBOSE on something that just deletes one row, and wait however long it takes. The printout should show how much time is

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 9:14 AM, Matt Brock wrote: > Hello. > > We're intending to deploy PostgreSQL on Linux with SSD drives which would be > in a RAID 1 configuration with Hardware RAID. > > My first question is essentially: are there any issues we need to be aware of > when running PostgreSQL

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-10 Thread Tom Lane
Larry Rosenman writes: > Any ideas on how to figure out if we ARE getting seqscan check plans, > and better fix it? Try an EXPLAIN ANALYZE VERBOSE on something that just deletes one row, and wait however long it takes. The printout should show how much time is taken in the implementation trigge

[GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Matt Brock
Hello. We're intending to deploy PostgreSQL on Linux with SSD drives which would be in a RAID 1 configuration with Hardware RAID. My first question is essentially: are there any issues we need to be aware of when running PostgreSQL 9 on CentOS 6 on a server with SSD drives in a Hardware RAID 1

Re: [GENERAL] PG in cash till machines

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 8:43 AM, Carlos Henrique Reimer wrote: > Hi, > > We are developing a solution which will run in thousands of small cash till > machines running Linux and we would like to use PostgreSQL but there is a > insecurity feeling regarding the solution basically because these boxes

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-10 Thread Larry Rosenman
On 2013-05-09 16:43, Larry Rosenman wrote: On 2013-05-09 16:40, Tom Lane wrote: Larry Rosenman writes: On 2013-05-09 16:22, Tom Lane wrote: Perhaps it's blocked on a lock? Did you look into pg_locks? Did you note whether the process was consuming CPU time and/or doing IO? all the locks were cl

Re: [GENERAL] FATAL: database "a/system_data" does not exist

2013-05-10 Thread Tom Lane
Adrian Klaver writes: > On 05/10/2013 12:13 AM, sumita wrote: >> This error is getting logged at an interval of 2 minutes and 10 seconds >> 2013-05-10 00:22:50 GMT:[4180]FATAL: database "a/system_data" does not >> exist > A suggestion, turn up your log_statement to 'all' to see if you can > cat

[GENERAL] PG in cash till machines

2013-05-10 Thread Carlos Henrique Reimer
Hi, We are developing a solution which will run in thousands of small cash till machines running Linux and we would like to use PostgreSQL but there is a insecurity feeling regarding the solution basically because these boxes would be exposed to an insecure environment and insecure procedures like

Re: [GENERAL] PL/R etc.

2013-05-10 Thread Merlin Moncure
On Fri, May 10, 2013 at 4:41 AM, Mark Morgan Lloyd wrote: > I don't know whether anybody active on the list has R (and in particular > PL/R) experience, but just in case... :-) > > i) Something like APL can operate on an array with minimal regard for > index order, i.e. operations across the arr

Re: [GENERAL] FATAL: database "a/system_data" does not exist

2013-05-10 Thread Adrian Klaver
On 05/10/2013 12:13 AM, sumita wrote: This error is getting logged at an interval of 2 minutes and 10 seconds 2013-05-10 00:22:50 GMT:[4180]FATAL: database "a/system_data" does not exist 2013-05-10 00:25:00 GMT:[4657]FATAL: database "a/system_data" does not exist 2013-05-10 00:27:10 GMT:[

Re: [GENERAL] Storing small image files

2013-05-10 Thread Nelson Green
On Fri, May 10, 2013 at 5:24 AM, Misa Simic wrote: > 2013/5/10 Eduardo Morras > >> >> Hi Nelson. I worked with images and Postgresql, and want to add some >> comments: >> >> On Thu, 9 May 2013 13:40:15 -0500 >> Nelson Green wrote: >> > OK, this is kind of convoluted, but I got a couple of test

Re: [GENERAL] Storing small image files

2013-05-10 Thread Nelson Green
On Fri, May 10, 2013 at 2:59 AM, Eduardo Morras wrote: > > Hi Nelson. I worked with images and Postgresql, and want to add some > comments: > > On Thu, 9 May 2013 13:40:15 -0500 > Nelson Green wrote: > > OK, this is kind of convoluted, but I got a couple of test cases that > work > > for me. The

Re: [GENERAL] FATAL: database "a/system_data" does not exist

2013-05-10 Thread Chris Angelico
On Fri, May 10, 2013 at 5:13 PM, sumita wrote: > This error is getting logged at an interval of 2 minutes and 10 seconds > 2013-05-10 00:22:50 GMT:[4180]FATAL: database "a/system_data" does not > exist > 2013-05-10 00:25:00 GMT:[4657]FATAL: database "a/system_data" does not > exist > 2013-05-

Re: [GENERAL] Storing small image files

2013-05-10 Thread Misa Simic
2013/5/10 Eduardo Morras > > Hi Nelson. I worked with images and Postgresql, and want to add some > comments: > > On Thu, 9 May 2013 13:40:15 -0500 > Nelson Green wrote: > > OK, this is kind of convoluted, but I got a couple of test cases that > work > > for me. The steps to make the first one a

[GENERAL] PL/R etc.

2013-05-10 Thread Mark Morgan Lloyd
I don't know whether anybody active on the list has R (and in particular PL/R) experience, but just in case... :-) i) Something like APL can operate on an array with minimal regard for index order, i.e. operations across the array are as easily-expressed and as efficient as operations down t

Re: [GENERAL] Storing small image files

2013-05-10 Thread Thomas Kellerer
Nelson Green wrote on 09.05.2013 19:05: On Thu, May 9, 2013 at 10:51 AM, Achilleas Mantzios wrote: then here : http://www.dbforums.com/postgresql/1666200-insert-jpeg-files-into-bytea-column.html Thanks Achilleas. I usually do the physical design in vi using sql scripts, and I like to include a

Re: [GENERAL] Storing small image files

2013-05-10 Thread Eduardo Morras
Hi Nelson. I worked with images and Postgresql, and want to add some comments: On Thu, 9 May 2013 13:40:15 -0500 Nelson Green wrote: > OK, this is kind of convoluted, but I got a couple of test cases that work > for me. The steps to make the first one are below. > > First I took one of the phot

[GENERAL] Large amount of serialization errors in transactions

2013-05-10 Thread Vegard Bønes
Hi! I have a problem understanding how transactions with serializable isolation level works in postgresql. What exactly may cause a serialization error? My problem is a system where process one adds data to a database. Shortly afterwards, process two reads and possibly modifies the same data (k

Re: [GENERAL] FATAL: database "a/system_data" does not exist

2013-05-10 Thread sumita
This error is getting logged at an interval of 2 minutes and 10 seconds 2013-05-10 00:22:50 GMT:[4180]FATAL: database "a/system_data" does not exist 2013-05-10 00:25:00 GMT:[4657]FATAL: database "a/system_data" does not exist 2013-05-10 00:27:10 GMT:[5394]FATAL: database "a/system_data" does