[GENERAL] Fetching last n records from Posgresql

2016-03-29 Thread Deole, Pushkar (Pushkar)
Hi, Does PostgreSQL support a query to fetch last 'n' records that match the selection criteria. I am trying to fetch records from a table with start date that falls in last 30 days, however, I want to fetch the oldest 'n' records and not the recent ones. I know there is a LIMIT clause which I

Re: [GENERAL] pg_restore casts check constraints differently

2016-03-29 Thread David G. Johnston
On Tue, Mar 29, 2016 at 7:40 PM, Tom Lane wrote: > "David G. Johnston" writes: > > On Tue, Mar 29, 2016 at 2:45 PM, Tom Lane wrote: > >> It's not really different. What you're seeing is pg_dump (or actually > >> ruleutils.c) choosing to dump some implicit casts explicitly to ensure > >> that t

Re: [GENERAL] pg_restore casts check constraints differently

2016-03-29 Thread Tom Lane
"David G. Johnston" writes: > On Tue, Mar 29, 2016 at 2:45 PM, Tom Lane wrote: >> It's not really different. What you're seeing is pg_dump (or actually >> ruleutils.c) choosing to dump some implicit casts explicitly to ensure >> that the expression is parsed the same way next time. > ​Why don

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread Adrian Klaver
On 03/29/2016 03:25 PM, Stephen Constable wrote: Sorry, my client environment is Linux. Hmm, so I was reading win32.c wrong. It is mapping a Windows error message to that string. My current theory is that my clients are running out of available ephemeral ports, like in this thread: http://

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread John R Pierce
On 3/29/2016 3:25 PM, Stephen Constable wrote: Sorry, my client environment is Linux. My current theory is that my clients are running out of available ephemeral ports, like in this thread: http://dba.stackexchange.com/questions/59650/pgbouncer-works-great-but-occasionally-becomes-unavailable

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread Stephen Constable
Sorry, my client environment is Linux. My current theory is that my clients are running out of available ephemeral ports, like in this thread: http://dba.stackexchange.com/questions/59650/pgbouncer-works-great-but-occasionally-becomes-unavailable (but I"m not currently using pg bouncer). I tried

Re: [GENERAL] pg_restore casts check constraints differently

2016-03-29 Thread David G. Johnston
On Tue, Mar 29, 2016 at 2:45 PM, Tom Lane wrote: > Joshua Ma writes: > > This might not be a common case, but we're using pg_dump in a testing > > environment to check migrations - 1) we initialize the db from HEAD, > > pg_dump it, 2) we initialize the db from migration_base.sql, apply > > migra

Re: [GENERAL] pg_restore casts check constraints differently

2016-03-29 Thread Tom Lane
Joshua Ma writes: > This might not be a common case, but we're using pg_dump in a testing > environment to check migrations - 1) we initialize the db from HEAD, > pg_dump it, 2) we initialize the db from migration_base.sql, apply > migrations, pg_dump it, and 3) compare the two dumps to verify tha

Re: [GENERAL] Partitioning and ORM tools

2016-03-29 Thread Brian Fehrle
Here is a working example of trigger based partitioning with a view and 'do instead' that works with ORM tools using the affected rows return (example attached). The key things that make it work are: 1. RETURN NEW; (in the function after inserting into the partition) 2. INSTEAD OF INSERT (in th

[GENERAL] pg_restore casts check constraints differently

2016-03-29 Thread Joshua Ma
This might not be a common case, but we're using pg_dump in a testing environment to check migrations - 1) we initialize the db from HEAD, pg_dump it, 2) we initialize the db from migration_base.sql, apply migrations, pg_dump it, and 3) compare the two dumps to verify that our migrations are correc

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread Adrian Klaver
On 03/29/2016 01:28 PM, Stephen Constable wrote: My apologies, I'm not sure what part of the networking stack the messages are coming from. It also states: """ could not connect to server: Cannot assign requested address Is the server running on host "" and accepting TCP/IP connections on port ?

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread John R Pierce
On 3/29/2016 1:28 PM, Stephen Constable wrote: """ This error is only printed under a 32-job load, never a single job load. The processes are indeed connecting over a local network. I have only enabled the logging of connections and disconnections since I figured that would be the most telling

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread Stephen Constable
My apologies, I'm not sure what part of the networking stack the messages are coming from. It also states: """ could not connect to server: Cannot assign requested address Is the server running on host "" and accepting TCP/IP connections on port ? """ This error is only printed under a 32-job load

Re: [GENERAL] debugging server connection issue

2016-03-29 Thread Adrian Klaver
On 03/29/2016 01:10 PM, Stephen Constable wrote: Hi All, I'm a new-ish sysadmin working on porting legacy scientific code from a local server/client to new supercomputer environment. My work is mostly done, except that my postgres database doesn't seem to be able to keep up with the new environ

[GENERAL] debugging server connection issue

2016-03-29 Thread Stephen Constable
Hi All, I'm a new-ish sysadmin working on porting legacy scientific code from a local server/client to new supercomputer environment. My work is mostly done, except that my postgres database doesn't seem to be able to keep up with the new environment. The application is written in-house in a mix

Re: [GENERAL] How to quote the COALESCE function?

2016-03-29 Thread Pavel Stehule
Hi 2016-03-29 10:30 GMT+02:00 Roman Scherer : > Tom, Jerry, I'm going to do the same as the `quote_identifier` > function of Postgres does, only quote if necessary. > > Thanks for your explanation, Roman. > The coalesce is one few functions implemented by special rule in PostgreSQL parser. Some

[GENERAL] [ANN] pgsql v1.0: PostgreSQL ftplugin for Vim

2016-03-29 Thread Lifepillar
See: http://www.vim.org/scripts/script.php?script_id=5358 This Vim plugin provides syntax highlighting and auto-completion support for PostgreSQL version 9.4 or above and for some of its extensions: - PL/pgSQL; - any other language, like PL/Python, PL/R, etc...; - PostGIS 2.2 (including PostGIS

Re: [GENERAL] How to quote the COALESCE function?

2016-03-29 Thread Roman Scherer
Tom, Jerry, I'm going to do the same as the `quote_identifier` function of Postgres does, only quote if necessary. Thanks for your explanation, Roman. On Tue, Mar 29, 2016 at 1:31 AM, Jerry Sievers wrote: > Roman Scherer writes: > > > Hello, > > > > I'm building a DSL in Clojure for SQL and s

Re: [GENERAL] pg_largeobject

2016-03-29 Thread Jerome Wagner
I am not saying that this will solve your problem (I never tried id even though I keep it in my radar), but this project seems to implement something close to what Daniel is describing: https://github.com/andreasbaumann/pgfuse + it gives you a FUSE wrapper so the client can use fs calls. the pro

Re: [GENERAL] pg_largeobject

2016-03-29 Thread Daniel Verite
Sridhar N Bamandlapally wrote: > due to size limitation BYTEA was not considered You could adopt for a custom table the same kind of structure that pg_largeobject has, that is an ordered series of BYTEA chunks. # \d pg_largeobject Table "pg_catalog.pg_largeobject" Column | Type | Mod

Re: [GENERAL] pg_largeobject

2016-03-29 Thread Alvaro Aguayo Garcia-Rada
Amongst all my tries, I also tried that. I created two tables, one for basic file data, and another for file content(splitted in pages, as in large objects), but the performance was almost the same as with pg_largeobject; he great difference was that, with my own tables, I could replicate withou

Re: [GENERAL] pg_largeobject

2016-03-29 Thread Sridhar N Bamandlapally
We are doing application/database migration compatible with postgresql on cloud, DR/replication also in plan at present I feel need of configurable multi-table storage instead of pg_largeobject only Thanks Sridhar On Tue, Mar 29, 2016 at 6:08 PM, Alvaro Aguayo Garcia-Rada < aagu...@opensysperu.

Re: [GENERAL] More correlated (?) index woes

2016-03-29 Thread Melvin Davidson
On Tue, Mar 29, 2016 at 6:47 AM, Geoff Winkless wrote: > On 28 March 2016 at 20:23, I wrote: > >> Table pa has 7522676 rows, 4834042 of which have field1 NULL, so it's >> absolutely not reasonable to expect this to be an optimal strategy. >> ​ >> > It occurred to me that even though the majority

Re: [GENERAL] pg_largeobject

2016-03-29 Thread Alvaro Aguayo Garcia-Rada
Some time ago I had to setup a replicated file system between multiple linux servers. I tried everything I could based on postgres, including large objects, but everything was significantly slower than a regular filesystem. My conclussion: postgres is not suitable for storing large files efficie

Re: [GENERAL] Horrible/never returning performance using stable function on WHERE clause

2016-03-29 Thread Achilleas Mantzios
Hello David On 29/03/2016 14:04, David Rowley wrote: On 29 March 2016 at 20:01, Achilleas Mantzios wrote: We are using PostgreSQL 9.3. Shouldn't the optimizer use a single call to get_machdef_sister_defids in get_machdef_sister_defids_maxrh ?? It shouldn't be up to the optimizer to evaluate a

Re: [GENERAL] pg_largeobject

2016-03-29 Thread Sridhar N Bamandlapally
all media files are stored in database with size varies from 1MB - 5GB based on media file types and user-group we storing in different tables, but PostgreSQL store OID/Large-object in single table (pg_largeobject), 90% of database size is with table pg_largeobject due to size limitation BYTEA wa

Re: [GENERAL] Horrible/never returning performance using stable function on WHERE clause

2016-03-29 Thread David Rowley
On 29 March 2016 at 20:01, Achilleas Mantzios wrote: > We are using PostgreSQL 9.3. Shouldn't the optimizer use a single call to > get_machdef_sister_defids in get_machdef_sister_defids_maxrh ?? It shouldn't be up to the optimizer to evaluate a STABLE function. Only IMMUTABLE functions will be ev

Re: [GENERAL] More correlated (?) index woes

2016-03-29 Thread Geoff Winkless
On 28 March 2016 at 20:23, I wrote: > Table pa has 7522676 rows, 4834042 of which have field1 NULL, so it's > absolutely not reasonable to expect this to be an optimal strategy. > ​ > It occurred to me that even though the majority of values are NULL, there are ​ ​1691 unique values in pa.field1,

Re: [GENERAL] pg_largeobject

2016-03-29 Thread John R Pierce
On 3/29/2016 2:13 AM, Sridhar N Bamandlapally wrote: Hi pg_largeobject is creating performance issues as it grow due to single point storage(for all tables) is there any alternate apart from bytea ? like configuration large-object-table at table-column level and oid PK(primary key) stored a

[GENERAL] pg_largeobject

2016-03-29 Thread Sridhar N Bamandlapally
Hi pg_largeobject is creating performance issues as it grow due to single point storage(for all tables) is there any alternate apart from bytea ? like configuration large-object-table at table-column level and oid PK(primary key) stored at pg_largeobject Thanks Sridhar

[GENERAL] Horrible/never returning performance using stable function on WHERE clause

2016-03-29 Thread Achilleas Mantzios
Hello list, I have written some functions to extract some data from our DB, from an hierarchical structure, the problem is that if a function doing lookups is defined as STABLE in the WHERE clause the performance is horrible. What I am trying to achieve is given a specific node in an hierarchical