Re: [HACKERS] checkpointer continuous flushing

2015-08-31 Thread Amit Kapila
On Mon, Aug 31, 2015 at 12:40 PM, Fabien COELHO wrote: > > > Hello Amit, > >> IBM POWER-8 24 cores, 192 hardware threads >> RAM = 492GB > > > Wow! Thanks for trying the patch on such high-end hardware! > > About the disks: what kind of HDD (RAID? speed?)? HDD write cache? > Speed of Reads - Timin

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2015-08-31 Thread Etsuro Fujita
On 2015/08/30 13:06, David Rowley wrote: It's likely not worth changing if there's cases when it'll be slower, but curiosity got the better of me and I wondered how extreme a case it would take to actually see a slowdown, and per my benchmark results the first used column would have to be about a

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

2015-08-31 Thread Michael Paquier
On Sun, Aug 30, 2015 at 1:06 AM, Jim Nasby wrote: > Steps to reproduce: > Download https://github.com/BlueTreble/test_factory/archive/crash.zip > Unzip, cd into directory > pgxn install pgtap (or just make test) FWIW, make test fails: ! ERROR: 42703: column "c_data_table_name" does not exist ! LI

Re: [HACKERS] [PATCH] SQL function to report log message

2015-08-31 Thread Pavel Stehule
2015-09-01 7:20 GMT+02:00 Jim Nasby : > On 8/31/15 11:59 PM, Pavel Stehule wrote: > >> The transformation: text -> error level is common task - and PLpgSQL it >> does in pl_gram.y. My idea is to add new function to error utils named >> "parse_error_level" and use it from PLpgSQL and from your code

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-31 Thread Pavel Stehule
Hi 2015-08-31 19:09 GMT+02:00 Shulgin, Oleksandr : > On Mon, Aug 31, 2015 at 12:35 PM, Pavel Stehule > wrote: > >> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >>> >>> Ah, thanks! Somehow I've missed this mail. You didn

Re: [HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Satoshi Nagayasu
On 2015/09/01 14:01, Tom Lane wrote: > Satoshi Nagayasu writes: >> On 2015/09/01 13:41, Peter Geoghegan wrote: >>> If you want to use the queryId field directly, which I recall you >>> mentioning before, then that's harder. There is simply no contract >>> among extensions for "owning" a queryId. B

Re: [HACKERS] buffer README is out of date

2015-08-31 Thread Jim Nasby
On 8/29/15 5:52 PM, Jeff Janes wrote: ! Obtaining the necessary lock is done by the bufmgr routines ! LockBufferForCleanup() or ConditionalLockBufferForCleanup(). ! They first get an exclusive lock and then check to see if the shared pin ! count is currentl

Re: [HACKERS] [PATCH] SQL function to report log message

2015-08-31 Thread Jim Nasby
On 8/31/15 11:59 PM, Pavel Stehule wrote: The transformation: text -> error level is common task - and PLpgSQL it does in pl_gram.y. My idea is to add new function to error utils named "parse_error_level" and use it from PLpgSQL and from your code. Wouldn't it be better to create an ENUM of err

Re: [HACKERS] [PATCH] SQL function to report log message

2015-08-31 Thread Pavel Stehule
2015-09-01 6:59 GMT+02:00 Pavel Stehule : > > > 2015-08-31 20:43 GMT+02:00 dinesh kumar : > >> Hi, >> >> On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule >> wrote: >> >>> Hi >>> >>> I am starting to work review of this patch >>> >>> 2015-07-13 9:54 GMT+02:00 dinesh kumar : >>> Hi All, >>>

Re: [HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Tom Lane
Satoshi Nagayasu writes: > On 2015/09/01 13:41, Peter Geoghegan wrote: >> If you want to use the queryId field directly, which I recall you >> mentioning before, then that's harder. There is simply no contract >> among extensions for "owning" a queryId. But when the fingerprinting >> code is moved

Re: [HACKERS] [PATCH] SQL function to report log message

2015-08-31 Thread Pavel Stehule
2015-08-31 20:43 GMT+02:00 dinesh kumar : > Hi, > > On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule > wrote: > >> Hi >> >> I am starting to work review of this patch >> >> 2015-07-13 9:54 GMT+02:00 dinesh kumar : >> >>> Hi All, >>> >>> Greetings for the day. >>> >>> Would like to discuss on below

Re: [HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Satoshi Nagayasu
On 2015/09/01 13:41, Peter Geoghegan wrote: On Mon, Aug 31, 2015 at 9:29 PM, Satoshi Nagayasu wrote: BTW, I'm interested in improving the queryid portability now because I'd like to use it in other extensions. :) That's the reason why I'm looking at query jumbling here. Are you interested in

Re: [HACKERS] WIP: About CMake v2

2015-08-31 Thread Tom Lane
Noah Misch writes: > On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote: >> If it allows us to get rid of our custom MSVC scripts, it's a huge >> benefit, for sure -- that has been a huge pain in the neck since day >> one. > Moreover, I suggest beginning with a patch that replaces the

[HACKERS] Speed up Clog Access by increasing CLOG buffers

2015-08-31 Thread Amit Kapila
After reducing ProcArrayLock contention in commit (0e141c0fbb211bdd23783afa731e3eef95c9ad7a), the other lock which seems to be contentious in read-write transactions is CLogControlLock. In my investigation, I found that the contention is mainly due to two reasons, one is that while writing the tra

Re: [HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Peter Geoghegan
On Mon, Aug 31, 2015 at 9:29 PM, Satoshi Nagayasu wrote: > BTW, I'm interested in improving the queryid portability now because > I'd like to use it in other extensions. :) > That's the reason why I'm looking at query jumbling here. Are you interested in having the query fingerprinting/jumbling i

Re: [HACKERS] WIP: About CMake v2

2015-08-31 Thread Noah Misch
On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote: > If it allows us to get rid of our custom MSVC scripts, it's a huge > benefit, for sure -- that has been a huge pain in the neck since day > one. Moreover, I suggest beginning with a patch that replaces the src/tools/msvc build syste

Re: [HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Satoshi Nagayasu
On 2015/09/01 12:36, Peter Geoghegan wrote: On Mon, Aug 31, 2015 at 8:32 PM, Satoshi Nagayasu wrote: Why don't we use relation name (with looking up the catalog) on query jumbling? For performance reason? I think that there is a good case for preferring this behavior. While it is a little c

Re: [HACKERS] perlcritic

2015-08-31 Thread Michael Paquier
On Tue, Sep 1, 2015 at 12:57 PM, Peter Eisentraut wrote: > We now have 80+ Perl files in our tree, and it's growing. Some of those > files were originally written for Perl 4, and the coding styles and > quality are quite, uh, divergent. So I figured it's time to clean up > that code a bit. I ra

[HACKERS] Unicode mapping scripts cleanup

2015-08-31 Thread Peter Eisentraut
Here is a series of patches to clean up the Unicode mapping script business in src/backend/utils/mb/Unicode/. It overlaps with the perlcritic work that I recently wrote about, except that these pieces are not strictly related to Perl, but wrong comments, missing makefile pieces, and such. I disco

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Pavan Deolasee
On Tue, Sep 1, 2015 at 3:17 AM, Robert Haas wrote: > > > It seems to me that sharding consists of (1) breaking your data set up > into shards, (2) possibly replicating some of those shards onto > multiple machines, and then (3) being able to access the remote data > from local queries. As far a

[HACKERS] perlcritic

2015-08-31 Thread Peter Eisentraut
We now have 80+ Perl files in our tree, and it's growing. Some of those files were originally written for Perl 4, and the coding styles and quality are quite, uh, divergent. So I figured it's time to clean up that code a bit. I ran perlcritic over the tree and cleaned up all the warnings at leve

Re: [HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Peter Geoghegan
On Mon, Aug 31, 2015 at 8:32 PM, Satoshi Nagayasu wrote: > Why don't we use relation name (with looking up the catalog) > on query jumbling? For performance reason? I think that there is a good case for preferring this behavior. While it is a little confusing that pg_stat_statements does not chan

[HACKERS] pg_stat_statements query jumbling question

2015-08-31 Thread Satoshi Nagayasu
Hi, I have a question on jumbling queries in pg_stat_statements. I found that JumbleRangeTable() uses relation oid in RangeTblEntry. Obviously, this would result different queryid when the table gets re-created (dropped and created). Why don't we use relation name (with looking up the catalog)

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Etsuro Fujita
On 2015/09/01 9:54, Bruce Momjian wrote: On Mon, Aug 31, 2015 at 05:10:11PM -0700, Josh Berkus wrote: As far as (3) is concerned, why wouldn't we use the foreign data wrapper interface, and specifically postgres_fdw? That interface was designed for the explicit purpose of allowing access to rem

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Michael Paquier
On Tue, Sep 1, 2015 at 4:23 AM, Peter Eisentraut wrote: > On 8/31/15 9:13 AM, Andres Freund wrote: >> I'm just saying that we should strive to behave at least somewhat >> consistently, and change everything at once, not piecemal. Because the >> latter will not decrease the pain of migrating to a n

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Alvaro Herrera
Bruce Momjian wrote: > My hope is that many FDW improvements will benefit sharding and > non-sharding workloads, but I bet some improvements are going to be > sharding-specific. I would say we are still in the exploratory stage, > but based on the number of people who care about this feature and

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-31 Thread Peter Eisentraut
On 8/25/15 11:32 PM, Joe Conway wrote: > 1.) pg_controldata() function and pg_controldata view added I don't think dumping out whatever pg_controldata happens to print as a bunch of text fields is very sophisticated. We have functionality to compute with WAL positions, for example, and they won't

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-31 Thread Peter Eisentraut
On 8/24/15 9:50 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 08/23/2015 08:58 PM, Michael Paquier wrote: >>> I think that's a good thing to have, now I have concerns about making >>> this data readable for non-superusers. Cloud deployments of Postgres >>> are logically going to block the acc

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-31 Thread Peter Eisentraut
On 8/20/15 9:59 AM, Andrew Dunstan wrote: > The regression tests thus passed, but should not have. It occurred to me > that if we had a test like > > select pg_config('configure') ~ '--with-libxml' as has_xml; > > in the xml tests then this failure mode would be detected. This particular cas

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 05:10:11PM -0700, Josh Berkus wrote: > > As far as (3) is concerned, why > > wouldn't we use the foreign data wrapper interface, and specifically > > postgres_fdw? That interface was designed for the explicit purpose of > > allowing access to remote data sources, and a lot

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Marc Munro
On Mon, 2015-08-31 at 22:21 +, Robert Haas wrote: > It seems to me that sharding consists of (1) breaking your data set up > into shards, (2) possibly replicating some of those shards onto > multiple machines, and then (3) being able to access the remote data > from local queries. [...] I be

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Josh Berkus
On 08/31/2015 02:47 PM, Robert Haas wrote: > On Mon, Aug 31, 2015 at 4:16 PM, Josh Berkus wrote: >> First, let me put out there that I think the horizontal scaling project >> which has buy-in from the community and we're working on is infinitely >> better than the one we're not working on or is an

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread David E. Wheeler
On Aug 31, 2015, at 4:58 PM, Tom Lane wrote: > In any case, there is plenty of precedent for hard-coding knowledge about > specific version updates into pg_upgrade. The question here is whether > it's feasible to handle extensions that way. I think we could reasonably > expect to know about cas

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Tom Lane
Andrew Dunstan writes: > On 08/31/2015 07:32 PM, Bruce Momjian wrote: >> Still, I don't know how many people are doing this, but the right fix is >> to get the names of the modules that are superceeded and tell pg_upgrade >> to skip them. > I don't think this knowledge should be hardcoded in pg_u

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Andrew Dunstan
On 08/31/2015 07:32 PM, Bruce Momjian wrote: On Mon, Aug 31, 2015 at 07:28:00PM -0400, Andrew Dunstan wrote: On 08/31/2015 07:21 PM, David E. Wheeler wrote: On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: I think it would help if its noted somewhere in the document as it would have help

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Smitha Pamujula
pg_upgrade skipping the modules makes the most sense to me as well. On Mon, Aug 31, 2015 at 4:32 PM, Bruce Momjian wrote: > On Mon, Aug 31, 2015 at 07:28:00PM -0400, Andrew Dunstan wrote: > > > > > > On 08/31/2015 07:21 PM, David E. Wheeler wrote: > > >On Aug 31, 2015, at 4:20 PM, Bruce Momjian

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 07:28:00PM -0400, Andrew Dunstan wrote: > > > On 08/31/2015 07:21 PM, David E. Wheeler wrote: > >On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: > > > >>>I think it would help if its noted somewhere in the document as it would > >>>have > >>>helped us save some time un

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Sumedh Pathak
Hi Bruce, Sumedh from Citus Data here. > August, 2015: While speaking at SFPUG, Citus Data approached me about joining the FDW sharding team. They have been invited to the September 1 meeting, as have the XC and XL people. I'd like to add a clarification. We already tried the FDW APIs for pg_s

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Andrew Dunstan
On 08/31/2015 07:21 PM, David E. Wheeler wrote: On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: I think it would help if its noted somewhere in the document as it would have helped us save some time understanding why it was failing and why it was looking for json_build. The problem is tha

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread David E. Wheeler
On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: >> I think it would help if its noted somewhere in the document as it would have >> helped us save some time understanding why it was failing and why it was >> looking for json_build. > > The problem is that this is a rare case where you had an

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 04:03:20PM -0700, Smitha Pamujula wrote: > Thank you Bruce. So far installing it before have been working well so we will > continue with that plan.  > > I think it would help if its noted somewhere in the document as it would have > helped us save some time understanding w

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Smitha Pamujula
Thank you Bruce. So far installing it before have been working well so we will continue with that plan. I think it would help if its noted somewhere in the document as it would have helped us save some time understanding why it was failing and why it was looking for json_build. On Mon, Aug 31,

Re: [HACKERS] WIP: Access method extendability

2015-08-31 Thread Alexander Korotkov
Hackers, there is next revision of patches providing access method extendability. Now it's based on another patch which reworks access method interface. http://www.postgresql.org/message-id/capphfdsxwzmojm6dx+tjnpyk27kt4o7ri6x_4oswcbyu1rm...@mail.gmail.com Besides access method interface, major c

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Bruce Momjian
On Tue, Jul 14, 2015 at 09:48:59AM -0700, Smitha Pamujula wrote: > This error will go away only if I install the new json_build94. > > I was under the impression that we dont need to get the json_build > libraries for 94. But

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Robert Haas
On Mon, Aug 31, 2015 at 4:16 PM, Josh Berkus wrote: > First, let me put out there that I think the horizontal scaling project > which has buy-in from the community and we're working on is infinitely > better than the one we're not working on or is an underresourced fork. > So we're in agreement on

Re: [HACKERS] security labels on databases are bad for dump & restore

2015-08-31 Thread Bruce Momjian
On Tue, Jul 28, 2015 at 04:23:36PM -0300, Alvaro Herrera wrote: > Josh Berkus wrote: > > On 07/28/2015 11:58 AM, Robert Haas wrote: > > > I'd be strongly in favour of teaching GRANT, SECURITY LABEL, COMMENT > > >> ON DATABASE, etc to recognise CURRENT_DATABASE as a keyword. Then > > >> dumping them

Re: [HACKERS] Should \o mean "everything?"

2015-08-31 Thread David Fetter
On Mon, Aug 31, 2015 at 07:18:02PM +, Kevin Grittner wrote: > David Fetter wrote: > > > In a failed attempt to send the output of \pset to a pipe, I > > noticed that for reasons I find difficult to explain, not every > > output gets redirected with \o. > > > > At first blush, I'd consider thi

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Joshua D. Drake
On 08/31/2015 01:16 PM, Josh Berkus wrote: All, Bruce: I'm also going to pontificate that, for a future solution, we should not focus on write *IO*, but rather on CPU and RAM. The reason for this thinking is that, with the latest improvements in hardware and 9.5 improvements, it's increasingl

Re: [HACKERS] [PROPOSAL] Table Partition

2015-08-31 Thread My Life
> There is already a recent proposal on hackers about partition support in > PostgreSQL by Amit Langote. > You will find the thread at > http://www.postgresql.org/message-id/55d3093c.5010...@lab.ntt.co.jp. Actually, I have seen this design before, and it was not just a design, it has been im

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Josh Berkus
All, Bruce: First, let me put out there that I think the horizontal scaling project which has buy-in from the community and we're working on is infinitely better than the one we're not working on or is an underresourced fork. So we're in agreement on that. However, I think there's a lot of room f

Re: [HACKERS] Should \o mean "everything?"

2015-08-31 Thread David Fetter
On Mon, Aug 31, 2015 at 07:18:02PM +, Kevin Grittner wrote: > David Fetter wrote: > > > In a failed attempt to send the output of \pset to a pipe, I > > noticed that for reasons I find difficult to explain, not every > > output gets redirected with \o. > > > > At first blush, I'd consider thi

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Qingqing Zhou
On Mon, Aug 31, 2015 at 2:12 AM, Oleg Bartunov wrote: > > AFAIK, XC/XL has already some customers and that is an additional pressure > on their development team, which is now called X2. I don't exactly know how > internal Huawei's MPPDB is connected to XC/XL. > Huawei's MPPDB is based on PG-XC an

Re: [HACKERS] Should \o mean "everything?"

2015-08-31 Thread Kevin Grittner
David Fetter wrote: > In a failed attempt to send the output of \pset to a pipe, I > noticed that for reasons I find difficult to explain, not every > output gets redirected with \o. > > At first blush, I'd consider this inconsistency as a bug. > > What have I missed? The documentation says: |

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Peter Eisentraut
On 8/31/15 9:13 AM, Andres Freund wrote: > I'm just saying that we should strive to behave at least somewhat > consistently, and change everything at once, not piecemal. Because the > latter will not decrease the pain of migrating to a new model in a > relevant way while making the system harder to

Re: [HACKERS] Anybody have icc for IA64?

2015-08-31 Thread Alvaro Herrera
Tom Lane wrote: > After pushing 2c713d6e, I realized that the buildfarm isn't going to tell > me anything useful about it, because the change only makes a difference > for icc on ia64, and we have no such members in the buildfarm. (We've > got icc, and we've got ia64, but not both in the same plac

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Alexander Korotkov
On Mon, Aug 31, 2015 at 9:48 PM, Mason S wrote: > >> We also a bit disappointed by Huawei position about CSN patch, we hoped >> to use for our XTM. >> > > Disappointed in what way? Moving to some sort of CSN approach seems to > open things up for different future ideas. In the short term, it wo

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 02:48:31PM -0400, Mason S wrote: > I assume that future work around PG sharding probably would be more likely to > be accepted with the FDW approach. One could perhaps work on pushing down > joins, aggregates and order by, then look at any optimizations gained if code > is m

Re: [HACKERS] Is "WIN32" #defined in Cygwin builds?

2015-08-31 Thread Tom Lane
Andrew Dunstan writes: > On 08/31/2015 02:21 PM, Tom Lane wrote: >> I started wondering about $subject because we are fairly schizophrenic >> about whether we believe this. > No, and we've made sure not to do that ourselves, or at least I hope we > have. OK, thanks. I was wondering whether I'd

[HACKERS] Anybody have icc for IA64?

2015-08-31 Thread Tom Lane
After pushing 2c713d6e, I realized that the buildfarm isn't going to tell me anything useful about it, because the change only makes a difference for icc on ia64, and we have no such members in the buildfarm. (We've got icc, and we've got ia64, but not both in the same place.) It's fairly worriso

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Mason S
> > > We also a bit disappointed by Huawei position about CSN patch, we hoped > to use for our XTM. > Disappointed in what way? Moving to some sort of CSN approach seems to open things up for different future ideas. In the short term, it would mean replacing potentially large snapshots and longe

Re: [HACKERS] Is "WIN32" #defined in Cygwin builds?

2015-08-31 Thread Andrew Dunstan
On 08/31/2015 02:21 PM, Tom Lane wrote: I started wondering about $subject because we are fairly schizophrenic about whether we believe this. For example, only a few lines apart in dirmod.c, there are #if defined(WIN32) || defined(__CYGWIN__) #if defined(WIN32) && !defined(__CYGWIN__) Presu

Re: [HACKERS] [PATCH] SQL function to report log message

2015-08-31 Thread dinesh kumar
Hi, On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule wrote: > Hi > > I am starting to work review of this patch > > 2015-07-13 9:54 GMT+02:00 dinesh kumar : > >> Hi All, >> >> Greetings for the day. >> >> Would like to discuss on below feature here. >> >> Feature: >> Having an SQL function, to

[HACKERS] Is "WIN32" #defined in Cygwin builds?

2015-08-31 Thread Tom Lane
I started wondering about $subject because we are fairly schizophrenic about whether we believe this. For example, only a few lines apart in dirmod.c, there are #if defined(WIN32) || defined(__CYGWIN__) #if defined(WIN32) && !defined(__CYGWIN__) Presumably, one of these could be simplified, but

[HACKERS] Should \o mean "everything?"

2015-08-31 Thread David Fetter
Folks, In a failed attempt to send the output of \pset to a pipe, I noticed that for reasons I find difficult to explain, not every output gets redirected with \o. At first blush, I'd consider this inconsistency as a bug. What have I missed? Cheers, David. -- David Fetter http://fetter.org/ P

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 12:35 PM, Pavel Stehule wrote: > >>> >>> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >>> >> >> Ah, thanks! Somehow I've missed this mail. You didn't add the patch to >> a commitfest back then I think? >> > > I h

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread YUriy Zhuravlev
On Monday 31 August 2015 17:48:50 Andres Freund wrote: > Additionally it's, for default pgbench, really mostly a bottlneck after > GetSnapshotData() is fixed. You can make it a problem much earlier if > you have index nested loops over a lot of rows. 100 000 000 is a lot? Simple select query from p

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-31 Thread Andres Freund
On 2015-08-19 15:14:03 -0700, Josh Berkus wrote: > Asking users to refactor their applications to add OFFSET 0 is a bit > painful, if we could take care of it via a backwards-compatibility GUC. > We have many users who are specifically using the CTE optimization > barrier to work around planner fa

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-31 Thread Qingqing Zhou
On Thu, Aug 27, 2015 at 1:01 PM, Qingqing Zhou wrote: > On Wed, Aug 26, 2015 at 5:28 PM, Tom Lane wrote: >> >> After looking at the code a bit, IMO the most reasonable thing to do is to >> include this transformation in inline_set_returning_functions(), perhaps >> renaming it to something like in

Re: [HACKERS] Missing latex-longtable value

2015-08-31 Thread Bruce Momjian
On Tue, Jul 7, 2015 at 03:21:50PM -0400, Bruce Momjian wrote: > On Tue, Jul 7, 2015 at 01:05:09PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Tue, Jul 7, 2015 at 11:48:13AM -0400, Tom Lane wrote: > > >> It's a bug. Back-patch as needed. > > > > > Doesn't that cause translation s

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread YUriy Zhuravlev
On Monday 31 August 2015 17:54:17 Tomas Vondra wrote: > So does this mean it's worth testing the patch on x86 > or not, in it's current state? Its realy intersting. But you need have true 64 cores without HT. (32 core +HT not have effect) -- YUriy Zhuravlev Postgres Professional: http://www.po

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Dmitry Vasilyev
We did not got any affect on core 64 with smt = 8, and we not have a 64 -cpu x86 machine with disable HT feature. You can set scale > 1000 and with shared_buffers >> size of index pgbench_accounts_pkey. You can also increase the concurrency: not only access top of b-tree index, but also to a speci

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Andres Freund
On 2015-08-31 17:54:17 +0200, Tomas Vondra wrote: > [scratches head] So does this mean it's worth testing the patch on x86 or > not, in it's current state? You could try if you're interested. But I don't think it's super meaningful. The patch is just a POC and rather widely incorrect. Don't get m

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Tomas Vondra
On 08/31/2015 05:48 PM, Andres Freund wrote: On 2015-08-31 17:43:08 +0200, Tomas Vondra wrote: Well, I could test the patch on a x86 machine with 4 sockets (64 cores), but I wonder whether it makes sense at this point, as the patch really is not correct (judging by what Andres says). Additio

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Andres Freund
On 2015-08-31 17:43:08 +0200, Tomas Vondra wrote: > Well, I could test the patch on a x86 machine with 4 sockets (64 cores), but > I wonder whether it makes sense at this point, as the patch really is not > correct (judging by what Andres says). Additionally it's, for default pgbench, really mostl

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Tomas Vondra
On 08/31/2015 12:54 PM, YUriy Zhuravlev wrote: Hello hackers Recently, we were given access to the test server is IBM, 9119-MHE with 8 CPUs * 8 cores * 8 threads. We decided to take advantage of this and to find bottlenecks for read scalability (pgbench -S). All detail you can read here: http

Re: [HACKERS] [PROPOSAL] Effective storage of duplicates in B-tree index.

2015-08-31 Thread Tomas Vondra
Hi, On 08/31/2015 09:41 AM, Anastasia Lubennikova wrote: Hi, hackers! I'm going to begin work on effective storage of duplicate keys in B-tree index. The main idea is to implement posting lists and posting trees for B-tree index pages as it's already done for GIN. In a nutshell, effective stori

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Andres Freund
On 2015-08-31 10:48:01 -0400, Tom Lane wrote: > The problem with this proposal is that it will add far more bloat of > the latter sort than currently-useful information; and the ratio will > get worse over time. If we add that information in sane way we should be able to remove it automatically af

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread David G. Johnston
On Mon, Aug 31, 2015 at 10:39 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane wrote: > >> >> It'll be a real >> mess if we do that for everything. >> > > I share the fear that it could become messy, but it doesn't necessary > *have to* be

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Tom Lane
"Shulgin, Oleksandr" writes: > On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane wrote: >> TBH, I think this is a horrid idea. We occasionally manually add remarks >> like "since version x.y, Postgres does this". Inevitably, that just bulks >> up the documentation; and it starts to look seriously silly

Re: [HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread Tom Lane
Kevin Grittner writes: > Pavel Raiskup wrote: >> It's been reported [1] that postmaster fails to start against staled >> postmaster.pid after (e.g.) power outage on Fedora, > Was the other newly started process another PostgreSQL cluster? > Was it launched under the same OS user? Yes, that's wh

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane wrote: > "Shulgin, Oleksandr" writes: > > I often find it pity that our docs are missing any information on since > > when a certain GUC setting, SQL-level command or function was introduced. > > It would be nice if we could make a script that would pars

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-31 Thread Rahila Syed
Hello, On Jul 16, 2015 1:48 AM, "Rahila Syed" wrote: > > Hello, > > Please find attached updated patch >with an interface to calculate command progress in pgstat.c. This interface currently implements VACUUM progress tracking . I have added this patch to CommitFest 2015-09. It is marked as Waiti

Re: [HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread David G. Johnston
On Mon, Aug 31, 2015 at 10:20 AM, Kevin Grittner wrote: > Pavel Raiskup wrote: > > > It's been reported [1] that postmaster fails to start against staled > > postmaster.pid after (e.g.) power outage on Fedora, its due to init > system > > parallelism and "some" other newly started process can al

Re: [HACKERS] Buildfarm failure from overly noisy warning message

2015-08-31 Thread Tom Lane
Jeff Janes writes: > On Tue, Jul 28, 2015 at 2:38 PM, Tom Lane wrote: >> Rather than remove the "sending signal" elog entirely, I reduced it to >> DEBUG1; that will avoid log chatter for normal cases but the info can >> still be obtained at need. > This part doesn't seem right to me. Now the au

Re: [HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread Kevin Grittner
Pavel Raiskup wrote: > It's been reported [1] that postmaster fails to start against staled > postmaster.pid after (e.g.) power outage on Fedora, its due to init system > parallelism and "some" other newly started process can already have allocated > the same PID as the old postmaster had -- and

[HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread Pavel Raiskup
This is most likely just a request for brainstorm. It's been reported [1] that postmaster fails to start against staled postmaster.pid after (e.g.) power outage on Fedora, its due to init system parallelism and "some" other newly started process can already have allocated the same PID as the old p

[HACKERS] snapshot too old, configured by time

2015-08-31 Thread Kevin Grittner
As discussed when the "proof of concept" patch was submitted during 9.5 development, here is a version intended to be considered for commit to 9.6, with the following changes: 1. It is configured using time rather than number of transactions. Not only was there unanimous agreement here that this w

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Michael Paquier
On Mon, Aug 31, 2015 at 10:31 PM, Andres Freund wrote: > On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: >> Is it correct to switch 2015-09 commitfest to inprogress now? No, this was not correct, it is expected to begin tomorrow. > Yea, isn't it only starting the 15th? If my memory does

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Tom Lane
"Shulgin, Oleksandr" writes: > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-level command or function was introduced. > It would be nice if we could make a script that would parse the sgml files > and for every symbol it finds it would a

Re: [HACKERS] WIP: About CMake v2

2015-08-31 Thread YUriy Zhuravlev
Thanks all hackers. I have not heard of fundamental problems and continue its development. :) -- YUriy Zhuravlev Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 3:30 PM, Andres Freund wrote: > Hi, > > On 2015-08-31 13:06:04 +0200, Shulgin, Oleksandr wrote: > > I often find it pity that our docs are missing any information on since > > when a certain GUC setting, SQL-level command or function was introduced. > > Same here. Not sure

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 3:17 PM, Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > 31.08.2015 14:06, Shulgin, Oleksandr пишет: > > Hello, > > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-level command or function was introdu

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Andreas Karlsson
On 08/31/2015 03:34 PM, Alexander Korotkov wrote: +1 for making these dates more explicit Yeah, I think being explicit would make life easier for newcomers. Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgres

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Magnus Hagander
On Mon, Aug 31, 2015 at 3:34 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Mon, Aug 31, 2015 at 4:31 PM, Andres Freund wrote: > >> On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: >> > Is it correct to switch 2015-09 commitfest to inprogress now? >> >> Yea, isn't it only

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Alexander Korotkov
On Mon, Aug 31, 2015 at 4:31 PM, Andres Freund wrote: > On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: > > Is it correct to switch 2015-09 commitfest to inprogress now? > > Yea, isn't it only starting the 15th? AFICS, on the last developer meeting it was decided to start commitfests in

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Andres Freund
On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: > Is it correct to switch 2015-09 commitfest to inprogress now? Yea, isn't it only starting the 15th? Can we add an option to display days in the CF app? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Andres Freund
Hi, On 2015-08-31 13:06:04 +0200, Shulgin, Oleksandr wrote: > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-level command or function was introduced. Same here. Not sure how to display it without getting disturbing the 'flow' of the docs

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Alexander Korotkov
On Mon, Aug 31, 2015 at 3:16 PM, Michael Paquier wrote: > On Mon, Aug 31, 2015 at 9:13 PM, Magnus Hagander wrote: > > Anyway - that CF is closed, but we seem to not have *any* CF open at this > > point. Should we not make 2015-11 open? > > Yeah, switched. Is it correct to switch 2015-09 commitf

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Anastasia Lubennikova
31.08.2015 14:06, Shulgin, Oleksandr пишет: Hello, I often find it pity that our docs are missing any information on since when a certain GUC setting, SQL-level command or function was introduced. Clicking through the "this page in other versions" links at the top of a webpage does help, bu

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Andres Freund
On 2015-08-31 09:06:27 -0400, Stephen Frost wrote: > Perhaps it really isn't moving the bar all that much but at least for a > number of our users, it's increasing what they have to be worrying about > ("well, we knew usernames were an issue, but now we also have to worry > about system usersnames

  1   2   >