Re: [HACKERS] updated emacs configuration

2013-06-14 Thread Daniel Farina
On Thu, Jun 13, 2013 at 6:27 PM, Peter Eisentraut pete...@gmx.net wrote: First, I propose adding a .dir-locals.el file to the top-level directory with basic emacs settings. These get applied automatically. This especially covers the particular tab and indentation settings that PostgreSQL

Re: [HACKERS] [PATCH] Add transforms feature

2013-06-14 Thread Cédric Villemain
Peter Eisentraut pete...@gmx.net a écrit : A transform is an SQL object that supplies to functions for converting between data types and procedural languages. For example, a transform could arrange that hstore is converted to an appropriate hash or dictionary object in PL/Perl or PL/Python.

[HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-14 Thread Kyotaro HORIGUCHI
Hello, Postgresql estimates the number of live tuples after the vacuum has left some buffers unscanned. This estimation does well for most cases, but makes completely different result with a strong imbalance of tuple density. For example, create table t (a int, b int); insert into t (select a,

[HACKERS] Add visibility map information to pg_freespace.

2013-06-14 Thread Kyotaro HORIGUCHI
Helle, I've added visibility map information to pg_freespace for my utility. This looks like this, postgres=# select * from pg_freespace('t'::regclass); blkno | avail | all_visible ---+---+- 0 | 7424 | t 1 | 7424 | t 2 | 7424 | t 3 | 7424 | t 4

Re: [HACKERS] Improvement of checkpoint IO scheduler for stable transaction responses

2013-06-14 Thread KONDO Mitsumasa
(2013/06/12 23:07), Robert Haas wrote: On Mon, Jun 10, 2013 at 3:48 PM, Simon Riggs si...@2ndquadrant.com wrote: On 10 June 2013 11:51, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I create patch which is improvement of checkpoint IO scheduler for stable transaction responses. Looks

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-14 Thread Kyotaro HORIGUCHI
Sorry, I made an mistake. Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp: Overall algorithm could be illistrated as below, - summing up used bytes, max offnum(PageGetMaxOffsetNumber), Not max offnum, the number of linp's used after page vacuum. maximum free bytes for tuple data ,

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Benedikt Grundmann
On Fri, Jun 14, 2013 at 10:11 AM, Samrat Revagade revagade.sam...@gmail.com wrote: Hello, We have already started a discussion on pgsql-hackers for the problem of taking fresh backup during the failback operation here is the link for that:

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Samrat Revagade
That will not happen if there is inconsistency in between both the servers. Please refer to the discussions on the link provided in the first post: http://www.postgresql.org/message-id/caf8q-gxg3pqtf71nvece-6ozraew5pwhk7yqtbjgwrfu513...@mail.gmail.com Regards, Samrat Revgade

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Heikki Linnakangas
On 14.06.2013 12:11, Samrat Revagade wrote: We have already started a discussion on pgsql-hackers for the problem of taking fresh backup during the failback operation here is the link for that:

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Pavan Deolasee
On Fri, Jun 14, 2013 at 4:12 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Robert Haas pointed out in that thread that it has a problem with hint bits that are not WAL-logged, I liked that tool a lot until Robert pointed out the above problem. I thought this is a show stopper

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Pavan Deolasee
On Fri, Jun 14, 2013 at 2:51 PM, Benedikt Grundmann bgrundm...@janestreet.com wrote: A alternative proposal (which will probably just reveal my lack of understanding about what is or isn't possible with WAL). Provide a way to restart the master so that it rolls back the WAL changes that the

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Heikki Linnakangas
On 14.06.2013 14:06, Pavan Deolasee wrote: On Fri, Jun 14, 2013 at 4:12 PM, Heikki Linnakangashlinnakan...@vmware.com wrote: Robert Haas pointed out in that thread that it has a problem with hint bits that are not WAL-logged, I liked that tool a lot until Robert pointed out the above

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Greg Stark
On Fri, Jun 14, 2013 at 12:20 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: For pg_rewind, we wouldn't actually need a full-page image for hint bit updates, just a small record saying hey, I touched this page. And you'd only need to write that the first time a page is touched after a

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Marko Kreen
On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum across a number of rows. This is something I've wished for a number of times. I think the primary use case is to

[HACKERS] Issue with PGC_BACKEND parameters

2013-06-14 Thread Amit Kapila
I had observed one problem with PGC_BACKEND parameters while testing patch for ALTER SYSTEM command. Problem statement: If I change PGC_BACKEND parameters directly in postgresql.conf and then do pg_reload_conf() and reconnect, it will still show the old value. Detailed steps

Re: [HACKERS] SPGist triple parity concept doesn't work

2013-06-14 Thread Teodor Sigaev
Anyway I now think that we might be better off with the other idea of abandoning an insertion and retrying if we get a lock conflict. done, look at the patch. I was faced with the fact that my mail is considered spam by postgresql.org, so I repeat some hthoughts from previous mail: I

Re: [HACKERS] [PATCH] Add transforms feature

2013-06-14 Thread Peter Eisentraut
On 6/14/13 3:46 AM, Cédric Villemain wrote: You kept PGXS style makefile... I know, but that's a separate issue that hasn't been decided yet. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

2013-06-14 Thread Peter Eisentraut
On 6/13/13 9:20 PM, amul sul wrote: Agree, only if we consider these contrib module is always gonna deployed with the postgresql. But, what if user going to install such module elsewhere i.e. not from contrib directory of pg source. Why would anyone do that? -- Sent via pgsql-hackers

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

2013-06-14 Thread Amit Langote
On Fri, Jun 14, 2013 at 9:35 PM, Peter Eisentraut pete...@gmx.net wrote: On 6/13/13 9:20 PM, amul sul wrote: Agree, only if we consider these contrib module is always gonna deployed with the postgresql. But, what if user going to install such module elsewhere i.e. not from contrib directory

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: Well, time will tell I guess. The biggest overhead with the checksums is exactly the WAL-logging of hint bits. Refresh my memory as to why we need to WAL-log hints for checksumming? I just had my nose in the part of the checksum patch that

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Tom Lane
Marko Kreen mark...@gmail.com writes: On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum across a number of rows. It's more efficient to calculate per-row md5,

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Amit Kapila
On Friday, June 14, 2013 2:42 PM Samrat Revagade wrote: Hello, We have already started a discussion on pgsql-hackers for the problem of taking fresh backup during the failback operation here is the link for that:   http://www.postgresql.org/message-id/CAF8Q-Gxg3PQTf71NVECe-6OzRaew5pWhk7yQtb

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Heikki Linnakangas
On 14.06.2013 16:08, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: Well, time will tell I guess. The biggest overhead with the checksums is exactly the WAL-logging of hint bits. Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: 1.

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 14.06.2013 16:08, Tom Lane wrote: Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: So it's not that we actually need to log the individual hint bit changes, it's that we need to WAL-log a full page image

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Benedikt Grundmann
On Fri, Jun 14, 2013 at 2:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Andres Freund
On 2013-06-14 09:08:15 -0400, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: Well, time will tell I guess. The biggest overhead with the checksums is exactly the WAL-logging of hint bits. Refresh my memory as to why we need to WAL-log hints for checksumming? I just

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

2013-06-14 Thread Andrew Dunstan
On 06/14/2013 08:35 AM, Peter Eisentraut wrote: On 6/13/13 9:20 PM, amul sul wrote: Agree, only if we consider these contrib module is always gonna deployed with the postgresql. But, what if user going to install such module elsewhere i.e. not from contrib directory of pg source. Why would

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Heikki Linnakangas
On 14.06.2013 16:21, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: On 14.06.2013 16:08, Tom Lane wrote: Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: So it's not that we actually need to log the individual hint bit changes, it's

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Andres Freund
On 2013-06-14 09:21:52 -0400, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 14.06.2013 16:08, Tom Lane wrote: Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: So it's not that we actually need to log the individual hint bit

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Marko Kreen mark...@gmail.com writes: On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum across a number of rows.

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Heikki Linnakangas
On 14.06.2013 16:15, Andres Freund wrote: On 2013-06-14 09:08:15 -0400, Tom Lane wrote: I just had my nose in the part of the checksum patch that tediously copies entire pages out of shared buffers to avoid possible instability of the hint bits while we checksum and write the page. I am

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Andrew Dunstan
On 06/14/2013 09:40 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Marko Kreen mark...@gmail.com writes: On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Andres Freund
On 2013-06-14 09:21:52 -0400, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 14.06.2013 16:08, Tom Lane wrote: Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: So it's not that we actually need to log the individual hint bit

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Heikki Linnakangas
On 14.06.2013 17:01, Andres Freund wrote: At this point we have a page that has been modified without an FPI. But it's not marked dirty, so it won't be written out without further cause. Which might be fine since there's no cause to write out the page and there probably won't be anyone doing

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Andres Freund
On 2013-06-14 16:58:38 +0300, Heikki Linnakangas wrote: On 14.06.2013 16:15, Andres Freund wrote: On 2013-06-14 09:08:15 -0400, Tom Lane wrote: I just had my nose in the part of the checksum patch that tediously copies entire pages out of shared buffers to avoid possible instability of the

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

2013-06-14 Thread Greg Smith
On 6/12/13 3:19 AM, Fabien COELHO wrote: If you are still worried: if you run the very same command without throttling and measure the same latency, does the same thing happens at the end? My guess is that it should be yes. If it is no, I'll try out pgbench-tools. It looks like it happens

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Stephen Frost
* Andrew Dunstan (and...@dunslane.net) wrote: I'd rather go the other way, processing the records without having to process them otherwise at all. Turning things into text must slow things down, surely. That's certainly an interesting idea also.. Thanks, Stephen

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Dean Rasheed
On 14 June 2013 14:14, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum across a number

Re: [HACKERS] Add visibility map information to pg_freespace.

2013-06-14 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: Helle, I've added visibility map information to pg_freespace for my utility. This makes sense to me. I only lament the fact that this makes the module a misnomer. Do we want to 1) rename the module (how inconvenient), 2) create a separate module for this (surely

[HACKERS] Buildfarm client 4.11 released

2013-06-14 Thread Andrew Dunstan
Version 4.11 of the PostgreSQL Buildfarm client has been released. It can be downloaded from http://www.pgbuildfarm.org/downloads/releases/build-farm-4_11.tgz Changes since 4.10: * Turn down module cleanup verbosity * Add check for rogue postmasters. * Add pseudo-branch targets

Re: [HACKERS] Add visibility map information to pg_freespace.

2013-06-14 Thread Andres Freund
On 2013-06-14 10:22:19 -0400, Alvaro Herrera wrote: Kyotaro HORIGUCHI wrote: Helle, I've added visibility map information to pg_freespace for my utility. This makes sense to me. +1 I only lament the fact that this makes the module a misnomer. Do we want to 1) rename the module

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Greg Stark
On Fri, Jun 14, 2013 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: So it's not that we actually need to log the individual hint bit changes, it's that we need to WAL-log a full page image on the first update after a checkpoint, so as to recover from torn-page cases. Which one are we doing?

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Fri, Jun 14, 2013 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: So it's not that we actually need to log the individual hint bit changes, it's that we need to WAL-log a full page image on the first update after a checkpoint, so as to recover from torn-page

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: On 14 June 2013 14:14, Tom Lane t...@sss.pgh.pa.us wrote: Personally I'd be a bit inclined to xor the per-row md5's rather than sum them, but that's a small matter. But this would be a much riskier thing to do with a single column, because if you

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Dean Rasheed
On 14 June 2013 15:19, Stephen Frost sfr...@snowman.net wrote: * Andrew Dunstan (and...@dunslane.net) wrote: I'd rather go the other way, processing the records without having to process them otherwise at all. Turning things into text must slow things down, surely. That's certainly an

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Andres Freund
On 2013-06-14 15:49:31 +0100, Dean Rasheed wrote: On 14 June 2013 15:19, Stephen Frost sfr...@snowman.net wrote: * Andrew Dunstan (and...@dunslane.net) wrote: I'd rather go the other way, processing the records without having to process them otherwise at all. Turning things into text must

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Hannu Krosing
On 06/14/2013 04:47 PM, Tom Lane wrote: Dean Rasheed dean.a.rash...@gmail.com writes: On 14 June 2013 14:14, Tom Lane t...@sss.pgh.pa.us wrote: Personally I'd be a bit inclined to xor the per-row md5's rather than sum them, but that's a small matter. But this would be a much riskier thing to

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-14 Thread Jeff Davis
On Thu, 2013-06-13 at 20:09 -0400, Tom Lane wrote: What I propose we do about this is reduce backend/storage/page/checksum.c to something like #include postgres.h #include storage/checksum.h #include storage/checksum_impl.h moving all the code currently in the file into a new .h file.

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-14 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: I have a question about the commit though: shouldn't both functions be static if they are in a .h file? Otherwise, it could lead to naming conflicts. I suppose it's wrong to include the implementation file twice, but it still might be confusing if someone

[HACKERS] refresh materialized view concurrently

2013-06-14 Thread Kevin Grittner
Attached is a patch for REFRESH MATERIALIZED VIEW CONCURRENTLY for 9.4 CF1.  The goal of this patch is to allow a refresh without interfering with concurrent reads, using transactional semantics. It is my hope to get this committed during this CF to allow me to focus on incremental maintenance

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-14 Thread Andres Freund
On 2013-06-14 11:59:04 -0400, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: I have a question about the commit though: shouldn't both functions be static if they are in a .h file? Otherwise, it could lead to naming conflicts. I suppose it's wrong to include the implementation file

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Jeff Davis
On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: Jeff Davis has a patch pending (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std flag down to MarkBufferDirtyHint() for exactly that reason. I thought we were on track committing that, but rereading the thread it

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Andres Freund
On 2013-06-14 09:21:12 -0700, Jeff Davis wrote: On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: Jeff Davis has a patch pending (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std flag down to MarkBufferDirtyHint() for exactly that reason. I thought we were on

[HACKERS] pg_restore -l with a directory archive

2013-06-14 Thread Fujii Masao
Hi, When I ran pg_restore -l with the directory arhicve input, I found that its format is wrongly reported as UNKNOWN. $ pg_dump -F d -f hoge $ pg_restore -l hoge ; ; Archive created at Sat Jun 15 01:38:14 2013 ; dbname: postgres ; TOC Entries: 9 ; Compression: -1 ; Dump Version:

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Dean Rasheed
On 14 June 2013 16:09, Hannu Krosing ha...@2ndquadrant.com wrote: What skytools/pgq/londiste uses for comparing tables on master and slave is query like this select sum(hashtext(t.*::text)) from yourtable t; This is non-modulo sum and does not use md5 but relies on whatever the hashtext()

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-14 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-06-14 11:59:04 -0400, Tom Lane wrote: Ah, you are right, I forgot the #ifndef CHECKSUM_IMPL_H dance. Will fix in a bit. That won't help against errors if it's included in two different files/translation units though. Good point, but

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-14 Thread Jeff Davis
On Sat, 2013-05-25 at 13:55 -0500, Jon Nelson wrote: Ack. I've revised the patch to always have the GUC (for now), default to false, and if configure can't find posix_fallocate (or the user disables it by way of pg_config_manual.h) then it remains a GUC that simply can't be changed. Why have

Re: [HACKERS] request a new feature in fuzzystrmatch

2013-06-14 Thread Liming Hu
On Tue, Jun 11, 2013 at 3:19 PM, Liming Hu dawnin...@gmail.com wrote: On Tue, Jun 11, 2013 at 2:56 PM, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/11/2013 02:23 PM, Liming Hu wrote: On Tue, Jun 11, 2013 at 1:57 PM, Alvaro Herrera

Re: [HACKERS] request a new feature in fuzzystrmatch

2013-06-14 Thread David Fetter
On Fri, Jun 14, 2013 at 10:08:24AM -0700, Liming Hu wrote: On Tue, Jun 11, 2013 at 3:19 PM, Liming Hu dawnin...@gmail.com wrote: On Tue, Jun 11, 2013 at 2:56 PM, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/11/2013 02:23 PM, Liming Hu wrote:

Re: [HACKERS] request a new feature in fuzzystrmatch

2013-06-14 Thread Joshua D. Drake
On 06/14/2013 10:11 AM, David Fetter wrote: ok, thanks, I will wait. Hi Joe, Do you have some time in the weekend to help me submit the patch? Thanks, Liming Liming, Is your git skill good enough to create a patch vs. PostgreSQL's git master? If so, send that and once it's hit the

Re: [HACKERS] request a new feature in fuzzystrmatch

2013-06-14 Thread David Fetter
On Fri, Jun 14, 2013 at 10:14:14AM -0700, Joshua D. Drake wrote: On 06/14/2013 10:11 AM, David Fetter wrote: ok, thanks, I will wait. Hi Joe, Do you have some time in the weekend to help me submit the patch? Thanks, Liming Liming, Is your git skill good enough to create a patch

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-14 Thread Greg Smith
On 6/14/13 1:06 PM, Jeff Davis wrote: Why have a GUC here at all? Perhaps this was already discussed, and I missed it? Is it just for testing purposes, or did you intend for it to be in the final version? You have guessed correctly! I suggested it stay in there only to make review

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-14 Thread Jeff Davis
On Tue, 2013-06-11 at 12:58 -0400, Stephen Frost wrote: My main question is really- would this be useful for extending *relations*? Apologies if it's already been discussed; I do plan to go back and read the threads about this more fully, but I wanted to voice my support for using

[HACKERS] another error perhaps to be enhanced

2013-06-14 Thread Joshua D. Drake
ERROR: index foo_idx We should probably add the schema. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579 PostgreSQL Support, Training, Professional Services and Development High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc For my dreams of your image

Re: [HACKERS] Add visibility map information to pg_freespace.

2013-06-14 Thread Peter Geoghegan
On Fri, Jun 14, 2013 at 7:23 AM, Andres Freund and...@2ndquadrant.com wrote: 3). All the others seem to inflict unneccesary pain for not all that much gain. +1. You might want to add a historical note about the name to the pg_freespace documentation, though. -- Peter Geoghegan -- Sent via

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

2013-06-14 Thread Greg Smith
I don't have this resolved yet, but I think I've identified the cause. Updating here mainly so Fabien doesn't duplicate my work trying to track this down. I'm going to keep banging at this until it's resolved now that I got this far. Here's a slow transaction: 1371226017.568515 client 1

Re: [HACKERS] another error perhaps to be enhanced

2013-06-14 Thread Michael Glaesemann
On Jun 14, 2013, at 13:38, Joshua D. Drake j...@commandprompt.com wrote: ERROR: index foo_idx We should probably add the schema. I've noticed similar issues with functions. I'd like to see those schema-qualified as well. JD -- Command Prompt, Inc. -

Re: [HACKERS] another error perhaps to be enhanced

2013-06-14 Thread Peter Geoghegan
I think you'll need to better describe what you mean here. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] another error perhaps to be enhanced

2013-06-14 Thread Joshua D. Drake
On 06/14/2013 10:47 AM, Peter Geoghegan wrote: I think you'll need to better describe what you mean here. postgres=# create schema foo; CREATE SCHEMA postgres=# create schema bar; CREATE SCHEMA postgres=# create table foo.foo(id serial); NOTICE: CREATE TABLE will create implicit sequence

Re: [HACKERS] another error perhaps to be enhanced

2013-06-14 Thread Peter Geoghegan
On Fri, Jun 14, 2013 at 10:54 AM, Joshua D. Drake j...@commandprompt.com wrote: Now, with the error previously shown, which one_idx needs to be reindexed? Well, you didn't show an actual error message. But if you \set VERBOSITY verbose within psql while connected to a 9.3 server, you'll get

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-14 Thread Jeff Davis
On Fri, 2013-06-14 at 13:21 -0400, Greg Smith wrote: I'm planning to duplicate Jon's test program on a few machines here, and then see if that turns into a useful latency improvement for clients. I'm trying to get this pgbench rate limit stuff working first though, because one of the tests

Re: [HACKERS] another error perhaps to be enhanced

2013-06-14 Thread Joshua D. Drake
On 06/14/2013 11:01 AM, Peter Geoghegan wrote: On Fri, Jun 14, 2013 at 10:54 AM, Joshua D. Drake j...@commandprompt.com wrote: Now, with the error previously shown, which one_idx needs to be reindexed? Well, you didn't show an actual error message. ERROR: index foo_idx Is not an error

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

2013-06-14 Thread Fabien COELHO
pgbench already has a \sleep command, and the way that delay is handled happens inside threadRun() instead. The pausing of the rate limit throttle needs to operate in the same place. It does operate at the same place. The throttling is performed by inserting a sleep first thing when

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

2013-06-14 Thread Josh Berkus
On 06/12/2013 02:03 PM, Joshua D. Drake wrote: What concerns me is we seem to be trying to make this easy. It isn't supposed to be easy. This is hard stuff. Smart people built it and it takes a smart person to run it. When did it become a bad thing to be something that smart people need to

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

2013-06-14 Thread Joshua D. Drake
On 06/14/2013 11:16 AM, Josh Berkus wrote: On 06/12/2013 02:03 PM, Joshua D. Drake wrote: What concerns me is we seem to be trying to make this easy. It isn't supposed to be easy. This is hard stuff. Smart people built it and it takes a smart person to run it. When did it become a bad thing

Re: [HACKERS] SPGist triple parity concept doesn't work

2013-06-14 Thread Tom Lane
Teodor Sigaev teo...@sigaev.ru writes: Anyway I now think that we might be better off with the other idea of abandoning an insertion and retrying if we get a lock conflict. done, look at the patch. Looks good, committed with some cosmetic adjustments. We definetly need new idea of locking

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

2013-06-14 Thread Fabien COELHO
Hello Greg, I think that the weirdness really comes from the way transactions times are measured, their interactions with throttling, and latent bugs in the code. One issue is that the throttling time was included in the measure, but not the first time because txn_begin is not set at the

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

2013-06-14 Thread Greg Smith
On 6/14/13 3:50 PM, Fabien COELHO wrote: I think that the weirdness really comes from the way transactions times are measured, their interactions with throttling, and latent bugs in the code. measurement times, no; interactions with throttling, no. If it was either of those I'd have finished

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

2013-06-14 Thread Fabien COELHO
I think that the weirdness really comes from the way transactions times are measured, their interactions with throttling, and latent bugs in the code. measurement times, no; interactions with throttling, no. If it was either of those I'd have finished this off days ago. Latent bugs,

[HACKERS] [RFC] Minmax indexes

2013-06-14 Thread Alvaro Herrera
Hi, This is a preliminary proposal for Minmax indexes. I'm experimenting with the code, but it's too crude to post yet, so here's a document explaining what they are and how they work, so that reviewers can poke holes to have the design improved. My intention is to have a patch to show for CF2,

Re: [HACKERS] single-user vs standalone in docs and messages

2013-06-14 Thread Robert Haas
On Thu, Jun 13, 2013 at 6:10 PM, Jeff Janes jeff.ja...@gmail.com wrote: Some places in the docs and elog hints refer to standalone backends, while the official name as used in app-postgres.html is single-user mode, and in fact standalone does not appear on that page. This tries to standardize

[HACKERS] stray SIGALRM

2013-06-14 Thread Richard Poole
In 9.3beta1, a backend will receive a SIGALRM after authentication_timeout seconds, even if authentication has been successful. Most of the time this doesn't hurt anyone, but there are cases, such as when the backend is doing the open() of a backend copy, when it breaks things and results in an

[HACKERS] logical changeset generation v5

2013-06-14 Thread Andres Freund
Hi! I am rather pleased to announce the next version of the changeset extraction patchset. Thanks to help from a large number of people I think we are slowly getting to the point where it is getting committable. Since the last submitted version (20121115002746.ga7...@awork2.anarazel.de) a large

[HACKERS] GIN improvements part2: fast scan

2013-06-14 Thread Alexander Korotkov
Hackes, attached patch implementing fast scan technique for GIN. This is second patch of GIN improvements, see the 1st one here: http://www.postgresql.org/message-id/capphfduxv-il7aedwpw0w5fxrwgakfxijwm63_hzujacrxn...@mail.gmail.com This patch allow to skip parts of posting trees when their scan

Re: [HACKERS] stray SIGALRM

2013-06-14 Thread Tom Lane
Richard Poole rich...@2ndquadrant.com writes: In 9.3beta1, a backend will receive a SIGALRM after authentication_timeout seconds, even if authentication has been successful. Most of the time this doesn't hurt anyone, but there are cases, such as when the backend is doing the open() of a

Re: [HACKERS] [RFC] Minmax indexes

2013-06-14 Thread Josh Berkus
Alvaro, This sounds really interesting, and I can see the possibilities. However ... Value changes in columns that are part of a minmax index, and tuple insertion in summarized pages, would invalidate the stored min/max values. To support this, each minmax index has a validity map; a range

[HACKERS] GIN improvements part 3: ordering in index

2013-06-14 Thread Alexander Korotkov
Hackers, attached patch implementing ordering inside GIN index. This is third patch of GIN improvements, see previous two: http://www.postgresql.org/message-id/capphfduxv-il7aedwpw0w5fxrwgakfxijwm63_hzujacrxn...@mail.gmail.com

Re: [HACKERS] extensible external toast tuple support

2013-06-14 Thread Andres Freund
On 2013-05-31 23:42:51 -0400, Robert Haas wrote: On Thu, May 30, 2013 at 7:42 AM, Andres Freund and...@2ndquadrant.com wrote: In http://archives.postgresql.org/message-id/20130216164231.GA15069%40awork2.anarazel.de I presented the need for 'indirect' toast tuples which point into memory

Re: [HACKERS] [RFC] Minmax indexes

2013-06-14 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: To avoid this, a table having a minmax index would be configured so that inserts only go to the page(s) at the end of the table; this avoids frequent invalidation of ranges in the middle of the table. We provide a table reloption that tweaks the FSM

Re: [HACKERS] extensible external toast tuple support

2013-06-14 Thread Alvaro Herrera
Andres Freund escribió: Here's the updated version. It shouldn't contain any obvious WIP pieces anymore, although I think it needs some more documentation. I am just not sure where to add it yet, postgres.h seems like a bad place :/ How about a new file, say src/include/access/toast.h? --

Re: [HACKERS] extensible external toast tuple support

2013-06-14 Thread Andres Freund
On 2013-06-14 19:14:15 -0400, Alvaro Herrera wrote: Andres Freund escribió: Here's the updated version. It shouldn't contain any obvious WIP pieces anymore, although I think it needs some more documentation. I am just not sure where to add it yet, postgres.h seems like a bad place :/

Re: [HACKERS] pluggable compression support

2013-06-14 Thread Josh Berkus
On 06/14/2013 04:01 PM, Andres Freund wrote: It still contains a guc as described in the above message to control the algorithm used for compressing new tuples but I think we should remove that guc after testing. Did you add the storage attribute? -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] pluggable compression support

2013-06-14 Thread Andres Freund
On 2013-06-14 17:12:01 -0700, Josh Berkus wrote: On 06/14/2013 04:01 PM, Andres Freund wrote: It still contains a guc as described in the above message to control the algorithm used for compressing new tuples but I think we should remove that guc after testing. Did you add the storage

[HACKERS] [PATCH] Revive line type

2013-06-14 Thread Peter Eisentraut
Complete the implementations of line_in, line_out, line_recv, line_send. Remove comments and error messages about the line type not being implemented. Add regression tests for existing line operators and functions. --- This just revives existing functionality, doesn't add anything new. One thing

Re: [HACKERS] [RFC] Minmax indexes

2013-06-14 Thread Greg Stark
On Fri, Jun 14, 2013 at 11:28 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Re-summarization is relatively expensive, because the complete page range has to be scanned. That doesn't sound too bad to me. It just means there's a downside to having larger page ranges. I would expect the page

Re: [HACKERS] pluggable compression support

2013-06-14 Thread Josh Berkus
No. I think as long as we only have pglz and one new algorithm (even if that is lz4 instead of the current snappy) we should just always use the new algorithm. Unless I missed it nobody seemed to have voiced a contrary position? For testing/evaluation the guc seems to be sufficient. Then

Re: [HACKERS] pluggable compression support

2013-06-14 Thread Andres Freund
On 2013-06-14 17:35:02 -0700, Josh Berkus wrote: No. I think as long as we only have pglz and one new algorithm (even if that is lz4 instead of the current snappy) we should just always use the new algorithm. Unless I missed it nobody seemed to have voiced a contrary position? For

Re: [HACKERS] dynamic background workers

2013-06-14 Thread Michael Paquier
On Sat, Jun 15, 2013 at 6:00 AM, Robert Haas robertmh...@gmail.com wrote: The second patch, dynamic-bgworkers-v1.patch, revises the background worker API to allow background workers to be started dynamically. This requires some communication channel from ordinary workers to the postmaster,

Re: [HACKERS] pluggable compression support

2013-06-14 Thread Robert Haas
On Fri, Jun 14, 2013 at 8:45 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-06-14 17:35:02 -0700, Josh Berkus wrote: No. I think as long as we only have pglz and one new algorithm (even if that is lz4 instead of the current snappy) we should just always use the new algorithm.

Re: [HACKERS] pluggable compression support

2013-06-14 Thread Joshua D. Drake
On 06/14/2013 06:56 PM, Robert Haas wrote: On Fri, Jun 14, 2013 at 8:45 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-06-14 17:35:02 -0700, Josh Berkus wrote: No. I think as long as we only have pglz and one new algorithm (even if that is lz4 instead of the current snappy) we

  1   2   >