Re: [HACKERS] Documentation for SET var_name FROM CURRENT
On Tue, Oct 1, 2013 at 9:01 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Amit Kapila escribió: While reading documentation for SET command, I observed that FROM CURRENT syntax and its description is missing from SET command's syntax page (http://www.postgresql.org/docs/devel/static/sql-set.html). Do you think that documentation should be updated for the same or is there any reason why it is not documented? Did you look at the commit message that introduced it? No, I just noticed while reading documentation that although syntax SET var_name FROM CURRENT works, but the documentation didn't have it. Commit meesage below explains why it is not there, but I think as it works, it might be better to mention in some form (as described by David in previous mail or some other variant). commit e7889b83b7059e776f0a3d76bbbdd98687f4592c Author: Tom Lane t...@sss.pgh.pa.us Date: Mon Sep 3 18:46:30 2007 + Support SET FROM CURRENT in CREATE/ALTER FUNCTION, ALTER DATABASE, ALTER ROLE. (Actually, it works as a plain statement too, but I didn't document that because it seems a bit useless.) Unify VariableResetStmt with VariableSetStmt, and clean up some ancient cruft in the representation of same. Thanks for pointing at right location. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Who is pgFoundery administrator?
Hi, I want to submit new project in pgFoundery project. I submitted new project which is WAL archive copy tool with directIO method in pgFoundery homepage 2 weeks ago, but it does not have approved and responded at all:-( Who is pgFoundery administrator or board member now? I would like to send e-mail them. At least, it does not have information and support page in pgFoundery homepage. Regards, -- Mitsumasa KONDO NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Who is pgFoundery administrator?
Hi, I want to submit new project in pgFoundery project. I submitted new project which is WAL archive copy tool with directIO method in pgFoundery homepage 2 weeks ago, but it does not have approved and responded at all:-( Who is pgFoundery administrator or board member now? I would like to send e-mail them. At least, it does not have information and support page in pgFoundery homepage. It's Marc (scra...@hub.org). -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Add use of asprintf()
Thank you Peter. When I try to apply the patch on latest PG source code on master branch, it gives the following error message i.e. asif$ git apply /Users/asif/core/Patch\:\ Add\ use\ of\ asprintf\(\)/v2-0001-Add-use-of-asprintf.patch /Users/asif/core/Patch: Add use of asprintf()/v2-0001-Add-use-of-asprintf.patch:76: trailing whitespace. /Users/asif/core/Patch: Add use of asprintf()/v2-0001-Add-use-of-asprintf.patch:77: trailing whitespace. for ac_func in asprintf crypt fls getopt getrusage inet_aton random rint srandom strerror strlcat strlcpy /Users/asif/core/Patch: Add use of asprintf()/v2-0001-Add-use-of-asprintf.patch:90: trailing whitespace. AC_REPLACE_FUNCS([asprintf crypt fls getopt getrusage inet_aton random rint srandom strerror strlcat strlcpy]) /Users/asif/core/Patch: Add use of asprintf()/v2-0001-Add-use-of-asprintf.patch:104: trailing whitespace. values[1] = psprintf(%s/%s, fctx-location, de-d_name); /Users/asif/core/Patch: Add use of asprintf()/v2-0001-Add-use-of-asprintf.patch:118: trailing whitespace. pg_asprintf(todo, fatal: git apply: bad git-diff - expected /dev/null on line 1557 Neither git nor patch command apply the patch successfully. Can you please guide ?. Thanks. Best Regards, Muhammad Asif Naeem On Wed, Oct 2, 2013 at 7:29 AM, Peter Eisentraut pete...@gmx.net wrote: On Tue, 2013-09-17 at 15:13 +0500, Asif Naeem wrote: I did put some time review the patch, please see my findings below i.e. Updated patch for this.
Re: [HACKERS] review: psql and pset without any arguments
Hello all there are no comments, so I'll close this topic This feature is ready for commit Regards Pavel Stehule 2013/10/1 Pavel Stehule pavel.steh...@gmail.com Hello all I am thinking so almost all is done I fixed a help and appended a simple test But it is a cosmetic changes. Comments? Regards Pavel Stehule 2013/9/30 Gilles Darold gilles.dar...@dalibo.com Le 30/09/2013 17:35, Peter Eisentraut a écrit : Please remove the tabs from the SGML files. Done. I've also fixed the typo reported by Ian. Here is the attached v4 patch. Thanks a lot for your review. Regards, -- Gilles Darold Administrateur de bases de données http://dalibo.com - http://dalibo.org
Re: [HACKERS] Who is pgFoundery administrator?
On Wed, Oct 2, 2013 at 5:37 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Who is pgFoundery administrator or board member now? I would like to send e-mail them. At least, it does not have information and support page in pgFoundery homepage. Why don't you consider github as a potential solution? You could use with it a bug tracker, a wiki and a website for documentation. git is also more pluggable when working with pg core. Creating an account as well as a new project does not take more than 5 minutes as well... -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Add use of asprintf()
On 10/2/13 5:12 AM, Asif Naeem wrote: Neither git nor patch command apply the patch successfully. Can you please guide ?. Thanks. Works for me. Check that your email client isn't mangling line endings. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No Index-Only Scan on Partial Index
On Tuesday, October 1, 2013, David E. Wheeler da...@justatheory.com wrote: On Oct 1, 2013, at 3:56 PM, Merlin Moncure mmonc...@gmail.com wrote: I don't think it has anything to do with the conditional index -- it's the functional based. For some reason postgres always wants to post filter (note the filter step below): postgres=# create index on try(upper_inf(irange)); CREATE INDEX Time: 12.001 ms postgres=# explain select * from try where upper_inf(irange); QUERY PLAN --- Index Scan using try_upper_inf_idx on try (cost=0.00..9.25 rows=33 width=40) Index Cond: (upper_inf(irange) = true) Filter: upper_inf(irange) Hrm. I get a seq scan for that query: create index on try(upper_inf(irange)); explain select * from try where upper_inf(irange); QUERY PLAN --- Seq Scan on try (cost=0.00..1887.00 rows=3 width=68) Filter: upper_inf(irange) True also if I just select the irange. Is the filter the issue, here? Turn off seq scan... merlin
Re: [HACKERS] pg_stat_statements: calls under-estimation propagation
Looks pretty good. Do you want to package up the patch with your change and do the honors and re-submit it? Thanks for helping out so much! Sure, will do. Need to add a bit of documentation explaining statistics session as well. I did some more basic testing around pg_stat_statements.max, now that we have clarity from Peter about its value being legitimate below 100. Seems to work fine, with pg_stat_statements =4 the max unique queries in the view are 4. On the 5th query the view holds just the latest unique query discarding the previous 4. Fujii had reported a segmentation fault in this scenario. Thank you for the patch regards Sameer -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] relscan_details.h
On Wed, Oct 2, 2013 at 08:59:42AM -0400, Noah Misch wrote: On Tue, Oct 01, 2013 at 10:12:05PM -0400, Bruce Momjian wrote: If we had not made massive cleanup changes years ago, our code would not be as good as it is today. By avoiding cleanup to reduce the burden on people who use our code, we are positioning our code on a slow decline in clarity. What I don't want to do is get into a mode where every code cleanup has to be verified that it isn't going to excessively burden outside code users. Cleanup is hard enough, and adding another check to that process makes cleanup even less likely. I don't disagree with that, but I would describe the proposal as a mild dirty-up for the sake of build performance, not a cleanup. OK, good enough. Thanks for the feedback. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Looking for information on our elephant
On Mon, Sep 23, 2013 at 12:40:06AM +0400, Oleg Bartunov wrote: I found in my archives several logos. The oldest one comes from postgres95 time. I recall that logo with cheetah was made by Bruce (?) No, I don't remember who did that cheetah. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] relscan_details.h
On Tue, Oct 01, 2013 at 10:12:05PM -0400, Bruce Momjian wrote: If we had not made massive cleanup changes years ago, our code would not be as good as it is today. By avoiding cleanup to reduce the burden on people who use our code, we are positioning our code on a slow decline in clarity. What I don't want to do is get into a mode where every code cleanup has to be verified that it isn't going to excessively burden outside code users. Cleanup is hard enough, and adding another check to that process makes cleanup even less likely. I don't disagree with that, but I would describe the proposal as a mild dirty-up for the sake of build performance, not a cleanup. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.
On Mon, Sep 30, 2013 at 8:15 AM, Peter Eisentraut pete...@gmx.net wrote: On 9/27/13 3:00 PM, Merlin Moncure wrote: Attached is simplified patch that replaces the spinlock with a read barrier based on a suggestion made by Andres offlist. This patch doesn't apply. works for me: merlin@mmoncure-ubuntu:~/pgdev/pgsql$ git reset --hard HEAD HEAD is now at 200ba16 Add regression test for bug fixed by recent refactoring. merlin@mmoncure-ubuntu:~/pgdev/pgsql$ patch -p1 buffer5.patch patching file src/backend/access/transam/xlog.c On Mon, Sep 30, 2013 at 7:51 PM, Ants Aasma a...@cybertec.at wrote: So we need a read barrier somewhere *after* reading the flag in RecoveryInProgress() and reading the shared memory structures, and in theory a full barrier if we are going to be writing data. wow -- thanks for your review and provided detail. Considering there are no examples of barrier instructions to date, I think some of your commentary should be included in the in-source documentation. In this particular case, a read barrier should be sufficient? By 'writing data', do you mean to the xlog control structure? This routine only sets a backend local flag so that should be safe? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Who is pgFoundery administrator?
On Wed, Oct 2, 2013 at 3:37 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi, I want to submit new project in pgFoundery project. I submitted new project which is WAL archive copy tool with directIO method in pgFoundery homepage 2 weeks ago, but it does not have approved and responded at all:-( Who is pgFoundery administrator or board member now? I would like to send e-mail them. At least, it does not have information and support page in pgFoundery homepage. I have not been able to get in contact with Marc either to help resolve a mailing list administration issue. Github is nice, but a lot of projects are still hosted on pgfoundry and it kind of sucks to be forced to move on this basis. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] relscan_details.h
Bruce Momjian escribió: On Wed, Oct 2, 2013 at 08:59:42AM -0400, Noah Misch wrote: On Tue, Oct 01, 2013 at 10:12:05PM -0400, Bruce Momjian wrote: If we had not made massive cleanup changes years ago, our code would not be as good as it is today. By avoiding cleanup to reduce the burden on people who use our code, we are positioning our code on a slow decline in clarity. What I don't want to do is get into a mode where every code cleanup has to be verified that it isn't going to excessively burden outside code users. Cleanup is hard enough, and adding another check to that process makes cleanup even less likely. I don't disagree with that, but I would describe the proposal as a mild dirty-up for the sake of build performance, not a cleanup. OK, good enough. Thanks for the feedback. Agreed. I will keep this patch in my local repo; I might present it again later as part of more invasive surgery. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones
Oskari Saarenmaa o...@ohmu.fi wrote: Add file cloning as an alternative data transfer method to pg_upgrade. Currently only btrfs is supported, but copy-on-write cloning is also available on at least ZFS. Cloning must be requested explicitly and if it isn't supported by the operating system or filesystem a fatal error is thrown. This provides upgrade performance similar to link mode while allowing the old cluster to be used even after the new one has been started. Please add the patch here to make sure it gets reviewed: https://commitfest.postgresql.org/action/commitfest_view/open For more information on the process, see: http://wiki.postgresql.org/wiki/CommitFest -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.
On 2013-10-02 08:39:59 -0500, Merlin Moncure wrote: wow -- thanks for your review and provided detail. Considering there are no examples of barrier instructions to date, I think some of your commentary should be included in the in-source documentation. I think most of it is in README.barrier Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones
On 02/10/13 17:18, Andrew Dunstan wrote: On 10/01/2013 06:31 PM, Oskari Saarenmaa wrote: Add file cloning as an alternative data transfer method to pg_upgrade. Currently only btrfs is supported, but copy-on-write cloning is also available on at least ZFS. Cloning must be requested explicitly and if it isn't supported by the operating system or filesystem a fatal error is thrown. So, just curious, why isn't ZFS supported? It's what I am more interested in, at least. No fundamental reason; I'm hoping ZFS will be supported in addition to btrfs, but I don't have any systems with ZFS filesystems at the moment so I haven't been able to test it or find out the mechanisms ZFS uses for cloning. On btrfs cloning is implemented with a custom btrfs-specific ioctl, ZFS probably has something similar which would be pretty easy to add on top of this patch. Added this patch to commitfest as suggested, https://commitfest.postgresql.org/action/patch_view?id=1251 / Oskari -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones
On 10/01/2013 06:31 PM, Oskari Saarenmaa wrote: Add file cloning as an alternative data transfer method to pg_upgrade. Currently only btrfs is supported, but copy-on-write cloning is also available on at least ZFS. Cloning must be requested explicitly and if it isn't supported by the operating system or filesystem a fatal error is thrown. So, just curious, why isn't ZFS supported? It's what I am more interested in, at least. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Prevent pg_basebackup -Fp -D -?
Right now, if you use pg_basebackup -Ft -D - you get a tarfile, written to stdout, for redirection. However, if you use: pg_basebackup -Fp -D - you get a plaintext (unpackaged) backup, in a directory called -. I can't think of a single usecase where this is a good idea. Therefor, I would suggest we simply throw an error in this case, instead of creating the directory. Only for the specific case of specifying exactly - as a directory. Comments? Also, if we do that, is this something we should consider backpatchable? It's not strictly speaking a bugfix, but I'd say it fixes some seriously annoying behavior. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.
On Wed, Oct 2, 2013 at 4:39 PM, Merlin Moncure mmonc...@gmail.com wrote:
On Mon, Sep 30, 2013 at 7:51 PM, Ants Aasma a...@cybertec.at wrote:
So we need a read barrier somewhere *after* reading the flag in
RecoveryInProgress() and reading the shared memory structures, and in
theory a full barrier if we are going to be writing data.
wow -- thanks for your review and provided detail. Considering there
are no examples of barrier instructions to date, I think some of your
commentary should be included in the in-source documentation.
In this particular case, a read barrier should be sufficient? By
'writing data', do you mean to the xlog control structure? This
routine only sets a backend local flag so that should be safe?
I haven't reviewed the code in as much detail to say if there is an
actual race here, I tend to think there's probably not, but the
specific pattern that I had in mind is that with the following actual
code:
Process A:
globalVar = 1;
write_barrier();
recoveryInProgress = false;
Process B:
if (!recoveryInProgress) {
globalVar = 2;
doWork();
}
If process B speculatively executes line 2 before reading the flag for
line 1, then it's possible that the store in process B is executed
before the store in process A, making globalVar move backwards. The
barriers as they are defined don't make this scenario impossible. That
said, I don't know of any hardware that would make speculatively
executed stores visible to non-speculative state, as I said, that
would be completely insane. However currently compilers consider it
completely legal to rewrite the code into the following form:
tmp = globalVar;
globalVar = 2;
if (!recoveryInProgress) {
doWork();
} else {
globalVar = tmp;
}
That still exhibits the same problem. An abstract read barrier would
not be enough here, as this requires a LoadStore barrier. However, the
control dependency is enough for the hardware and PostgreSQL
pg_read_barrier() is a full compiler barrier, so in practice a simple
pg_read_barrier() is enough.
Regards,
Ants Aasma
--
Cybertec Schönig Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] insert throw error when year field len 4 for timestamptz datatype
On Tue, Oct 1, 2013 at 7:52 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Sep 27, 2013 at 10:42:17AM +, Haribabu kommi wrote: If the changes are very high to deal all scenarios, I feel it is better do it only in scenarios where the use cases needs it, until it is not confusing users. The rest can be documented. Any other opinions/suggestions welcome. I have reviewed this patch and it is good. The problem is guessing if a number with 5+ digits is YMD, HMS, or a year. I have created a modified patch, attached, assumes a 5-digit number is a year, because YMD and HMS require at least six digits, and used your date/time test to control the other cases. I also added a few more regression tests. In an ideal world the interpretation of the tokens wouldn't depend on the order in which they appear. But we don't live in an ideal world, so maybe this is fine. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pluggable compression support
On Tue, Oct 1, 2013 at 9:56 PM, Daniel Farina dan...@heroku.com wrote: On Mon, Sep 30, 2013 at 1:49 PM, Huchev hugochevr...@gmail.com wrote: How come any compressor which could put some competition to pglz is systematically pushed out of the field on the ground of unverifiable legal risks ? Because pglz has been around for a while and has not caused patent trouble. The risks have been accepted and the downsides have not materialized. Were pglz were being written and distributed starting today, perhaps your reasoning would be more compelling, but as-is the pglz ship has sailed for quite some time and empirically it has not been a problem. That said, I hope the findings are in favor of lz4 or snappy integration. It does seem lz4 has picked up a slight edge. Yeah, I'm also in favor of a new compression format, whatever we can agree on. However, I'm uncertain we're actually moving toward that goal in any meaningful way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] logical changeset generation v6.1
On 2013-10-02 10:56:38 -0400, Robert Haas wrote: On Tue, Oct 1, 2013 at 1:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-01 10:07:19 -0400, Robert Haas wrote: - It seems that HeapSatisfiesHOTandKeyUpdate is now HeapSatisfiesHOTandKeyandCandidateKeyUpdate. Considering I think this was merely HeapSatisfiesHOTUpdate a year ago, it's hard not to be afraid that something unscalable is happening to this function. On a related node, any overhead added here costs broadly; I'm not sure if there's enough to worry about. Ok, I had to think a bit, but now I remember why I think these changes are not really problem: Neither the addition of keys nor candidate keys will add any additional comparisons since the columns compared for candidate keys are a subset of the set of key columns which in turn are a subset of the columns checked for HOT. Right? TBH, my primary concern was with maintainability more than performance. On performance, I think any time you add code it's going to cost somehow. However, it might not be enough to care about. The easy alternative seems to be to call such a function multiple times - which I think is prohibitive from a performance POV. More radically we could simply compute the overall set/bitmap of differening columns and then use bms_is_subset() to determine whether any index columns/key/ckey columns changed. But that will do comparisons we don't do today... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] logical changeset generation v6.1
On Wed, Oct 2, 2013 at 11:05 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-02 10:56:38 -0400, Robert Haas wrote: On Tue, Oct 1, 2013 at 1:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-01 10:07:19 -0400, Robert Haas wrote: - It seems that HeapSatisfiesHOTandKeyUpdate is now HeapSatisfiesHOTandKeyandCandidateKeyUpdate. Considering I think this was merely HeapSatisfiesHOTUpdate a year ago, it's hard not to be afraid that something unscalable is happening to this function. On a related node, any overhead added here costs broadly; I'm not sure if there's enough to worry about. Ok, I had to think a bit, but now I remember why I think these changes are not really problem: Neither the addition of keys nor candidate keys will add any additional comparisons since the columns compared for candidate keys are a subset of the set of key columns which in turn are a subset of the columns checked for HOT. Right? TBH, my primary concern was with maintainability more than performance. On performance, I think any time you add code it's going to cost somehow. However, it might not be enough to care about. The easy alternative seems to be to call such a function multiple times - which I think is prohibitive from a performance POV. More radically we could simply compute the overall set/bitmap of differening columns and then use bms_is_subset() to determine whether any index columns/key/ckey columns changed. But that will do comparisons we don't do today... Yeah, there may be no better alternative to doing things as you've done them here. It just looks grotty, so I was hoping we had a better idea. Maybe not. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] logical changeset generation v6.1
On 2013-10-02 11:06:59 -0400, Robert Haas wrote: On Wed, Oct 2, 2013 at 11:05 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-02 10:56:38 -0400, Robert Haas wrote: On Tue, Oct 1, 2013 at 1:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-01 10:07:19 -0400, Robert Haas wrote: - It seems that HeapSatisfiesHOTandKeyUpdate is now HeapSatisfiesHOTandKeyandCandidateKeyUpdate. Considering I think this was merely HeapSatisfiesHOTUpdate a year ago, it's hard not to be afraid that something unscalable is happening to this function. On a related node, any overhead added here costs broadly; I'm not sure if there's enough to worry about. Ok, I had to think a bit, but now I remember why I think these changes are not really problem: Neither the addition of keys nor candidate keys will add any additional comparisons since the columns compared for candidate keys are a subset of the set of key columns which in turn are a subset of the columns checked for HOT. Right? TBH, my primary concern was with maintainability more than performance. On performance, I think any time you add code it's going to cost somehow. However, it might not be enough to care about. The easy alternative seems to be to call such a function multiple times - which I think is prohibitive from a performance POV. More radically we could simply compute the overall set/bitmap of differening columns and then use bms_is_subset() to determine whether any index columns/key/ckey columns changed. But that will do comparisons we don't do today... Yeah, there may be no better alternative to doing things as you've done them here. It just looks grotty, so I was hoping we had a better idea. Maybe not. Imo the code now looks easier to understand - which is not saying much - than in 9.3/HEAD... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] logical changeset generation v6.1
On Tue, Oct 1, 2013 at 1:56 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-01 10:07:19 -0400, Robert Haas wrote: - It seems that HeapSatisfiesHOTandKeyUpdate is now HeapSatisfiesHOTandKeyandCandidateKeyUpdate. Considering I think this was merely HeapSatisfiesHOTUpdate a year ago, it's hard not to be afraid that something unscalable is happening to this function. On a related node, any overhead added here costs broadly; I'm not sure if there's enough to worry about. Ok, I had to think a bit, but now I remember why I think these changes are not really problem: Neither the addition of keys nor candidate keys will add any additional comparisons since the columns compared for candidate keys are a subset of the set of key columns which in turn are a subset of the columns checked for HOT. Right? TBH, my primary concern was with maintainability more than performance. On performance, I think any time you add code it's going to cost somehow. However, it might not be enough to care about. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Who is pgFoundery administrator?
direct_cp project approved, sorry for delay … as to the the mailing list issue, where did you send it? can you resend it to me here? On 2013-10-02, at 6:46 , Merlin Moncure mmonc...@gmail.com wrote: On Wed, Oct 2, 2013 at 3:37 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi, I want to submit new project in pgFoundery project. I submitted new project which is WAL archive copy tool with directIO method in pgFoundery homepage 2 weeks ago, but it does not have approved and responded at all:-( Who is pgFoundery administrator or board member now? I would like to send e-mail them. At least, it does not have information and support page in pgFoundery homepage. I have not been able to get in contact with Marc either to help resolve a mailing list administration issue. Github is nice, but a lot of projects are still hosted on pgfoundry and it kind of sucks to be forced to move on this basis. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] insert throw error when year field len 4 for timestamptz datatype
On Wed, Oct 2, 2013 at 11:00:30AM -0400, Robert Haas wrote: On Tue, Oct 1, 2013 at 7:52 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Sep 27, 2013 at 10:42:17AM +, Haribabu kommi wrote: If the changes are very high to deal all scenarios, I feel it is better do it only in scenarios where the use cases needs it, until it is not confusing users. The rest can be documented. Any other opinions/suggestions welcome. I have reviewed this patch and it is good. The problem is guessing if a number with 5+ digits is YMD, HMS, or a year. I have created a modified patch, attached, assumes a 5-digit number is a year, because YMD and HMS require at least six digits, and used your date/time test to control the other cases. I also added a few more regression tests. In an ideal world the interpretation of the tokens wouldn't depend on the order in which they appear. But we don't live in an ideal world, so maybe this is fine. Yes, earlier in the thread the original patch poster questioned whether he was going in the right direction, given the unusual hacks needed, but such hacks are standard operating procedure for date/time stuff. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Who is pgFoundery administrator?
On Wed, Oct 2, 2013 at 10:42 AM, Marc Fournier scra...@hub.org wrote: direct_cp project approved, sorry for delay … as to the the mailing list issue, where did you send it? can you resend it to me here? sure -- I had sent it initially to the www list then directly to you.. Trying to figure out what/who to contact -- I admin the libpqtypes mailing lists and I'm unable to authenticate through the moderator interface -- is there a way to reset or recover the admin password? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] ToDo: fast update of arrays with fixed length fields for PL/pgSQL
Hello this proposal is related to reported issue http://www.postgresql.org/message-id/e1vquta-0007y4...@wrigleys.postgresql.org We can directly modify any fields of int, float, double arrays (when result size will be immutable). This trick is used now for acceleration of some aggregates. Ideas, comments? Regards Pavel
Re: [HACKERS] ToDo: fast update of arrays with fixed length fields for PL/pgSQL
Hi, On 2013-10-02 18:56:51 +0200, Pavel Stehule wrote: Hello this proposal is related to reported issue http://www.postgresql.org/message-id/e1vquta-0007y4...@wrigleys.postgresql.org We can directly modify any fields of int, float, double arrays (when result size will be immutable). This trick is used now for acceleration of some aggregates. Ideas, comments? A specific array might be located directly on a buffer - starting to manipulate them inplace will lead to havoc in such scenarios. I don't think we have the infrastructure for detecting such cases. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ToDo: fast update of arrays with fixed length fields for PL/pgSQL
2013/10/2 Andres Freund and...@2ndquadrant.com Hi, On 2013-10-02 18:56:51 +0200, Pavel Stehule wrote: Hello this proposal is related to reported issue http://www.postgresql.org/message-id/e1vquta-0007y4...@wrigleys.postgresql.org We can directly modify any fields of int, float, double arrays (when result size will be immutable). This trick is used now for acceleration of some aggregates. Ideas, comments? A specific array might be located directly on a buffer - starting to manipulate them inplace will lead to havoc in such scenarios. I don't think we have the infrastructure for detecting such cases. If you can do a update of some array in plpgsql now, then you have to work with local copy only. It is a necessary precondition, and I am think it is valid. Pavel Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services
Re: [HACKERS] hstore extension version screwup
On 9/29/13 9:41 PM, Andrew Dunstan wrote: On 09/29/2013 10:38 PM, Peter Eisentraut wrote: On Sun, 2013-09-29 at 22:33 -0400, Andrew Dunstan wrote: Well if these are not meant to be changed then not being able to write them in your git repo might be a clue to that. Git doesn't support setting file permissions other than the executable bit, so this is a nonstarter. Oh, didn't know that, I've certainly know other SCM systems that do. We could potentially do it with git commit hooks, but the problem is that there's no way to force use of those on clients (a huge deficiency in git, imho). The best alternative I've been able to come up with is having hooks in a standard location in the repo and then there's one file that people would need to put into their home directory (under ~/.git I think) that would pull all of that stuff in. -- Jim C. Nasby, Data Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Who is pgFoundery administrator?
Merlin == Merlin Moncure mmonc...@gmail.com writes: Who is pgFoundery administrator or board member now? I would like to send e-mail them. At least, it does not have information and support page in pgFoundery homepage. Merlin I have not been able to get in contact with Marc either to Merlin help resolve a mailing list administration issue. Github is Merlin nice, but a lot of projects are still hosted on pgfoundry and Merlin it kind of sucks to be forced to move on this basis. Also, many pgfoundry projects had all their admins and members removed leaving them empty; no attempt seems to have been made to restore this (surely any project with no admins could have had them restored from a backup, rather than expecting every individual user to complain about it?) The help-with-pgfoundry forum was also broken last I looked. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.
On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma a...@cybertec.at wrote: I haven't reviewed the code in as much detail to say if there is an actual race here, I tend to think there's probably not, but the specific pattern that I had in mind is that with the following actual code: hm. I think there *is* a race. 2+ threads could race to the line: LocalRecoveryInProgress = xlogctl-SharedRecoveryInProgress; and simultaneously set the value of LocalRecoveryInProgress to false, and both engage InitXLOGAccess, which is destructive. The move operation is atomic, but I don't think there's any guarantee the reads to xlogctl-SharedRecoveryInProgress are ordered between threads without a lock. I don't think the memory barrier will fix this. Do you agree? If so, my earlier patch (recovery4) is another take on this problem, and arguable safer; the unlocked read is in a separate path that does not engage InitXLOGAccess() merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones
On Wed, Oct 2, 2013 at 05:23:31PM +0300, Oskari Saarenmaa wrote: On 02/10/13 17:18, Andrew Dunstan wrote: On 10/01/2013 06:31 PM, Oskari Saarenmaa wrote: Add file cloning as an alternative data transfer method to pg_upgrade. Currently only btrfs is supported, but copy-on-write cloning is also available on at least ZFS. Cloning must be requested explicitly and if it isn't supported by the operating system or filesystem a fatal error is thrown. So, just curious, why isn't ZFS supported? It's what I am more interested in, at least. No fundamental reason; I'm hoping ZFS will be supported in addition to btrfs, but I don't have any systems with ZFS filesystems at the moment so I haven't been able to test it or find out the mechanisms ZFS uses for cloning. On btrfs cloning is implemented with a custom btrfs-specific ioctl, ZFS probably has something similar which would be pretty easy to add on top of this patch. Added this patch to commitfest as suggested, https://commitfest.postgresql.org/action/patch_view?id=1251 What is the performance overhead of using a cloned data directory for a cluster? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.
On Wed, Oct 2, 2013 at 10:37 PM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma a...@cybertec.at wrote: I haven't reviewed the code in as much detail to say if there is an actual race here, I tend to think there's probably not, but the specific pattern that I had in mind is that with the following actual code: hm. I think there *is* a race. 2+ threads could race to the line: LocalRecoveryInProgress = xlogctl-SharedRecoveryInProgress; and simultaneously set the value of LocalRecoveryInProgress to false, and both engage InitXLOGAccess, which is destructive. The move operation is atomic, but I don't think there's any guarantee the reads to xlogctl-SharedRecoveryInProgress are ordered between threads without a lock. I don't think the memory barrier will fix this. Do you agree? If so, my earlier patch (recovery4) is another take on this problem, and arguable safer; the unlocked read is in a separate path that does not engage InitXLOGAccess() InitXLOGAccess only writes backend local variables, so it can't be destructive. In fact, it calls GetRedoRecPtr(), which does a spinlock acquisition cycle, which should be a full memory barrier. A read barrier after accessing SharedRecoveryInProgress is good enough, and it seems to be necessary to avoid a race condition for XLogCtl-ThisTimeLineID. Admittedly the race is so unprobable that in practice it probably doesn't matter. I just wanted to be spell out the correct way to do unlocked accesses as it can get quite confusing. I found Herb Sutter's atomic weapons talk very useful, thinking about the problem in terms of acquire and release makes it much clearer to me. Regards, Ants Aasma -- Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.
On Wed, Oct 2, 2013 at 3:43 PM, Ants Aasma a...@cybertec.at wrote: On Wed, Oct 2, 2013 at 10:37 PM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma a...@cybertec.at wrote: I haven't reviewed the code in as much detail to say if there is an actual race here, I tend to think there's probably not, but the specific pattern that I had in mind is that with the following actual code: hm. I think there *is* a race. 2+ threads could race to the line: LocalRecoveryInProgress = xlogctl-SharedRecoveryInProgress; and simultaneously set the value of LocalRecoveryInProgress to false, and both engage InitXLOGAccess, which is destructive. The move operation is atomic, but I don't think there's any guarantee the reads to xlogctl-SharedRecoveryInProgress are ordered between threads without a lock. I don't think the memory barrier will fix this. Do you agree? If so, my earlier patch (recovery4) is another take on this problem, and arguable safer; the unlocked read is in a separate path that does not engage InitXLOGAccess() InitXLOGAccess only writes backend local variables, so it can't be destructive. In fact, it calls GetRedoRecPtr(), which does a spinlock acquisition cycle, which should be a full memory barrier. A read barrier after accessing SharedRecoveryInProgress is good enough, and it seems to be necessary to avoid a race condition for XLogCtl-ThisTimeLineID. Admittedly the race is so unprobable that in practice it probably doesn't matter. I just wanted to be spell out the correct way to do unlocked accesses as it can get quite confusing. I found Herb Sutter's atomic weapons talk very useful, thinking about the problem in terms of acquire and release makes it much clearer to me. If we don't care about multiple calls to InitXLOGAccess() (for a backend) then we can get away with just a barrier. That's a pretty weird assumption though (even if 'improbable') and I think it should be well documented in the code, particularly since previous versions went though such trouble to do a proper check-set-init. I'm still leaning on my earlier idea (recovery4.patch) since it keeps the old semantics by exploiting the fact that the call site of interest (only) does not require a correct answer (that is, for the backend to think it's in recovery when it's not). That just defers the heap prune for a time and you don't have to mess around with barriers at all or be concerned about present or future issues caused by spurious extra calls to InitXLOGAccess(). The other code paths to RecoveryInProgress() appear not to be interesting from a spinlock perspective. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones
No fundamental reason; I'm hoping ZFS will be supported in addition to btrfs, but I don't have any systems with ZFS filesystems at the moment so I haven't been able to test it or find out the mechanisms ZFS uses for cloning. On btrfs cloning is implemented with a custom btrfs-specific ioctl, ZFS probably has something similar which would be pretty easy to add on top of this patch. Would you like a VM with ZFS on it? I'm pretty sure I can supply one. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_upgrade: support for btrfs copy-on-write clones
On 2013-10-02 17:32, Josh Berkus wrote: No fundamental reason; I'm hoping ZFS will be supported in addition to btrfs, but I don't have any systems with ZFS filesystems at the moment so I haven't been able to test it or find out the mechanisms ZFS uses for cloning. On btrfs cloning is implemented with a custom btrfs-specific ioctl, ZFS probably has something similar which would be pretty easy to add on top of this patch. Would you like a VM with ZFS on it? I'm pretty sure I can supply one. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com I can also supply SSH access to a FreeBSD 10 system that is totally ZFS. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: l...@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Who is pgFoundery administrator?
(2013/10/02 17:37), KONDO Mitsumasa wrote: I want to submit new project in pgFoundery project. Our new project was approved yesterday! Thanks very much for pgFoundery crew. Regards, -- Mitsumasa KONDO NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Prevent pg_basebackup -Fp -D -?
On Wed, Oct 2, 2013 at 11:31 PM, Magnus Hagander mag...@hagander.net wrote: Right now, if you use pg_basebackup -Ft -D - you get a tarfile, written to stdout, for redirection. However, if you use: pg_basebackup -Fp -D - you get a plaintext (unpackaged) backup, in a directory called -. I can't think of a single usecase where this is a good idea. Therefor, I would suggest we simply throw an error in this case, instead of creating the directory. Only for the specific case of specifying exactly - as a directory. Comments? Isn't this a non-problem? This behavior is in line with the documentation, so I would suspected that if directory name is specified as - in plain mode, it should create the folder with this name. Do you consider having a folder of this name an annoyance? Also, if we do that, is this something we should consider backpatchable? It's not strictly speaking a bugfix, but I'd say it fixes some seriously annoying behavior. This would change the spec of pg_basebackup, so no? Does the current behavior have potential security issues? My 2c. Regards, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Prevent pg_basebackup -Fp -D -?
On 10/02/2013 05:47 PM, Michael Paquier wrote: On Wed, Oct 2, 2013 at 11:31 PM, Magnus Hagander mag...@hagander.net wrote: Right now, if you use pg_basebackup -Ft -D - you get a tarfile, written to stdout, for redirection. However, if you use: pg_basebackup -Fp -D - you get a plaintext (unpackaged) backup, in a directory called -. I can't think of a single usecase where this is a good idea. Therefor, I would suggest we simply throw an error in this case, instead of creating the directory. Only for the specific case of specifying exactly - as a directory. Comments? I can see fixing this going forwards, but it doesn't seem worth backpatching. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Prevent pg_basebackup -Fp -D -?
On Oct 3, 2013 2:47 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Oct 2, 2013 at 11:31 PM, Magnus Hagander mag...@hagander.net wrote: Right now, if you use pg_basebackup -Ft -D - you get a tarfile, written to stdout, for redirection. However, if you use: pg_basebackup -Fp -D - you get a plaintext (unpackaged) backup, in a directory called -. I can't think of a single usecase where this is a good idea. Therefor, I would suggest we simply throw an error in this case, instead of creating the directory. Only for the specific case of specifying exactly - as a directory. Comments? Isn't this a non-problem? This behavior is in line with the documentation, so I would suspected that if directory name is specified as - in plain mode, it should create the folder with this name. Do you consider having a folder of this name an annoyance? Yes, that is exactly the point - i do consider that an annoyance, and i don't see the use case where you'd actually want it. I bet 100% of the users of that have been accidental, thinking they'd get the pipe, not the directory. Also, if we do that, is this something we should consider backpatchable? It's not strictly speaking a bugfix, but I'd say it fixes some seriously annoying behavior. This would change the spec of pg_basebackup, so no? Does the current behavior have potential security issues? No, there are no security issues that I can see. Just annoyance. And yes, I guess it would change the spec, so backpatching might be a bad idea.. /Magnus
