[HACKERS] pg_dump fails on domain constraint comments

Hello Alvaro, It looks like pg_dump emits incorrect text for domain constraint comments: Assuming the following structure, CREATE DOMAIN "dom" AS integer CONSTRAINT "dom_constraint" CHECK ((VALUE > 10)); COMMENT ON CONSTRAINT "dom_constraint" ON DOMAIN "dom" IS 'domain constraint

Re: [HACKERS] Speedup twophase transactions

> > On 11 Jan 2016, at 21:40, Jesper Pedersen wrote: > > I have done a run with the patch and it looks really great. > > Attached is the TPS graph - with a 1pc run too - and the perf profile as a > flame graph (28C/56T w/ 256Gb mem, 2 x RAID10 SSD). > Thanks for

Re: [HACKERS] Fuzzy substring searching with the pg_trgm extension

Alvaro Herrera writes: >> + >> show_substring_limit()show_substring_limit >> + >> set_substring_limit(real)set_substring_limit > I don't quite understand why aren't we using a custom GUC variable here. Presumably this is following the existing set_limit()

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

I wrote: > In further experimentation, I found out that even with this patch in > place, the plpython transform extensions still present a dump/reload > hazard: > ... > But I wonder why we couldn't make it do > LOAD 'plpython3'; I've verified that the attached additional patch makes everything

Re: [HACKERS] [PROPOSAL] Client Log Output Filtering

David Steele wrote: > Currently log messages generated by pgaudit can be made visible to the > client simply by altering client_min_messages. While this has not been a > showstopper for anyone it's ideal, either. > > The client authentication code sets the global variable ClientAuthInProgress >

Re: [HACKERS] Speedup twophase transactions

Hi, On January 11, 2016 10:46:01 PM GMT+01:00, Simon Riggs wrote: >On 11 January 2016 at 20:10, Andres Freund wrote: > >> On January 11, 2016 8:57:58 PM GMT+01:00, Simon Riggs >> wrote: >> >On 11 January 2016 at 18:43, Simon

Re: [HACKERS] Fuzzy substring searching with the pg_trgm extension

I gave a quick look through the patch and noticed a few minor things while trying to understand it. I think the test corpus isn't particularly interesting for how big it is. I'd rather have (a) a small corpus (say 100 words) with which to do detailed regression testing, and (b) some larger

Re: [HACKERS] Spelling corrections

On Sun, Jan 10, 2016 at 2:17 AM, David Rowley wrote: > Snap! > http://www.postgresql.org/message-id/CAKJS1f-AGgQaurTwhY=wkJjspgDcmDUE8Yx03XTXCDz=kae...@mail.gmail.com Great minds think alike. :-) -- Peter Geoghegan -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

Andres, Robert, are you still reviewing this patch? Aleksander Alekseev wrote: > Here is a clean version of the patch. Step 1 is an optimization. Step 2 > refactors this: > > HTAB * > ShmemInitHash(const char *name, /* table string name for shmem index */ > - long init_size, /*

Re: [HACKERS] ExecGather() + nworkers

On Sun, Jan 10, 2016 at 5:45 PM, Robert Haas wrote: > Well, in general, the parallel sort code doesn't really get to pick > whether or not a BackgroundWorkerSlot gets used or not. Whoever > created the parallel context decides how many workers to request, and > then the

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 20:10, Andres Freund wrote: > On January 11, 2016 8:57:58 PM GMT+01:00, Simon Riggs > wrote: > >On 11 January 2016 at 18:43, Simon Riggs wrote: > > >It's clear there are various additional tuning

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

On 2016/01/08 21:20, Rahila Syed wrote: >> I suspect you need to create a new CF entry for this patch in CF 2016-01. > > Unless I am missing something, there seems to be no entry for this patch > into CF 2016-01 page: https://commitfest.postgresql.org/8/. > Regrettably, we have exceeded the

Re: [HACKERS] [COMMITTERS] pgsql: Avoid pin scan for replay of XLOG_BTREE_VACUUM

On 11 January 2016 at 01:46, Robert Haas wrote: > /me crawls into a hole. > > Thanks. Far from it, thanks very much for thinking about it. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] PATCH: add pg_current_xlog_flush_location function

On Tue, Jan 12, 2016 at 6:35 AM, Simon Riggs wrote: > Comments in xlog.c say > > "In addition to the shared variable, each backend has a private copy of > LogwrtResult, which is updated when convenient." > It is therefore valid to update the value of both Write and Flush positions > at the same

Re: [HACKERS] extend pgbench expressions with functions

On Thu, Jan 7, 2016 at 7:00 PM, Fabien COELHO wrote: > Obviously all this is possible in the grand scale of things, but this is > also significant work for a tiny benefit, if any. I rather see "debug" as a > simple tool for debugging a script with "pgbench -t 1" (i.e. one or few > transactions),

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

On Jan 12, 2016 11:22 AM, "Amit Langote" wrote: > > On 2016/01/12 10:30, Amit Langote wrote: > > I'm slightly concerned that the latest patch doesn't incorporate any > > revisions to the original pgstat interface per Robert's comments in [1]. > > I meant to say

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 22:24, Andres Freund wrote: > Please wait till at least tomorrow evening, so I can have a meaningful > look. > No problem, make sure to look at 2pc_optimize.v4.patch -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] Proposal: BSD Authentication support

I've attached the latest version of this patch. I've fixed up an issue with the configuration scripts that I missed. On 08/01/16 12:40, Marisa Emerson wrote: There's a port for PAM, but we would prefer to use BSD Auth as its quite a lot cleaner and is standard on OpenBSD. I've attached an

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 23:11, Stas Kelvich wrote: > > > > On 11 Jan 2016, at 21:43, Simon Riggs wrote: > > > > Have you measured lwlocking as a problem? > > > > Yes. GXACT locks that wasn’t even in perf top 10 on dual Xeon moves to the > first

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

On 2016/01/12 10:30, Amit Langote wrote: > I'm slightly concerned that the latest patch doesn't incorporate any > revisions to the original pgstat interface per Robert's comments in [1]. I meant to say "originally proposed pgstat interface on this thread". Thanks, Amit -- Sent via

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

Hi Sudhir, On Jan 7, 2016 3:02 AM, "Sudhir Lonkar-2" wrote: > > Hello, > >Please find attached patch addressing following comments > I have tested this patch. > It is showing empty (null) in phase column of pg_stat_vacuum_progress, when > I switched to a unprivileged

Re: [HACKERS] pg_conversion seems rather strangely defined

>> I used to had a customer who needs to have different client and >> database encoding than the default. That is, a slightly different >> mapping between Shift-JIS and other database encoding. Due to >> unfortunate historical reasons, there are several Shift-JIS variants >> (in addition to the

Re: [HACKERS] Speedup twophase transactions

On Tue, Jan 12, 2016 at 4:57 AM, Simon Riggs wrote: > Performance tests for me show that the patch is effective; my results match > Jesper's roughly in relative numbers. > > My robustness review is that the approach and implementation are safe. > > It's clear there are

Re: [HACKERS] Speedup twophase transactions

On Tue, Jan 12, 2016 at 3:35 PM, Michael Paquier wrote: > On Tue, Jan 12, 2016 at 4:57 AM, Simon Riggs wrote: >> Performance tests for me show that the patch is effective; my results match >> Jesper's roughly in relative numbers. >> >> My

Re: [HACKERS] Weighted Stats

On Fri, Jan 08, 2016 at 04:37:36PM +1100, Haribabu Kommi wrote: > On Mon, Dec 21, 2015 at 1:50 PM, David Fetter wrote: > > On Sun, Dec 20, 2015 at 06:13:33PM -0600, Jim Nasby wrote: > >> On 11/2/15 5:46 PM, David Fetter wrote: > >> >I'd like to add weighted statistics to

Re: [HACKERS] PATCH: add pg_current_xlog_flush_location function

On 11 January 2016 at 12:01, Tomas Vondra wrote: > > On 01/11/2016 06:30 AM, Michael Paquier wrote: > >> >> Updating LogwrtResult directly when calling your new function >> GetXLogFlushRecPtr() does not strike me as a particularly good idea >> per this portion in

Re: [HACKERS] Speedup twophase transactions

> On 10 Jan 2016, at 12:15, Simon Riggs wrote: > > So we've only optimized half the usage? We're still going to cause > replication delays. Yes, replica will go through old procedures of moving data to and from file. > We can either > > 1) Skip fsyncing the

[HACKERS] Need help on pgcrypto

Trying to find a documentation which will make me understand how to secure a password column. I want the encryption to be one way and it should not be decrypted. I am able to find discrete documents but nothing of them gets me there. Please help. - -- Thanks, Rajan. -- View this message

Re: [HACKERS] looking for an array-extract-item-as-it operator

(ops, sending to the pgsql-hackers, see the complete thread below) Adding a formal suggestion after discussion: to include a fast array_getarray() function! CREATE FUNCTION array_getarray( m anyarray, idx int ) RETURNS anyarray AS $f$ -- this is a slow workaround for an (need for)

Re: [HACKERS] strange CREATE INDEX tab completion cases

On Mon, Jan 11, 2016 at 2:16 AM, Peter Eisentraut wrote: > On 12/13/15 9:16 AM, Michael Paquier wrote: >> Please see the attached to address those things (and others) with >> extra fixes for a couple of comments. > > I have ported these changes to the new world order and divided

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

On Mon, Jan 11, 2016 at 4:31 AM, Joshua D. Drake wrote: > On 01/10/2016 06:19 PM, Robert Haas wrote: > > [snip] > > No arguments with what was written above. +1. Very well written. > > > If we had an "issue" tracker rather than a bug tracker, I'd expect a >> lot more

Re: [HACKERS] Need help on pgcrypto

On 1/11/16, Michael Paquier wrote: > On Mon, Jan 11, 2016 at 10:03 PM, rajan wrote: >> Trying to find a documentation which will make me understand how to secure >> a >> password column. I want the encryption to be one way and it should not be >>

Re: [HACKERS] checkpointer continuous flushing

On 2016-01-09 16:49:56 +0100, Fabien COELHO wrote: > > Hello Andres, > > >Hm. New theory: The current flush interface does the flushing inside > >FlushBuffer()->smgrwrite()->mdwrite()->FileWrite()->FlushContextSchedule(). > >The > >problem with that is that at that point we (need to) hold a

Re: [HACKERS] PATCH: add pg_current_xlog_flush_location function

On 01/11/2016 06:30 AM, Michael Paquier wrote: Updating LogwrtResult directly when calling your new function GetXLogFlushRecPtr() does not strike me as a particularly good idea per this portion in XLogFlush(): > /* Quick exit if already known flushed */ if (record <=

Re: [HACKERS] Need help on pgcrypto

On Mon, Jan 11, 2016 at 10:03 PM, rajan wrote: > Trying to find a documentation which will make me understand how to secure a > password column. I want the encryption to be one way and it should not be > decrypted. > > I am able to find discrete documents but nothing of them

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

On Mon, Jan 11, 2016 at 6:19 AM, Amit Kapila wrote: > On Sun, Jan 10, 2016 at 11:55 PM, Tom Lane wrote: > > > > Some of the Windows buildfarm members occasionally fail like this: > > > > LOG: could not bind IPv4 socket: No error > > HINT: Is

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

On 01/11/2016 03:31 AM, Magnus Hagander wrote: We can argue about if it's actually an easier management interface ;) (as long as it is manageable via email as well as web?) I'd agree with Robert in that it will cause some more such bickering -- but only because the discussions become

Re: [HACKERS] proposal: PL/Pythonu - function ereport

On Fri, Jan 8, 2016 at 7:56 AM, Pavel Stehule wrote: >> 3. PLy_error__init__ is used for BaseError but always sets an >> attribute called spidata, I would expect that to only happen for >> SPIError not for BaseError. You'll need to pick some other way of >> attaching the

Re: [HACKERS] Accessing non catalog table in backend

On Tue, Jan 5, 2016 at 11:09 AM, Tom Lane wrote: > Atri Sharma writes: > > I fail to see the relevance of which node is getting evaluated (its a > Plan > > node BTW) for this question. The concern I had was around using SPI > inside > > executor and its

Re: [HACKERS] Accessing non catalog table in backend

On Mon, Jan 11, 2016 at 10:48 PM, Atri Sharma wrote: > Sorry, I missed this email. > > I was specifically targeting accessing tables inside Node evaluation hence > do not want to add new nodes. > > Thanks for your inputs! > > Regards, > > Atri > > On Tue, Jan 5, 2016 at

Re: [HACKERS] Accessing non catalog table in backend

Sorry, I missed this email. I was specifically targeting accessing tables inside Node evaluation hence do not want to add new nodes. Thanks for your inputs! Regards, Atri On Tue, Jan 5, 2016 at 11:43 AM, Amit Langote wrote: > On 2016/01/05 14:30, Atri Sharma

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

I wonder, > --- 2166,2213 > } > > /* > ! * If rel is a base relation, detect whether any system columns are > ! * requested from the rel. (If rel is a join relation, rel->relid will > be > ! * 0, but there can be no Var in the target list with relid 0, so we

Re: [HACKERS] 2016-01 Commitfest

These are the numbers after one week of commitfest work: Needs review: 65. Waiting on Author: 14. Ready for Committer: 6. Committed: 14. Total: 99. The attentive reader might notice that we grew one more patch since last week, which is the "VACUUM progress checker" thingy that has been

[HACKERS] Making plpython 2 and 3 coexist a bit better

Commit 803716013dc1350f installed a safeguard against loading plpython2 and plpython3 at the same time, stating + It is not allowed to use PL/Python based on Python 2 and PL/Python + based on Python 3 in the same session, because the symbols in the + dynamic modules would clash, which could

Re: [HACKERS] proposal: PL/Pythonu - function ereport

On 1/11/16 12:33 PM, Pavel Stehule wrote: 1. break compatibility and SPIError replace by Error At this point I've lost track... what's the incompatibility between the two? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

On 1/11/16 11:51 AM, Tom Lane wrote: We could ameliorate the first of these cases by putting the can't-run- with-two-pythons error back up to FATAL rather than ERROR, but I'm not sure that that would be a net improvement --- FATAL errors aren't very friendly. In any case errors of the second

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 12:58, Stas Kelvich wrote: > > > On 10 Jan 2016, at 12:15, Simon Riggs wrote: > > > > So we've only optimized half the usage? We're still going to cause > replication delays. > > Yes, replica will go through old procedures

Re: [HACKERS] 2016-01 Commitfest

On Mon, Jan 11, 2016 at 6:38 AM, Alvaro Herrera wrote: > * Partial sort That shouldn't have been in "needs review" state. Fixed. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

I wrote: > I think we might be able to improve this if we don't do the check in > plpython's _PG_init(), but delay it until we have a need to call into > libpython; which we could skip doing at _PG_init() time, at the cost > of perhaps one more flag check per plpython function call to see if >

Re: [HACKERS] Speedup twophase transactions

Hi, On 2016-01-09 15:29:11 +, Simon Riggs wrote: > Hmm, I was just preparing this for commit. Just read downthread that you want to commit this soon. Please hold of for a while, this doesn't really look ready to me. I don't have time for a real review right now, but I'll try to get to it

Re: [HACKERS] Speedup twophase transactions

On 2016-01-11 20:03:18 +0100, Andres Freund wrote: > More generally, I'm doubtful that the approach of reading data from WAL > as proposed here is a very good idea. It seems better to "just" dump the > entire 2pc state into *one* file at checkpoint time. Or better: After determining the

Re: [HACKERS] 2016-01 Commitfest

On 11 January 2016 at 14:38, Alvaro Herrera wrote: > * Simon Riggs > Fix handling on XLOG_RUNNING_XACTS generated by bgwriter on idle > systems > > I assume this means they intend to commit them in some reasonable > timeframe (possibly after some rework). If this

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-01-11 17:05 GMT+01:00 Catalin Iacob : > On Fri, Jan 8, 2016 at 7:56 AM, Pavel Stehule > wrote: > >> 3. PLy_error__init__ is used for BaseError but always sets an > >> attribute called spidata, I would expect that to only happen for > >>

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-01-11 19:41 GMT+01:00 Jim Nasby : > On 1/11/16 12:33 PM, Pavel Stehule wrote: > >> 1. break compatibility and SPIError replace by Error >> > > At this point I've lost track... what's the incompatibility between the > two? > the name and internal format (but this

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

Jim Nasby writes: > Something that's always concerned me about functions in other languages > is that any kind of snafu in the function/language can hose the backend, > which you may or may not detect. I've used other databases that (by > default) spin up a separate

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

On 1/11/16 1:00 PM, Tom Lane wrote: There's at least one PL/Java implementation that does that. The interprocess communication overhead is pretty awful, IIRC. Don't know what they do about nested calls. You'd think that pipes wouldn't be that much overhead... >Obviously this is a lot more

Re: [HACKERS] strange CREATE INDEX tab completion cases

Alvaro Herrera writes: > One thing I just noticed is that CREATE INDEX CONCURRENTLY cannot be > used within CREATE SCHEMA, so perhaps the lines that match the > CONCURRENTLY keyword should use Matches() rather than TailMatches(). > Similarly (but perhaps this is not

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 19:07, Andres Freund wrote: > On 2016-01-11 20:03:18 +0100, Andres Freund wrote: > > More generally, I'm doubtful that the approach of reading data from WAL > > as proposed here is a very good idea. It seems better to "just" dump the > > entire 2pc state

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 18:51, Jesper Pedersen wrote: > On 01/10/2016 04:15 AM, Simon Riggs wrote: > >> One concern that come into my mind while reading updated >>> patch is about creating extra bool field in GlobalTransactionData >>> structure. While this improves

Re: [HACKERS] strange CREATE INDEX tab completion cases

Peter Eisentraut wrote: > On 12/13/15 9:16 AM, Michael Paquier wrote: > > Please see the attached to address those things (and others) with > > extra fixes for a couple of comments. > > I have ported these changes to the new world order and divided them up > into more logical changes that are

Re: [HACKERS] Speedup twophase transactions

On 2016-01-11 19:15:23 +, Simon Riggs wrote: > On 11 January 2016 at 19:03, Andres Freund wrote: > > > Hi, > > > > On 2016-01-09 15:29:11 +, Simon Riggs wrote: > > > Hmm, I was just preparing this for commit. > > > > Just read downthread that you want to commit this

Re: [HACKERS] PATCH: Extending the HyperLogLog API a bit

Tomas Vondra wrote: > Attached is v2 of the patch, adding the comments. Looks pretty reasonable to me. I'm not sure we want to push this ahead of the bloom filter stuff, but it looks ready to commit otherwise. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 18:43, Simon Riggs wrote: > I'm looking to commit what we have now. > Here is the patch in its "final" state after my minor additions, edits and review. Performance tests for me show that the patch is effective; my results match Jesper's roughly in

Re: [HACKERS] Speedup twophase transactions

On January 11, 2016 8:57:58 PM GMT+01:00, Simon Riggs wrote: >On 11 January 2016 at 18:43, Simon Riggs wrote: >It's clear there are various additional tuning opportunities, but the >objective of the current patch to improve performance is very,

Re: [HACKERS] proposal: PL/Pythonu - function ereport

On 1/11/16 12:46 PM, Pavel Stehule wrote: 2016-01-11 19:41 GMT+01:00 Jim Nasby >: On 1/11/16 12:33 PM, Pavel Stehule wrote: 1. break compatibility and SPIError replace by Error At this point I've lost track... what's

Re: [HACKERS] Speedup twophase transactions

On 11 January 2016 at 19:03, Andres Freund wrote: > Hi, > > On 2016-01-09 15:29:11 +, Simon Riggs wrote: > > Hmm, I was just preparing this for commit. > > Just read downthread that you want to commit this soon. Please hold of > for a while, this doesn't really look ready

Re: [HACKERS] checkpointer continuous flushing

On 2016-01-11 14:45:16 +0100, Andres Freund wrote: > On 2016-01-09 16:49:56 +0100, Fabien COELHO wrote: > > >Hm. New theory: The current flush interface does the flushing inside > > >FlushBuffer()->smgrwrite()->mdwrite()->FileWrite()->FlushContextSchedule(). > > > The > > >problem with that is

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

Jim Nasby writes: > On 1/11/16 1:00 PM, Tom Lane wrote: >> The question of whether to do ERROR or FATAL remains open. I'm not sure >> I have a strong preference either way. > If they both get loaded is there risk of bad data happening? Personally, > I'll take a