Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-02-22 Thread Amit Langote
Fujita-san, On Thu, Feb 22, 2018 at 8:49 PM, Etsuro Fujita wrote: > (2018/02/22 11:52), Amit Langote wrote: >> I wonder why partition_index needs to be made part of this API? > > The reason for that is because I think the FDW might want to look at the > partition info stored in mtstate->mt_partit

RTLD_GLOBAL (& JIT inlining)

2018-02-22 Thread Andres Freund
Hi Peter, All, First question: Why do we currently use RTLD_GLOBAL loading extension libraries, but take pains ([1]) to make things work without RTLD_GLOBAL. It seems like it'd be both safer to RTLD_LOCAL on platforms supporting it (or equivalent), as well as less error-prone because we'd be les

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Feb 23, 2018 at 7:56 PM, Amit Kapila wrote: > On Fri, Feb 23, 2018 at 8:48 AM, Thomas Munro > wrote: >> On Fri, Feb 23, 2018 at 3:29 PM, Amit Kapila wrote: >>> By the way, in which case leader can exit early? As of now, we do >>> wait for workers to end both before the query is finished

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Amit Kapila
On Fri, Feb 23, 2018 at 8:48 AM, Thomas Munro wrote: > On Fri, Feb 23, 2018 at 3:29 PM, Amit Kapila wrote: >> On Thu, Feb 22, 2018 at 10:35 PM, Robert Haas wrote: >>> On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro PS I noticed that for BecomeLockGroupMember() we say "If we can't join t

Re: committing inside cursor loop

2018-02-22 Thread Simon Riggs
On 20 February 2018 at 14:45, Tom Lane wrote: > Peter Eisentraut writes: >> Here is a patch that allows COMMIT inside cursor loops in PL/pgSQL. As >> alluded to in earlier threads, this is done by converting such cursors >> to holdable automatically. A special flag "auto-held" marks such >> cur

Re: check error messages in SSL tests

2018-02-22 Thread Michael Paquier
On Thu, Feb 22, 2018 at 08:34:30AM -0500, Peter Eisentraut wrote: > I noticed that a couple of test cases in the SSL tests fail to connect > not for the reason that the tests think they should. Here is a patch to > augment the test setup so that a test for connection rejection also > checks that w

Re: Translations contributions urgently needed

2018-02-22 Thread Tom Lane
Alvaro Herrera writes: > Please join pgsql-translat...@postgresql.org. What surprises me about this thread is that apparently the sad state of the v10 translations wasn't already discussed on that list? I have no objection to calling for more translation volunteers on this list --- in fact, prob

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Amit Langote
On Fri, Feb 23, 2018 at 11:32 AM, Alvaro Herrera wrote: > Amit Langote wrote: >> On 2018/02/23 8:52, Alvaro Herrera wrote: >> > We could mitigate the performance loss to some extent by adding more to >> > RelationData. For example, a "is_partition" boolean would help: skip >> > searching pg_inher

Re: Translations contributions urgently needed

2018-02-22 Thread Craig Ringer
On 23 February 2018 at 09:50, Tatsuo Ishii wrote: > > I was wondering a bit about how translations are maintained. Experience > > with another highly internationalised project years ago showed that many > > people who are willing to volunteer as translators are NOT willing to > > interact with gi

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> On Fri, Feb 23, 2018 at 09:51:02AM +0900, Tatsuo Ishii wrote: >> I'm not working on message translations. Hotta-san is actively >> working on the translations here: >> >> https://github.com/hotta/pg-nls-ja > > If you need help here, feel free to ping me. I have done translation to > Japanese

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> Michael Paquier wrote: >> On Fri, Feb 23, 2018 at 09:51:02AM +0900, Tatsuo Ishii wrote: >> > I'm not working on message translations. Hotta-san is actively >> > working on the translations here: >> > >> > https://github.com/hotta/pg-nls-ja >> >> If you need help here, feel free to ping me. I

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Feb 23, 2018 at 3:29 PM, Amit Kapila wrote: > On Thu, Feb 22, 2018 at 10:35 PM, Robert Haas wrote: >> On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro >>> PS I noticed that for BecomeLockGroupMember() we say "If we can't >>> join the lock group, the leader has gone away, so just exit quietl

SSL passphrase prompt external command

2018-02-22 Thread Peter Eisentraut
Here is a patch that adds a way to specify an external command for obtaining SSL passphrases. There is a new GUC setting ssl_passphrase_command. Right now, we rely on the OpenSSL built-in prompting mechanism, which doesn't work in some situations, including under systemd. This patch allows a con

Re: Translations contributions urgently needed

2018-02-22 Thread Michael Paquier
On Thu, Feb 22, 2018 at 11:45:11PM -0300, Alvaro Herrera wrote: > How many repositories do we need, though? There is already one in > git.postgresql.org -- see https://babel.postgresql.org/ Centralizing things into postgresql.org is nice for the long term. > Please join pgsql-translat...@postgre

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
How does: On 2018-02-23 11:48:16 +0900, Michael Paquier wrote: > On Thu, Feb 22, 2018 at 11:24:37AM -0800, Andres Freund wrote: > > I suspect I'm going to get some grief for this, but I think the time has > > come to bite the bullet and support changing databases in the same > > process... > > I'

Re: Online enabling of checksums

2018-02-22 Thread Michael Paquier
On Thu, Feb 22, 2018 at 11:24:37AM -0800, Andres Freund wrote: > I suspect I'm going to get some grief for this, but I think the time has > come to bite the bullet and support changing databases in the same > process... I'd like to see that. Last time this has been discussed, and Robert complaine

Re: Translations contributions urgently needed

2018-02-22 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Feb 23, 2018 at 09:51:02AM +0900, Tatsuo Ishii wrote: > > I'm not working on message translations. Hotta-san is actively > > working on the translations here: > > > > https://github.com/hotta/pg-nls-ja > > If you need help here, feel free to ping me. I have done

Re: Translations contributions urgently needed

2018-02-22 Thread Michael Paquier
On Fri, Feb 23, 2018 at 09:51:02AM +0900, Tatsuo Ishii wrote: > I'm not working on message translations. Hotta-san is actively > working on the translations here: > > https://github.com/hotta/pg-nls-ja If you need help here, feel free to ping me. I have done translation to Japanese for other th

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/02/23 8:52, Alvaro Herrera wrote: > > We could mitigate the performance loss to some extent by adding more to > > RelationData. For example, a "is_partition" boolean would help: skip > > searching pg_inherits for a relation that is not a partition. > > Unless I'm miss

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Amit Kapila
On Thu, Feb 22, 2018 at 10:35 PM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro >> PS I noticed that for BecomeLockGroupMember() we say "If we can't >> join the lock group, the leader has gone away, so just exit quietly" >> but for various other similar things we spew errors

Re: [bug fix] Cascaded standby cannot start after a clean shutdown

2018-02-22 Thread Michael Paquier
On Thu, Feb 22, 2018 at 04:55:38PM +0900, Michael Paquier wrote: > I am definitely ready to buy that it can be possible to have garbage > being read the length field which can cause allocate_recordbuf to fail > as that's the only code path in xlogreader.c which does such an > allocation. Still, it

Re: Incorrect grammar

2018-02-22 Thread Etsuro Fujita
(2018/02/23 0:09), Robert Haas wrote: On Thu, Feb 22, 2018 at 6:54 AM, Etsuro Fujita wrote: Here is a tiny patch to fix $SUBJECT in a comment in execPartition.c. Committed. Thanks! Best regards, Etsuro Fujita

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Amit Langote
Hi David. On 2018/02/23 0:11, David Rowley wrote: > On 23 February 2018 at 01:15, David Rowley > wrote: >> One problem that I'm facing now is down to the way I'm gathering the >> ParamIds that match the partkeys. As you'll see from the patch I've >> added a 'paramids' field to PartitionPruneCont

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Amit Langote
On 2018/02/23 8:52, Alvaro Herrera wrote: > We could mitigate the performance loss to some extent by adding more to > RelationData. For example, a "is_partition" boolean would help: skip > searching pg_inherits for a relation that is not a partition. Unless I'm missing something, doesn't rd_rel->r

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> I was wondering a bit about how translations are maintained. Experience > with another highly internationalised project years ago showed that many > people who are willing to volunteer as translators are NOT willing to > interact with git (or at the time, svn) and the other tools many of us take

Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread Amit Langote
On 2018/02/22 20:28, David Rowley wrote: > On 22 February 2018 at 22:48, Amit Langote > wrote: >>> I'm having to add some NULL handling there for the run-time pruning >>> patch but wondered if it was also required for your patch. >> >> Hmm, not sure why. Can you explain a bit more? > > hmm, yeah

Re: Translations contributions urgently needed

2018-02-22 Thread Craig Ringer
On 23 February 2018 at 08:51, Tatsuo Ishii wrote: > > I also noticed the same thing few days ago, by the fact that RPM > > release of PG10 doesn't contain psql translation. > > > > @10 > ja | pg_basebackup,pg_resetxlog, > plpython,pltcl,postgres,psql > > @9.4> ja | pg_basebackup,pg_

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Amit Langote
On 2018/02/23 1:10, Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: >>> Attached is an updated version for that. >> >> Thanks for updating the patch. > > Committed with a few changes. The big one was that I got rid of the > local variable is_update in ExecSetupPartiti

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> I also noticed the same thing few days ago, by the fact that RPM > release of PG10 doesn't contain psql translation. > > @10 > ja | pg_basebackup,pg_resetxlog,plpython,pltcl,postgres,psql > @9.4> ja | pg_basebackup,pg_resetxlog > > I'm not sure how the translations are maintained

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Amit Langote
On 2018/02/23 1:53, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote >> wrote: Attached is an updated version for that. >>> >>> Thanks for updating the patch. >> >> Committed with a few changes. > > I propose to tweak a few comments to PartitionTupl

Re: Translations contributions urgently needed

2018-02-22 Thread Kyotaro HORIGUCHI
# my clone repository has been corrupted...sigh. At Thu, 22 Feb 2018 18:32:15 -0500, Robert Haas wrote in > On Thu, Feb 22, 2018 at 5:54 PM, Tatsuo Ishii wrote: > >> I have found that Japanese language support for the database server > >> has been dropped for 10. This is because it fell below

Re: PostgreSQL crashes with SIGSEGV

2018-02-22 Thread Peter Geoghegan
On Mon, Feb 12, 2018 at 6:15 PM, Peter Geoghegan wrote: > * For 9.5 and 9.6, the approach taken in bugfix commit d8589946d > should be taken even further -- we should always copy. Moreover, we > should always copy within tuplesort_getdatum(), for the same reasons. > > * For 9.5, 9.6, 10, and maste

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
>> Was that mentioned in the 10.0 release note? I didn't know that. > > I don't think we ever mention translations that are added or dropped > in minor release notes. I think it would better for users to be noticed that in the release notes because the message translations are apparently visible

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Tom Lane
Alexander Kuzmenkov writes: >> The third possibility is to decide that create_mergejoin_plan is being >> overly paranoid and it's okay to extract merge details from a "redundant" >> path key even though it specifies the opposite sort order from what the >> current merge clause seems to need. > Th

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Alvaro Herrera
Alvaro Herrera wrote: > Another option is to rethink this feature from the ground up: instead of > cloning catalog rows for each children, maybe we should have the trigger > lookup code, when running DML on the child relation (the partition), > obtain trigger entries not only for the child relatio

Re: Translations contributions urgently needed

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 5:54 PM, Tatsuo Ishii wrote: >> I have found that Japanese language support for the database server >> has been dropped for 10. This is because it fell below the 80% of >> strings translated requirement, so it was shipped without Japanese. > > Was that mentioned in the 10.

Re: [HACKERS] Pluggable storage

2018-02-22 Thread Robert Haas
On Fri, Feb 16, 2018 at 5:56 AM, Alexander Korotkov wrote: > BTW, EnterpriseDB announces zheap table access method (heap with undo log) > [2]. I think this is great, and I'm looking forward for publishing zheap in > mailing lists. But I'm concerning about its compatibility with pluggable > table

Re: Typo in predicate.c comment

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 3:10 PM, Thomas Munro wrote: > Here's a tiny patch to fix a typo. I have pushed a tiny commit containing it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Alexander Kuzmenkov
El 22/02/18 a las 21:03, Tom Lane escribió: The third possibility is to decide that create_mergejoin_plan is being overly paranoid and it's okay to extract merge details from a "redundant" path key even though it specifies the opposite sort order from what the current merge clause seems to need.

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> I have found that Japanese language support for the database server > has been dropped for 10. This is because it fell below the 80% of > strings translated requirement, so it was shipped without Japanese. Was that mentioned in the 10.0 release note? I didn't know that. Best regards, -- Tatsuo

Re: file cloning in pg_upgrade and CREATE DATABASE

2018-02-22 Thread Peter Eisentraut
On 2/21/18 18:57, Tomas Vondra wrote: > Nice improvement, of course. How does that affect performance on the > cloned database? If I understand this correctly, it essentially enables > CoW on the files, so what's the overhead on that? It'd be unfortunate to > speed up CREATE DATABASE only to get de

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Peter Geoghegan
On Thu, Feb 22, 2018 at 1:14 PM, Tomas Vondra wrote: > OK, thanks for reminding me about SBF and for the discussion. > > At this point I'll probably focus on the other parts though - > determining selectivity of the join, etc. Which I think is crucial, and > we need to get that right even for accu

Re: [HACKERS] Partition-wise aggregation/grouping

2018-02-22 Thread Robert Haas
On Thu, Feb 8, 2018 at 8:05 AM, Jeevan Chalke wrote: > In this attached version, I have rebased my changes over new design of > partially_grouped_rel. The preparatory changes of adding > partially_grouped_rel are in 0001. I spent today hacking in 0001; results attached. The big change from your

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Tomas Vondra
On 02/22/2018 09:52 PM, Claudio Freire wrote: > On Thu, Feb 22, 2018 at 5:11 PM, Tomas Vondra > wrote: >> On 02/22/2018 08:33 PM, Claudio Freire wrote: >>> That's kinda slow to do per-item. I tried to "count" distinct items by >>> checking the BF before adding (don't add redundantly), but that's

Re: Add PGDLLIMPORT to enable_hashagg

2018-02-22 Thread Andres Freund
On 2018-02-21 11:41:31 -0800, Brian Cloutier wrote: > On Wed, Feb 21, 2018 at 10:14 AM, Andres Freund wrote: > > > Could you take the relevant commit, backport it to the > > relevant branches, resolve conflicts, make possibly appropriate > > adaptions, and post? > > > > The original commit touch

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Claudio Freire
On Thu, Feb 22, 2018 at 5:11 PM, Tomas Vondra wrote: > On 02/22/2018 08:33 PM, Claudio Freire wrote: >> That's kinda slow to do per-item. I tried to "count" distinct items by >> checking the BF before adding (don't add redundantly), but that's less >> precise than a HLL in my experience. > > But y

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 15:24:50 -0500, Tom Lane wrote: > You could possibly make it work with more aggressive refactoring, but > I remain of the opinion that this is a fundamentally bad idea anyhow. > A GUC of this kind is just ripe for abuse, and I don't think it's solving > any problem we really need

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 9:23 PM, Andres Freund wrote: > Hi, > > On 2018-02-22 21:16:02 +0100, Magnus Hagander wrote: > > You could do that, but then you've moving the complexity to managing that > > list in shared memory instead. > > Maybe I'm missing something, but how are you going to get quick

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
On 2018-02-22 15:24:50 -0500, Tom Lane wrote: > Magnus Hagander writes: > > I hacked up an attempt to do this. It does seem to work in the very simple > > case, but it does requiring changing the order in InitPostgres() to load > > the startup packet before validating those. > > I doubt that's sa

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Magnus Hagander writes: > I hacked up an attempt to do this. It does seem to work in the very simple > case, but it does requiring changing the order in InitPostgres() to load > the startup packet before validating those. I doubt that's safe. It requires, to name just one thing, an assumption th

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 21:16:02 +0100, Magnus Hagander wrote: > You could do that, but then you've moving the complexity to managing that > list in shared memory instead. Maybe I'm missing something, but how are you going to get quick parallel processing if you don't have a shmem piece? You can't assi

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:52 PM, Andres Freund wrote: > > > On February 22, 2018 11:44:17 AM PST, Magnus Hagander > wrote: > >On Thu, Feb 22, 2018 at 8:41 PM, Andres Freund > >wrote: > >In this particular case that would at least phase 1 simplify it because > >we'd only need one process instead

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 9:09 PM, Tom Lane wrote: > Andres Freund writes: > > The more important part I think is that we solve it via a GUC that can > > be used outside of bgworkers. > > Are you proposing an "ignore_datallowconn" GUC? That's a remarkably > bad idea. We don't have infrastructure

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Tomas Vondra
On 02/22/2018 08:33 PM, Claudio Freire wrote: > On Thu, Feb 22, 2018 at 12:45 PM, Tomas Vondra > wrote: >> >> >> On 02/22/2018 12:44 PM, Claudio Freire wrote: >>> ... >>> >>> An HLL can be used to estimate set size, the paper makes no >>> mention of it, probably assuming only distinct items are ad

Re: Online enabling of checksums

2018-02-22 Thread Peter Eisentraut
On 2/22/18 12:38, Magnus Hagander wrote: > I'm not entirely sure which the others ones are. Auto-Vacuum obviously > is one, which doesn't use the worker infrastructure. But I'm not sure > which the others are referring to?  autovacuum, subscription workers, auto prewarm -- Peter Eisentraut

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Andres Freund writes: > The more important part I think is that we solve it via a GUC that can > be used outside of bgworkers. Are you proposing an "ignore_datallowconn" GUC? That's a remarkably bad idea. We don't have infrastructure that would allow it to be set at an appropriate scope. I can

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Magnus Hagander writes: > On Thu, Feb 22, 2018 at 8:26 PM, Andres Freund wrote: >> What's the argument against? > Complexity for the bgw usecase. They'd be completely different implementations and code paths, no? For pg_upgrade to use such a thing it'd need to be a connection parameter of some

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
On February 22, 2018 11:44:17 AM PST, Magnus Hagander wrote: >On Thu, Feb 22, 2018 at 8:41 PM, Andres Freund >wrote: >In this particular case that would at least phase 1 simplify it because >we'd only need one process instead of worker/launcher. However, if we'd >ever want to parallellize it -

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Tom Lane
I wrote: > The third possibility is to decide that create_mergejoin_plan is being > overly paranoid and it's okay to extract merge details from a "redundant" > path key even though it specifies the opposite sort order from what the > current merge clause seems to need. This is scary at first glanc

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
On 2018-02-22 20:30:02 +0100, Magnus Hagander wrote: > Complexity for the bgw usecase. It now has to construct a key/value pair > with proper escaping (well, for this one flag it would be easy, but if we > do that wouldn't we also support the other config params? Were you thinking > we'd have *just

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:41 PM, Andres Freund wrote: > > On 2018-02-22 20:30:52 +0100, Magnus Hagander wrote: > > On Thu, Feb 22, 2018 at 8:24 PM, Andres Freund > wrote: > > > I suspect I'm going to get some grief for this, but I think the time > has > > > come to bite the bullet and support ch

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 20:30:52 +0100, Magnus Hagander wrote: > On Thu, Feb 22, 2018 at 8:24 PM, Andres Freund wrote: > > I suspect I'm going to get some grief for this, but I think the time has > > come to bite the bullet and support changing databases in the same > > process... > > > > Hey, I can't

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Claudio Freire
On Thu, Feb 22, 2018 at 12:45 PM, Tomas Vondra wrote: > > > On 02/22/2018 12:44 PM, Claudio Freire wrote: >> Let me reiterate, you can avoid both issues with scalable bloom filters[1]. >> > > I'm afraid it's not as straight-forward as "Use scalable bloom filters!" > > This is not merely a question

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:24 PM, Andres Freund wrote: > On 2018-02-22 08:22:48 -0500, Peter Eisentraut wrote: > > On 2/21/18 15:53, Magnus Hagander wrote: > > > *Two new functions are added, pg_enable_data_checksums() and > > > pg_disable_data_checksums(). The disable one is easy -- it just chang

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:26 PM, Andres Freund wrote: > Hi, > > On 2018-02-22 20:24:03 +0100, Magnus Hagander wrote: > > In a background worker you can just set the parameter using > > SetConfigOption(), no? That seems a lot easier than turning things in to > a > > kv pair and back... > > Sure, b

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 20:24:03 +0100, Magnus Hagander wrote: > In a background worker you can just set the parameter using > SetConfigOption(), no? That seems a lot easier than turning things in to a > kv pair and back... Sure, but, it doesn't seem bad to offer the option to only allow this for code

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
On 2018-02-22 08:22:48 -0500, Peter Eisentraut wrote: > On 2/21/18 15:53, Magnus Hagander wrote: > > *Two new functions are added, pg_enable_data_checksums() and > > pg_disable_data_checksums(). The disable one is easy -- it just changes > > to disable. The enable one will change the state to inpro

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:17 PM, Andres Freund wrote: > Hi, > > On 2018-02-22 19:01:35 +0100, Magnus Hagander wrote: > > In working on the checksumhelper patch, we came across wanting a > background > > worker to be allowed to bypass datallowconn for a database. Right now we > > didn't take care

Re: Allow workers to override datallowconn

2018-02-22 Thread Simon Riggs
On 22 February 2018 at 18:24, Tom Lane wrote: >> Are there any other caveats in doing that this actually makes it dangerous >> to just allow bypassing it for extensions? > > Don't think so; we autovacuum such DBs anyway don't we? Yeh, there is already precedent that should mean it is easy/defaul

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 19:01:35 +0100, Magnus Hagander wrote: > In working on the checksumhelper patch, we came across wanting a background > worker to be allowed to bypass datallowconn for a database. Right now we > didn't take care of that, and just said "you have to ALTER TABLE" first. I suspect yo

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 7:24 PM, Tom Lane wrote: > Magnus Hagander writes: > > Attached is a patch that adds new Override versions of the functions to > > connect to a database from a background worker. > > > Another option would be to just add the parameter directly to the regular > > connectio

Re: [HACKERS] PoC: full merge join on comparison clause

2018-02-22 Thread Alexander Kuzmenkov
Here are some updates on this patch. I split it into two parts. The preparatory part contains some mechanical changes to prepare for the main part. Most importantly, a new field is added, `RestrictInfo.is_mj_equality`. It is a marker of mergejoinable equality clauses, and `RestrictInfo.mergeop

Re: SHA-2 functions

2018-02-22 Thread Peter Eisentraut
On 2/22/18 01:05, Michael Paquier wrote: > On Wed, Feb 21, 2018 at 03:45:17PM -0500, Peter Eisentraut wrote: >> On 2/20/18 23:04, Michael Paquier wrote: >>> I think that crypto_hash.c or hash_crypt.c would be adapted as well. >>> crypt.c is too much generic, so including both concepts in the name i

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Magnus Hagander writes: > Attached is a patch that adds new Override versions of the functions to > connect to a database from a background worker. > Another option would be to just add the parameter directly to the regular > connection function, and not create separate functions. But that would

Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
In working on the checksumhelper patch, we came across wanting a background worker to be allowed to bypass datallowconn for a database. Right now we didn't take care of that, and just said "you have to ALTER TABLE" first. Specifically for this usecase that is OK, but not paticularly user friendly.

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Tom Lane
Alexander Kuzmenkov writes: > explain select * from j1_tbl full join (select * from j2_tbl order by > j2_tbl.i desc, j2_tbl.k) j2_tbl on j1_tbl.i = j2_tbl.i and j1_tbl.i = > j2_tbl.k; > ERROR: left and right pathkeys do not match in mergejoin Nice example. There are several places that we cou

Re: Translations contributions urgently needed

2018-02-22 Thread Ioseph Kim
Hi, I’m translating to Korean since 10 years over. I think this issue should be left to the local community. It may be omitted in some versions, or added in some versions. because, it depends on volunteering. Let It Be! Regards Ioseph. On Thu, Feb 22, 2018 at 05:29:11PM +, Thom Brown wrote:

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 4:47 PM, Andrey Borodin wrote: > Hello, Magnus, Peter! > > I'm excited that this feature emerged, thanks for the patch. Hope it will > help to fix some mistakes made during initdb long time ago... > > 22 февр. 2018 г., в 18:22, Peter Eisentraut com> написал(а): > > On 2/2

Re: Translations contributions urgently needed

2018-02-22 Thread Thom Brown
On 22 February 2018 at 17:24, Magnus Hagander wrote: > On Thu, Feb 22, 2018 at 6:20 PM, Thom Brown wrote: >> >> Hi, >> >> I have found that Japanese language support for the database server >> has been dropped for 10. This is because it fell below the 80% of >> strings translated requirement, so

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2018-02-22 Thread Nikolay Shaplov
В письме от 3 сентября 2017 11:45:43 пользователь Alvaro Herrera написал: > I think we should split this in at least two commits, I've added a third part of this patch to commitfest: https://www.postgresql.org/message-id/flat/2083183.Rn7qOxG4Ov@x200m#2083183.Rn7qOxG4Ov@x200m To finally commit the

Re: Translations contributions urgently needed

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 6:20 PM, Thom Brown wrote: > Hi, > > I have found that Japanese language support for the database server > has been dropped for 10. This is because it fell below the 80% of > strings translated requirement, so it was shipped without Japanese. > This isn't true of all comp

Re: Translations contributions urgently needed

2018-02-22 Thread Pavel Stehule
2018-02-22 18:20 GMT+01:00 Thom Brown : > Hi, > > I have found that Japanese language support for the database server > has been dropped for 10. This is because it fell below the 80% of > strings translated requirement, so it was shipped without Japanese. > This isn't true of all components, but

Translations contributions urgently needed

2018-02-22 Thread Thom Brown
Hi, I have found that Japanese language support for the database server has been dropped for 10. This is because it fell below the 80% of strings translated requirement, so it was shipped without Japanese. This isn't true of all components, but it seems quite alarming that we've pushed out Postgr

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro wrote:> > The best solution I have come up with so far is to add a reference > count to SERIALIZABLEXACT. I toyed with putting the refcount into the > DSM instead, but then I ran into problems making that work when you > have a query with multiple Gat

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 11:53 AM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote >> wrote: >> >> Attached is an updated version for that. >> > >> > Thanks for updating the patch. >> >> Committed with a few changes. > > I propose to tweak a few comments

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: > >> Attached is an updated version for that. > > > > Thanks for updating the patch. > > Committed with a few changes. I propose to tweak a few comments to PartitionTupleRouting, as attached. -- Álvaro Herrera

[PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2018-02-22 Thread Nikolay Shaplov
This is part or my bigger patch https://www.postgresql.org/message-id/flat/2146419.veIEZdk4E4@x200m#2146419.veIEZdk4E4@x200m we've decided to commit by smaller parts. Now in postgres an StdRdOptions structure is used as binary represenations of reloptions for heap, toast, and some indexes. It h

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Robert Haas
On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote wrote: >> Attached is an updated version for that. > > Thanks for updating the patch. Committed with a few changes. The big one was that I got rid of the local variable is_update in ExecSetupPartitionTupleRouting. That saved a level of indentation o

Re: Online enabling of checksums

2018-02-22 Thread Andrey Borodin
Hello, Magnus, Peter! I'm excited that this feature emerged, thanks for the patch. Hope it will help to fix some mistakes made during initdb long time ago... > 22 февр. 2018 г., в 18:22, Peter Eisentraut > написал(а): > > On 2/21/18 15:53, Magnus Hagander wrote: >> *Two new functions are adde

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Tomas Vondra
On 02/22/2018 12:44 PM, Claudio Freire wrote: > On Wed, Feb 21, 2018 at 11:21 PM, Tomas Vondra > wrote: >> On 02/21/2018 02:10 AM, Peter Geoghegan wrote: >>> ... >>> I misunderstood. I would probably do something like double or triple >>> the original rows estimate instead, though. The estimate

Re: phasing out pg_pltemplate?

2018-02-22 Thread Tom Lane
Peter Eisentraut writes: > What is the plan for pg_pltemplate? Is there a roadmap to get rid of > it? (It's not currently blocking anything for me. I'm just wondering.) I think it's just waiting for someone to put in the effort to make it unnecessary. It seems like the extension mechanism cou

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread David Rowley
On 23 February 2018 at 01:15, David Rowley wrote: > One problem that I'm facing now is down to the way I'm gathering the > ParamIds that match the partkeys. As you'll see from the patch I've > added a 'paramids' field to PartitionPruneContext and I'm populating > this when the clauses are being pr

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Jesper Pedersen
Hi David, On 02/21/2018 04:06 AM, David Rowley wrote: I've attached v11 of the patch. Are UPDATE and DELETE suppose to be supported ? With -- test.sql -- CREATE TABLE test (a integer NOT NULL, b integer) PARTITION BY HASH(a); CREATE TABLE test_p00 PARTITION OF test FOR VALUES WITH (MODULUS

Re: Incorrect grammar

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 6:54 AM, Etsuro Fujita wrote: > Here is a tiny patch to fix $SUBJECT in a comment in execPartition.c. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: PlaceHolderVars in pushed down child-join cause error

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 7:41 AM, Ashutosh Bapat wrote: > postgres_fdw isn't expected to push down joins with placeholder vars. > But the check for that in foreign_join_ok() only considers > joinrel->relids. For a child-join relids contains the child relids but > PlaceHolderInfo refers to the top p

Re: pgsql: Avoid valgrind complaint about write() of uninitalized bytes.

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 3:18 PM, Peter Geoghegan wrote: > On Wed, Feb 21, 2018 at 10:55 AM, Peter Geoghegan wrote: >> Sure, but it looks like it has the exact same underlying cause to the >> LogicalTapeFreeze() issue. It shouldn't be very hard to write an >> equivalent patch for LogicalTapeRewind

Re: Possible performance regression in version 10.1 with pgbench read-write tests.

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 10:03 PM, Mithun Cy wrote: > seeing futex in the call stack andres suggested that following commit could > be the reason for regression > > commit ecb0d20a9d2e09b7112d3b192047f711f9ff7e59 > Author: Tom Lane > Date: 2016-10-09 18:03:45 -0400 > > Use unnamed POSIX sema

Re: [doc fix] Correct calculation of vm.nr_hugepages

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 3:28 PM, Justin Pryzby wrote: > On Wed, Feb 21, 2018 at 03:14:57PM -0500, Robert Haas wrote: >> On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki >> wrote: >> > Thanks, I'd like to take this. >> >> Why are these values so large? The example in the documentation shows >

Re: pgsql: Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes.

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 8:31 AM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 12:35 AM, Tom Lane wrote: >> Robert Haas writes: >>> Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes. >> >> This seems to have produced some plan instability in the buildfarm. > > I was worried about that.

check error messages in SSL tests

2018-02-22 Thread Peter Eisentraut
I noticed that a couple of test cases in the SSL tests fail to connect not for the reason that the tests think they should. Here is a patch to augment the test setup so that a test for connection rejection also checks that we get the expected error message. -- Peter Eisentraut http:

  1   2   >