Re: Oddity in tuple routing for foreign partitions

On 2018/04/17 11:13, Kyotaro HORIGUCHI wrote: >>> Also, I removed the CheckValidResultRel check from ExecInitRoutingInfo >>> and added that to ExecInitPartitionInfo right after the> InitResultRelInfo >>> call, >>> because it would be better to abort the >>> operation as soon as we find the partiti

Re: Oddity in tuple routing for foreign partitions

(2018/04/17 16:10), Amit Langote wrote: On 2018/04/17 11:13, Kyotaro HORIGUCHI wrote: If I'm reading this correctly, ExecInitParititionInfo calls ExecInitRoutingInfo so currently CheckValidityResultRel is called for the child when partrel is created in ExecPrepareTupleRouting. But the move of Ch

Re: Built-in connection pooling

On 13.04.2018 19:07, Nikolay Samokhvalov wrote: On Fri, Apr 13, 2018 at 2:59 AM, Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote: Development in built-in connection pooling will be continued in https://github.com/postgrespro/postgresql.builtin_pool.git

Re: ON CONFLICT DO UPDATE for partitioned tables

On 2018/04/17 4:10, Alvaro Herrera wrote: > Amit Langote wrote: > >> The solution I came up with is to call map_variable_attnos() directly, >> instead of going through map_partition_varattnos() every time, after first >> creating the attribute map ourselves. > > Yeah, sounds good. I added a twea

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

(2018/02/20 18:13), Ashutosh Bapat wrote: Here's patchset implementing this solution. 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its callers. 0002 fixes a similar bug for regular partitioned tables. The patch has testcase. The commit message explains the bug in more det

Re: ON CONFLICT DO UPDATE for partitioned tables

On 2018/04/17 16:45, Amit Langote wrote: > Instead of doing this, I think we should try to make > convert_tuples_by_name_map() a bit smarter by integrating the logic in > convert_tuples_by_name() that's used conclude if no tuple conversion is > necessary. So, if it turns that the tuples descriptor

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

On Tue, Apr 17, 2018 at 2:05 PM, Etsuro Fujita wrote: > (2018/02/20 18:13), Ashutosh Bapat wrote: >> >> Here's patchset implementing this solution. >> >> 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its >> callers. >> >> 0002 fixes a similar bug for regular partitioned table

Test coverage for mark_invalid_subplans_as_finished

Alvaro pinged me off list to mention the coverage tool indicates that there's no test coverage for mark_invalid_subplans_as_finished() [1]. The attached patch aims to put that right. This function would only ever be called during a Parallel Append in a query which has some exec Params being compar

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

(2018/04/17 18:43), Ashutosh Bapat wrote: On Tue, Apr 17, 2018 at 2:05 PM, Etsuro Fujita wrote: (2018/02/20 18:13), Ashutosh Bapat wrote: Here's patchset implementing this solution. 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its callers. 0002 fixes a similar bug fo

Re: WIP: Covering + unique indexes.

On Mon, Apr 16, 2018 at 1:05 AM, Peter Geoghegan wrote: > Attached patch makes the changes that I talked about, and a few > others. The commit message has full details. The general direction of > the patch is that it documents our assumptions, and verifies them in > more cases. Most of the change

Re: pg_recvlogical broken in back branches

2018-04-17 3:38 GMT-03:00 Michael Paquier : > The exact same fix has already applied on all stable branches: > Sorry about the noise. I've only checked the REL9_6_8 tag and the tarball. -- Euler Taveira Timbira - http://www.timbira.com.br/ PostgreSQL: Cons

Re: Proposal: Adding json logging

On 4/16/18 23:12, Michael Paquier wrote: >> I have also had good success using syslog. While syslog is not very >> structured, the setting syslog_split_messages allows sending log entries >> that include newlines in one piece, which works well if you have some >> kind of full-text search engine at

Re: Proposal: Adding json logging

David Arnold wrote: > Interesting, does that implicitly mean the whole log event would get > transmitted as a "line" (with CRLF) in CSV. To me it's implied by the doc at: https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG > In the aff

remove quoting hacks and simplify bootscanner.l

For the bootstrap data conversion, it was desirable for postgres.bki to remain unchanged, so some ugly quoting hacks were added to genbki.pl to match the quoting conventions in the DATA() lines. At this point, it's possible (and worthwhile I think) to remove those, and along the way simplify the to

Re: Proposal: Adding json logging

>To me it's implied by the doc at: https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG Additionally this still depends on the way some middleware might choose to stream data. Can we really be sure the risk is minimal that any middleware would ha

Re: Proposal: Adding json logging

This discussion is thriving, and long and behold, we've got an opinion from Eduardo (fluent-bit): https://github.com/fluent/fluent-bit/issues/564#issuecomment-381844419 >Also consider that in not all scenarios full multiline logs are flushed right away, sometimes there are delays. I think this l

Re: [HACKERS] proposal: schema variables

Hello Pavel, On Thu, Oct 26, 2017 at 09:21:24AM +0200, Pavel Stehule wrote: > I hope so this proposal is good enough and simple. > > Comments, notes? As I understood variables are stored in pg_class table. Did you consider storing variables in a special catalog table? It can be named as pg_varia

Re: Proposal: Adding json logging

One issue I haven't seen mentioned in this thread is the translation status of the server message (as well as its encoding): it's possible to receive messages in some random language if the lc_message setting is changed. Requiring that lc_messages must always be set to some English locale seems li

Re: ON CONFLICT DO UPDATE for partitioned tables

Amit Langote wrote: > Attached find a patch that does that. When working on this, I noticed > that when recursing for inheritance children, ATPrepAlterColumnType() > would use a AlterTableCmd (cmd) that's already scribbled on as if it were > the original. While I agree that the code here is in p

Re: ON CONFLICT DO UPDATE for partitioned tables

Amit Langote wrote: > I just confirmed my hunch that this wouldn't somehow do the right thing > when the OID system column is involved. Like this case: This looks too big a patch to pursue now. I'm inclined to just remove the equalTupdesc changes. -- Álvaro Herrerahttps://www.

Re: Test coverage for mark_invalid_subplans_as_finished

Thanks for that, pushed. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Proposal: Adding json logging

Alvaro, just to clarify for me, do you refer to the messages generated by https://github.com/postgres/postgres/blob/master/src/backend/utils/error/elog.c or other messages? Standardizing on UTF8 seems a good option. Assuming it* is* a problem, I would classify this as another second-order problem,

Re: Speedup of relation deletes during recovery

On Fri, Mar 30, 2018 at 11:46 AM, Michael Paquier wrote: > On Fri, Mar 30, 2018 at 11:19:58AM +0900, Kyotaro HORIGUCHI wrote: >> At Fri, 30 Mar 2018 08:31:29 +0900, Fujii Masao >> wrote in >>> When multiple relations are deleted at the same transaction, >>> the files of those relations are dele

Re: Typos from Covering Index patch

On 11/04/18 03:52, Michael Paquier wrote: Hi all, I am catching up with new features so I have begun going through Covering indexes. While reading the code, I have noticed a couple of things: 1) Some typos. 2) An inconsistent variable name in pg_dump. Committed, thanks! - Heikki

Re: Typos from Covering Index patch

On 11/04/18 03:52, Michael Paquier wrote: Hi all, I am catching up with new features so I have begun going through Covering indexes. While reading the code, I have noticed a couple of things: 1) Some typos. 2) An inconsistent variable name in pg_dump. Committed, thanks! - Heikki

Re: Sample values for pg_stat_statements

On 03/10/2018 03:02 PM, Tomas Vondra wrote: > Hi, > > I've looked at this patch today. I like the idea / intent in general, as > it helps with some investigation tasks. That being said, I have a couple > of questions/comments based on read through the patch: Thanks! Attached is a patch addressin

Re: Speedup of relation deletes during recovery

On Fri, Mar 30, 2018 at 12:18 PM, Tsunakawa, Takayuki wrote: > From: Fujii Masao [mailto:masao.fu...@gmail.com] >> When multiple relations are deleted at the same transaction, the files of >> those relations are deleted by one call to smgrdounlinkall(), which leads >> to scan whole shared_buffers

Re: Deadlock in multiple CIC.

I wrote: > So we can now refine the problem statement to "SnapshotResetXmin isn't > doing what it's supposed to". No idea why yet. 9.4 is using a simple > RegisteredSnapshots counter which 9.5 has replaced with a pairing heap, > so you'd think the newer code would be *more* likely to have bugs...

Re: [HACKERS] proposal: schema variables

Hi 2018-04-17 16:14 GMT+02:00 Arthur Zakirov : > Hello Pavel, > > On Thu, Oct 26, 2017 at 09:21:24AM +0200, Pavel Stehule wrote: > > I hope so this proposal is good enough and simple. > > > > Comments, notes? > > As I understood variables are stored in pg_class table. Did you consider > storing v

Re: WIP: Covering + unique indexes.

On Tue, Apr 17, 2018 at 3:12 AM, Alexander Korotkov wrote: > Hmm, what do you think about making BTreeTupGetNAtts() take tupledesc > argument, not relation> It anyway doesn't need number of key attributes, > only total number of attributes. Then _bt_isequal() would be able to use > BTreeTupGetNA

[ANN] PG-Strom v2.0 is released

Hello, PG-Strom v2.0 is released. It is an extension module for PostgreSQL to accelerate reporting and analytics workloads towards large scale data set using GPU. Major enhancement in PG-Strom v2.0 includes: - Overall redesign of the internal infrastructure to manage GPU and stabilization - CPU+

pgindent run soon?

Now that feature freeze is past, I wonder if it's time to run pgindent. Last year we did a run immediately after beta1, plus one just before branching off REL_10_STABLE. The value of an early run, IMO, is to get most of the changes in place so that people have a stable base to work from while reb

Re: Built-in connection pooling

Understood. One more question. Have you considered creation of pooling tool as a separate, not built-in tool, but being shipped with Postgres — like psql is shipped in packages usually called “postgresql-client-XX” which makes psql the default tool to work in terminal? I constantly hear opinion fr

Re: WIP: a way forward on bootstrap data

On 4/6/18, Tom Lane wrote: > I experimented with converting all frontend code to include just the > catalog/pg_foo_d.h files instead of catalog/pg_foo.h, as per the > proposed new policy. I soon found that we'd overlooked one thing: > some clients expect to see the relation OID macros, eg > Large

reloption to prevent VACUUM from truncating empty pages at the end of relation

Hi, I'd like to propose to add $SUBJECT for performance improvement. When VACUUM tries to truncate the trailing empty pages, it scans shared_buffers to invalidate the pages-to-truncate during holding an AccessExclusive lock on the relation. So if shared_buffers is huge, other transactions need to

Re: WIP: a way forward on bootstrap data

John Naylor writes: > On 4/6/18, Tom Lane wrote: >> Some of the CATALOG lines spill well past 80 characters with this, >> although many of the affected ones already were overlength, eg ... > Thinking about this some more, a way occurred to me to shorten the > CATALOG lines while still treating a

Re: remove quoting hacks and simplify bootscanner.l

John Naylor writes: > For the bootstrap data conversion, it was desirable for postgres.bki > to remain unchanged, so some ugly quoting hacks were added to > genbki.pl to match the quoting conventions in the DATA() lines. At > this point, it's possible (and worthwhile I think) to remove those, > an

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

Fujii Masao writes: > When VACUUM tries to truncate the trailing empty pages, it scans > shared_buffers > to invalidate the pages-to-truncate during holding an AccessExclusive lock on > the relation. So if shared_buffers is huge, other transactions need to wait > for > a very long time before ac

Re: Deadlock in multiple CIC.

Tom Lane wrote: > It's still not entirely clear what's happening on okapi, but in the > meantime I've thought of an easily-reproducible way to cause similar > failures in any branch. That is to run CREATE INDEX CONCURRENTLY > with default_transaction_isolation = serializable. Then, snapmgr.c > w

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

On Tue, Apr 17, 2018 at 11:09 AM, Tom Lane wrote: > So rather than a klugy solution that only fixes > VACUUM (and not very well, requiring user intervention and an unpleasant > tradeoff), we ought to look at ways to avoid needing a whole-pool scan to > find the pages belonging to one relation. In

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

Tom Lane wrote: > Fujii Masao writes: > > When VACUUM tries to truncate the trailing empty pages, it scans > > shared_buffers > > to invalidate the pages-to-truncate during holding an AccessExclusive lock > > on > > the relation. So if shared_buffers is huge, other transactions need to wait > >

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

Alvaro Herrera writes: > Andres was working on a radix tree structure to fix this problem, but > that seems to be abandoned now, and it seems a major undertaking. While > I agree that the proposed solution is a wart, it seems much better than > no solution at all. Can we consider Fujii's proposa

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

On 2018-04-17 15:09:18 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > Andres was working on a radix tree structure to fix this problem, but > > that seems to be abandoned now, and it seems a major undertaking. I hope to re-ignite work on that later in the v12 cycle. But realistically that me

Re: Gotchas about pg_verify_checksums

Hi Michael, On Mon, Apr 16, 2018 at 11:30:30AM +0900, Michael Paquier wrote: > On Thu, Apr 12, 2018 at 05:47:29AM +0900, Michael Paquier wrote: > > On Wed, Apr 11, 2018 at 10:21:29PM +0200, Daniel Gustafsson wrote: > >> Naming it pg_checksums, with only verification as an option, seems to me to >

Append's first_partial_plan

David Rowley wrote in https://postgr.es/m/CAKJS1f8o2Yd=rOP=et3a0fwgf+gsaokfsu6enhngztpv7nn...@mail.gmail.com > I've made another pass over the nodeAppend.c code and I'm unable to > see what might cause this, although I did discover a bug where > first_partial_plan is not set taking into account t

Re: Setting rpath on llvmjit.so?

On Sat, Apr 14, 2018 at 5:13 PM, Andres Freund wrote: > The half unmaintainedness of autoconf (no release in five years counts > as that imo), sure makes it look like a good idea to move on to cmake or > such at some point... I don't necessarily see that as a reason to move. If we're hitting bug

Repeated crashes in GENERATED ... AS IDENTITY tests

woodlouse just showed a failure that looked a bit familiar: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=woodlouse&dt=2018-04-17%2016%3A42%3A43 I trawled the buildfarm logs for similar failures, and there are several: sysname | snapshot | stage |

Re: Setting rpath on llvmjit.so?

Robert Haas writes: > ... But switching build systems > won't be frictionless, I think we have a nominee for Understatement of the Year. regards, tom lane

Re: Setting rpath on llvmjit.so?

On 2018-04-17 15:56:14 -0400, Robert Haas wrote: > On Sat, Apr 14, 2018 at 5:13 PM, Andres Freund wrote: > > The half unmaintainedness of autoconf (no release in five years counts > > as that imo), sure makes it look like a good idea to move on to cmake or > > such at some point... > > I don't ne

Re: Setting rpath on llvmjit.so?

Hi, On 2018-04-16 11:26:14 +0900, Yuriy Zhuravlev wrote: > My cmake branch still working and I supporting stable postgres releases: > https://github.com/stalkerg/postgres_cmake > Anyway, my branch exist, cmake working fine maybe without minor features > like generating documentation. > We can st

Re: pgsql: Store 2PC GID in commit/abort WAL recs for logical decoding

On 10/04/18 03:24, Michael Paquier wrote: + /* twophase_gid follows if XINFO_HAS_GID. As a null-terminated string. */ + /* xl_xact_origin follows if XINFO_HAS_ORIGIN, stored unaligned! */ Worth mentioning that the first one is also unaligned with your patch? Hmm. 'twophase_gid' is actually

Re: partitioning code reorganization

Amit Langote wrote: > 0001-Make-copying-of-cached-partitioning-info-more-con.patch > 0002-Cache-all-partitioning-info-under-one-memory-cont.patch > 0003-Cache-partsupfunc-separately-from-PartitionKey.patch I'd rather not do these patches now, unless there is some pressing reason to (eg. some bug

Re: [sqlsmith] Unpinning error in parallel worker

On Wed, Mar 29, 2017, at 10:50, Robert Haas wrote: > On Wed, Mar 29, 2017 at 1:31 AM, Thomas Munro > wrote: > > I considered whether the error message could be improved but it > > matches the message for an existing similar case (where you try to > > attach to an unknown handle). > > Ugh, OK. I

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 10, 2018 at 05:54:40PM +0100, Greg Stark wrote: > On 10 April 2018 at 02:59, Craig Ringer wrote: > > > Nitpick: In most cases the kernel reserves disk space immediately, > > before returning from write(). NFS seems to be the main exception > > here. > > I'm kind of puzzled by this. S

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Mon, Apr 9, 2018 at 03:42:35PM +0200, Tomas Vondra wrote: > On 04/09/2018 12:29 AM, Bruce Momjian wrote: > > > > An crazy idea would be to have a daemon that checks the logs and > > stops Postgres when it seems something wrong. > > > > That doesn't seem like a very practical way. It's better

Re: Append's first_partial_plan

On 18 April 2018 at 07:52, Alvaro Herrera wrote: > While looking at this patch I became curious as to why do we even have > first_partial_plan in the first place; it seems to require some strange > contortions in the code. Wouldn't it be simpler to have two lists, one > for non-partial and anothe

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Mon, Apr 9, 2018 at 03:42:35PM +0200, Tomas Vondra wrote: > On 04/09/2018 12:29 AM, Bruce Momjian wrote: > > > > An crazy idea would be to have a daemon that checks the logs and > > stops Postgres when it seems something wrong. > > > > That doesn't seem like a very practical way. It's better

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 2018-04-17 17:29:17 -0400, Bruce Momjian wrote: > Also, if we are relying on WAL, we have to make sure WAL is actually > safe with fsync, and I am betting only the O_DIRECT methods actually > are safe: > > #wal_sync_method = fsync# the default is the first > option >

Re: Append's first_partial_plan

David Rowley wrote: > On 18 April 2018 at 07:52, Alvaro Herrera wrote: > > While looking at this patch I became curious as to why do we even have > > first_partial_plan in the first place; it seems to require some strange > > contortions in the code. Wouldn't it be simpler to have two lists, one

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 2018-04-17 17:32:45 -0400, Bruce Momjian wrote: > On Mon, Apr 9, 2018 at 03:42:35PM +0200, Tomas Vondra wrote: > > That doesn't seem like a very practical way. It's better than nothing, > > of course, but I wonder how would that work with containers (where I > > think you may not have access to

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Mon, Apr 9, 2018 at 12:25:33PM -0700, Peter Geoghegan wrote: > On Mon, Apr 9, 2018 at 12:13 PM, Andres Freund wrote: > > Let's lower the pitchforks a bit here. Obviously a grand rewrite is > > absurd, as is some of the proposed ways this is all supposed to > > work. But I think the case we're

Re: pruning disabled for array, enum, record, range type partition keys

Amit Langote wrote: > Ah, I think I got it after staring at the (btree) index code for a bit. > > What pruning code got wrong is that it's comparing the expression type > (type of the constant arg that will be compared with partition bound > datums when pruning) with the partopcintype to determin

Re: [sqlsmith] Unpinning error in parallel worker

On Wed, Apr 18, 2018 at 8:52 AM, Jonathan Rudenberg wrote: > Hundreds of queries stuck with a wait_event of DynamicSharedMemoryControlLock > and pg_terminate_backend did not terminate the queries. > > In the log: > >> FATAL: cannot unpin a segment that is not pinned Thanks for the report. That

Re: [sqlsmith] Unpinning error in parallel worker

On Tue, Apr 17, 2018, at 18:38, Thomas Munro wrote: > I don't have any theories about how that could be going wrong right > now, but I'm looking into it. Thank you! > > I don't have a backtrace yet, but I will provide them if/when the issue > > happens again. > > Thanks, that would be much

Re: [sqlsmith] Unpinning error in parallel worker

On Wed, Apr 18, 2018 at 11:01 AM, Jonathan Rudenberg wrote: > On Tue, Apr 17, 2018, at 18:38, Thomas Munro wrote: >> Thanks, that would be much appreciated, as would any clues about what >> workload you're running. Do you know what the query plan looks like >> for the queries that crashed? > > Ye

Re: [sqlsmith] Unpinning error in parallel worker

On Tue, Apr 17, 2018, at 19:31, Thomas Munro wrote: > On Wed, Apr 18, 2018 at 11:01 AM, Jonathan Rudenberg > wrote: > > On Tue, Apr 17, 2018, at 18:38, Thomas Munro wrote: > >> Thanks, that would be much appreciated, as would any clues about what > >> workload you're running. Do you know what the

Should we add GUCs to allow partition pruning to be disabled?

In PG10 the planner's partition pruning could be disabled by changing the constraint_exclusion GUC to off. This is still the case for PG11, but only for UPDATE and DELETE queries. There is currently no way to disable partition pruning for SELECT. Should we allow this? To make this a bit more com

Re: Gotchas about pg_verify_checksums

On Tue, Apr 17, 2018 at 09:43:47PM +0200, Michael Banck wrote: > So I decided to add some support for earlier version in my version of > the program, and pushed it to https://github.com/credativ/pg_checksums > if anybody is interested in that. I have to admit that it is quite less > fancy than your

Re: Typos from Covering Index patch

On Tue, Apr 17, 2018 at 11:57:20AM -0400, Heikki Linnakangas wrote: > Committed, thanks! Thanks, Heikki. -- Michael signature.asc Description: PGP signature

Re: Should we add GUCs to allow partition pruning to be disabled?

On Wed, Apr 18, 2018 at 12:07:18PM +1200, David Rowley wrote: > In PG10 the planner's partition pruning could be disabled by changing > the constraint_exclusion GUC to off. This is still the case for PG11, > but only for UPDATE and DELETE queries. There is currently no way to > disable partition p

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

On Tue, Apr 17, 2018 at 12:12:26PM -0700, Andres Freund wrote: > On 2018-04-17 15:09:18 -0400, Tom Lane wrote: >> Alvaro Herrera writes: >>> Andres was working on a radix tree structure to fix this problem, but >>> that seems to be abandoned now, and it seems a major undertaking. > > I hope to re

Re: Should we add GUCs to allow partition pruning to be disabled?

On Tue, Apr 17, 2018 at 5:42 PM, Justin Pryzby wrote: > On Wed, Apr 18, 2018 at 12:07:18PM +1200, David Rowley wrote: > > In PG10 the planner's partition pruning could be disabled by changing > > the constraint_exclusion GUC to off. This is still the case for PG11, > > but only for UPDATE and DE

Re: Should we add GUCs to allow partition pruning to be disabled?

On 18 April 2018 at 13:03, David G. Johnston wrote: > My initial reaction is that we need to fix the bug introduced in v10 - > leaving constraint_exclusion working as it has historically and not affect > the new-as-of-10 ability to prune (maybe better termed as skip...) > partitions known during e

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

On April 17, 2018 6:00:59 PM PDT, Michael Paquier wrote: >On Tue, Apr 17, 2018 at 12:12:26PM -0700, Andres Freund wrote: >> On 2018-04-17 15:09:18 -0400, Tom Lane wrote: >>> Alvaro Herrera writes: Andres was working on a radix tree structure to fix this problem, >but that seems to be

Re: Speedup of relation deletes during recovery

On Wed, Apr 18, 2018 at 12:46:58AM +0900, Fujii Masao wrote: > Yes, I think. And, I found that smgrdounlinkfork() is also dead code. > Per the discussion [1], this unused function was left intentionally. > But it's still dead code since 2012, so I'd like to remove it. Patch attached. Indeed, it's

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

On Tue, Apr 17, 2018 at 06:13:31PM -0700, Andres Freund wrote: > Not sure what you mean? Do you need help on it? I suggest that I could undertake the proposed patch and submit it earlier in the development cycle of v12. -- Michael signature.asc Description: PGP signature

Re: Should we add GUCs to allow partition pruning to be disabled?

On Tue, Apr 17, 2018 at 6:12 PM, David Rowley wrote: > On 18 April 2018 at 13:03, David G. Johnston > wrote: > > My initial reaction is that we need to fix the bug introduced in v10 - > > leaving constraint_exclusion working as it has historically and not > affect > > the new-as-of-10 ability to

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

On 2018-04-18 10:46:51 +0900, Michael Paquier wrote: > On Tue, Apr 17, 2018 at 06:13:31PM -0700, Andres Freund wrote: > > Not sure what you mean? > > Do you need help on it? I suggest that I could undertake the proposed > patch and submit it earlier in the development cycle of v12. I think it's

Re: pruning disabled for array, enum, record, range type partition keys

Thanks for the review. On 2018/04/18 7:11, Alvaro Herrera wrote: > Amit Langote wrote: > >> Ah, I think I got it after staring at the (btree) index code for a bit. >> >> What pruning code got wrong is that it's comparing the expression type >> (type of the constant arg that will be compared with

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

Hello. At Mon, 16 Apr 2018 17:05:28 +0530, Ashutosh Bapat wrote in > Hi, > Consider this scenario > > postgres=# CREATE TABLE plt (a int, b int) PARTITION BY LIST(a); > postgres=# CREATE TABLE plt_p1 PARTITION OF plt FOR VALUES IN (1); > postgres=# CREATE TABLE plt_p2 PARTITION OF plt FOR VAL

Re: ON CONFLICT DO UPDATE for partitioned tables

On 2018/04/18 0:02, Alvaro Herrera wrote: > Amit Langote wrote: > >> Attached find a patch that does that. When working on this, I noticed >> that when recursing for inheritance children, ATPrepAlterColumnType() >> would use a AlterTableCmd (cmd) that's already scribbled on as if it were >> the o

Re: ON CONFLICT DO UPDATE for partitioned tables

On 2018/04/18 0:04, Alvaro Herrera wrote: > Amit Langote wrote: > >> I just confirmed my hunch that this wouldn't somehow do the right thing >> when the OID system column is involved. Like this case: > > This looks too big a patch to pursue now. I'm inclined to just remove > the equalTupdesc ch

Re: Excessive PostmasterIsAlive calls slow down WAL redo

On Wed, Apr 11, 2018 at 10:22 PM, Heikki Linnakangas wrote: >> On Tue, Apr 10, 2018 at 12:53 PM, Andres Freund >> wrote: >>> That person said he'd work on adding an equivalent of linux' >>> prctl(PR_SET_PDEATHSIG) to FreeBSD. Here is an implementation of Andres's idea for Linux, and also for pat

Re: Oddity in tuple routing for foreign partitions

On 2018/04/17 16:41, Etsuro Fujita wrote: > In the INSERT/COPY-tuple-routing case, as explained by Amit, the > RTE at that position in the EState's range table is the one for the > partitioned table of a given partition, so the statement would be true.  > BUT in the UPDATE-tuple-routing case, the R

Double-writes, take two?

Hi all, Back in 2012, Dan Scales, who was working on VMware Postgres, has posted a patch aimed at removing the need of full-page writes by introducing the concept of double writes using a double-write buffer approach in order to fix torn page problems: https://www.postgresql.org/message-id/1962493

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Thu, Apr 12, 2018 at 5:53 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush > > of TOAST errors post a crash recovery, nearly causing an outage. Most > > errors r

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

Hi, I'd like to disentangle two related topics. For "I want PostmasterIsAlive() to go faster using signals on platforms that can support that", please see over here: https://www.postgresql.org/message-id/flat/7261eb39-0369-f2f4-1bb5-62f3b6083...@iki.fi#7261eb39-0369-f2f4-1bb5-62f3b6083...@iki.fi

Re: Oddity in tuple routing for foreign partitions

On 2018/04/17 11:13, Kyotaro HORIGUCHI wrote: >>> Also, I removed the CheckValidResultRel check from ExecInitRoutingInfo >>> and added that to ExecInitPartitionInfo right after the> InitResultRelInfo >>> call, >>> because it would be better to abort the >>> operation as soon as we find the partiti

Re: Oddity in tuple routing for foreign partitions

(2018/04/17 16:10), Amit Langote wrote: On 2018/04/17 11:13, Kyotaro HORIGUCHI wrote: If I'm reading this correctly, ExecInitParititionInfo calls ExecInitRoutingInfo so currently CheckValidityResultRel is called for the child when partrel is created in ExecPrepareTupleRouting. But the move of Ch

Re: Built-in connection pooling

On 13.04.2018 19:07, Nikolay Samokhvalov wrote: On Fri, Apr 13, 2018 at 2:59 AM, Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> wrote: Development in built-in connection pooling will be continued in https://github.com/postgrespro/postgresql.builtin_pool.git

Re: ON CONFLICT DO UPDATE for partitioned tables

On 2018/04/17 4:10, Alvaro Herrera wrote: > Amit Langote wrote: > >> The solution I came up with is to call map_variable_attnos() directly, >> instead of going through map_partition_varattnos() every time, after first >> creating the attribute map ourselves. > > Yeah, sounds good. I added a twea

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

(2018/02/20 18:13), Ashutosh Bapat wrote: Here's patchset implementing this solution. 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its callers. 0002 fixes a similar bug for regular partitioned tables. The patch has testcase. The commit message explains the bug in more det

Re: ON CONFLICT DO UPDATE for partitioned tables

On 2018/04/17 16:45, Amit Langote wrote: > Instead of doing this, I think we should try to make > convert_tuples_by_name_map() a bit smarter by integrating the logic in > convert_tuples_by_name() that's used conclude if no tuple conversion is > necessary. So, if it turns that the tuples descriptor

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

On Tue, Apr 17, 2018 at 2:05 PM, Etsuro Fujita wrote: > (2018/02/20 18:13), Ashutosh Bapat wrote: >> >> Here's patchset implementing this solution. >> >> 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its >> callers. >> >> 0002 fixes a similar bug for regular partitioned table

Test coverage for mark_invalid_subplans_as_finished

Alvaro pinged me off list to mention the coverage tool indicates that there's no test coverage for mark_invalid_subplans_as_finished() [1]. The attached patch aims to put that right. This function would only ever be called during a Parallel Append in a query which has some exec Params being compar

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

(2018/04/17 18:43), Ashutosh Bapat wrote: On Tue, Apr 17, 2018 at 2:05 PM, Etsuro Fujita wrote: (2018/02/20 18:13), Ashutosh Bapat wrote: Here's patchset implementing this solution. 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its callers. 0002 fixes a similar bug fo

Re: WIP: Covering + unique indexes.

On Mon, Apr 16, 2018 at 1:05 AM, Peter Geoghegan wrote: > Attached patch makes the changes that I talked about, and a few > others. The commit message has full details. The general direction of > the patch is that it documents our assumptions, and verifies them in > more cases. Most of the change

Re: pg_recvlogical broken in back branches

2018-04-17 3:38 GMT-03:00 Michael Paquier : > The exact same fix has already applied on all stable branches: > Sorry about the noise. I've only checked the REL9_6_8 tag and the tarball. -- Euler Taveira Timbira - http://www.timbira.com.br/ PostgreSQL: Cons

Re: Proposal: Adding json logging

On 4/16/18 23:12, Michael Paquier wrote: >> I have also had good success using syslog. While syslog is not very >> structured, the setting syslog_split_messages allows sending log entries >> that include newlines in one piece, which works well if you have some >> kind of full-text search engine at

Re: Proposal: Adding json logging

David Arnold wrote: > Interesting, does that implicitly mean the whole log event would get > transmitted as a "line" (with CRLF) in CSV. To me it's implied by the doc at: https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG > In the aff

  1   2   >