Re: [HACKERS] SCRAM protocol documentation

2017-08-10 Thread Álvaro Hernández Tortosa
On 11/08/17 03:57, Peter Eisentraut wrote: The SCRAM protocol documentation (https://www.postgresql.org/docs/devel/static/sasl-authentication.html) states "To avoid confusion, the client should use pg_same_as_startup_message as the username in the client-first-message." However, the client im

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-08-10 Thread Maksim Milyutin
10.08.17 23:01, Robert Haas wrote: On Wed, Apr 19, 2017 at 5:25 AM, Maksim Milyutin wrote: Ok, thanks for the feedback. Then I'll use a new relkind for local partitioned index in further development. Any update on this? I'll continue to work soon. Sorry for so long delay. -- Regards, Maks

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-10 Thread Amit Kapila
On Wed, Aug 9, 2017 at 2:58 PM, Amit Kapila wrote: > On Mon, Aug 7, 2017 at 5:50 PM, Ashutosh Sharma wrote: > > 7. > _hash_kill_items(IndexScanDesc scan) > { > .. > + /* > + * If page LSN differs it means that the page was modified since the > + * last read. killedItems could be not valid so LP_D

[HACKERS] coverage analysis improvements

2017-08-10 Thread Peter Eisentraut
Here is a patch series with some significant reworking and adjusting of how the coverage analysis tools are run. The result should be that the "make coverage" runs are faster and more robust, the results are more accurate, and the code is simpler. Please see the individual patches for details. I

Re: [HACKERS] dubious error message from partition.c

2017-08-10 Thread Amit Langote
(Sorry Robert for the duplicate message, I mistakenly didn't hit Reply All) On Fri, Aug 11, 2017 at 2:54 Robert Haas wrote: > On Wed, Aug 9, 2017 at 10:14 PM, Amit Langote > wrote: > >> That seems like overkill. I'm good with "greater than or equal to". > > > > Attached updated patch has "grea

Re: [HACKERS] WIP: Failover Slots

2017-08-10 Thread Craig Ringer
On 11 August 2017 at 01:02, Robert Haas wrote: > Well, > anybody's welcome to write code without discussion and drop it to the > list, but if people don't like it, that's the risk you took by not > discussing it first. > Agreed, patches materializing doesn't mean they should be committed, and t

Re: [HACKERS] Thoughts on unit testing?

2017-08-10 Thread Craig Ringer
On 11 August 2017 at 07:13, Thomas Munro wrote: > On Fri, Aug 11, 2017 at 10:35 AM, Andres Freund > wrote: > > On 2017-08-11 09:53:23 +1200, Thomas Munro wrote: > >> One idea that keeps coming back to me is that we could probably extend > >> our existing regression tests to cover C tests with au

Re: [HACKERS] Getting server crash on Windows when using ICU collation

2017-08-10 Thread Peter Eisentraut
On 8/10/17 22:21, Peter Geoghegan wrote: > On Thu, Aug 10, 2017 at 3:57 PM, Peter Geoghegan wrote: >> On Thu, Aug 10, 2017 at 11:02 AM, Robert Haas wrote: >>> On Fri, Jun 23, 2017 at 1:14 AM, Ashutosh Sharma >>> wrote: Okay, attached is the patch which first detects the platform type and >

Re: [HACKERS] Getting server crash on Windows when using ICU collation

2017-08-10 Thread Peter Geoghegan
On Thu, Aug 10, 2017 at 3:57 PM, Peter Geoghegan wrote: > On Thu, Aug 10, 2017 at 11:02 AM, Robert Haas wrote: >> On Fri, Jun 23, 2017 at 1:14 AM, Ashutosh Sharma >> wrote: >>> Okay, attached is the patch which first detects the platform type and >>> runs the uconv commands accordingly from th

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

2017-08-10 Thread Noah Misch
On Thu, Aug 03, 2017 at 10:45:50AM -0400, Robert Haas wrote: > On Wed, Aug 2, 2017 at 11:47 PM, Noah Misch wrote: > > postmaster algorithms rely on the PG_SETMASK() calls preventing that. > > Without > > such protection, duplicate bgworkers are an understandable result. I caught > > several oth

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread Amit Kapila
On Fri, Aug 11, 2017 at 5:01 AM, AP wrote: > On Thu, Aug 10, 2017 at 01:12:25PM -0400, Robert Haas wrote: >> On Thu, Aug 10, 2017 at 6:41 AM, AP wrote: >> > The index is 135GB rather than 900GB (from memory/give or take). >> >> Whoa. Big improvement. > > > As an aside, btree for the above is aro

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread Amit Kapila
On Thu, Aug 10, 2017 at 4:11 PM, AP wrote: > On Sun, Aug 06, 2017 at 04:32:29PM +1000, AP wrote: >> On Sat, Aug 05, 2017 at 04:41:24PM +0530, Amit Kapila wrote: >> > > (On another note, I committed these patches.) >> > >> > Thanks. >> >> Seconded. :) >> >> Now uploading data with fillfactor of 90.

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-10 Thread Tom Lane
Noah Misch writes: > On Tue, Aug 08, 2017 at 07:25:37PM -0400, Peter Eisentraut wrote: >> I don't think I can usefully contribute to this. Could someone else >> take it? > If nobody volunteers, you could always resolve this by reverting 1e8a850 and > successors. I think you're blaming the victi

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-10 Thread Augustine, Jobin
I am in an effort to create independent build environment on Windows to test the patch from Andres. I shall come back with result possibly in another 24 hours. -Jobin On Fri, Aug 11, 2017 at 7:25 AM, Andres Freund wrote: > On 2017-08-10 18:51:07 -0700, Noah Misch wrote: > > On Tue, Aug 08, 2017

[HACKERS] SCRAM protocol documentation

2017-08-10 Thread Peter Eisentraut
The SCRAM protocol documentation (https://www.postgresql.org/docs/devel/static/sasl-authentication.html) states "To avoid confusion, the client should use pg_same_as_startup_message as the username in the client-first-message." However, the client implementation in libpq doesn't actually do that,

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-10 Thread Andres Freund
On 2017-08-10 18:51:07 -0700, Noah Misch wrote: > On Tue, Aug 08, 2017 at 07:25:37PM -0400, Peter Eisentraut wrote: > > On 8/7/17 21:06, Noah Misch wrote: > > >> That would fit. Until v10 (commit 1e8a850), PQconnectStart() had no > > >> in-tree > > >> callers outside of libpq itself. > > > [Actio

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread AP
On Thu, Aug 10, 2017 at 01:12:25PM -0400, Robert Haas wrote: > On Thu, Aug 10, 2017 at 6:41 AM, AP wrote: > > The index is 135GB rather than 900GB (from memory/give or take). > > Whoa. Big improvement. Not a good direct comparison in general but it fits my workload. The 900GB was fillfactor 10

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-10 Thread Noah Misch
On Tue, Aug 08, 2017 at 07:25:37PM -0400, Peter Eisentraut wrote: > On 8/7/17 21:06, Noah Misch wrote: > >> That would fit. Until v10 (commit 1e8a850), PQconnectStart() had no > >> in-tree > >> callers outside of libpq itself. > > [Action required within three days. This is a generic notificatio

Re: [HACKERS] Thoughts on unit testing?

2017-08-10 Thread Thomas Munro
On Fri, Aug 11, 2017 at 11:13 AM, Thomas Munro wrote: > Just create a .test.c file and type "TEST(my_math, > factorial) { EXPECT_EQ(6, factorial(3)); }" ... Of course that would really need to #include "something/test_macros.h" and "something/factorial.h", and EXPECT_EQ probably doesn't make much

Re: [HACKERS] Thoughts on unit testing?

2017-08-10 Thread Thomas Munro
On Fri, Aug 11, 2017 at 10:35 AM, Andres Freund wrote: > On 2017-08-11 09:53:23 +1200, Thomas Munro wrote: >> One idea that keeps coming back to me is that we could probably extend >> our existing regression tests to cover C tests with automatic >> discovery/minimal boilerplate. > > What's your de

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-10 Thread Michael Paquier
On Thu, Aug 10, 2017 at 9:00 PM, Robert Haas wrote: > On Thu, Aug 10, 2017 at 2:50 AM, Andreas Seltenreich > wrote: >> Will do. Won't miss this chance to try out discostu's extension >> pg_rage_terminator[1] :-) >> [1] https://github.com/disco-stu/pg_rage_terminator > > Oh, that's *awesome*.

Re: [HACKERS] Getting server crash on Windows when using ICU collation

2017-08-10 Thread Peter Geoghegan
On Thu, Aug 10, 2017 at 11:02 AM, Robert Haas wrote: > On Fri, Jun 23, 2017 at 1:14 AM, Ashutosh Sharma > wrote: >> Okay, attached is the patch which first detects the platform type and >> runs the uconv commands accordingly from the corresponding icu bin >> path. Please have a look and let me

Re: [HACKERS] Thoughts on unit testing?

2017-08-10 Thread Andres Freund
Hi, On 2017-08-11 09:53:23 +1200, Thomas Munro wrote: > The current regression tests, isolation tests and TAP tests are very > good(though I admit my experience with TAP is limited), but IMHO we > are lacking support for C-level unit testing. Complicated, fiddly > things with many states, interac

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

2017-08-10 Thread Mark Dilger
> On Aug 10, 2017, at 11:20 AM, Robert Haas wrote: > > On Wed, Jul 5, 2017 at 12:14 AM, Mark Dilger wrote: >> I can understand this, but wonder if I could use something like >> >> FOR I TOTALLY PROMISE TO USE ALL ROWS rec IN EXECUTE sql LOOP >> ... >> END LOOP; > > Actually, what you'd need i

[HACKERS] Thoughts on unit testing?

2017-08-10 Thread Thomas Munro
Hi hackers, The current regression tests, isolation tests and TAP tests are very good (though I admit my experience with TAP is limited), but IMHO we are lacking support for C-level unit testing. Complicated, fiddly things with many states, interactions, edge cases etc can be hard to get full tes

Re: [HACKERS] emergency outage requiring database restart

2017-08-10 Thread Merlin Moncure
On Thu, Aug 10, 2017 at 12:01 PM, Ants Aasma wrote: > On Wed, Jan 18, 2017 at 4:33 PM, Merlin Moncure wrote: >> On Wed, Jan 18, 2017 at 4:11 AM, Ants Aasma wrote: >>> On Wed, Jan 4, 2017 at 5:36 PM, Merlin Moncure wrote: Still getting checksum failures. Over the last 30 days, I see the >

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-08-10 Thread Robert Haas
On Wed, Apr 19, 2017 at 5:25 AM, Maksim Milyutin wrote: > Ok, thanks for the feedback. Then I'll use a new relkind for local > partitioned index in further development. Any update on this? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

2017-08-10 Thread Robert Haas
On Wed, Jul 5, 2017 at 12:14 AM, Mark Dilger wrote: > I can understand this, but wonder if I could use something like > > FOR I TOTALLY PROMISE TO USE ALL ROWS rec IN EXECUTE sql LOOP > ... > END LOOP; Actually, what you'd need is: FOR I TOTALLY PROMISE TO USE ALL ROWS AND IT IS OK TO BUFFER THE

Re: [HACKERS] Getting server crash on Windows when using ICU collation

2017-08-10 Thread Robert Haas
On Fri, Jun 23, 2017 at 1:14 AM, Ashutosh Sharma wrote: > Okay, attached is the patch which first detects the platform type and > runs the uconv commands accordingly from the corresponding icu bin > path. Please have a look and let me know for any issues. Thanks. Should this be on the open items

Re: [HACKERS] Default Partition for Range

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 6:56 AM, Jeevan Ladhe wrote: >> This looks like a problem with the default list partitioning patch. I >> think "true" is what we expect to see here in both cases. > > In case of list partition if there is only default partition, then there is > no > specific value set that

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-10 Thread Robert Haas
On Tue, Aug 8, 2017 at 10:48 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Aug 8, 2017 at 4:34 PM, Tom Lane wrote: >>> In the meantime, I think my vote would be to remove AtEOXact_CatCache. > >> In all supported branches? > > Whatever we do about this issue, I don't feel a need to do it f

Re: [HACKERS] dubious error message from partition.c

2017-08-10 Thread Robert Haas
On Wed, Aug 9, 2017 at 10:14 PM, Amit Langote wrote: >> That seems like overkill. I'm good with "greater than or equal to". > > Attached updated patch has "greater than or equal to". Committed, with one small change which I hope will be considered an improvement. Your proposed primary error mes

Re: [HACKERS] parallel documentation improvements

2017-08-10 Thread Robert Haas
On Tue, Aug 1, 2017 at 2:43 PM, Robert Haas wrote: > In commit 054637d2e08cda6a096f48cc99696136a06f4ef5, I updated the > parallel query documentation to reflect recently-committed parallel > query features. However, a few more things got committed after that. > Most of the attached patch consists

Re: [HACKERS] Comment typo in plannodes.h

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 8:25 AM, Etsuro Fujita wrote: > Here is a small patch for fixing a comment typo in plannodes.h: s/all the > partitioned table/all the partitioned tables/. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 6:41 AM, AP wrote: > The index is 135GB rather than 900GB (from memory/give or take). Whoa. Big improvement. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] WIP: Failover Slots

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 2:38 AM, Craig Ringer wrote: > Yep, so again, you're pushing slots "up" the tree, by name, with a 1:1 > correspondence, and using globally unique slot names to manage state. Yes, that's what I'm imagining. (Whether I should instead be imagining something else is the impor

Re: [HACKERS] emergency outage requiring database restart

2017-08-10 Thread Ants Aasma
On Wed, Jan 18, 2017 at 4:33 PM, Merlin Moncure wrote: > On Wed, Jan 18, 2017 at 4:11 AM, Ants Aasma wrote: >> On Wed, Jan 4, 2017 at 5:36 PM, Merlin Moncure wrote: >>> Still getting checksum failures. Over the last 30 days, I see the >>> following. Since enabling checksums FWICT none of the

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-08-10 Thread Josh Berkus
On 08/09/2017 10:49 PM, Michael Paquier wrote: > On Fri, Aug 4, 2017 at 8:19 AM, Masahiko Sawada wrote: >> On Fri, Jul 28, 2017 at 2:24 PM, Noah Misch wrote: >>> This item appears under "decisions to recheck mid-beta". If anyone is going >>> to push for a change here, now is the time. >> >> It h

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-08-10 Thread Tom Lane
Alexander Korotkov writes: > ... > You have random mix of tabs and spaces here. It's worth running pgindent over your code before submitting. It should be pretty easy to set that up nowadays, see src/tools/pgindent/README. (If you find any portability problems while trying to install pgindent, p

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Tom Lane
Ashutosh Sharma writes: > On Thu, Aug 10, 2017 at 8:06 PM, Tom Lane wrote: >> Yeah ... however, if that's there, then there's something wrong with >> Ashutosh's explanation, because that means we *are* building with >> _USE_32BIT_TIME_T in 32-bit builds. It's just getting there in a >> roundabou

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-08-10 Thread Craig Ringer
On 10 August 2017 at 23:25, Robert Haas wrote: > On Mon, Aug 7, 2017 at 2:06 AM, Craig Ringer > wrote: > > I think so - specifically, that it's a leftover from a revision where the > > xid limit was advanced before clog truncation. > > > > I'll be finding time in the next couple of days to look

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-08-10 Thread Alexander Korotkov
On Thu, Aug 10, 2017 at 7:37 AM, Michael Paquier wrote: > On Wed, Aug 9, 2017 at 6:38 PM, Robert Haas wrote: > > The patch doesn't really conform to our coding standards, though, so > > you need to clean it up (or, if you're not sure what you need to do, > > you need to have someone who knows ho

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-08-10 Thread Robert Haas
On Mon, Aug 7, 2017 at 2:06 AM, Craig Ringer wrote: > I think so - specifically, that it's a leftover from a revision where the > xid limit was advanced before clog truncation. > > I'll be finding time in the next couple of days to look more closely and > ensure that's all it is. A couple of days

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Ashutosh Sharma
On Thu, Aug 10, 2017 at 8:06 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Aug 10, 2017 at 8:30 AM, Ashutosh Sharma >> wrote: >>> So, the question is should we allow the preprocessor option >>> '_USE_32BIT_TIME_T' to be defined implicitly or not in postgresql >>> provided we are using Vi

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 10, 2017 at 10:36 AM, Tom Lane wrote: >> Yeah ... however, if that's there, then there's something wrong with >> Ashutosh's explanation, because that means we *are* building with >> _USE_32BIT_TIME_T in 32-bit builds. It's just getting there in a >> roundabout w

Re: [HACKERS] parallelize queries containing initplans

2017-08-10 Thread Amit Kapila
On Wed, Aug 9, 2017 at 6:51 PM, Haribabu Kommi wrote: > > On Wed, Aug 9, 2017 at 9:26 PM, Amit Kapila wrote: >> > > By the way, I tested the patch with by DML support for parallel patch to > check the returning of clause of insert, and all the returning clause init > plans > are parallel plans wi

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-08-10 Thread Alexander Korotkov
On Thu, Aug 10, 2017 at 3:30 PM, Sokolov Yura wrote: > On 2017-07-31 18:56, Sokolov Yura wrote: > >> Good day, every one. >> >> In attempt to improve performance of YCSB on zipfian distribution, >> it were found that significant time is spent in XidInMVCCSnapshot in >> scanning snapshot->xip arra

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-08-10 Thread Alexander Korotkov
On Wed, Aug 9, 2017 at 7:38 PM, Robert Haas wrote: > On Tue, Aug 1, 2017 at 4:00 PM, Ildus K > wrote: > > It's a workaround. DatumGetTSVector and > > DatumGetTSVectorCopy will upgrade tsvector on the fly if it > > has old format. > > Hmm, that seems like a real fix, not just a workaround. If yo

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 10:36 AM, Tom Lane wrote: > Yeah ... however, if that's there, then there's something wrong with > Ashutosh's explanation, because that means we *are* building with > _USE_32BIT_TIME_T in 32-bit builds. It's just getting there in a > roundabout way. (Or, alternatively, th

Re: [HACKERS] reload-through-the-top-parent switch the partition table

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 3:47 AM, Rushabh Lathia wrote: >> (1) seems like a pretty arbitrary restriction, so I don't like that >> option. (2) would hurt performance in some use cases. Do we have an >> option (3)? > > How about protecting option 2) with the load-via-partition-root protection. > Me

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 10, 2017 at 8:30 AM, Ashutosh Sharma > wrote: >> So, the question is should we allow the preprocessor option >> '_USE_32BIT_TIME_T' to be defined implicitly or not in postgresql >> provided we are using Visual C compiler version > 8.0. I think this >> should mak

Re: [HACKERS] Funny WAL corruption issue

2017-08-10 Thread Chris Travers
On Thu, Aug 10, 2017 at 3:17 PM, Vladimir Rusinov wrote: > > > On Thu, Aug 10, 2017 at 1:48 PM, Aleksander Alekseev < > a.aleks...@postgrespro.ru> wrote: > >> I just wanted to point out that a hardware issue or third party software >> issues (bugs in FS, software RAID, ...) could not be fully exc

Re: [HACKERS] Comment in snapbuild.c file

2017-08-10 Thread Alvaro Herrera
Masahiko Sawada wrote: > Hi all, > > In snapbuild.c file, there is a comment as follows. > >* NB: Because of that xmax can be lower than xmin, because we only >* increase xmax when a catalog modifying transaction commits. While odd >* looking, it's correct and actually more efficient

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 8:30 AM, Ashutosh Sharma wrote: > So, the question is should we allow the preprocessor option > '_USE_32BIT_TIME_T' to be defined implicitly or not in postgresql > provided we are using Visual C compiler version > 8.0. I think this > should make plperl compatible with perl

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 5:43 AM, Thomas Munro wrote: >> Do you think we solving this problem is a prerequisite for >> partition-wise join? Or should we propose that patch as a separate >> enhancement? > > No, I'm not proposing anything yet. For now I just wanted to share > this observation about

Re: [HACKERS] Funny WAL corruption issue

2017-08-10 Thread Vladimir Rusinov
On Thu, Aug 10, 2017 at 1:48 PM, Aleksander Alekseev < a.aleks...@postgrespro.ru> wrote: > I just wanted to point out that a hardware issue or third party software > issues (bugs in FS, software RAID, ...) could not be fully excluded from > the list of suspects. According to the talk by Christophe

[HACKERS] Foreign tables privileges not shown in information_schema.table_privileges

2017-08-10 Thread Nicolas Thauvin
Hello, The information_schema.table_privileges view filters on regular tables and views. Foreign tables are not shown in this view but they are in other views of the information_schema like tables or column_privileges. Is it intentional? A patch is attached if not. Thanks -- Nicolas Thauvin htt

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

2017-08-10 Thread Sokolov Yura
On 2017-07-18 20:20, Sokolov Yura wrote: On 2017-06-05 16:22, Sokolov Yura wrote: Good day, everyone. This patch improves performance of contended LWLock. Patch makes lock acquiring in single CAS loop: 1. LWLock->state is read, and ability for lock acquiring is detected. If there is possibili

Re: [HACKERS] Funny WAL corruption issue

2017-08-10 Thread Aleksander Alekseev
Hi Chris, > I ran into a funny situation today regarding PostgreSQL replication and > wal corruption and wanted to go over what I think happened and what I > wonder about as a possible solution. Sad story. Unfortunately I have no idea what could be a reason nor can I suggest a good way to find it

Re: [HACKERS] Funny WAL corruption issue

2017-08-10 Thread Chris Travers
> Yes. Exactly the same output until a certain point where pg_xlogdump dies > with an error. That is the same LSN where PostgreSQL died with an error on > restart. > One other thing that is possibly relevant after reading through the last bug report is the error pgxlogdumo gives: pg_xlogdump:

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-08-10 Thread Sokolov Yura
On 2017-07-31 18:56, Sokolov Yura wrote: Good day, every one. In attempt to improve performance of YCSB on zipfian distribution, it were found that significant time is spent in XidInMVCCSnapshot in scanning snapshot->xip array. While overall CPU time is not too noticable, it has measurable impac

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-10 Thread Ashutosh Sharma
On Thu, Aug 10, 2017 at 1:55 AM, Robert Haas wrote: > On Tue, Aug 8, 2017 at 12:15 PM, Sandeep Thakkar > wrote: >> I copied and pasted that portion of the build log into file build.log >> (attached) for Windows 32bit and Windows 64bit. > > Can you also share the output of 'perl -V' on each system

Re: [HACKERS] Funny WAL corruption issue

2017-08-10 Thread Chris Travers
Sorry, meant to reply all. On Thu, Aug 10, 2017 at 2:19 PM, Vladimir Borodin wrote: > Hi, Chris. > > 10 авг. 2017 г., в 15:09, Chris Travers > написал(а): > > Hi; > > I ran into a funny situation today regarding PostgreSQL replication and > wal corruption and wanted to go over what I think happ

[HACKERS] Comment typo in plannodes.h

2017-08-10 Thread Etsuro Fujita
Hi, Here is a small patch for fixing a comment typo in plannodes.h: s/all the partitioned table/all the partitioned tables/. Best regards, Etsuro Fujita diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index f1a1b24..7c51e7f 100644 --- a/src/include/nodes/plannodes.h

Re: [HACKERS] Funny WAL corruption issue

2017-08-10 Thread Vladimir Borodin
Hi, Chris. > 10 авг. 2017 г., в 15:09, Chris Travers написал(а): > > Hi; > > I ran into a funny situation today regarding PostgreSQL replication and wal > corruption and wanted to go over what I think happened and what I wonder > about as a possible solution. > > Basic information is custom-

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-08-10 Thread Ashutosh Bapat
On Thu, Aug 10, 2017 at 3:13 PM, Thomas Munro wrote: > On Thu, Aug 10, 2017 at 6:23 PM, Ashutosh Bapat > wrote: >> Your patch didn't improve planning time without partition-wise join, >> so it's something good to have along-with partition-wise join. Given >> that Bitmapsets are used in other part

[HACKERS] Funny WAL corruption issue

2017-08-10 Thread Chris Travers
Hi; I ran into a funny situation today regarding PostgreSQL replication and wal corruption and wanted to go over what I think happened and what I wonder about as a possible solution. Basic information is custom-build PostgreSQL 9.6.3 on Gentoo, on a ~5TB database with variable load. Master datab

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-10 Thread Robert Haas
On Thu, Aug 10, 2017 at 2:50 AM, Andreas Seltenreich wrote: > Will do. Won't miss this chance to try out discostu's extension > pg_rage_terminator[1] :-) > [1] https://github.com/disco-stu/pg_rage_terminator Oh, that's *awesome*. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The En

Re: [HACKERS] Walsender timeouts and large transactions

2017-08-10 Thread Sokolov Yura
On 2017-08-09 16:23, Petr Jelinek wrote: On 02/08/17 19:35, Yura Sokolov wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:n

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread AP
On Sun, Aug 06, 2017 at 04:32:29PM +1000, AP wrote: > On Sat, Aug 05, 2017 at 04:41:24PM +0530, Amit Kapila wrote: > > > (On another note, I committed these patches.) > > > > Thanks. > > Seconded. :) > > Now uploading data with fillfactor of 90. I'll know in 2-3 days > if the new patches are suc

Re: [HACKERS] Default Partition for Range

2017-08-10 Thread Jeevan Ladhe
Hi Robert, Beena, On Wed, Aug 9, 2017 at 5:53 PM, Robert Haas wrote: > On Wed, Aug 9, 2017 at 8:18 AM, Rajkumar Raghuwanshi > wrote: > > --difference in the description of default partition in case of list vs > > range > > > > create table lp (a int) partition by list(a); > > create table lp_d

[HACKERS] Fix number skipping in to_number

2017-08-10 Thread Oliver Ford
Prevents an issue where numbers can be skipped in the to_number() function when the format mask contains a "G" or a "," but the input string doesn't contain a separator. This resolves the TODO item "Fix to_number() handling for values not matching the format string". == Change == Currently, if th

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-10 Thread Beena Emerson
Hi Amit, On Thu, Aug 10, 2017 at 7:41 AM, Amit Langote wrote: > On 2017/08/05 2:25, Robert Haas wrote: >> Concretely, my proposal is: >> >> P.S. While I haven't reviewed 0002 in detail, I think the concept of >> minimizing what needs to be built in RelationGetPartitionDispatchInfo >> is a very go

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-08-10 Thread Thomas Munro
On Thu, Aug 10, 2017 at 6:23 PM, Ashutosh Bapat wrote: > Your patch didn't improve planning time without partition-wise join, > so it's something good to have along-with partition-wise join. Given > that Bitmapsets are used in other parts of code as well, the > optimization may affect those parts

Re: [HACKERS] Creating backup history files for backups taken from standbys

2017-08-10 Thread Masahiko Sawada
On Thu, Aug 10, 2017 at 4:50 PM, Michael Paquier wrote: > On Thu, Aug 10, 2017 at 9:45 AM, Masahiko Sawada > wrote: >> Thank you for the patch. Regarding to creating the backup history file >> on stanbys, is there any difference from the patch posted on earlier >> thread? > > That's a rebased ve

Re: [HACKERS] Creating backup history files for backups taken from standbys

2017-08-10 Thread Michael Paquier
On Thu, Aug 10, 2017 at 9:45 AM, Masahiko Sawada wrote: > Thank you for the patch. Regarding to creating the backup history file > on stanbys, is there any difference from the patch posted on earlier > thread? That's a rebased version on top of what has been applied, except that I have fixed an i

Re: [HACKERS] reload-through-the-top-parent switch the partition table

2017-08-10 Thread Rushabh Lathia
On Wed, Aug 9, 2017 at 1:20 AM, Robert Haas wrote: > On Tue, Aug 8, 2017 at 8:48 AM, Rushabh Lathia > wrote: > > It seems like with we set the numParents and parents only for the > > dumpable objects (flagInhTables()). Current patch relies on the > numParents > > and parents to get the root part

Re: [HACKERS] Creating backup history files for backups taken from standbys

2017-08-10 Thread Masahiko Sawada
On Thu, Aug 10, 2017 at 3:56 PM, Michael Paquier wrote: > Hi all, > > In the recent thread related to a bug in pg_stop_backup when waiting > for WAL segments to be archived, it has been mentioned that it would > be nice to get backup history files generated as well on standbys: > https://www.postg

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-10 Thread Michael Paquier
On Thu, Aug 10, 2017 at 2:52 AM, Michael Paquier wrote: > With a minimal maintenance effort we can be careful enough. I think > that a comment for example in pgstat.c about the usage uniqueness > would be an adapted answer. By the way, let's discuss that on a new thread. I'll try to come up with

Re: [HACKERS] More race conditions in logical replication

2017-08-10 Thread Michael Paquier
On Tue, Aug 8, 2017 at 8:11 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Mon, Aug 7, 2017 at 8:14 PM, Alvaro Herrera >> wrote: >> > BTW, I noticed that the PG_WAIT_LOCK value that we're using for wait >> > event here (and in the replication slot case) is bogus. We probably >> > need som