Re: "could not reattach to shared memory" on buildfarm member dory

2018-04-24 Thread Noah Misch
On Tue, Apr 24, 2018 at 11:37:33AM +1200, Thomas Munro wrote: > On Tue, Apr 24, 2018 at 11:18 AM, Stephen Frost wrote: > > Greetings, > > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> So far, dory has failed three times with essentially identical symptoms: > >> > >> 2018-04-23 19:57:10.624 GMT [

Re: "could not reattach to shared memory" on buildfarm member dory

2018-04-24 Thread Magnus Hagander
On Tue, Apr 24, 2018 at 1:18 AM, Stephen Frost wrote: > Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > So far, dory has failed three times with essentially identical symptoms: > > > > 2018-04-23 19:57:10.624 GMT [2240] FATAL: could not reattach to shared > memory (key=0190

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-04-24 Thread Michael Paquier
On Sat, Apr 21, 2018 at 12:25:27PM +1200, Thomas Munro wrote: > Here's a new version, because FreeBSD's new interface changed slightly. I have been looking at the proposed set for Linux, and the numbers are here. By replaying 1GB worth of WAL after a pgbench run with the data folder on a tmpfs th

[RFC] Add an until-0 loop in psql

2018-04-24 Thread Pierre Ducroquet
Hi When running database migrations with .sql files on a live database, it's not uncommon to have to run a migration in a loop to prevent a big lock on a table. For instance if one want to delete some old datas from a big table one would write : DELETE FROM big_table WHERE id IN (SELECT id FRO

Re: community bonding

2018-04-24 Thread Aleksander Alekseev
Hello Charles, > Thanks for selecting my project as one of GSoC student projects! Pretty > exciting and honor to join the development for PostgreSQL (the best > database in the world :)). Welcome! > 1. What ide or command line tools do you guys used most for PostgreSQL > development? Personally

Re: Built-in connection pooling

2018-04-24 Thread Konstantin Knizhnik
On 23.04.2018 21:56, Robert Haas wrote: On Fri, Jan 19, 2018 at 11:59 AM, Tomas Vondra wrote: Hmmm, that's unfortunate. I guess you'll have process the startup packet in the main process, before it gets forked. At least partially. I'm not keen on a design that would involve doing more stuff

Re: using expression syntax for partition bounds

2018-04-24 Thread Kyotaro HORIGUCHI
Thanks. I have almost missed this. At Mon, 23 Apr 2018 11:44:14 +0900, Amit Langote wrote in > On 2018/04/23 11:37, Amit Langote wrote: > > I tried to update the patch to do things that way. I'm going to create a > > new entry in the next CF titled "generalized expression syntax for > > parti

Re: [RFC] Add an until-0 loop in psql

2018-04-24 Thread Pavel Stehule
Hi 2018-04-24 9:58 GMT+02:00 Pierre Ducroquet : > Hi > > When running database migrations with .sql files on a live database, it's > not > uncommon to have to run a migration in a loop to prevent a big lock on a > table. > For instance if one want to delete some old datas from a big table one >

GCC 8 warnings

2018-04-24 Thread Devrim Gündüz
Hi, While building stable releases and v11 on Fedora 28, I am seeing some warnings. What is the project's policy about fixing those warnings in older branches? To contribute to world peace, I did not attach the text to the email. Here is what I see in today's git snapshot: https://gunduz.org/t

Re: minor fix for acquire_inherited_sample_rows

2018-04-24 Thread Amit Langote
Hi. On 2018/04/24 0:16, Alvaro Herrera wrote: > Hello Amit > > Amit Langote wrote: > >> acquire_inherited_sample_rows() currently uses equalTupleDescs() being >> false as the condition for going to tupconv.c to determine whether tuple >> conversion is needed. But equalTupleDescs() will always r

Re: Make description of heap records more talkative for flags

2018-04-24 Thread Jehan-Guillaume de Rorthais
Hi all, Bellow a 1¢ on feedback from a side project about this. On Mon, 23 Apr 2018 12:37:20 -0300 Alvaro Herrera wrote: > Michael Paquier wrote: > > On Thu, Apr 12, 2018 at 08:49:03PM -0700, Andres Freund wrote: > > > OTOH, that also kinda bloats the output noticeably... I'm somewhat > > > i

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2018-04-24 Thread Kyotaro HORIGUCHI
Thank you very much for looking this! At Mon, 23 Apr 2018 03:41:47 -0400, Heikki Linnakangas wrote in <89e33d4f-5c75-0738-3dcb-44c4df59e...@iki.fi> > On 18/01/18 20:54, Kyotaro HORIGUCHI wrote: > > At Thu, 18 Jan 2018 11:52:52 -0800, Andres Freund > > wrote in <20180118195252.hyxgkb3krcqjz...@a

Re: [HACKERS] Custom compression methods

2018-04-24 Thread Alexander Korotkov
On Mon, Apr 23, 2018 at 7:34 PM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > IMHO end-user do not have skills and time to create their own compression > algorithms. And without knowledge of specific of particular data set, > it is very hard to implement something more efficient than

obsoleting plpython2u and defaulting plpythonu to plpython3u

2018-04-24 Thread Pavel Raiskup
Forwarding to hackers. Hope that's OK after few days. Pavel -- Forwarded Message -- Subject: obsoleting plpython2u and defaulting plpythonu to plpython3u Date: Thursday, April 19, 2018, 11:56:40 AM CEST From: Pavel Raiskup To: pgsql-general Per current plpython docs: T

Minor comment update in execPartition.c

2018-04-24 Thread Etsuro Fujita
Hi, Here is a comment for ExecInitPartitionInfo: 296 * ExecInitPartitionInfo 297 * Initialize ResultRelInfo and other information for a partition if not 298 * already done I think we should remove the words "if not already done" from that comment because 1) that function is calle

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-04-24 Thread Etsuro Fujita
(2018/04/17 19:00), Etsuro Fujita wrote: (2018/04/17 18:43), Ashutosh Bapat wrote: Here's updated patch-set. Will review. I started reviewing this. o 0001-Handle-ConvertRowtypeExprs-in-pull_vars_clause.patch: + else if (IsA(node, ConvertRowtypeExpr)) + { +#ifdef USE_ASSERT_CHECKING +

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Andrey Borodin
> 24 апр. 2018 г., в 11:31, Юрий Соколов написал(а): > > Before implementing algorithms within PostgreSQL it will be great to test > them outside with real traces. > > I think, there should be mechamism to collect traces from real-world > postgresql instalations: every read and write access.

Re: Built-in connection pooling

2018-04-24 Thread Merlin Moncure
On Mon, Apr 23, 2018 at 3:14 PM, Robert Haas wrote: > In other words, transparent connection pooling is going to require > some new mechanism, which third-party code will have to know about, > for tracking every last bit of session state that might need to be > preserved or cleared. That's going

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Amit Langote wrote: > That's neat! Definitely agree that we should call ExecInitExpr just once > here. The patch looks good too, except the long line. How about this as a small tweak? Determine the array index using a macro, which serves as documentation. -- Álvaro Herrerahtt

Re: [sqlsmith] Unpinning error in parallel worker

2018-04-24 Thread Jonathan Rudenberg
On Fri, Apr 20, 2018, at 00:42, Thomas Munro wrote: > On Wed, Apr 18, 2018 at 11:43 AM, Jonathan Rudenberg > wrote: > > On Tue, Apr 17, 2018, at 19:31, Thomas Munro wrote: > >> On Wed, Apr 18, 2018 at 11:01 AM, Jonathan Rudenberg > >> wrote: > >> > Yep, I think I know approximately what it look

Re: Built-in connection pooling

2018-04-24 Thread Adam Brusselback
On Tue, Apr 24, 2018 at 9:52 AM, Merlin Moncure wrote: > > Why does it have to be completely transparent? As long as the feature > is optional (say, a .conf setting) the tradeoffs can be managed. It's > a reasonable to expect to exchange some functionality for pooling; > pgbouncer provides a 're

Re: Extending a bit string

2018-04-24 Thread Tom Lane
Evan Carroll writes: > Currently the behavior of bit-string extensions is pretty insane. You've provided no support for this assertion, much less any defense of why your proposed semantics change is any less insane. Also, if I understood you correctly, you want to change the semantics of casting

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-24 Thread Bruce Momjian
On Wed, Apr 18, 2018 at 12:07:52PM +0530, Pavan Deolasee wrote: > > > On Thu, Apr 12, 2018 at 5:53 AM, Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush >

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-24 Thread Peter Eisentraut
On 4/18/18 02:37, Pavan Deolasee wrote: > While I agree that we should clean it up, I wonder if changing error > text would be a good idea. These errors are being reported by a very > long time and if we change the text, we might forget the knowledge about > the past reports. It's presumably fixed

Re: Extending a bit string

2018-04-24 Thread Evan Carroll
> > That's SQL:99 6.22 general rule 11) c). > (SV and TD are the source value and the target datatype for a cast.) > > In hindsight, it would likely be more consistent with this if we'd > considered bitstrings to be LSB first when coercing them to/from integer, > but whoever stuck that behavior in

Re: Corrupted btree index on HEAD because of covering indexes

2018-04-24 Thread Teodor Sigaev
Perfect! I would like to commit it but have some suggestions: 1) TRUNCATE bttest_multi; INSERT INTO bttest_multi SELECT i, i%2 FROM generate_series(1, 10) as i; SELECT bt_index_parent_check('bttest_multi_idx', true); to improve test stability it would be better to disable autovacuum: ALTER

Re: Transform for pl/perl

2018-04-24 Thread Peter Eisentraut
On 4/10/18 10:31, Dagfinn Ilmari Mannsåker wrote: > Also, it doesn't parse back in as jsonb either: > > =# select jsonbnan()::text::json; > ERROR: invalid input syntax for type json > DETAIL: Token "NaN" is invalid. > CONTEXT: JSON data, line 1: NaN > > And it's inconsistent wi

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Robert Haas
On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera wrote: > Robert, I think this is your turf, per 3d956d9562aa. Are you looking > into it? Thanks for the ping. I just had a look over the proposed patch and I guess I don't like it very much. Temporarily modifying the range table in place and the

Re: Built-in connection pooling

2018-04-24 Thread Konstantin Knizhnik
On 23.04.2018 23:14, Robert Haas wrote: On Wed, Apr 18, 2018 at 9:41 AM, Heikki Linnakangas wrote: Well, may be I missed something, but i do not know how to efficiently support 1. Temporary tables 2. Prepared statements 3. Sessoin GUCs with any external connection pooler (with pooling level o

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Pushed. Thanks! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Corrupted btree index on HEAD because of covering indexes

2018-04-24 Thread Peter Geoghegan
On Tue, Apr 24, 2018 at 9:06 AM, Teodor Sigaev wrote: > Perfect! Thanks! > I would like to commit it but have some suggestions: I attach a revised version, which has changes based on your feedback. > to improve test stability it would be better to disable autovacuum: > ALTER TABLE bttest_multi

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Andres Freund
On 2018-04-19 12:04:35 +1200, David Rowley wrote: > On 19 April 2018 at 03:13, Robert Haas wrote: > > 10% is more than a "slight" improvement, I'd say! It's certainly got > > to be worth avoiding the repeated calls to ExecInitExpr, whatever we > > do about the memory contexts. Yea, that seems im

Re: Toast issues with OldestXmin going backwards

2018-04-24 Thread Peter Eisentraut
On 4/23/18 00:33, Amit Kapila wrote: > Yeah, right, I have missed the point that they can be vacuumed > separately, however, I think that decision is somewhat questionable. Manually vacuuming the TOAST table was a way to repair the recently fixed TOAST bug, so it's kind of useful. -- Peter Eisen

Fsync request queue

2018-04-24 Thread Andres Freund
Hi, While thinking about the at the fsync mess, I started looking at the fsync request queue. I was primarily wondering whether we can keep FDs open long enough (by forwarding them to the checkpointer) to guarantee that we see the error. But that's mostly irrelevant for what I'm wondering about he

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Юрий Соколов
вт, 24 апр. 2018 г., 15:16 Andrey Borodin : > > > > 24 апр. 2018 г., в 11:31, Юрий Соколов > написал(а): > > > > Before implementing algorithms within PostgreSQL it will be great to > test them outside with real traces. > > > > I think, there should be mechamism to collect traces from real-world

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Andres Freund
Hi, On 2018-04-24 17:16:47 +0500, Andrey Borodin wrote: > But, I think that cost of development of real page eviction strategy > itself is neglectable small compared to infrastructure changes needed > by any non-CS5 strategy. What problems are you seeing? This isn't a lot of code? Greetings, An

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Andres Freund wrote: > What I wonder, after skimming this change, is where the relevant > expression context is reset? That's not really related to this change > but the wider thread, I just noticed it while looking at this. Do you mean ResetExprContext? We use the plan's context, so it should

Re: lazy detoasting

2018-04-24 Thread Peter Eisentraut
On 4/11/18 11:33, Tom Lane wrote: > (Wanders away wondering what Peter has done about toasted parameter > values for procedures in general ...) I'm not sure. How can a procedure have a toasted parameter value? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: Transform for pl/perl

2018-04-24 Thread Andrew Dunstan
On 04/24/2018 12:17 PM, Peter Eisentraut wrote: > On 4/10/18 10:31, Dagfinn Ilmari Mannsåker wrote: >> Also, it doesn't parse back in as jsonb either: >> >> =# select jsonbnan()::text::json; >> ERROR: invalid input syntax for type json >> DETAIL: Token "NaN" is invalid. >> CONTE

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Robert Haas
On Tue, Apr 24, 2018 at 12:21 PM, Robert Haas wrote: > On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera > wrote: >> Robert, I think this is your turf, per 3d956d9562aa. Are you looking >> into it? > > Thanks for the ping. I just had a look over the proposed patch and I > guess I don't like it v

Re: [sqlsmith] Unpinning error in parallel worker

2018-04-24 Thread Thomas Munro
On Wed, Apr 25, 2018 at 2:21 AM, Jonathan Rudenberg wrote: > This issue happened again in production, here are the stack traces from three > we grabbed before nuking the >400 hanging backends. > > [...] > #4 0x55fccb93b21c in LWLockAcquire+188() at > /usr/lib/postgresql/10/bin/postgres at l

Re: [sqlsmith] Unpinning error in parallel worker

2018-04-24 Thread Jonathan Rudenberg
On Tue, Apr 24, 2018, at 16:06, Thomas Munro wrote: > On Wed, Apr 25, 2018 at 2:21 AM, Jonathan Rudenberg > wrote: > > This issue happened again in production, here are the stack traces from > > three we grabbed before nuking the >400 hanging backends. > > > > [...] > > #4 0x55fccb93b21c i

Re: Extending a bit string

2018-04-24 Thread Evan Carroll
> In hindsight, it would likely be more consistent with this if we'd > considered bitstrings to be LSB first when coercing them to/from integer, > but whoever stuck that behavior in didn't think about it. Too late to > change that now I'm afraid, though perhaps we could provide non-cast > conversi

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-24 Thread Alvaro Herrera
Jim Finnerty wrote: > The patch looks good to me, David. > Thanks for checking! It's already pushed. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-24 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/04/24 6:10, Alvaro Herrera wrote: > > Not really sure how best to handle that one. For starters, I think it need > > to stop mentioning the GUC name in the title; > > Hmm, "Constraint Exclusion" that's used in the title is a concept, not a > GUC, although pretty clo

Re: obsoleting plpython2u and defaulting plpythonu to plpython3u

2018-04-24 Thread Peter Eisentraut
On 4/24/18 07:13, Pavel Raiskup wrote: > .. the status quo seems to be bit optimistic with the "distant future", > and we should start thinking about dropping plpython2 support, same as > upstream (a bit optimistically too, IMO) does [1]. I don't think we are going to drop Python 2 support anytime

Re: Minor comment update in execPartition.c

2018-04-24 Thread Amit Langote
Fujita-san, On 2018/04/24 20:14, Etsuro Fujita wrote: > Hi, > > Here is a comment for ExecInitPartitionInfo: > > 296 * ExecInitPartitionInfo > 297 * Initialize ResultRelInfo and other information for a > partition if not > 298 * already done > > I think we should remove the word

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Kyotaro HORIGUCHI
Hello. At Tue, 24 Apr 2018 15:49:20 -0400, Robert Haas wrote in > On Tue, Apr 24, 2018 at 12:21 PM, Robert Haas wrote: > > On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera > > wrote: > >> Robert, I think this is your turf, per 3d956d9562aa. Are you looking > >> into it? > > > > Thanks for th

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Amit Langote
On 2018/04/25 4:49, Robert Haas wrote: > On Tue, Apr 24, 2018 at 12:21 PM, Robert Haas wrote: >> On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera >> wrote: >>> Robert, I think this is your turf, per 3d956d9562aa. Are you looking >>> into it? >> >> Thanks for the ping. I just had a look over the

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Kyotaro HORIGUCHI
I forgot to mention that. At Wed, 25 Apr 2018 10:17:02 +0900, Amit Langote wrote in > On 2018/04/25 4:49, Robert Haas wrote: > > On Tue, Apr 24, 2018 at 12:21 PM, Robert Haas wrote: > >> On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera > >> wrote: > >>> Robert, I think this is your turf, per

Re: Minor comment update in execPartition.c

2018-04-24 Thread Alvaro Herrera
Amit Langote wrote: > > I think we should remove the words "if not already done" from that > > comment because 1) that function is called if the partition wasn't > > already initialized and 2) that function assumes that. Attached is a > > small patch for removing the words. > > Thanks, sounds fi

Typo in JIT documentation

2018-04-24 Thread Michael Paquier
Hi all, I just found $subject: the server was compiled without --with-llvm), -JIT will not performed, even if considered to be +JIT will not be performed, even if considered to be beneficial based on the above criteria. Setting Thanks, -- Michael diff --git a/doc/src/sgml/jit.

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Amit Langote
Oops, really meant to send the "+1 to the root -> rte refactoring" comment and the rest in the same email. On 2018/04/25 4:49, Robert Haas wrote: > I have done some refactoring to avoid that. See attached. > > I didn't really get beyond the refactoring stage with this today. > This version still

Re: Minor comment update in execPartition.c

2018-04-24 Thread Etsuro Fujita
(2018/04/25 11:05), Alvaro Herrera wrote: Amit Langote wrote: I think we should remove the words "if not already done" from that comment because 1) that function is called if the partition wasn't already initialized and 2) that function assumes that. Attached is a small patch for removing the

Re: Built-in connection pooling

2018-04-24 Thread Christophe Pettus
> On Apr 24, 2018, at 06:52, Merlin Moncure wrote: > Why does it have to be completely transparent? Well, we have non-transparent connection pooling now, in the form of pgbouncer, and the huge fleet of existing application-stack poolers. The main reason to move it into core is to avoid the l

Re: Fsync request queue

2018-04-24 Thread Heikki Linnakangas
On 24/04/18 21:00, Andres Freund wrote: Right now if the queue is full and can't be compacted we end up fsync()ing on every single write, rather than once per checkpoint afaict. That's a fairly horrible. For the case that there's no space in the map, I'd suggest to just do 10% or so of the fsync

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-04-24 Thread Thomas Munro
On Tue, Apr 24, 2018 at 7:37 PM, Michael Paquier wrote: > I have been looking at the proposed set for Linux, and the numbers are > here. By replaying 1GB worth of WAL after a pgbench run with the data > folder on a tmpfs the recovery time goes from 33s to 28s, so that's a > nice gain. Thanks for

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Andrey Borodin
> 24 апр. 2018 г., в 23:14, Andres Freund написал(а): > > On 2018-04-24 17:16:47 +0500, Andrey Borodin wrote: >> But, I think that cost of development of real page eviction strategy >> itself is neglectable small compared to infrastructure changes needed >> by any non-CS5 strategy. > > What pr

Racing DEADLOCK on PostgreSQL 9.3

2018-04-24 Thread Nick Dro
Hi, I have a stock table.   One of the users in the system ran this query:  update stock set quantity=quantity-5 where stockid=100  (from his client application). On the same time I ran from pg-admin this query:   do $$beginalter table stock disable trigger stock_aftertrigger;update stock set stock

Re: Racing DEADLOCK on PostgreSQL 9.3

2018-04-24 Thread Heikki Linnakangas
On 25/04/18 09:21, Nick Dro wrote: Hi, I have a stock table. One of the users in the system ran this query:  update stock set quantity=quantity-5 where stockid=100  (from his client application). On the same time I ran from pg-admin this query: do $$ begin alter table stock disable trigger stoc

Re: "could not reattach to shared memory" on buildfarm member dory

2018-04-24 Thread Noah Misch
On Tue, Apr 24, 2018 at 11:37:33AM +1200, Thomas Munro wrote: > On Tue, Apr 24, 2018 at 11:18 AM, Stephen Frost wrote: > > Greetings, > > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> So far, dory has failed three times with essentially identical symptoms: > >> > >> 2018-04-23 19:57:10.624 GMT [

Re: "could not reattach to shared memory" on buildfarm member dory

2018-04-24 Thread Magnus Hagander
On Tue, Apr 24, 2018 at 1:18 AM, Stephen Frost wrote: > Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > So far, dory has failed three times with essentially identical symptoms: > > > > 2018-04-23 19:57:10.624 GMT [2240] FATAL: could not reattach to shared > memory (key=0190

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-04-24 Thread Michael Paquier
On Sat, Apr 21, 2018 at 12:25:27PM +1200, Thomas Munro wrote: > Here's a new version, because FreeBSD's new interface changed slightly. I have been looking at the proposed set for Linux, and the numbers are here. By replaying 1GB worth of WAL after a pgbench run with the data folder on a tmpfs th

[RFC] Add an until-0 loop in psql

2018-04-24 Thread Pierre Ducroquet
Hi When running database migrations with .sql files on a live database, it's not uncommon to have to run a migration in a loop to prevent a big lock on a table. For instance if one want to delete some old datas from a big table one would write : DELETE FROM big_table WHERE id IN (SELECT id FRO

Re: community bonding

2018-04-24 Thread Aleksander Alekseev
Hello Charles, > Thanks for selecting my project as one of GSoC student projects! Pretty > exciting and honor to join the development for PostgreSQL (the best > database in the world :)). Welcome! > 1. What ide or command line tools do you guys used most for PostgreSQL > development? Personally

Re: Built-in connection pooling

2018-04-24 Thread Konstantin Knizhnik
On 23.04.2018 21:56, Robert Haas wrote: On Fri, Jan 19, 2018 at 11:59 AM, Tomas Vondra wrote: Hmmm, that's unfortunate. I guess you'll have process the startup packet in the main process, before it gets forked. At least partially. I'm not keen on a design that would involve doing more stuff

Re: using expression syntax for partition bounds

2018-04-24 Thread Kyotaro HORIGUCHI
Thanks. I have almost missed this. At Mon, 23 Apr 2018 11:44:14 +0900, Amit Langote wrote in > On 2018/04/23 11:37, Amit Langote wrote: > > I tried to update the patch to do things that way. I'm going to create a > > new entry in the next CF titled "generalized expression syntax for > > parti

Re: [RFC] Add an until-0 loop in psql

2018-04-24 Thread Pavel Stehule
Hi 2018-04-24 9:58 GMT+02:00 Pierre Ducroquet : > Hi > > When running database migrations with .sql files on a live database, it's > not > uncommon to have to run a migration in a loop to prevent a big lock on a > table. > For instance if one want to delete some old datas from a big table one >

GCC 8 warnings

2018-04-24 Thread Devrim Gündüz
Hi, While building stable releases and v11 on Fedora 28, I am seeing some warnings. What is the project's policy about fixing those warnings in older branches? To contribute to world peace, I did not attach the text to the email. Here is what I see in today's git snapshot: https://gunduz.org/t

Re: minor fix for acquire_inherited_sample_rows

2018-04-24 Thread Amit Langote
Hi. On 2018/04/24 0:16, Alvaro Herrera wrote: > Hello Amit > > Amit Langote wrote: > >> acquire_inherited_sample_rows() currently uses equalTupleDescs() being >> false as the condition for going to tupconv.c to determine whether tuple >> conversion is needed. But equalTupleDescs() will always r

Re: Make description of heap records more talkative for flags

2018-04-24 Thread Jehan-Guillaume de Rorthais
Hi all, Bellow a 1¢ on feedback from a side project about this. On Mon, 23 Apr 2018 12:37:20 -0300 Alvaro Herrera wrote: > Michael Paquier wrote: > > On Thu, Apr 12, 2018 at 08:49:03PM -0700, Andres Freund wrote: > > > OTOH, that also kinda bloats the output noticeably... I'm somewhat > > > i

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2018-04-24 Thread Kyotaro HORIGUCHI
Thank you very much for looking this! At Mon, 23 Apr 2018 03:41:47 -0400, Heikki Linnakangas wrote in <89e33d4f-5c75-0738-3dcb-44c4df59e...@iki.fi> > On 18/01/18 20:54, Kyotaro HORIGUCHI wrote: > > At Thu, 18 Jan 2018 11:52:52 -0800, Andres Freund > > wrote in <20180118195252.hyxgkb3krcqjz...@a

Re: [HACKERS] Custom compression methods

2018-04-24 Thread Alexander Korotkov
On Mon, Apr 23, 2018 at 7:34 PM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > IMHO end-user do not have skills and time to create their own compression > algorithms. And without knowledge of specific of particular data set, > it is very hard to implement something more efficient than

obsoleting plpython2u and defaulting plpythonu to plpython3u

2018-04-24 Thread Pavel Raiskup
Forwarding to hackers. Hope that's OK after few days. Pavel -- Forwarded Message -- Subject: obsoleting plpython2u and defaulting plpythonu to plpython3u Date: Thursday, April 19, 2018, 11:56:40 AM CEST From: Pavel Raiskup To: pgsql-general Per current plpython docs: T

Minor comment update in execPartition.c

2018-04-24 Thread Etsuro Fujita
Hi, Here is a comment for ExecInitPartitionInfo: 296 * ExecInitPartitionInfo 297 * Initialize ResultRelInfo and other information for a partition if not 298 * already done I think we should remove the words "if not already done" from that comment because 1) that function is calle

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-04-24 Thread Etsuro Fujita
(2018/04/17 19:00), Etsuro Fujita wrote: (2018/04/17 18:43), Ashutosh Bapat wrote: Here's updated patch-set. Will review. I started reviewing this. o 0001-Handle-ConvertRowtypeExprs-in-pull_vars_clause.patch: + else if (IsA(node, ConvertRowtypeExpr)) + { +#ifdef USE_ASSERT_CHECKING +

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Andrey Borodin
> 24 апр. 2018 г., в 11:31, Юрий Соколов написал(а): > > Before implementing algorithms within PostgreSQL it will be great to test > them outside with real traces. > > I think, there should be mechamism to collect traces from real-world > postgresql instalations: every read and write access.

Re: Built-in connection pooling

2018-04-24 Thread Merlin Moncure
On Mon, Apr 23, 2018 at 3:14 PM, Robert Haas wrote: > In other words, transparent connection pooling is going to require > some new mechanism, which third-party code will have to know about, > for tracking every last bit of session state that might need to be > preserved or cleared. That's going

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Amit Langote wrote: > That's neat! Definitely agree that we should call ExecInitExpr just once > here. The patch looks good too, except the long line. How about this as a small tweak? Determine the array index using a macro, which serves as documentation. -- Álvaro Herrerahtt

Re: [sqlsmith] Unpinning error in parallel worker

2018-04-24 Thread Jonathan Rudenberg
On Fri, Apr 20, 2018, at 00:42, Thomas Munro wrote: > On Wed, Apr 18, 2018 at 11:43 AM, Jonathan Rudenberg > wrote: > > On Tue, Apr 17, 2018, at 19:31, Thomas Munro wrote: > >> On Wed, Apr 18, 2018 at 11:01 AM, Jonathan Rudenberg > >> wrote: > >> > Yep, I think I know approximately what it look

Re: Built-in connection pooling

2018-04-24 Thread Adam Brusselback
On Tue, Apr 24, 2018 at 9:52 AM, Merlin Moncure wrote: > > Why does it have to be completely transparent? As long as the feature > is optional (say, a .conf setting) the tradeoffs can be managed. It's > a reasonable to expect to exchange some functionality for pooling; > pgbouncer provides a 're

Re: Extending a bit string

2018-04-24 Thread Tom Lane
Evan Carroll writes: > Currently the behavior of bit-string extensions is pretty insane. You've provided no support for this assertion, much less any defense of why your proposed semantics change is any less insane. Also, if I understood you correctly, you want to change the semantics of casting

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-24 Thread Bruce Momjian
On Wed, Apr 18, 2018 at 12:07:52PM +0530, Pavan Deolasee wrote: > > > On Thu, Apr 12, 2018 at 5:53 AM, Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush >

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-24 Thread Peter Eisentraut
On 4/18/18 02:37, Pavan Deolasee wrote: > While I agree that we should clean it up, I wonder if changing error > text would be a good idea. These errors are being reported by a very > long time and if we change the text, we might forget the knowledge about > the past reports. It's presumably fixed

Re: Extending a bit string

2018-04-24 Thread Evan Carroll
> > That's SQL:99 6.22 general rule 11) c). > (SV and TD are the source value and the target datatype for a cast.) > > In hindsight, it would likely be more consistent with this if we'd > considered bitstrings to be LSB first when coercing them to/from integer, > but whoever stuck that behavior in

Re: Corrupted btree index on HEAD because of covering indexes

2018-04-24 Thread Teodor Sigaev
Perfect! I would like to commit it but have some suggestions: 1) TRUNCATE bttest_multi; INSERT INTO bttest_multi SELECT i, i%2 FROM generate_series(1, 10) as i; SELECT bt_index_parent_check('bttest_multi_idx', true); to improve test stability it would be better to disable autovacuum: ALTER

Re: Transform for pl/perl

2018-04-24 Thread Peter Eisentraut
On 4/10/18 10:31, Dagfinn Ilmari Mannsåker wrote: > Also, it doesn't parse back in as jsonb either: > > =# select jsonbnan()::text::json; > ERROR: invalid input syntax for type json > DETAIL: Token "NaN" is invalid. > CONTEXT: JSON data, line 1: NaN > > And it's inconsistent wi

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Robert Haas
On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera wrote: > Robert, I think this is your turf, per 3d956d9562aa. Are you looking > into it? Thanks for the ping. I just had a look over the proposed patch and I guess I don't like it very much. Temporarily modifying the range table in place and the

Re: Built-in connection pooling

2018-04-24 Thread Konstantin Knizhnik
On 23.04.2018 23:14, Robert Haas wrote: On Wed, Apr 18, 2018 at 9:41 AM, Heikki Linnakangas wrote: Well, may be I missed something, but i do not know how to efficiently support 1. Temporary tables 2. Prepared statements 3. Sessoin GUCs with any external connection pooler (with pooling level o

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Pushed. Thanks! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Corrupted btree index on HEAD because of covering indexes

2018-04-24 Thread Peter Geoghegan
On Tue, Apr 24, 2018 at 9:06 AM, Teodor Sigaev wrote: > Perfect! Thanks! > I would like to commit it but have some suggestions: I attach a revised version, which has changes based on your feedback. > to improve test stability it would be better to disable autovacuum: > ALTER TABLE bttest_multi

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Andres Freund
On 2018-04-19 12:04:35 +1200, David Rowley wrote: > On 19 April 2018 at 03:13, Robert Haas wrote: > > 10% is more than a "slight" improvement, I'd say! It's certainly got > > to be worth avoiding the repeated calls to ExecInitExpr, whatever we > > do about the memory contexts. Yea, that seems im

Re: Toast issues with OldestXmin going backwards

2018-04-24 Thread Peter Eisentraut
On 4/23/18 00:33, Amit Kapila wrote: > Yeah, right, I have missed the point that they can be vacuumed > separately, however, I think that decision is somewhat questionable. Manually vacuuming the TOAST table was a way to repair the recently fixed TOAST bug, so it's kind of useful. -- Peter Eisen

Fsync request queue

2018-04-24 Thread Andres Freund
Hi, While thinking about the at the fsync mess, I started looking at the fsync request queue. I was primarily wondering whether we can keep FDs open long enough (by forwarding them to the checkpointer) to guarantee that we see the error. But that's mostly irrelevant for what I'm wondering about he

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Юрий Соколов
вт, 24 апр. 2018 г., 15:16 Andrey Borodin : > > > > 24 апр. 2018 г., в 11:31, Юрий Соколов > написал(а): > > > > Before implementing algorithms within PostgreSQL it will be great to > test them outside with real traces. > > > > I think, there should be mechamism to collect traces from real-world

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-24 Thread Andres Freund
Hi, On 2018-04-24 17:16:47 +0500, Andrey Borodin wrote: > But, I think that cost of development of real page eviction strategy > itself is neglectable small compared to infrastructure changes needed > by any non-CS5 strategy. What problems are you seeing? This isn't a lot of code? Greetings, An

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Andres Freund wrote: > What I wonder, after skimming this change, is where the relevant > expression context is reset? That's not really related to this change > but the wider thread, I just noticed it while looking at this. Do you mean ResetExprContext? We use the plan's context, so it should

Re: lazy detoasting

2018-04-24 Thread Peter Eisentraut
On 4/11/18 11:33, Tom Lane wrote: > (Wanders away wondering what Peter has done about toasted parameter > values for procedures in general ...) I'm not sure. How can a procedure have a toasted parameter value? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: Transform for pl/perl

2018-04-24 Thread Andrew Dunstan
On 04/24/2018 12:17 PM, Peter Eisentraut wrote: > On 4/10/18 10:31, Dagfinn Ilmari Mannsåker wrote: >> Also, it doesn't parse back in as jsonb either: >> >> =# select jsonbnan()::text::json; >> ERROR: invalid input syntax for type json >> DETAIL: Token "NaN" is invalid. >> CONTE

Re: Oddity in tuple routing for foreign partitions

2018-04-24 Thread Robert Haas
On Tue, Apr 24, 2018 at 12:21 PM, Robert Haas wrote: > On Mon, Apr 23, 2018 at 11:25 AM, Alvaro Herrera > wrote: >> Robert, I think this is your turf, per 3d956d9562aa. Are you looking >> into it? > > Thanks for the ping. I just had a look over the proposed patch and I > guess I don't like it v

  1   2   >