Re: [HACKERS] question about replication docs

2017-06-06 Thread Peter Eisentraut
On 5/29/17 09:11, Dave Cramer wrote: > The following makes an explicit reference to the simple query protocol > being the only protocol allowed in walsender mode. It is my > understanding this is true for logical replication as well ?? It's the same thing (for this purpose). -- Peter Eisentraut

[HACKERS] question about replication docs

2017-05-29 Thread Dave Cramer
The following makes an explicit reference to the simple query protocol being the only protocol allowed in walsender mode. It is my understanding this is true for logical replication as well ?? 51.3. Streaming Replication Protocol To initiate streaming replication, the frontend sends the replicati

Re: [HACKERS] Question about toasting code

2017-05-07 Thread Mat Arye
On Sun, May 7, 2017 at 3:48 PM, Tom Lane wrote: > Mat Arye writes: > > This is in arrayfuncs.c:5022 (postgre 9.6.2) > > > /* > > * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if > > * it's varlena. (You might think that detoasting is not needed here > > * because constr

Re: [HACKERS] Question about toasting code

2017-05-07 Thread Tom Lane
Mat Arye writes: > This is in arrayfuncs.c:5022 (postgre 9.6.2) > /* > * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if > * it's varlena. (You might think that detoasting is not needed here > * because construct_md_array can detoast the array elements later. > * However,

[HACKERS] Question about toasting code

2017-05-07 Thread Mat Arye
Hi, I am trying to create a custom aggregate and have run across some puzzling code while trying to figure out how to implement it. This is in arrayfuncs.c:5022 (postgre 9.6.2) /* * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if * it's varlena. (You might think that det

Re: [HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 1:45 AM, David G. Johnston wrote: > The first write to a page after a checkpoint is always recorded in the WAL > as a full page write. Every WAL file since the checkpoint must also be > copied to the backed up system. The replay of those WAL files is what > brings the rem

Re: [HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread David G. Johnston
On Tue, Apr 25, 2017 at 9:08 AM, Rui Hai Jiang wrote: > When pg_basebackup is launched, a checkpoint is created first, then all > files are transferred to the pg_basebackup client. Is it possible that a > data page(say page-N) in a data file is changed after the checkpoint and > before the pg_b

[HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread Rui Hai Jiang
Hello, I'm checking how the pg_basebackup works and I got a question(maybe there are no such issues): When pg_basebackup is launched, a checkpoint is created first, then all files are transferred to the pg_basebackup client. Is it possible that a data page(say page-N) in a data file is change

[HACKERS] Question about one of the old Autonomous Transaction approach

2017-04-06 Thread Vaishnavi Prabakaran
Hi All, Regarding the discussion about Autonomous transaction in below message ID, long time ago, it has been specified that having a new structure "Struct PGAutonomousXACT" was rejected in PGCon hackers meeting. Can anyone know why is it been rejected? What is the disadvantage/problem identified

[HACKERS] Question about memory contexts in ReindexMultipleTables()

2017-02-17 Thread Andreas Karlsson
Hi, When working on REINDEX CONCURRENTLY I noticed that the new memory context created in the ReindexMultipleTables() seems pointless. The purpose claimed in the code for introducing the ReindexMultipleTables context is to make sure the list we build with relation IDs survive the commit, sin

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-21 Thread Masahiko Sawada
On Mon, Oct 17, 2016 at 10:04 PM, Kevin Grittner wrote: > On Sun, Oct 16, 2016 at 9:26 PM, Masahiko Sawada > wrote: > >> When I set old_snapshot_threshold = 0 I got error at step #3, which >> means that the error is occurred without table pruning. > > The "snapshot too old" error can happen with

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-19 Thread Kevin Grittner
On Wed, Oct 19, 2016 at 11:11 AM, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 11:08:28AM -0500, Kevin Grittner wrote: >> On Wed, Oct 19, 2016 at 10:04 AM, Bruce Momjian wrote: >> >>> Slide 10 of this presentation has an example showing >>> old_snapshot_threshold set to '1min': >>> >>>

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-19 Thread Bruce Momjian
On Wed, Oct 19, 2016 at 11:08:28AM -0500, Kevin Grittner wrote: > On Wed, Oct 19, 2016 at 10:04 AM, Bruce Momjian wrote: > > > Slide 10 of this presentation has an example showing > > old_snapshot_threshold set to '1min': > > > > http://momjian.us/main/writings/pgsql/features.pdf > > If

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-19 Thread Kevin Grittner
On Wed, Oct 19, 2016 at 10:04 AM, Bruce Momjian wrote: > Slide 10 of this presentation has an example showing > old_snapshot_threshold set to '1min': > > http://momjian.us/main/writings/pgsql/features.pdf If the presentation is intending to show reasonable values for production use, that

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-19 Thread Bruce Momjian
On Mon, Oct 17, 2016 at 08:04:43AM -0500, Kevin Grittner wrote: > > We have regression test for this feature but it sets > > old_snapshot_threshold = 0, I doubt about we can test it properly. > > Am I missing something? > > This is a hard feature to test properly, and certainly hard to test > with

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-17 Thread Kevin Grittner
On Sun, Oct 16, 2016 at 9:26 PM, Masahiko Sawada wrote: > When I set old_snapshot_threshold = 0 I got error at step #3, which > means that the error is occurred without table pruning. The "snapshot too old" error can happen without pruning, but only because there is no way to tell the difference

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-16 Thread Masahiko Sawada
On Fri, Oct 14, 2016 at 11:29 PM, Kevin Grittner wrote: > On Fri, Oct 14, 2016 at 8:53 AM, Amit Kapila wrote: >> On Fri, Oct 14, 2016 at 1:40 PM, Masahiko Sawada >> wrote: > >>> For example, I set old_snapshot_threshold = 1min and prepare a table >>> and two terminals. >>> And I did the followi

[HACKERS] Question on "record type has not been registered"

2016-10-16 Thread Jim Nasby
I'm working on a function(attached) that returns a bitmask of NULL fields in a record. It works fine if I feed it a row directly, but fails in this case: select record_nulls(r), expected, CASE WHEN record_nulls(r) <> expected THEN 'BAD' END AS bad, r from (values(row(NULL,NULL,NULL,2,2,NULL,2

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-14 Thread Kevin Grittner
On Fri, Oct 14, 2016 at 8:53 AM, Amit Kapila wrote: > On Fri, Oct 14, 2016 at 1:40 PM, Masahiko Sawada > wrote: >> For example, I set old_snapshot_threshold = 1min and prepare a table >> and two terminals. >> And I did the followings steps. >> >> 1. [Terminal 1] Begin transaction and get snapsh

Re: [HACKERS] Question about behavior of snapshot too old feature

2016-10-14 Thread Amit Kapila
On Fri, Oct 14, 2016 at 1:40 PM, Masahiko Sawada wrote: > Hi all, > > I have a question about behavior of snapshot too old feature. > > For example, I set old_snapshot_threshold = 1min and prepare a table > and two terminals. > And I did the followings steps. > > 1. [Terminal 1] Begin transaction

[HACKERS] Question about behavior of snapshot too old feature

2016-10-14 Thread Masahiko Sawada
Hi all, I have a question about behavior of snapshot too old feature. For example, I set old_snapshot_threshold = 1min and prepare a table and two terminals. And I did the followings steps. 1. [Terminal 1] Begin transaction and get snapshot data and wait. BEGIN TRANSACTION ISOLATION LEVEL R

Re: [HACKERS] Question / requests.

2016-10-10 Thread Francisco Olarte
On Mon, Oct 10, 2016 at 4:51 AM, Jim Nasby wrote: > On 10/5/16 9:58 AM, Francisco Olarte wrote: >> Is the system catalog a bottleneck for people who has real use for >> paralell vacuum? I mean, to me someone who does this must have a very >> big db on a big iron. If that does not consist of thousa

Re: [HACKERS] Question / requests.

2016-10-09 Thread Jim Nasby
On 10/5/16 9:58 AM, Francisco Olarte wrote: Is the system catalog a bottleneck for people who has real use for paralell vacuum? I mean, to me someone who does this must have a very big db on a big iron. If that does not consist of thousands and thousands of smallish relations, it will normally be

Re: [HACKERS] Question / requests.

2016-10-07 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte > wrote: > > On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote: > >> On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera > >> wrote: > > ... > >>> I wonder if the real answer isn't just to disallow -f with parallel > >>> vacuumin

Re: [HACKERS] Question about pg_control usage

2016-10-07 Thread Jeff Janes
On Fri, Oct 7, 2016 at 8:24 AM, Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > Hi, hackers! > > I am examining various global variables in ShmemVariableCache, > pg_control and pg_controldata. To force and debug xid wraparound, > I've implemented a function, that allows to set next

Re: [HACKERS] Question about pg_control usage

2016-10-07 Thread Tom Lane
Anastasia Lubennikova writes: > What is the purpose of having CheckPoint stored in control file then? IIRC, we use some but not all of the fields. It seemed prudent (and simpler) to keep a copy of the whole thing. regards, tom lane -- Sent via pgsql-hackers mailing li

Re: [HACKERS] Question / requests.

2016-10-07 Thread Francisco Olarte
Robert: On Fri, Oct 7, 2016 at 3:20 PM, Robert Haas wrote: > On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte > wrote: >> On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote: >>> On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera >>> wrote: >> ... I wonder if the real answer isn't just to dis

[HACKERS] Question about pg_control usage

2016-10-07 Thread Anastasia Lubennikova
Hi, hackers! I am examining various global variables in ShmemVariableCache, pg_control and pg_controldata. To force and debug xid wraparound, I've implemented a function, that allows to set nextXid value in control file manually, but it doesn't work as expected. After an investigation I found t

Re: [HACKERS] Question / requests.

2016-10-07 Thread Robert Haas
On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte wrote: > On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote: >> On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera >> wrote: > ... >>> I wonder if the real answer isn't just to disallow -f with parallel >>> vacuuming. >> Seems like we should figure o

Re: [HACKERS] Question / requests.

2016-10-05 Thread Francisco Olarte
On Tue, Oct 4, 2016 at 7:50 PM, Robert Haas wrote: > On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera > wrote: ... >> I wonder if the real answer isn't just to disallow -f with parallel >> vacuuming. > Seems like we should figure out which catalog tables are needed in > order to perform a VACUUM,

Re: [HACKERS] Question / requests.

2016-10-04 Thread Robert Haas
On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Fri, Sep 30, 2016 at 11:20 AM, Francisco Olarte >> wrote: >> > After some messages due to vacuumdb auto-deadlocking itself on the >> > system tables when doing paralell vacuum of a full database I >> > suggested addi

Re: [HACKERS] Question / requests.

2016-10-04 Thread Francisco Olarte
On Mon, Oct 3, 2016 at 11:44 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Fri, Sep 30, 2016 at 11:20 AM, Francisco Olarte >> What messages are you seeing, exactly? "auto-deadlocking" isn't a thing. > https://www.postgresql.org/message-id/57EBC9AE.2060302%40163.com Besides that even the do

Re: [HACKERS] Question / requests.

2016-10-03 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Sep 30, 2016 at 11:20 AM, Francisco Olarte > wrote: > > After some messages due to vacuumdb auto-deadlocking itself on the > > system tables when doing paralell vacuum of a full database I > > suggested adding some flags to make vacuumdb process schemas. I was > > aske

Re: [HACKERS] Question / requests.

2016-10-03 Thread Robert Haas
On Fri, Sep 30, 2016 at 11:20 AM, Francisco Olarte wrote: > After some messages due to vacuumdb auto-deadlocking itself on the > system tables when doing paralell vacuum of a full database I > suggested adding some flags to make vacuumdb process schemas. I was > asked wether I could write a patch

Re: [HACKERS] Question / requests.

2016-09-30 Thread Vitaly Burovoy
On 9/30/16, Francisco Olarte wrote: > Hello everyone. Hello, Francisco! > Also, although I feel confident in my coding I have zero knowledge of > developing for postgres, It is easy enough and all important steps are documented in the wiki. Also some interesting things can be found in presentat

[HACKERS] Question / requests.

2016-09-30 Thread Francisco Olarte
Hello everyone. I've been using the bugs/general mailing lists for a while, but never been on hackers, so please take that into account. After some messages due to vacuumdb auto-deadlocking itself on the system tables when doing paralell vacuum of a full database I suggested adding some flags to m

Re: [HACKERS] Question about MVCC-unsafe commands

2016-08-23 Thread Antonin Houska
Robert Haas wrote: > On Tue, Aug 23, 2016 at 10:10 AM, Antonin Houska wrote: > CLUSTER preserves xmin/xmax when rewriting, but ALTER TABLE does not. Indeed, CLUSTER was the command I picked for my experiments. I didn't expect such a subtle difference. Thanks for answer. -- Antonin Houska Cybe

Re: [HACKERS] Question about MVCC-unsafe commands

2016-08-23 Thread Robert Haas
On Tue, Aug 23, 2016 at 10:10 AM, Antonin Houska wrote: > I'm failing to understand why [1] mentions "table-rewriting forms of ALTER > TABLE" besides TRUNCATE command. > > For TRUNCATE it seems clear: if transaction A takes the snapshot before it > accesses the table first time (typically because

[HACKERS] Question about MVCC-unsafe commands

2016-08-23 Thread Antonin Houska
I'm failing to understand why [1] mentions "table-rewriting forms of ALTER TABLE" besides TRUNCATE command. For TRUNCATE it seems clear: if transaction A takes the snapshot before it accesses the table first time (typically because isolation level is at least REPEATABLE READ) and transaction B man

Re: [HACKERS] Question about an inconsistency - 3

2016-07-19 Thread Tom Lane
"pet...@gmail.com" writes: > In file postgres/postgresql-9.4.4/src/timezone/zic.c > function stringzone line 2091we have > if (stringrule(result, stdrp, dstrp->r_stdoff, zp->z_gmtoff) != 0) > Is it ok to have as the 3rd argument dstrp->r_stdoff or should we > have stdrp->r_stdoff? In line 2085 d

Re: [HACKERS] Question about an inconsistency - 2

2016-07-19 Thread Tom Lane
"pet...@gmail.com" writes: > In file postgresql-9.4.4/src/backend/utils/adt/format_type.c > function format_type_internal line 159, shouldn’t be used > array_base_type instead of type_oid? IIRC, that was intentional. Supposing there's a pg_type row with a corrupted typelem value, it's more use

[HACKERS] Question about an inconsistency - 2

2016-07-19 Thread pet...@gmail.com
Hi, In file postgresql-9.4.4/src/backend/utils/adt/format_type.c function format_type_internal line 159, shouldn’t be used array_base_type instead of type_oid? In line 153 it is searched for array_base_type and thus shouldn’t we use it (ie., array_base_type) to report that the type is not found.

[HACKERS] Question about an inconsistency - 3

2016-07-19 Thread pet...@gmail.com
Hi, In file postgres/postgresql-9.4.4/src/timezone/zic.c function stringzone line 2091we have if (stringrule(result, stdrp, dstrp->r_stdoff, zp->z_gmtoff) != 0) Is it ok to have as the 3rd argument dstrp->r_stdoff or should we have stdrp->r_stdoff? In line 2085 dstrp is used in both arguments.

Re: [HACKERS] Question about an inconsistency - 1

2016-07-06 Thread Tom Lane
"pet...@gmail.com" writes: > I have a question regarding the source code in file pg_proc.h > (postgresql-9.4.4). > At line 1224 (copied below) why the 28th identifier is timestamp_eq? > DATA(insert OID = 1152 ( timestamptz_eq PGNSP PGUID 12 1 0 0 0 f f f t t f > i 2 0 16 "1184 1184" _null_

[HACKERS] Question about an inconsistency - 1

2016-07-06 Thread pet...@gmail.com
Hi, I have a question regarding the source code in file pg_proc.h (postgresql-9.4.4). At line 1224 (copied below) why the 28th identifier is timestamp_eq? DATA(insert OID = 1152 ( timestamptz_eq PGNSP PGUID 12 1 0 0 0 f f f t t f i 2 0 16 "1184 1184" _null_ _null_ _null_ _null_ timestamp_eq

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-06-23 Thread Bruce Momjian
On Thu, Jun 23, 2016 at 06:42:57AM +, Tsunakawa, Takayuki wrote: > > From: Bruce Momjian [mailto:br...@momjian.us] > > We have this text in src/tools/RELEASE_CHANGES: > > ... > > This is saying running against a mismatched minor version should be fine > > for a binary. > > Thanks for a good ra

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-06-22 Thread Tsunakawa, Takayuki
> From: Bruce Momjian [mailto:br...@momjian.us] > We have this text in src/tools/RELEASE_CHANGES: > ... > This is saying running against a mismatched minor version should be fine > for a binary. Thanks for a good rationale. > I know this thread is old but it bounced around a lot of ideas. I thi

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-06-22 Thread Bruce Momjian
On Mon, May 30, 2016 at 03:04:24AM +, Tsunakawa, Takayuki wrote: > Hello, > > I'd like to ask you about the policy of application binary compatibility. > And have a suggestion as well. > > QUESTION > == > > My customer asked me if the followi

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-06-01 Thread Michael Meskes
> However, the problem I pointed out is that when the new library is > incompatible with the older one, say the major version of libpq.dll > changes from 5 to 6, the application user and/or developer cannot > notice the incompatibility immediately and easily.  On Unix/Linux, > the application fails

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Craig Ringer
On 1 June 2016 at 13:09, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org [mailto: > pgsql-hackers-ow...@postgresql.org] On Behalf Of Craig Ringer > > While that's probably OK, it's not especially desirable. The typical > Windows deployment model involves the application bund

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Craig Ringer While that's probably OK, it's not especially desirable. The typical Windows deployment model involves the application bundling all its direct dependencies except when those are provide

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Craig Ringer
On 30 May 2016 at 11:04, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > Hello, > > I'd like to ask you about the policy of application binary compatibility. > And have a suggestion as well. > > QUESTION > == > > My customer asked me

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Tsunakawa, Takayuki
> From: Michael Meskes [mailto:mes...@postgresql.org] > e.g. a random hit from google:=C2=A0https://www.bottomupcs.com/libra > ries_and_the_linker.xhtml > > There even is a wikipedia page about > it:=C2=A0https://en.wikipedia.org/wiki/ > Soname Thank you for good pointers. The former is particul

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Tsunakawa, Takayuki
> From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Marco Atzeri > on cygwin the postgresql binary package already include the library > versions: > >/usr/bin/cygecpg-6.dll >/usr/bin/cygecpg_compat-3.dll >/usr/bin/cygpgtypes-3.dll >

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Michael Meskes
> I couldn't find appropriate system documentation.  Regarding Linux, I > remember I saw some HOWTO on tldp.org website which explains the > concept of shared library soname, but it's not very friendly for > users who just want to know the application binary compatibility > policy of PostgreSQL.  A

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Marco Atzeri
On 31/05/2016 08:10, Tsunakawa, Takayuki wrote: From: Michael Meskes [mailto:mes...@postgresql.org] Yes, but Windows users probably don't understand or know it. So, I suggested explicitly describing the application binary compatibility policy in the PostgreSQL manual. What do you think about

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-30 Thread Tsunakawa, Takayuki
> From: Michael Meskes [mailto:mes...@postgresql.org] > > Yes, but Windows users probably don't understand or know it. So, I > > suggested explicitly describing the application binary compatibility > > policy in the PostgreSQL manual. What do you think about it? > > Couldn't you point your custo

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-30 Thread Michael Meskes
> > Sorry I fail to understand what you mean with "legal"? Are you > > wondering > > about license restrictions? There are none. > > Sorry, I just meant "correct" or "valid". Ah ok, thanks for clarifying. > > As for compatibility, that's what major version numbers are for. > > This is > > not a

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-30 Thread Tsunakawa, Takayuki
> From: pgsql-hackers-ow...@postgresql.org > Sorry I fail to understand what you mean with "legal"? Are you wondering > about license restrictions? There are none. Sorry, I just meant "correct" or "valid". > As for compatibility, that's what major version numbers are for. This is > not a Postgre

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-30 Thread Michael Meskes
Hi, > - Build an embedded SQL C application with PostgreSQL 9.2. > - Distribute the app without ecpg nor libpq libraries.  Require users > to install PostgreSQL client which includes ecpg and libpq libraries. > - Use the app with newer PostgreSQL major versions without rebuilding > the app.  That

[HACKERS] Question and suggestion about application binary compatibility policy

2016-05-29 Thread Tsunakawa, Takayuki
Hello, I'd like to ask you about the policy of application binary compatibility. And have a suggestion as well. QUESTION == My customer asked me if the following usage is correct. - Build an embedded SQL C application with PostgreSQL 9.2. - Dist

Re: [HACKERS] Question about Restart point and checkpoint_segments

2016-02-18 Thread Benoit Lobréau
Sorry for the noise, I got my answer. 2016-02-16 18:02 GMT+01:00 Benoit Lobréau : > Hello, > > I am using a hot_standby setup on PostgreSQL 9.1 > While I was testing, I found out that only checkpoint_timeout (+ a > checkpoint since the last restart point) could trigger a restart point. > > The co

[HACKERS] Question about Restart point and checkpoint_segments

2016-02-16 Thread Benoit Lobréau
Hello, I am using a hot_standby setup on PostgreSQL 9.1 While I was testing, I found out that only checkpoint_timeout (+ a checkpoint since the last restart point) could trigger a restart point. The code (bgwriter.c) seems to confirm this: /* * Check progress against WAL segments written

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Michal Novotny
Hi Andres, thanks a lot. I've managed to run DROP TABLE and then cancel process using pg_cancel_backend(autovacuum_pid) and it passed and dropped the 5T table. Thanks a lot! Michal On 01/12/2016 12:37 PM, Andres Freund wrote: > Hi, > > On 2016-01-12 12:17:01 +0100, Michal Novotny wrote: >> than

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Michal Novotny
Hi Andres, On 01/12/2016 12:37 PM, Andres Freund wrote: > Hi, > > On 2016-01-12 12:17:01 +0100, Michal Novotny wrote: >> thanks a lot for your reply. Unfortunately I've found out most it didn't >> really start DROP TABLE yet and it's locked on autovacuum running for >> the table and even if I kil

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Andres Freund
Hi, On 2016-01-12 12:17:01 +0100, Michal Novotny wrote: > thanks a lot for your reply. Unfortunately I've found out most it didn't > really start DROP TABLE yet and it's locked on autovacuum running for > the table and even if I kill the process it's autostarting again and again. Start the DROP T

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Michal Novotny
On 01/12/2016 12:20 PM, Andres Freund wrote: > On 2016-01-12 12:17:09 +0100, Pavel Stehule wrote: >> 2016-01-12 12:14 GMT+01:00 Michal Novotny : >> >>> Hi Pavel, >>> thanks for the information. I've been doing more investigation of this >>> issue and there's autovacuum running on the table however

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Pavel Stehule
2016-01-12 12:22 GMT+01:00 Marko Tiikkaja : > On 12/01/16 12:17, Pavel Stehule wrote: > >> 2016-01-12 12:14 GMT+01:00 Michal Novotny : >> >> Hi Pavel, >>> thanks for the information. I've been doing more investigation of this >>> issue and there's autovacuum running on the table however it's >>> a

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Marko Tiikkaja
On 12/01/16 12:17, Pavel Stehule wrote: 2016-01-12 12:14 GMT+01:00 Michal Novotny : Hi Pavel, thanks for the information. I've been doing more investigation of this issue and there's autovacuum running on the table however it's automatically starting even if there is "autovacuum = off" in the p

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Andres Freund
On 2016-01-12 12:17:09 +0100, Pavel Stehule wrote: > 2016-01-12 12:14 GMT+01:00 Michal Novotny : > > > Hi Pavel, > > thanks for the information. I've been doing more investigation of this > > issue and there's autovacuum running on the table however it's > > automatically starting even if there is

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Pavel Stehule
2016-01-12 12:14 GMT+01:00 Michal Novotny : > Hi Pavel, > thanks for the information. I've been doing more investigation of this > issue and there's autovacuum running on the table however it's > automatically starting even if there is "autovacuum = off" in the > postgresql.conf configuration file

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Michal Novotny
Hi Andres, thanks a lot for your reply. Unfortunately I've found out most it didn't really start DROP TABLE yet and it's locked on autovacuum running for the table and even if I kill the process it's autostarting again and again. Is there any way how to do the DROP TABLE and bypass/disable autovac

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Michal Novotny
Hi Pavel, thanks for the information. I've been doing more investigation of this issue and there's autovacuum running on the table however it's automatically starting even if there is "autovacuum = off" in the postgresql.conf configuration file. The test of rm 5T file was fast and not taking 24 ho

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Andres Freund
Hi Michal, This isn't really a question for -hackers, the list for postgres development. -general or -performance would have been more appropriate. On 2016-01-12 11:57:05 +0100, Michal Novotny wrote: > I've discovered an issue with dropping a large table (~5T). I was > thinking drop table is fast

Re: [HACKERS] Question about DROP TABLE

2016-01-12 Thread Pavel Stehule
Hi 2016-01-12 11:57 GMT+01:00 Michal Novotny : > Dear PostgreSQL Hackers, > I've discovered an issue with dropping a large table (~5T). I was > thinking drop table is fast operation however I found out my assumption > was wrong. > > Is there any way how to tune it to drop a large table in the mat

[HACKERS] Question about DROP TABLE

2016-01-12 Thread Michal Novotny
Dear PostgreSQL Hackers, I've discovered an issue with dropping a large table (~5T). I was thinking drop table is fast operation however I found out my assumption was wrong. Is there any way how to tune it to drop a large table in the matter of seconds or minutes? Any configuration variable in the

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-17 Thread Alvaro Herrera
konstantin knizhnik wrote: > The transaction is normally committed in xlog, so that it can always be > recovered in case of node fault. > But before setting correspondent bit(s) in CLOG and releasing locks we first > contact arbiter to get global status of transaction. > If it is successfully lo

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-17 Thread Kevin Grittner
On Tuesday, November 17, 2015 12:43 AM, konstantin knizhnik wrote: > On Nov 16, 2015, at 11:21 PM, Kevin Grittner wrote: >> If you are saying that DTM tries to roll back a transaction after >> any participating server has entered the RecordTransactionCommit() >> critical section, then IMO it is

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-17 Thread konstantin knizhnik
On Nov 17, 2015, at 10:44 AM, Amit Kapila wrote: > > I think the general idea is that if Commit is WAL logged, then the > operation is considered to committed on local node and commit should > happen on any node, only once prepare from all nodes is successful. > And after that transaction is not

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread Atri Sharma
> I think the general idea is that if Commit is WAL logged, then the > operation is considered to committed on local node and commit should > happen on any node, only once prepare from all nodes is successful. > And after that transaction is not supposed to abort. But I think you are > trying to o

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread Amit Kapila
On Tue, Nov 17, 2015 at 12:12 PM, konstantin knizhnik < k.knizh...@postgrespro.ru> wrote: > Thank you for your response. > > > On Nov 16, 2015, at 11:21 PM, Kevin Grittner wrote: > > I'm not entirely clear on what you're saying here. I admit I've > not kept in close touch with the distributed pro

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread konstantin knizhnik
Thank you for your response. On Nov 16, 2015, at 11:21 PM, Kevin Grittner wrote: > I'm not entirely clear on what you're saying here. I admit I've > not kept in close touch with the distributed processing discussions > lately -- is there a write-up and/or diagram to give an overview of > where w

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread Kevin Grittner
On Monday, November 16, 2015 2:47 AM, Konstantin Knizhnik wrote: > Some time ago at PgConn.Vienna we have proposed eXtensible >Transaction Manager API (XTM). > The idea is to be able to provide custom implementation of >transaction managers as standard Postgres extensions, > primary goal

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread Konstantin Knizhnik
On 11/16/2015 10:54 PM, Alvaro Herrera wrote: Konstantin Knizhnik wrote: But you may notice that original TransactionIdSetTreeStatus function is void - it is not intended to return anything. It is called in RecordTransactionCommit in critical section where it is not expected that commit may fai

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread Alvaro Herrera
Konstantin Knizhnik wrote: > But you may notice that original TransactionIdSetTreeStatus function is void > - it is not intended to return anything. > It is called in RecordTransactionCommit in critical section where it is not > expected that commit may fail. > But in case of DTM transaction may b

[HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

2015-11-16 Thread Konstantin Knizhnik
Hello, Some time ago at PgConn.Vienna we have proposed eXtensible Transaction Manager API (XTM). The idea is to be able to provide custom implementation of transaction managers as standard Postgres extensions, primary goal is implementation of distritibuted transaction manager. It should not o

Re: [HACKERS] Question about TEMP tables

2015-03-17 Thread David G. Johnston
On Tuesday, March 17, 2015, Воронин Дмитрий wrote: > > Make sure to show your full command(s) and the full, exact text of any > errors. > > OK, I use PostgreSQL version 9.4.1. > > I create cluster 'main' and connect to it. After cluster init we have > those shemas: > > postgres=# SELECT nspname

Re: [HACKERS] Question about TEMP tables

2015-03-17 Thread Воронин Дмитрий
> Make sure to show your full command(s) and the full, exact text of any > errors. OK, I use PostgreSQL version 9.4.1. I create cluster 'main' and connect to it. After cluster init we have those shemas: postgres=# SELECT nspname FROM pg_namespace ; nspname p

[HACKERS] Question about TEMP tables

2015-03-17 Thread Воронин Дмитрий
Hello, all. We can create temp namespaces and temp objects that contains it. So, for example, temp table will be create at pg_temp_N (N - backendID). But afrer cluster init we have pg_temp_1 and pg_toast_temp_1 namespaces with OIDs 11333 and 11334. Those namespaces are visible from any cluster

Re: [HACKERS] Question about TEMP tables

2015-03-16 Thread Craig Ringer
On 16 March 2015 at 16:31, Dmitry Voronin wrote: > Hello, all. > > We can create temp namespaces and temp objects that contains it. So, for > example, temp table will be create at pg_temp_N (N - backendID). But afrer > cluster init we have pg_temp_1 and pg_toast_temp_1 namespaces with OIDs > 1133

[HACKERS] Question about TEMP tables

2015-03-16 Thread Dmitry Voronin
Hello, all. We can create temp namespaces and temp objects that contains it. So, for example, temp table will be create at pg_temp_N (N - backendID). But afrer cluster init we have pg_temp_1 and pg_toast_temp_1 namespaces with OIDs 11333 and 11334. Those namespaces are visible from any cluster

[HACKERS] Question about TEMP tables

2015-03-16 Thread Dmitry Voronin
Hello, all.We can create temp namespaces and temp objects that contains it. So, for example, temp table will be create at pg_temp_N (N - backendID). But afrer cluster init we have pg_temp_1 and pg_toast_temp_1 namespaces with OIDs 11333 and 11334. Those namespaces are visible from any cluster datab

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-10 Thread Noah Misch
I'm okay with any of the proposed designs or with dropping the idea. Closing the loop on a few facts: On Sat, Mar 07, 2015 at 04:34:41PM -0600, Jim Nasby wrote: > If we go that route, does it still make sense to explicitly use > repalloc_huge? It will just cut over to that at some point (128M?) a

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-09 Thread Andres Freund
On 2015-03-09 17:12:22 -0500, Jim Nasby wrote: > That will be significantly more code than a simple repalloc, but as long as > people are OK with that I can go that route. I still would like to see some actual evidence of need for change before we invest more time/code here. Greetings, Andres Fr

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-09 Thread Jim Nasby
On 3/9/15 12:28 PM, Alvaro Herrera wrote: Robert Haas wrote: On Sat, Mar 7, 2015 at 5:49 PM, Andres Freund wrote: On 2015-03-05 15:28:12 -0600, Jim Nasby wrote: I was thinking the simpler route of just repalloc'ing... the memcpy would suck, but much less so than the extra index pass. 64M gets

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-09 Thread Alvaro Herrera
Robert Haas wrote: > On Sat, Mar 7, 2015 at 5:49 PM, Andres Freund wrote: > > On 2015-03-05 15:28:12 -0600, Jim Nasby wrote: > >> I was thinking the simpler route of just repalloc'ing... the memcpy would > >> suck, but much less so than the extra index pass. 64M gets us 11M tuples, > >> which prob

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-09 Thread Robert Haas
On Sat, Mar 7, 2015 at 5:49 PM, Andres Freund wrote: > On 2015-03-05 15:28:12 -0600, Jim Nasby wrote: >> I was thinking the simpler route of just repalloc'ing... the memcpy would >> suck, but much less so than the extra index pass. 64M gets us 11M tuples, >> which probably isn't very common. > > T

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-07 Thread Jim Nasby
On 3/7/15 4:49 PM, Andres Freund wrote: On 2015-03-05 15:28:12 -0600, Jim Nasby wrote: I was thinking the simpler route of just repalloc'ing... the memcpy would suck, but much less so than the extra index pass. 64M gets us 11M tuples, which probably isn't very common. That has the chance of co

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-07 Thread Andres Freund
On 2015-03-05 15:28:12 -0600, Jim Nasby wrote: > I was thinking the simpler route of just repalloc'ing... the memcpy would > suck, but much less so than the extra index pass. 64M gets us 11M tuples, > which probably isn't very common. That has the chance of considerably increasing the peak memory

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-07 Thread Jim Nasby
On 3/7/15 12:48 AM, Noah Misch wrote: On Sat, Mar 07, 2015 at 12:46:42AM -0500, Tom Lane wrote: Noah Misch writes: On Thu, Mar 05, 2015 at 03:28:12PM -0600, Jim Nasby wrote: I was thinking the simpler route of just repalloc'ing... the memcpy would suck, but much less so than the extra index p

  1   2   3   4   5   6   7   8   >