Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Michael Paquier
On Fri, Nov 17, 2017 at 11:14 AM, Andres Freund wrote: > On 2017-11-17 11:09:56 +0900, Michael Paquier wrote: >> when redirection_done is switched to true because the first process >> generating a message to the syslogger pipe needs to open it first if >> not done yet? &g

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Michael Paquier
ue because the first process generating a message to the syslogger pipe needs to open it first if not done yet? So you'd need proper locking to save from race conditions. Or is the first message redirected message always generated by the postmaster or the syslogger? I don't recall that this

Re: [GENERAL] Build in function to verify email addresses

2017-11-16 Thread Michael Nolan
On Thu, Nov 16, 2017 at 1:56 AM, Nick Dro wrote: > I beleieve that every information system has the needs to send emails. > Currently PostgreSQL doesn't have a function which gets TEXT and return > true if it's valid email address (x...@yyy.com / .co.ZZ) > Do you believe such function should exis

Re: [GENERAL] PostgreSQL walsender process doesn't exist after "pg_ctl stop -m fast"

2017-11-13 Thread Michael Paquier
f the source server was idle. In passing, improve the rather weak comments in this area, and slightly rearrange some related code for better readability. Back-patch to 9.4 where this code was introduced. Discussion: https://postgr.es/m/14154.1498781...@sss.pgh.pa.us -- Michael -- Sent via pgs

Re: [GENERAL] archive_command not being executed

2017-11-13 Thread Michael Paquier
andby you'd like to get the archives from. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Postgresql and github

2017-11-09 Thread Michael Paquier
a mirror of the official repository in git.postgresql.org: https://github.com/postgres/postgres There is as well a section with release tarballs (so do the facilities offered by community by the way). -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make

Re: [GENERAL] Naming conventions for column names

2017-11-07 Thread Michael Paquier
rsonally tend to take with a pinch of salt such proposals though if there are no good reasons behind a switch other than because it-is-beautiful, so I agree with Álvaro that it is good to be careful here. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes

Re: [GENERAL] Fwd: standby stop replicating, then picked back up

2017-11-07 Thread Michael Paquier
y to recover from an earlier point. Strategies to adopt mainly depend on if taking a full backup is more costly than a range of WAL segments, so the data folder size of the primary instance matters as a decision-making parameter. -- Michael -- Sent via pgsql-general mailing list (pgsql-gener

Re: [GENERAL] [HACKERS] SSL and Encryption

2017-11-02 Thread Michael Paquier
5802: https://tools.ietf.org/html/rfc5802 https://tools.ietf.org/html/rfc7677 -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] PG 10 - Trouble with overlap of range partition of two dimensions

2017-10-31 Thread Michael Paquier
>= 1, and the second partition allows source_no >= 3 which overlaps with the first one. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Backup strategy using 'wal_keep_segments'

2017-10-31 Thread Michael Paquier
e using something like pg_receivexlog, pg_stat_replication is the way to go to monitor the archiving progress. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] starting PG command line options vs postgresql.con

2017-10-30 Thread Michael Paquier
ust copy it temporarily. The current way of doing things gives the best of both worlds. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Can't build ODBC -- odbc_config not found

2017-10-27 Thread Michael Paquier
; So it must be looking for a file called "odic_config"? It's not 100% > clear what is needed. Are you sure that you installed unixodbc? Normally an install of this package, be it on Linux or macos should install this command as well. You can enforce a path to it using ODBC_CONFI

Re: [GENERAL] Allow only certain query on replication slave

2017-10-24 Thread Michael Paquier
can be used in the planner hook to check for those functions and accept them. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Postgres 9.6 fails to start on VMWare

2017-10-23 Thread Michael Nolan
On Mon, Oct 23, 2017 at 3:14 AM, Martin Moore wrote: > Same server. I tried a few times. > > I didn’t move the db separately, but did a ‘dd’ to copy the disk to an > imagefile which was converted and loaded into VMWare. > > I ‘believed’ that this should keep the low level disk structure the same,

Re: [GENERAL] Postgres 9.6 fails to start on VMWare

2017-10-22 Thread Michael Paquier
n this world (I am referring to quiesced snapshot & co). -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Backup strategy using 'wal_keep_segments'

2017-10-22 Thread Michael Paquier
logic is copying. So I would advise to not do that. Instead of using the archive command, you should also consider using pg_receivexlog combined with a replication slot. This brings way more control with the error handling. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgres

Re: [GENERAL] Weird performance difference

2017-10-22 Thread Michael Paquier
_autovacuum(). -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] could not fdatasync log file: Input/output error

2017-10-17 Thread Michael Paquier
ly under way to get something improved in Postgres 11. I should actually spare some time to look more at the patch concepts.. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] wal_retrieve_retry_interval

2017-10-16 Thread Michael Paquier
am to archive after failure Ludovic, is there some event happening between those two logs? Something that could explain a longer delay is the time it takes for a WAL receiver to be considered as started (see WALRCV_STARTUP_TIMEOUT). -- Michael -- Sent via pgsql-general mailing list (pgsql-gene

Re: [GENERAL] could not fdatasync log file: Input/output error

2017-10-16 Thread Michael Paquier
is host at this point, this looks like a file system or a disk issue. Before doing anything you should stop the database, and make a cold copy of the data folder on which you could work on if you don't have a live backup. This wiki page is wise on the matter: http://wiki.postgresql.org/wiki/Co

Re: [GENERAL] Using cp to back up a database?

2017-10-09 Thread Michael Paquier
ups :) It depends on what you are trying to achieve, pg_dump can be fine for small-ish databases. By relying on both logical (pg_dump) and physical backups (base backups) brings more insurance in face of a disaster. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.or

Re: [GENERAL] Regarding scram authentication libpq version 10 or more

2017-10-06 Thread Michael Paquier
iple versions of PostgreSQL in parallel, and the client you are using may not be the client you think it is. I suggest that you check the infrastructure of your host as well as the package repository you are using. Good luck. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresq

Re: [GENERAL] Regarding scram authentication libpq version 10 or more

2017-10-06 Thread Michael Paquier
r using password_encryption = 'scram-sha-256', and configure pg_hba.conf with a correct entry. If you are upgrading from an existing instance, you need to make sure that users with passwords are updated with proper SCRAM-hashed entries. -- Michael -- Sent via pgsql-general mailing l

Re: [GENERAL] pg_rewind copy so much data

2017-09-29 Thread Michael Paquier
formation in WAL about changes cannot cover all data in the database. The only thing I have in mind able to create this much amount of data using this less WAL is a CREATE DATABASE using as template an existing database. Based on the information you are giving here this is the best guess I can do.

Re: [GENERAL] pg_rewind copy so much data

2017-09-28 Thread Michael Paquier
On Fri, Sep 29, 2017 at 1:06 PM, Hung Phan wrote: > I used tablespace to store data and it seems to be that pg_rewind copied > everthing in the tablespace. Today I found an article posted by you (Michael > Paquier) and you said that there was no tablespace support. If so, is there &g

Re: [GENERAL] Logical decoding client has the power to crash the server

2017-09-21 Thread Michael Paquier
can make a direct comparison and decide if a slot can be dropped or not. Make sure that things are casted to bigint though. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] 10 beta 4 foreign table partition check constraint broken?

2017-09-20 Thread Michael Paquier
foo_1 ( > id INT NOT NULL, > nameTEXT > ); So here I think that you should add a CHECK constraint to this table, and that the behavior of your example works as expected. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make chan

Re: [GENERAL] Logical decoding client has the power to crash the server

2017-09-20 Thread Michael Paquier
to review the access permissions to your instance. Postgres 9.6 offers better access control to system functions, so you could be granted access to just those resources to be fine using a SQL session. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To

Re: [GENERAL] VM-Ware Backup of VM safe?

2017-09-20 Thread Michael Paquier
: quiesced and non-quiesced. You definitely want a quiesced snapshot when taking a backup so as the system gets into a consistent state when working on it. There should be an option related to that on the vSphere client managing the VM, so make sure that quiesced is enabled. -- Michael -- Sent v

Re: [GENERAL] Logical decoding client has the power to crash the server

2017-09-20 Thread Michael Paquier
ved from any crash. Note that advancing a slot is doable for a physical slot, but advancing a logical slot is trickier (not sure if that's doable actually but Andres can comment on that) because it involves being sure that the catalog_xmin is still preserved so as past logical changes can be l

Re: [GENERAL] pg_rewind issue

2017-09-18 Thread Michael Paquier
n error, you may not copy or distribute any part of it or > otherwise disclose its contents to anyone. Please advise the sender of your > incorrect receipt of this correspondence. The contents of this mailing list are public. -- Michael -- Sent via pgsql-general mailing list (pgsql-gen

Re: [GENERAL] "Canceling authentication due to timeout" with idle transaction and reindex

2017-09-15 Thread Michael Paquier
hid which is looked up at authentication. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] pg_rewind copy so much data

2017-09-14 Thread Michael Paquier
abase after the standy is promoted or something like that? -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] looking for a globally unique row ID

2017-09-14 Thread Michael Paquier
d 3 can be found with the usage of a serial column. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Postgres DB is failed due to pg_Xlog is continues full.

2017-09-13 Thread Michael Paquier
llow it to recover from the crash. You are going to need more space at the end. And yes, upgrade as well. Lagging 7 major releases behind cannot be an excuse. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.

Re: [GENERAL] pg_rewind copy so much data

2017-09-13 Thread Michael Paquier
ersion of Postgres 9.5 you are using as well. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] pg_rewind copy so much data

2017-09-12 Thread Michael Paquier
more efforts would be needed to re-sync it. That's only an assumption without data with clear numbers, numbers that could be found using the --debug messages of pg_rewind. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscript

Re: [GENERAL] WAL & ready files retained after turning off log shipping

2017-09-12 Thread Michael Paquier
mand telling then? If those .ready files are here, it means that you are not able to archive correctly segments. It seems to me that at the end you should try to just set archive_command = '/bin/true', this would solve all your problems, and trick the server correctly... -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] contrecord is requested

2017-09-11 Thread Michael Paquier
LOG page header cannot be found where it should. You may want to check your WAL segments. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] pg_rewind issue

2017-09-07 Thread Michael Paquier
nce step 5 would not be necessary. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] WAL & ready files retained after turning off log shipping

2017-09-07 Thread Michael Paquier
u don't intend to do archiving any more, you can just flush all the > .ready files (and .done if any) without much thought. It would be less risky to do that as a two-time move: - First change archive_command to /sbin/true and let all archives be switched to .done. - And then disable

Re: [GENERAL] CREATE TABLE LIKE including all not including storage parameters?

2017-09-05 Thread Michael Paquier
ch are defined here by toast: https://www.postgresql.org/docs/devel/static/storage-toast.html -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] RemoveIPC problem

2017-08-31 Thread Michael Paquier
ng as you don't run Postgres on scissors with things like fsync = off or full_page_writes = off, there should be no risk with the data consistency. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/

Re: [GENERAL] Table create time

2017-08-31 Thread Michael Paquier
> available during an event trigger, albeit perhaps I am missing something? The function pg_event_trigger_ddl_commands() returns classid and objid, which should map to respectively pg_class and the relation created for a CREATE TABLE query, no? -- Michael -- Sent via pgsql-general mai

Re: [GENERAL] Table create time

2017-08-31 Thread Michael Paquier
st save the timestamp value of now() in a custom table with the name and/or OID of the relation involved. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Unlogged Crash Detection

2017-08-29 Thread Michael Paquier
gger for example. There is no perfect method I am afraid. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Unlogged Crash Detection

2017-08-29 Thread Michael Paquier
erfect, but it give an indication. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Unlogged Crash Detection

2017-08-29 Thread Michael Paquier
lation files of unlogged tables are all removed, and replaced by the init forknum which represents their initial state. You can see by yourself ResetUnloggedRelations & friends in reinit.c. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make chang

Re: [GENERAL] Extension coverage

2017-08-27 Thread Michael Paquier
GXS provides. Here is a simple Makefile I have used for coverage testing with an extension: https://github.com/michaelpq/pg_plugins/blob/master/decoder_raw/Makefile If you enforce abs_top_srcdir=$(pwd) with make coverage, or coverage-html if you want, then both are able to work properly. At least f

Re: [GENERAL] Retrieving query results

2017-08-27 Thread Michael Paquier
On Sun, Aug 27, 2017 at 12:12 AM, Tom Lane wrote: > Michael Paquier writes: >> On Fri, Aug 25, 2017 at 8:10 AM, Tom Lane wrote: >>> I think the real problem occurs where we realloc the array bigger. > >> Looking at the surroundings, I think that it would be nic

Re: [GENERAL] Retrieving query results

2017-08-26 Thread Michael Paquier
t; SIZE_MAX / sizeof(PGresAttValue *)) + return FALSE; Looking at the surroundings, I think that it would be nice to have pqAddTuple and PQsetvalue set an error message with this patch. The user can see now that those would only properly report on OOM, but if we add more types of errors proper err

Re: [GENERAL] Extension coverage

2017-08-25 Thread Michael Paquier
ltas/extra/bin/genhtml --show-details --legend --output-directory=coverage --title=PostgreSQL --num-spaces=4 --prefix= `find . -name lcov.info -print` genhtml: Option prefix requires an argument Use genhtml --help to get usage information But this bit is easy enough to fix by enforcing the value of

Re: [GENERAL] Retrieving query results

2017-08-24 Thread Michael Paquier
retrieve even 1G rows, let alone more ... Yeah, looking at the code we would just need to check if ntups gets negative (well, equal to INT_MIN) after being incremented. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://

Re: [GENERAL] pg_rewind - restore new slave failed to startup during recovery

2017-08-22 Thread Michael Paquier
On Tue, Aug 22, 2017 at 11:39 PM, Magnus Hagander wrote: > On Tue, Aug 22, 2017 at 3:06 AM, Michael Paquier > wrote: >> That flow looks correct to me. No I think that you should trigger >> manually a checkpoint after step 2 on the promoted standby so as its >> control fi

Re: [GENERAL] Retrieving query results

2017-08-22 Thread Michael Paquier
and getting out this much data is not network-wise anyway. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] [BUGS] Fwd: PostgreSQL 9.4.13 is facing issue in shutting down

2017-08-21 Thread Michael Paquier
some confirmation activity. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] pg_rewind - restore new slave failed to startup during recovery

2017-08-21 Thread Michael Paquier
areful about the contents of your archives. Now a failover done correctly is a tricky thing, which could likely fail if you don't issue a checkpoint immediately on the promoted standby if pg_rewind is kicked in the process before an automatic checkpoint happens (because of timeout or volum

Re: [GENERAL] Deleting unwanted wal files

2017-08-17 Thread Michael Paquier
ve_cleanup_command is part of recovery.conf, which gets loaded by the server at the beginning of recovery by the startup process, so the command will get executed continuously on a standby. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your

Re: [GENERAL] Deleting unwanted wal files

2017-08-17 Thread Michael Paquier
tation is here: https://www.postgresql.org/docs/devel/static/pgarchivecleanup.html If a single archive is not cross-used among multiple standbys, you could use it with archive_cleanup_command is recovery.conf to remove unneeded WAL segments. -- Michael -- Sent via pgsql-general mailing list

Re: [GENERAL] Begginers question

2017-08-15 Thread Michael Paquier
d. Note that this depends also on the values of checkpoint_segments (max_wal_size/min_wal_size in Postgres 9.5 and onwards). -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] WAL replication wrong collate

2017-08-15 Thread Michael Paquier
ade the state you saw happen? -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] WAL replication wrong collate

2017-08-14 Thread Michael Paquier
hly two phases: - Log the new pg_database record. - Copy the directory of the template database to the new database. So collations are conserved at replay. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] pg_stat_statements -- Historical Query

2017-08-09 Thread Michael Paquier
ta of pg_stat_statements and then invoke pg_stat_statements_reset to put everything back to zero. Then you would just need to do your analysis work based on the amount of data copied into your custom table. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make c

Re: [GENERAL] Lifetime of PQexecPrepared() returned value

2017-08-04 Thread Michael Paquier
On Fri, Aug 4, 2017 at 9:12 PM, Igor Korot wrote: > Am I missing something? How do I fix the crash? Based on what I can see here, I see nothing wrong. Now it is hard to reach any conclusion with the limited information you are providing. -- Michael -- Sent via pgsql-general mailing l

Re: [GENERAL] standby database crash

2017-08-01 Thread Michael Paquier
tabase seems to be too time consuming. You may want to check the validity of the so-said WAL segment as well. Corrupted data could come from it. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] vacuum on streaming replication

2017-07-31 Thread Michael Paquier
so as tuples that a standby may need to avoid conflicts for its transactions are not removed. So VACUUM may result in less cleanup depending on the read load on the standby. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription

Re: [GENERAL] Fwd: getting error while parsing log file using pgbadger

2017-07-26 Thread Michael Paquier
nk that you should first try to update PERL5LIB so as it points to the location where the module has been installed. Good luck! It does not sound complicated to me to address anyway. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscripti

Re: [GENERAL] Fwd: getting error while parsing log file using pgbadger

2017-07-26 Thread Michael Paquier
_XS-1.31/CSV_XS.pm This is defined in the requirements of pgbadger's README when parsing csv files: https://github.com/dalibo/pgbadger/blob/master/README -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-26 Thread Michael Paquier
On Mon, Jul 24, 2017 at 9:08 PM, Jeff Janes wrote: > On Sun, Jul 16, 2017 at 8:47 AM, Michael Paquier > wrote: >> What do you think about the patch attached? > > Looks OK. Should it mention specifically "On a hot standby" rather than "On > a standby"? O

Re: [GENERAL] pg_dump not dropping event trigger

2017-07-23 Thread Michael Paquier
This has been fixed by the following commit, which will be present in the next round of minor releases planned for the second week of August (https://www.postgresql.org/developer/roadmap/): https://git.postgresql.org/pg/commitdiff/93f039b4944fdf806f029ed46cf192bc9021d8e7 -- Michael -- Sent via pgs

Re: [GENERAL] Logging at schema level

2017-07-20 Thread Michael Paquier
it touches N schemas, making the exercise part of parsing. I think that it would be possible to use the parser hook to achieve that actually, as you need extra lookups for WITH clauses and such. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to yo

Re: [GENERAL] Streaming Replication archive_command is really needed?

2017-07-20 Thread Michael Paquier
On Fri, Jul 21, 2017 at 8:15 AM, Andreas Kretschmer wrote: > Am 21.07.2017 um 08:01 schrieb Michael Paquier: >> "No" is not completely exact and lacks in details. There are two cases >> where having an archive is helpful: >> 1) The standby has disconnected f

Re: [GENERAL] Streaming Replication archive_command is really needed?

2017-07-20 Thread Michael Paquier
new base backup. 2) Backup strategies. Keeping a larger history set of WAL segments is helpful for incremental backups, which is partially the point actually raised upthread about PITR. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Logging at schema level

2017-07-20 Thread Michael Paquier
r, so you could leverage things at application level with some SET commands to switch them dynamically. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Two-phase commit case studies

2017-07-19 Thread Michael Paquier
es when a transaction involves writes to multiple nodes. Postgres JDBC has XA support by the way: https://jdbc.postgresql.org/documentation/faq.html#xa-support -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-16 Thread Michael Paquier
On Fri, Jul 14, 2017 at 9:11 PM, Jeff Janes wrote: > On Thu, Jul 13, 2017 at 10:38 AM, Michael Paquier > wrote: >> >> On Thu, Jul 13, 2017 at 7:23 PM, Jeff Janes wrote: >> > >> > I think that pg_stat_wal_receiver should be crossreferenced in >> > h

Re: [GENERAL] What is exactly a schema?

2017-07-14 Thread Michael Paquier
cumentation gives a good starting point: https://www.postgresql.org/docs/9.6/static/ddl-schemas.html -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-13 Thread Michael Paquier
On Thu, Jul 13, 2017 at 7:23 PM, Jeff Janes wrote: > On Thu, Jul 13, 2017 at 1:15 AM, Michael Paquier > wrote: >> >> On Thu, Jul 13, 2017 at 5:26 AM, Jeff Janes wrote: >> > >> > I think that none of the recovery information functions >> > >

Re: [GENERAL] Associating a basebackup and it's .backup WAL file

2017-07-13 Thread Michael Paquier
imestamps, and this meta data is written and saved when each backup is taken. This saves future lookups at all tarballs when doing cleanup of past backups. I am not sure about the more popular barman and pgBackrest since I know them less, but I would imagine they handle retention policies similarly.

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-13 Thread Michael Paquier
d data in catalogs is based on the shared memory state of the WAL senders and the WAL receiver, and those are wiped out at reconnection. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] debugging SSL connection problems

2017-07-11 Thread Michael Paquier
ct from > OpenSSL, which you can then use OpenSSL to inspect. You should be able to > use (I think) SSL_get_peer_certificate() to get at it. Yes that will work. The SSL context stored in PGconn offers enough entry point to access all the SSL-related data. -- Michael -- Sent via pgsql-gene

Re: [GENERAL] pg_start/stop_backup non-exclusive scripts to snapshot

2017-07-05 Thread Michael Paquier
undreds of man hours spent in developing those backup tools to be robust solutions, done by specialists on the matter. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] duplicate key value violates unique constraint and duplicated records

2017-07-02 Thread Michael Paquier
the community code, it is going to be hard to say if the problem comes from PostgreSQL itself or from something that has been changed there. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Suddenly - LOG: could not open file "postmaster.pid": No such file or directory

2017-06-22 Thread Michael Paquier
2 -0400 Perform an immediate shutdown if the postmaster.pid file is removed -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Error with DefineSavepoint:Unexpected state STARTED

2017-06-19 Thread Michael Paquier
message-id/0A3221C70F24FB45833433255569204D1F6BE40D@G01JPEXMBYT05 I have been thinking a bit about how to fix that, and wondered about using a new transaction status to track that, but that finished by being rather intrusive.. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgre

Re: [GENERAL] Replication slot and pg_rewind

2017-06-05 Thread Michael Paquier
bloat its pg_xlog with useless data. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Replication slot and pg_rewind

2017-06-05 Thread Michael Paquier
e the past WAL segments you won't be able to do a rewind of the previous primary because there is no way to know what are the blocks modified on the standby since the point of divergence. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To m

Re: [GENERAL] pg_rewind - enable wal_log_hints or data-checksums

2017-06-04 Thread Michael Paquier
data. Things get found more quickly. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] wal_retrieve_retry_interval

2017-05-29 Thread Michael Paquier
gs coming from a standby that is being reconnected to a promoted standby? In short you restarted this standby with a fresh recovery.conf. The WAL receiver would try to fetch history file data for validity checks when it notices that there is a timeline jump, when it needs to itself jump to a new t

Re: [GENERAL] wal_retrieve_retry_interval

2017-05-27 Thread Michael Paquier
ng it to 3s, but > it seems to take 15s still. Here are two log samples: Could you double-check your configuration? If I set wal_retrieve_retry_interval to 1s on a standby, I am able to see a connection attempt from a WAL receiver happening with this interval of time in the case of repetitive fa

Re: [GENERAL] Re: Is there possibility btree_redo with XLOG_BTREE_DELETE done between standby_redo and the end of backup

2017-05-25 Thread Michael Paquier
That's a good study and the code is well-commented, so I let you guess what are those conditions and how they are met during recovery. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Is there possibility btree_redo with XLOG_BTREE_DELETE done between standby_redo and the end of backup

2017-05-24 Thread Michael Paquier
ent starting a standalone backend with standby_mode on. Ahh. This one... Thanks. That may be related, but no way to say if that's related with this report as there is no information that a standalone backend is used. Perhaps that's the case though. So it could be possible as well t

Re: [GENERAL] Is there possibility btree_redo with XLOG_BTREE_DELETE done between standby_redo and the end of backup

2017-05-24 Thread Michael Paquier
icating so. Still I don't understand how this PANIC code can be reachable with community code. CountDBBackends() will normally return 0 if consistency is not reached, and postmaster will block incoming connections until a consistent state is reached. -- Michael -- Sent via pgsql-general m

[GENERAL] initdb with ignore hidden option

2017-05-04 Thread Feld, Michael (IMS)
Hi Postgres team, Our organization would like to make a feature request to ignore hidden files/directories when running an initdb. We use hidden directories for misc backup strategies, and it's cumbersome when needing to coordinate with our storage administrator when we are re-u

Re: [GENERAL] pgAdmin4 needs information of v10 SCRAM authentication

2017-04-23 Thread Michael Paquier
proof when building the last message for the client. I didn't check in details, but the routines are the same as in the message above. The format of the hashed password has changed a bit since commit 68e61ee though. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.

Re: [GENERAL] pg_basebackup ----xlog-method=stream

2017-04-21 Thread Michael Paquier
e have been integrated in pg_basebackup with 9.6. -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] pgAdmin4 needs information of v10 SCRAM authentication

2017-04-20 Thread Michael Paquier
On Fri, Apr 21, 2017 at 3:43 PM, Akshay Joshi wrote: >Thanks Michael, will check this. One thing I forgot to mention... Both StoredKey and ServerKey are now encoded in hex, but there is still an open item related to the handling of psql's \password on which I have written a patch t

Re: [GENERAL] pgAdmin4 needs information of v10 SCRAM authentication

2017-04-20 Thread Michael Paquier
crypt/decrypt the password for SCRAM > authentication. Can someone guide me out here. Here you go: https://www.postgresql.org/message-id/76ac7e67-4e3a-f4df-e087-fbac90151...@iki.fi -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Nice to have features: Percentage function

2017-04-16 Thread Michael Nolan
I also have some pre-defined percentage functions, they check the denominator and return null if it is zero, to avoid 'divide by zero' errors. -- Mike Nolan On Sun, Apr 16, 2017 at 11:37 AM, Melvin Davidson wrote: > > > On Sun, Apr 16, 2017 at 12:23 PM, Adrian Klaver > wrote: > >> On 04/15/2017

  1   2   3   4   5   6   7   8   9   10   >