Re: [GENERAL] BDR replication and table triggers
> However if I perform any INSERT, UPDATE or DELETE operations on > DB2 and these changes propagate over to DB1 via BDR I do not see DB1 firing > any triggers. Is this intended behavior? Yes. > My current understanding is that > BDR is unable to invoke Postgres triggers as it operates on the rows > directly, a layer below Postgres. Is this Correct? Yes. > Is there any mechanism > that exists that could provide notifications to a listening application when > BDR makes changes to the underlying database? You could listen to an underlying logical decoding stream, but it might be a bit fiddly and complex for your needs. Ideally we'd be able to fire triggers in BDR, but that's not implemented or on the current roadmap and there's no funded work on it at this point. There's some work to support it in pglogical though. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Implicit typecasting to numeric in psql
Payal Singhwrites: > I have a table with an integer column 'userid'. But I am not seeing an out > of range error when trying to get an id larger than possible in integer: > db=# explain select * from users where userid = > 21474836472871287898765456789::numeric; Cross-type comparisons are legal, in general, so this is a legal query. The fact that no rows could match is not relevant to that. > Also, when putting it in quotes or explicitly casting it to integer, I do > get the our of range message: > db=# select * from users where userid = > 21474836472344567898765456789::integer; > ERROR: integer out of range Well, sure. That number doesn't fit in an integer. > db=# explain select * from users where userid = '21474737377373737373'; > ERROR: value "21474737377373737373" is out of range for type integer The reason this fails is that the quoted literal initially has type "unknown", and the parser's heuristic for resolving the unknown is, in this case, to give it the same type as the operator's other input. So then it tries to convert 21474737377373737373 to integer. See https://www.postgresql.org/docs/current/static/typeconv-oper.html particularly rule 3f. > It seems when on psql and querying for a numeric type, postgres is not > checking the type of the column, but instead converting into the numeric > type that best matches the length: That's specified in the description of constants, https://www.postgresql.org/docs/current/static/sql-syntax-lexical.html#SQL-SYNTAX-CONSTANTS (see 4.1.2.6 about numeric constants). Numeric constants don't start out as "unknown" the way quoted literals do, because it's possible to make a reasonable determination of their type without any context. regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] BDR replication and table triggers
Why not using the logical decoding feature: https://www.postgresql.org/docs/9.4/static/logicaldecoding-example.html On both sides, you would have a process that regularly decodes the stream and emits notifications for event in tables you are insterested in. Sylvain 2017-05-02 18:18 GMT+02:00 Alvaro Aguayo Garcia-Rada < aagu...@opensysperu.com>: > Hi. > > It's not like BDR is unable to replicate triggers across the cluster: BDR > is not intended to do so. > > BDR replicates everything that happens inside a transaction; that includes > both SQL run directly from the application, as well as changes made by > triggers and extensions. As the changes are applied directly from the WAL, > no trigger is re-run on the other nodes. If the trigger is re-run, that > would lead to problems, such as duplicated rows. > > The only "problem", if it really is, is that BDR does not copy > notifications across the databases. As this may be seen as a problem, I > could also consider it as a chance to make the application more > self-conscious of the distributed environment it is running in. So I would > try one out of two alternatives: > > 1. Make the application listen to notifications on both databases, so it > will get notified of changes no matter where they happen > > 2. Instead of using notify, create a notification table, which your app > should scan periodically and act accordingly. > > Regards, > > Alvaro Aguayo > Jefe de Operaciones > Open Comb Systems E.I.R.L. > > Oficina: (+51-1) 3377813 | RPM: #034252 / (+51) 995540103 | RPC: (+51) > 954183248 > Website: www.ocs.pe > > - Original Message - > From: "jamesadams89"> To: "PostgreSql-general" > Sent: Wednesday, 26 April, 2017 07:48:03 > Subject: [GENERAL] BDR replication and table triggers > > Hi, > > I have some questions regarding how BDR interacts with triggers. > > I have two databases that are both joined to the same BDR group and > correctly replicating between one another sharing a table created as: > > create table testtable( > key varchar(16) NOT NULL PRIMARY KEY, > data jsonb > ); > > With the following trigger defined: > > CREATE OR REPLACE FUNCTION test_table_notify() > RETURNS TRIGGER AS > $$ > BEGIN > IF TG_OP='INSERT' OR TG_OP='UPDATE' THEN > PERFORM pg_notify( 'TestTable', TG_OP || ' ' || NEW.key ); > ELSE > PERFORM pg_notify( 'TestTable', TG_OP || ' ' || OLD.key ); > END IF; > RETURN NULL; > END; > $$ LANGUAGE plpgsql; > > > CREATE TRIGGER TestTableTrigger > AFTER INSERT OR UPDATE OR DELETE > on testtable > FOR EACH ROW > EXECUTE PROCEDURE test_table_notify(); > > I then have a client application listening on the 'TestTable' Notify on one > of the Databases: > > Client > ___ > | | > | A | > |___| > /\ >| > _|_ ___ > | | | | > |DB1|-|DB2| > |_ __| || > > If I perform any INSERT, UPDATE or DELETE operations directly on DB1 I see > the trigger on the table being fired as expected and Client Application 'A' > recieves the notify. I also see the changes propagate to DB2 via BDR as > expected. However if I perform any INSERT, UPDATE or DELETE operations on > DB2 and these changes propagate over to DB1 via BDR I do not see DB1 firing > any triggers. Is this intended behavior? My current understanding is that > BDR is unable to invoke Postgres triggers as it operates on the rows > directly, a layer below Postgres. Is this Correct? Is there any mechanism > that exists that could provide notifications to a listening application > when > BDR makes changes to the underlying database? > > Apologies if this is all a bit elementary, this is my first foray into BDR > and I was unable to find anything in the documentation that mentioned > triggers. > > Thanks for any input > > > > -- > View this message in context: http://www.postgresql-archive. > org/BDR-replication-and-table-triggers-tp5958463.html > Sent from the PostgreSQL - general mailing list archive at Nabble.com. > > > -- > Sent via pgsql-general mailing list (pgsql-general@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-general > > > -- > Sent via pgsql-general mailing list (pgsql-general@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-general >
[GENERAL] Implicit typecasting to numeric in psql
Hi, I have a table with an integer column 'userid'. But I am not seeing an out of range error when trying to get an id larger than possible in integer: db=# explain select * from users where userid = 21474836472871287898765456789::numeric; QUERY PLAN -- Seq Scan on users (cost=0.00..4047620.36 rows=431750 width=301) Filter: ((userid)::numeric = '21474836472871287898765456789'::numeric) (2 rows) I do see an error when using a bind variable though, just not in psql. Also, when putting it in quotes or explicitly casting it to integer, I do get the our of range message: db=# select * from users where userid = 21474836472344567898765456789::integer; ERROR: integer out of range db=# explain select * from users where userid = '21474737377373737373'; ERROR: value "21474737377373737373" is out of range for type integer LINE 1: ...lain select * from users where userid = '214747373... ^ db=# explain select * from users where userid = '2147'; QUERY PLAN --- Index Scan using userid_pkey on users (cost=0.57..8.59 rows=1 width=301) Index Cond: (userid = 2147) It seems when on psql and querying for a numeric type, postgres is not checking the type of the column, but instead converting into the numeric type that best matches the length: db=# explain select * from users where userid = 2147473737737373; QUERY PLAN --- Index Scan using userid_pkey on users (cost=0.57..8.59 rows=1 width=301) Index Cond: (userid = '2147473737737373'::bigint) (2 rows) db=# explain select * from users where userid = 21474737377373737373; QUERY PLAN -- Seq Scan on users (cost=0.00..4047620.36 rows=431750 width=301) Filter: ((userid)::numeric = '21474737377373737373'::numeric) (2 rows) Why is it that postgres checks the data type of the column when value is in quotes vs not checking when no quotes are used? Thanks, -- Payal Singh
Re: [GENERAL] Language support of postgresql
On 5/2/2017 11:41 AM, Tom Lane wrote: John R Piercewrites: I thought Postgres supported client_encodings of BIG5, GB18030, and GBK, all of which can be stored in the server using either UTF8 or MULE_INTERNAL (MultiLingual EMACS) encodings for internal storage ? Hm, there's MULE<=>BIG5 converters but I don't see any for GBK or GB18030. Also, it looks like the MULE<=>BIG5 converters do some re-encoding, so it's not clear to me whether they're lossless, which I assume is the concern driving this request. I based my statement on misreading the tables on here, https://www.postgresql.org/docs/current/static/multibyte.html but, now I see, MULE only supports big5 and EUC_CN. My limited readings earlier about BIG5 suggested its a mess of conflicting extensions, E-TEN and others, and the GB* stuff wasn't much better. Anyways, it seems to me like UTF8 is the correct server encoding for most all uses. -- john r pierce, recycling bits in santa cruz
Re: [GENERAL] Language support of postgresql
John R Piercewrites: > I thought Postgres supported client_encodings of BIG5, GB18030, and GBK, > all of which can be stored in the server using either UTF8 or > MULE_INTERNAL (MultiLingual EMACS) encodings for internal storage ? Hm, there's MULE<=>BIG5 converters but I don't see any for GBK or GB18030. Also, it looks like the MULE<=>BIG5 converters do some re-encoding, so it's not clear to me whether they're lossless, which I assume is the concern driving this request. Still, you're right, there's more than one way to skin this cat. Somebody could write an encoding converter that translates one of these ASCII-unsafe representations into an ASCII-safe format to be used internally in the backend, and then the reverse on the way out. regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Language support of postgresql
"Martel, Hong"writes: > As I understand, currently Postgres doesn$B!G(Bt support Chinese encoding > GBK and BIG5 on both server and client side, only UNICODE. Is it true? Are > there any plans for postgresql team to implement GBK and BIG5 encoding > anytime soon? Yes, and no. There's basically zero chance that we'll ever allow these ecodings as server-side encodings, because they aren't strict ASCII supersets (that is, not all bytes of a multibyte character are individually distinguishable from an ASCII character). The amount of work involved, and the ongoing hazard of security bugs that would ensue, is just prohibitive. We do however support them as client-side encodings with automatic translation to and from Unicode on the server. regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Language support of postgresql
On 4/28/2017 7:45 AM, Martel, Hong wrote: As I understand, currently Postgres doesn’t support Chinese encoding GBK and BIG5 on both server and client side, only UNICODE. Is it true? Are there any plans for postgresql team to implement GBK and BIG5 encoding anytime soon? Are there any alternative solutions for this besides switching our database to Oracle or others that support the encodings? One of our customers insists that we need to support all three encoding (BIG5, GB2312安and UNICODE). We would love to stick to Postgres if there is any alternative way to solve the problem without incurring big cost. I thought Postgres supported client_encodings of BIG5, GB18030, and GBK, all of which can be stored in the server using either UTF8 or MULE_INTERNAL (MultiLingual EMACS) encodings for internal storage ? -- john r pierce, recycling bits in santa cruz
Re: [GENERAL] BDR replication and table triggers
Hi. It's not like BDR is unable to replicate triggers across the cluster: BDR is not intended to do so. BDR replicates everything that happens inside a transaction; that includes both SQL run directly from the application, as well as changes made by triggers and extensions. As the changes are applied directly from the WAL, no trigger is re-run on the other nodes. If the trigger is re-run, that would lead to problems, such as duplicated rows. The only "problem", if it really is, is that BDR does not copy notifications across the databases. As this may be seen as a problem, I could also consider it as a chance to make the application more self-conscious of the distributed environment it is running in. So I would try one out of two alternatives: 1. Make the application listen to notifications on both databases, so it will get notified of changes no matter where they happen 2. Instead of using notify, create a notification table, which your app should scan periodically and act accordingly. Regards, Alvaro Aguayo Jefe de Operaciones Open Comb Systems E.I.R.L. Oficina: (+51-1) 3377813 | RPM: #034252 / (+51) 995540103 | RPC: (+51) 954183248 Website: www.ocs.pe - Original Message - From: "jamesadams89"To: "PostgreSql-general" Sent: Wednesday, 26 April, 2017 07:48:03 Subject: [GENERAL] BDR replication and table triggers Hi, I have some questions regarding how BDR interacts with triggers. I have two databases that are both joined to the same BDR group and correctly replicating between one another sharing a table created as: create table testtable( key varchar(16) NOT NULL PRIMARY KEY, data jsonb ); With the following trigger defined: CREATE OR REPLACE FUNCTION test_table_notify() RETURNS TRIGGER AS $$ BEGIN IF TG_OP='INSERT' OR TG_OP='UPDATE' THEN PERFORM pg_notify( 'TestTable', TG_OP || ' ' || NEW.key ); ELSE PERFORM pg_notify( 'TestTable', TG_OP || ' ' || OLD.key ); END IF; RETURN NULL; END; $$ LANGUAGE plpgsql; CREATE TRIGGER TestTableTrigger AFTER INSERT OR UPDATE OR DELETE on testtable FOR EACH ROW EXECUTE PROCEDURE test_table_notify(); I then have a client application listening on the 'TestTable' Notify on one of the Databases: Client ___ | | | A | |___| /\ | _|_ ___ | | | | |DB1|-|DB2| |_ __| || If I perform any INSERT, UPDATE or DELETE operations directly on DB1 I see the trigger on the table being fired as expected and Client Application 'A' recieves the notify. I also see the changes propagate to DB2 via BDR as expected. However if I perform any INSERT, UPDATE or DELETE operations on DB2 and these changes propagate over to DB1 via BDR I do not see DB1 firing any triggers. Is this intended behavior? My current understanding is that BDR is unable to invoke Postgres triggers as it operates on the rows directly, a layer below Postgres. Is this Correct? Is there any mechanism that exists that could provide notifications to a listening application when BDR makes changes to the underlying database? Apologies if this is all a bit elementary, this is my first foray into BDR and I was unable to find anything in the documentation that mentioned triggers. Thanks for any input -- View this message in context: http://www.postgresql-archive.org/BDR-replication-and-table-triggers-tp5958463.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
[GENERAL] Column rename in an extension update script
Hi all, I am coding an update script for an extension. And I am in trouble when trying to rename a column of an existing table. Just after the ALTER TABLE statement, I want to access this table. But at this time, the altered column is not visible with its new name. I wrote a simple test case to show this. Here is the shell script that can be easily adapted. # issue in postgres extension when trying to access a column that has been renamed inside an extension update script # export EXTDIR="/tmp" export PGDIR="/usr/local/pg962/share/postgresql/extension" export PGHOST=localhost export PGPORT=5496 export PGDATABASE='postgres' echo "create files for the extension" echo "--" cat >$EXTDIR/myextension.control <<*END* default_version= '1' directory= '$EXTDIR' *END* sudo ln -s $EXTDIR/myextension.control $PGDIR/myextension.control cat >$EXTDIR/myextension--1.sql <<*END* CREATE TABLE mytable (col_old INT); *END* cat >$EXTDIR/myextension--1--2.sql <<*END* ALTER TABLE mytable RENAME col_old TO col_new; UPDATE mytable SET col_new = 0; *END* echo "psql: run the test ==> FAILS" echo "" psql -a <<*END* select version(); CREATE EXTENSION myextension VERSION '1'; ALTER EXTENSION myextension UPDATE TO '2'; DROP EXTENSION IF EXISTS myextension; *END* echo "psql: similar statements outside extension ==> WORKS" echo "" psql -a <<*END* CREATE TABLE mytable (col_old INT); BEGIN; ALTER TABLE mytable RENAME col_old TO col_new; UPDATE mytable SET col_new = 0; COMMIT; DROP TABLE IF EXISTS mytable; *END* sudo rm $PGDIR/myextension.control rm $EXTDIR/myextension* And here is the result: create files for the extension -- psql: run the test ==> FAILS select version(); version - PostgreSQL 9.6.2 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609, 64-bit (1 row) CREATE EXTENSION myextension VERSION '1'; CREATE EXTENSION ALTER EXTENSION myextension UPDATE TO '2'; ERROR: column "col_new" of relation "mytable" does not exist DROP EXTENSION IF EXISTS myextension; DROP EXTENSION psql: similar statements outside extension ==> WORKS CREATE TABLE mytable (col_old INT); CREATE TABLE BEGIN; BEGIN ALTER TABLE mytable RENAME col_old TO col_new; ALTER TABLE UPDATE mytable SET col_new = 0; UPDATE 0 COMMIT; COMMIT DROP TABLE IF EXISTS mytable; DROP TABLE As you can see: - the error message is "ERROR: column "col_new" of relation "mytable" does not exist", while the ALTER TABLE statement doesn't return any error, - the same statements in a simple psql script works fine, - I reproduce this with all supported postgres versions. As a workaround, I perform the UPDATE statement before the ALTER TABLE operation, using of course the old column name. I probably do something wrong. But I can't see what. Thanks by advance for any piece of advise. Best regards. Philippe Beaudoin.
Re: [GENERAL] Can PG replace redis, amqp, s3 in the future?
On April 30, 2017 1:37:02 PM GMT+02:00, "Thomas Güttler"wrote: >Is is possible that PostgreSQL will replace these building blocks in >the future? > > - redis (Caching) > - rabbitmq (amqp) > - s3 (Blob storage) These are three very different sets of functionalities, each requiring a different approach. I am curious as to why you are thinking about having a single piece of software that does these three very different things. >One question is "is it possible?", then next "is it feasible?" Possible? Sure: p != 0 Probable? No Desirable? No >I think it would be great if I could use PG only and if I could >avoid the other types of servers. When you're holding a hammer, everything looks like a nail. But hammering screws doesn't get you very far. Sometimes you need a screwdriver and on other days a glue gun... >The benefit is not very obvious on the first sight. I think it will >saves you >time, money and energy only in the long run. > >What do you think? Do one thing(*) and do that thing well. Don't try to be everything to everyone. -- Thomas (Sent from my mobile device, please forgive brevity or typos.) -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] Language support of postgresql
Hi, As I understand, currently Postgres doesn’t support Chinese encoding GBK and BIG5 on both server and client side, only UNICODE. Is it true? Are there any plans for postgresql team to implement GBK and BIG5 encoding anytime soon? Are there any alternative solutions for this besides switching our database to Oracle or others that support the encodings? One of our customers insists that we need to support all three encoding (BIG5, GB2312 安and UNICODE). We would love to stick to Postgres if there is any alternative way to solve the problem without incurring big cost. Thank you very much for your time and attention. Sincerely, Hong Martel Software Developer This message is intended only for the addressee and may contain information that is company confidential or privileged. Any technical data in this message may be exported only in accordance with the U.S. International Traffic in Arms Regulations (22 CFR Parts 120-130) or the Export Administration Regulations (15 CFR Parts 730-774). Unauthorized use is strictly prohibited and may be unlawful. If you are not the intended recipient, or the person responsible for delivering to the intended recipient, you should not read, copy, disclose or otherwise use this message. If you have received this email in error, please delete it, and advise the sender immediately.
[GENERAL] Generalized pg_stat_statements?
When reviewing execution statistics, I am frequently interested in the behavior of "classes" of queries, rather than individual queries, for example queries which - Contain a join - Touch a specific column - Use POSIX regular expressions AFAIK this sort of summary/rollup information can't be computed reliably from pg_stat_statements because of rows being discarded in the face of a large number of (post normalization/jumble) distinct statements. Is there a way to retrieve these kinds of statistics in Postgres today? Thanks, Jacob
Re: [GENERAL] FDW table doesn't exist
Figured this out... someone changed the name of the column and took out the extra 'h' in acceptance_threshhold on the database the fdw is pointing -- View this message in context: http://www.postgresql-archive.org/FDW-table-doesn-t-exist-tp5958279p5958336.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
[GENERAL] BDR replication and table triggers
Hi, I have some questions regarding how BDR interacts with triggers. I have two databases that are both joined to the same BDR group and correctly replicating between one another sharing a table created as: create table testtable( key varchar(16) NOT NULL PRIMARY KEY, data jsonb ); With the following trigger defined: CREATE OR REPLACE FUNCTION test_table_notify() RETURNS TRIGGER AS $$ BEGIN IF TG_OP='INSERT' OR TG_OP='UPDATE' THEN PERFORM pg_notify( 'TestTable', TG_OP || ' ' || NEW.key ); ELSE PERFORM pg_notify( 'TestTable', TG_OP || ' ' || OLD.key ); END IF; RETURN NULL; END; $$ LANGUAGE plpgsql; CREATE TRIGGER TestTableTrigger AFTER INSERT OR UPDATE OR DELETE on testtable FOR EACH ROW EXECUTE PROCEDURE test_table_notify(); I then have a client application listening on the 'TestTable' Notify on one of the Databases: Client ___ | | | A | |___| /\ | _|_ ___ | | | | |DB1|-|DB2| |_ __| || If I perform any INSERT, UPDATE or DELETE operations directly on DB1 I see the trigger on the table being fired as expected and Client Application 'A' recieves the notify. I also see the changes propagate to DB2 via BDR as expected. However if I perform any INSERT, UPDATE or DELETE operations on DB2 and these changes propagate over to DB1 via BDR I do not see DB1 firing any triggers. Is this intended behavior? My current understanding is that BDR is unable to invoke Postgres triggers as it operates on the rows directly, a layer below Postgres. Is this Correct? Is there any mechanism that exists that could provide notifications to a listening application when BDR makes changes to the underlying database? Apologies if this is all a bit elementary, this is my first foray into BDR and I was unable to find anything in the documentation that mentioned triggers. Thanks for any input -- View this message in context: http://www.postgresql-archive.org/BDR-replication-and-table-triggers-tp5958463.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
[GENERAL] FDW table doesn't exist
Hi all, I have a FDW to another database but I started getting this error while trying to use it select * from aut_class limit 10; ERROR: column "acceptance_threshhold" does not exist HINT: Perhaps you meant to reference the column "aut_class.acceptance_threshold". Yet when I do some checking, it's obvious that the column does exist. acceptance_threshhold | numeric| | (column_name 'acceptance_threshhold') | main | | Any ideas on what I can do to fix this or debug this problem? -- View this message in context: http://www.postgresql-archive.org/FDW-table-doesn-t-exist-tp5958279.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general