date:20170323

Re: [COMMITTERS] pgsql: ICU support

2017-03-23 Thread Tom Lane

I wrote:
> The buildfarm's mostly happy, but "prion" still isn't, which
> suggests there's someplace in the new code that references an
> already-closed relcache entry.

Ah, scratch that, I was overthinking it.  The problem is pretty
clear from the error message:

+ ERROR:  collation "en-x-icu" for encoding "SQL_ASCII" does not exist

I can reproduce this with vanilla configure options if I'm running
with --with-icu and LANG=C.  In short, this regression test does not
work in C locale, and you need to find a way to disable it in that
environment.

regards, tom lane

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Avoid syntax error on platforms that have neither LOCALE_T nor I

2017-03-23 Thread Tom Lane

Avoid syntax error on platforms that have neither LOCALE_T nor ICU.

Buildfarm member anole sees this union as empty, and doesn't like it.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/457a4448732881b5008f7a3bcca76fc299075ac3

Modified Files
--
src/include/utils/pg_locale.h | 1 +
1 file changed, 1 insertion(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: ICU support

2017-03-23 Thread Tom Lane

The buildfarm's mostly happy, but "prion" still isn't, which
suggests there's someplace in the new code that references an
already-closed relcache entry.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: doc: add missing closing 'sect3' tag for ICU patch

2017-03-23 Thread Bruce Momjian

doc:  add missing closing 'sect3' tag for ICU patch

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/218747d2cf3cc1536ff77435e596280e0e739760

Modified Files
--
doc/src/sgml/charset.sgml | 1 +
1 file changed, 1 insertion(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Add ICU_FLAGS to one more place

2017-03-23 Thread Peter Eisentraut

Add ICU_FLAGS to one more place

Reported-by: Thomas Munro 

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/2e0c17dc7809992e37441a4e39072373f5d99e0d

Modified Files
--
src/backend/snowball/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix crash in ICU patch

2017-03-23 Thread Peter Eisentraut

Fix crash in ICU patch

This only happened with single-byte encodings.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/524e0f7ac8e143e377b8acff24796d7e37b3a9b1

Modified Files
--
src/backend/utils/adt/like.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: ICU support

2017-03-23 Thread Andrew Dunstan

On 03/23/2017 04:07 PM, Peter Eisentraut wrote:
> On 3/23/17 15:33, Peter Eisentraut wrote:
>> ICU support
> Seeing some crashes on the build farm, investigating.  If someone can
> reproduce locally and get a backtrace, let me know.
>

The buildfarm actually does that :-)

See for example

which has stuff like:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  Generic_Text_IC_like (collation=100, pat=0x10cf3f8, str=0x10cf388) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/like.c:197
197 if (pg_database_encoding_max_length() > 1 || locale->provider 
== COLLPROVIDER_ICU)
#0  Generic_Text_IC_like (collation=100, pat=0x10cf3f8, str=0x10cf388) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/like.c:197
#1  texticlike (fcinfo=) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/like.c:394
#2  0x005db832 in ExecMakeFunctionResultNoSets (fcache=0x106af48, 
econtext=0x106b828, isNull=0x7fff5884877c "") at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/executor/execQual.c:1866
#3  0x005dfe76 in ExecEvalExprSwitchContext 
(expression=expression@entry=0x106af48, econtext=, 
isNull=isNull@entry=0x7fff5884877c "") at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/executor/execQual.c:4228
#4  0x00671d29 in evaluate_expr (expr=, 
result_type=result_type@entry=16, result_typmod=result_typmod@entry=-1, 
result_collation=result_collation@entry=0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:4680
#5  0x006732ab in evaluate_function (context=0x7fff58848aa0, 
func_tuple=0x7fb799a56048, funcvariadic=0 '\000', args=0x10cf8a0, 
input_collid=100, result_collid=0, result_typmod=-1, result_type=16, 
funcid=1633) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:4237
#6  simplify_function (funcid=1633, result_type=16, 
result_typmod=result_typmod@entry=-1, result_collid=0, input_collid=100, 
args_p=args_p@entry=0x7fff588488f8, funcvariadic=0 '\000', process_args=1 
'\001', allow_non_const=1 '\001', context=0x7fff58848aa0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:3877
#7  0x00673b4a in eval_const_expressions_mutator (node=0x10cf410, 
context=0x7fff58848aa0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:2584
#8  0x00619be7 in expression_tree_mutator (node=node@entry=0x10cf460, 
mutator=mutator@entry=0x673a10 , 
context=context@entry=0x7fff58848aa0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/nodes/nodeFuncs.c:3017
#9  0x00673a50 in eval_const_expressions_mutator (node=0x10cf460, 
context=0x7fff58848aa0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:3527
#10 0x00619e4b in expression_tree_mutator (node=node@entry=0x10cf4b0, 
mutator=mutator@entry=0x673a10 , 
context=context@entry=0x7fff58848aa0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/nodes/nodeFuncs.c:2912
#11 0x00673a50 in eval_const_expressions_mutator (node=0x10cf4b0, 
context=context@entry=0x7fff58848aa0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:3527
#12 0x0067574f in eval_const_expressions (root=root@entry=0x10cf5d0, 
node=) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/util/clauses.c:2378
#13 0x00660d95 in preprocess_expression (root=root@entry=0x10cf5d0, 
expr=, kind=kind@entry=1) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/plan/planner.c:880
#14 0x0066519d in subquery_planner (glob=glob@entry=0x10cf198, 
parse=parse@entry=0x10cec38, parent_root=parent_root@entry=0x0, 
hasRecursion=hasRecursion@entry=0 '\000', 
tuple_fraction=tuple_fraction@entry=0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/plan/planner.c:593
#15 0x00666321 in standard_planner (parse=0x10cec38, cursorOptions=256, 
boundParams=) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/optimizer/plan/planner.c:306
#16 0x006efeec in pg_plan_query (querytree=querytree@entry=0x10cec38, 
cursorOptions=cursorOptions@entry=256, boundParams=boundParams@entry=0x0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/tcop/postgres.c:798
#17 0x006effce in pg_plan_queries (querytrees=, 
cursorOptions=cursorOptions@entry=256, boundParams=boundParams@entry=0x0) at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/tcop/postgres.c:864
#18 0x006f0467 in exec_simple_query (query_string=0x10cdd68 "SELECT 
'hawkeye' ILIKE 'h%' AS \"true\";") at 
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/backend/tcop/postgres.c:1029
#19 0x006f17cb in PostgresMain (argc=, 
argv=argv@entry=0x10784a0, dbname=0x1078298 "regression", username=) at

[COMMITTERS] pgsql: Fix enum definition.

2017-03-23 Thread Robert Haas

Fix enum definition.

Commit 249cf070e36721a65be74838c53acf8249faf935 assigned to one of
the labels in the middle the value that should have been assigned
to the first member of the enum.  Rushabh's patch didn't have that
defect as submitted, but I managed to mess it up while editing.
Repair.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/c23b186ae9000f66e2abf783fe820913305fb616

Modified Files
--
src/include/pgstat.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: ICU support

2017-03-23 Thread Peter Eisentraut

On 3/23/17 15:33, Peter Eisentraut wrote:
> ICU support

Seeing some crashes on the build farm, investigating.  If someone can
reproduce locally and get a backtrace, let me know.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: ICU support

2017-03-23 Thread Peter Eisentraut

ICU support

Add a column collprovider to pg_collation that determines which library
provides the collation data.  The existing choices are default and libc,
and this adds an icu choice, which uses the ICU4C library.

The pg_locale_t type is changed to a union that contains the
provider-specific locale handles.  Users of locale information are
changed to look into that struct for the appropriate handle to use.

Also add a collversion column that records the version of the collation
when it is created, and check at run time whether it is still the same.
This detects potentially incompatible library upgrades that can corrupt
indexes and other structures.  This is currently only supported by
ICU-provided collations.

initdb initializes the default collation set as before from the `locale
-a` output but also adds all available ICU locales with a "-x-icu"
appended.

Currently, ICU-provided collations can only be explicitly named
collations.  The global database locales are still always libc-provided.

ICU support is enabled by configure --with-icu.

Reviewed-by: Thomas Munro 
Reviewed-by: Andreas Karlsson 

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/eccfef81e1f73ee41f1d8bfe4fa4e80576945048

Modified Files
--
aclocal.m4   |1 +
config/pkg.m4|  275 ++
configure|  313 ++
configure.in |   35 +
doc/src/sgml/catalogs.sgml   |   19 +
doc/src/sgml/charset.sgml|  177 +++-
doc/src/sgml/func.sgml   |   17 +
doc/src/sgml/installation.sgml   |   15 +
doc/src/sgml/mvcc.sgml   |3 +-
doc/src/sgml/ref/alter_collation.sgml|   55 ++
doc/src/sgml/ref/create_collation.sgml   |   37 +-
src/Makefile.global.in   |4 +
src/backend/Makefile |2 +-
src/backend/catalog/pg_collation.c   |   52 +-
src/backend/commands/collationcmds.c |  288 +-
src/backend/common.mk|2 +
src/backend/nodes/copyfuncs.c|   13 +
src/backend/nodes/equalfuncs.c   |   11 +
src/backend/parser/gram.y|   18 +-
src/backend/regex/regc_pg_locale.c   |  110 ++-
src/backend/tcop/utility.c   |8 +
src/backend/utils/adt/formatting.c   |  453 +
src/backend/utils/adt/like.c |   53 +-
src/backend/utils/adt/pg_locale.c|  266 -
src/backend/utils/adt/selfuncs.c |8 +-
src/backend/utils/adt/varlena.c  |  179 +++-
src/backend/utils/mb/encnames.c  |   76 ++
src/bin/initdb/initdb.c  |3 +-
src/bin/pg_dump/pg_dump.c|   75 +-
src/bin/pg_dump/t/002_pg_dump.pl |2 +-
src/bin/psql/describe.c  |7 +-
src/include/catalog/pg_collation.h   |   25 +-
src/include/catalog/pg_collation_fn.h|2 +
src/include/catalog/pg_proc.h|3 +
src/include/commands/collationcmds.h |1 +
src/include/mb/pg_wchar.h|6 +
src/include/nodes/nodes.h|1 +
src/include/nodes/parsenodes.h   |   11 +
src/include/pg_config.h.in   |6 +
src/include/utils/pg_locale.h|   32 +-
src/test/regress/GNUmakefile |3 +
src/test/regress/expected/collate.icu.out| 1126 ++
src/test/regress/expected/collate.linux.utf8.out |   80 +-
src/test/regress/sql/collate.icu.sql |  433 +
src/test/regress/sql/collate.linux.utf8.sql  |   32 +
45 files changed, 3929 insertions(+), 409 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Track the oldest XID that can be safely looked up in CLOG.

2017-03-23 Thread Robert Haas

Track the oldest XID that can be safely looked up in CLOG.

This provides infrastructure for looking up arbitrary, user-supplied
XIDs without a risk of scary-looking failures from within the clog
module.  Normally, the oldest XID that can be safely looked up in CLOG
is the same as the oldest XID that can reused without causing
wraparound, and the latter is already tracked.  However, while
truncation is in progress, the values are different, so we must
keep track of them separately.

Craig Ringer, reviewed by Simon Riggs and by me.

Discussion: 
http://postgr.es/m/camsr+yhqiwnei0dactbos40t+v5s_+dst3pyv_8v2wnvh+x...@mail.gmail.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/ea42cc18c35381f639d45628d792e790ff39e271

Modified Files
--
doc/src/sgml/monitoring.sgml |  4 +++
src/backend/access/rmgrdesc/clogdesc.c   | 12 +++--
src/backend/access/transam/clog.c| 46 +---
src/backend/access/transam/transam.c |  4 +--
src/backend/access/transam/varsup.c  | 23 +++-
src/backend/access/transam/xlog.c| 11 
src/backend/commands/vacuum.c|  2 +-
src/backend/storage/lmgr/lwlocknames.txt |  1 +
src/include/access/clog.h|  8 +-
src/include/access/transam.h |  7 +
src/include/access/xlog_internal.h   |  2 +-
11 files changed, 102 insertions(+), 18 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Remove createlang and droplang

2017-03-23 Thread Peter Eisentraut

Remove createlang and droplang

They have been deprecated since PostgreSQL 9.1.

Reviewed-by: Magnus Hagander 
Reviewed-by: Daniel Gustafsson 

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/50c956add83963d7bbb367dd0b879fccddebd623

Modified Files
--
doc/src/sgml/installation.sgml|  12 +-
doc/src/sgml/plperl.sgml  |   3 +-
doc/src/sgml/plpython.sgml|   3 +-
doc/src/sgml/pltcl.sgml   |   7 +-
doc/src/sgml/ref/allfiles.sgml|   2 -
doc/src/sgml/ref/create_function.sgml |   1 -
doc/src/sgml/ref/create_language.sgml |  15 +-
doc/src/sgml/ref/createlang.sgml  | 291 --
doc/src/sgml/ref/drop_language.sgml   |   1 -
doc/src/sgml/ref/droplang.sgml| 288 -
doc/src/sgml/reference.sgml   |   2 -
doc/src/sgml/release-9.1.sgml |   4 +-
doc/src/sgml/xplang.sgml  |   9 +-
src/bin/scripts/.gitignore|   2 -
src/bin/scripts/Makefile  |   6 +-
src/bin/scripts/createlang.c  | 251 -
src/bin/scripts/droplang.c| 250 -
src/bin/scripts/nls.mk|   4 +-
src/bin/scripts/t/030_createlang.pl   |  25 ---
src/bin/scripts/t/060_droplang.pl |  23 ---
src/tools/msvc/Install.pm |   4 +-
21 files changed, 21 insertions(+), 1182 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Allow for parallel execution whenever ExecutorRun() is done only

2017-03-23 Thread Robert Haas

Allow for parallel execution whenever ExecutorRun() is done only once.

Previously, it was unsafe to execute a plan in parallel if
ExecutorRun() might be called with a non-zero row count.  However,
it's quite easy to fix things up so that we can support that case,
provided that it is known that we will never call ExecutorRun() a
second time for the same QueryDesc.  Add infrastructure to signal
this, and cross-checks to make sure that a caller who claims this is
true doesn't later reneg.

While that pattern never happens with queries received directly from a
client -- there's no way to know whether multiple Execute messages
will be sent unless the first one requests all the rows -- it's pretty
common for queries originating from procedural languages, which often
limit the result to a single tuple or to a user-specified number of
tuples.

This commit doesn't actually enable parallelism in any additional
cases, because currently none of the places that would be able to
benefit from this infrastructure pass CURSOR_OPT_PARALLEL_OK in the
first place, but it makes it much more palatable to pass
CURSOR_OPT_PARALLEL_OK in places where we currently don't, because it
eliminates some cases where we'd end up having to run the parallel
plan serially.

Patch by me, based on some ideas from Rafia Sabih and corrected by
Rafia Sabih based on feedback from Dilip Kumar and myself.

Discussion: 
http://postgr.es/m/ca+tgmobxehvhbjtwdupzm9bvslitj-kshxqj2um5gpdze9f...@mail.gmail.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/691b8d59281b5177f16fe80858df921f77a8e955

Modified Files
--
contrib/auto_explain/auto_explain.c |  9 +++---
contrib/pg_stat_statements/pg_stat_statements.c |  9 +++---
src/backend/commands/copy.c |  2 +-
src/backend/commands/createas.c |  2 +-
src/backend/commands/explain.c  |  2 +-
src/backend/commands/extension.c|  2 +-
src/backend/commands/matview.c  |  2 +-
src/backend/commands/portalcmds.c   |  2 +-
src/backend/commands/prepare.c  |  2 +-
src/backend/executor/execMain.c | 38 +
src/backend/executor/execParallel.c |  2 +-
src/backend/executor/functions.c|  2 +-
src/backend/executor/spi.c  |  2 +-
src/backend/tcop/postgres.c |  2 ++
src/backend/tcop/pquery.c   | 20 ++---
src/include/executor/execdesc.h |  3 ++
src/include/executor/executor.h |  7 +++--
src/include/tcop/pquery.h   |  2 +-
src/include/utils/portal.h  |  1 +
19 files changed, 73 insertions(+), 38 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Reduce page locking in GIN vacuum

2017-03-23 Thread Teodor Sigaev

Reduce page locking in GIN vacuum

GIN vacuum during cleaning posting tree can lock this whole tree for a long
time with by holding LockBufferForCleanup() on root. Patch changes it with
two ways: first, cleanup lock will be taken only if there is an empty page
(which should be deleted) and, second, it tries to lock only subtree, not the
whole posting tree.

Author: Andrey Borodin with minor editorization by me
Reviewed-by: Jeff Davis, me

https://commitfest.postgresql.org/13/896/

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/218f51584d5a9fcdf702bcc7f54b5b65e255c187

Modified Files
--
src/backend/access/gin/README  |  15 ++-
src/backend/access/gin/ginbtree.c  |   2 +-
src/backend/access/gin/ginvacuum.c | 236 -
src/include/access/gin_private.h   |   2 +
4 files changed, 145 insertions(+), 110 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Peter Eisentraut

On 3/23/17 10:25, Petr Jelinek wrote:
> On 23/03/17 15:17, Stephen Frost wrote:
>> Petr,
>>
>> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>>> On 23/03/17 14:47, Petr Jelinek wrote:
 I am looking into buildfarm failures.

 Looks like the clang issue (mylodon, longfin) is because we are missing
 extern in include/replication/worker_internal.h for ApplyCacheContext.

 Still analyzing the rest.
>>>
>>> Ah missing fields in copy/equal function for AlterSubscriptionStmt.
>>>
>>> Attached patch adds the above mentioned extern and fixes the copy and
>>> equal functions.
>>
>> Pretty sure I saw a complaint about a trailing comma in a struct or enum
>> too..
>>
> 
> Ah, missed that in the wall of text, thanks, fixed that too.

Also fixed.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Remove trailing comma from enum definition

2017-03-23 Thread Peter Eisentraut

Remove trailing comma from enum definition

Author: Petr Jelinek 

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/73561013e5aa44a1669b2cb4351f5e2b29485efb

Modified Files
--
src/include/replication/walreceiver.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Peter Eisentraut

On 3/23/17 10:12, Petr Jelinek wrote:
> On 23/03/17 14:47, Petr Jelinek wrote:
>> I am looking into buildfarm failures.
>>
>> Looks like the clang issue (mylodon, longfin) is because we are missing
>> extern in include/replication/worker_internal.h for ApplyCacheContext.
>>
>> Still analyzing the rest.
>>
> Ah missing fields in copy/equal function for AlterSubscriptionStmt.
> 
> Attached patch adds the above mentioned extern and fixes the copy and
> equal functions.

Fixed.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Assorted compilation and test fixes

2017-03-23 Thread Peter Eisentraut

Assorted compilation and test fixes

related to 7c4f52409a8c7d85ed169bbbc1f6092274d03920, per build farm

Author: Petr Jelinek 

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/128e6ee01dc9a8b7b0d3d3c8edc594ca9e51f993

Modified Files
--
src/backend/nodes/copyfuncs.c | 3 +++
src/backend/nodes/equalfuncs.c| 3 +++
src/include/replication/worker_internal.h | 2 +-
3 files changed, 7 insertions(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Minor spelling correction in comment

2017-03-23 Thread Simon Riggs

Minor spelling correction in comment
Jon Nelson

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/232c532213446701583e015ca55b5afb9c291445

Modified Files
--
src/backend/utils/adt/network.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Petr Jelinek

On 23/03/17 15:17, Stephen Frost wrote:
> Petr,
> 
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> On 23/03/17 14:47, Petr Jelinek wrote:
>>> I am looking into buildfarm failures.
>>>
>>> Looks like the clang issue (mylodon, longfin) is because we are missing
>>> extern in include/replication/worker_internal.h for ApplyCacheContext.
>>>
>>> Still analyzing the rest.
>>
>> Ah missing fields in copy/equal function for AlterSubscriptionStmt.
>>
>> Attached patch adds the above mentioned extern and fixes the copy and
>> equal functions.
> 
> Pretty sure I saw a complaint about a trailing comma in a struct or enum
> too..
> 

Ah, missed that in the wall of text, thanks, fixed that too.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 67c7de6..93d4eb2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4450,7 +4450,10 @@ _copyAlterSubscriptionStmt(const AlterSubscriptionStmt *from)
 {
 	AlterSubscriptionStmt *newnode = makeNode(AlterSubscriptionStmt);
 
+	COPY_SCALAR_FIELD(kind);
 	COPY_STRING_FIELD(subname);
+	COPY_STRING_FIELD(conninfo);
+	COPY_NODE_FIELD(publication);
 	COPY_NODE_FIELD(options);
 
 	return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4d8e7fe..6b40b56 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2199,7 +2199,10 @@ static bool
 _equalAlterSubscriptionStmt(const AlterSubscriptionStmt *a,
 			const AlterSubscriptionStmt *b)
 {
+	COMPARE_SCALAR_FIELD(kind);
 	COMPARE_STRING_FIELD(subname);
+	COMPARE_STRING_FIELD(conninfo);
+	COMPARE_NODE_FIELD(publication);
 	COMPARE_NODE_FIELD(options);
 
 	return true;
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index fb55c30..cc31bc6 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -175,7 +175,7 @@ typedef enum
 	WALRCV_OK_TUPLES,			/* Query returned tuples. */
 	WALRCV_OK_COPY_IN,			/* Query started COPY FROM. */
 	WALRCV_OK_COPY_OUT,			/* Query started COPY TO. */
-	WALRCV_OK_COPY_BOTH,		/* Query started COPY BOTH replication protocol. */
+	WALRCV_OK_COPY_BOTH			/* Query started COPY BOTH replication protocol. */
 } WalRcvExecStatus;
 
 /*
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index bf96d34..5bebca3 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -46,7 +46,7 @@ typedef struct LogicalRepWorker
 } LogicalRepWorker;
 
 /* Memory context for cached variables in apply worker. */
-MemoryContext			ApplyCacheContext;
+extern MemoryContextApplyCacheContext;
 
 /* libpqreceiver connection */
 extern struct WalReceiverConn	   *wrconn;

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Stephen Frost

Petr,

* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> On 23/03/17 14:47, Petr Jelinek wrote:
> > I am looking into buildfarm failures.
> > 
> > Looks like the clang issue (mylodon, longfin) is because we are missing
> > extern in include/replication/worker_internal.h for ApplyCacheContext.
> > 
> > Still analyzing the rest.
> 
> Ah missing fields in copy/equal function for AlterSubscriptionStmt.
> 
> Attached patch adds the above mentioned extern and fixes the copy and
> equal functions.

Pretty sure I saw a complaint about a trailing comma in a struct or enum
too..

Thanks!

Stephen


signature.asc
Description: Digital signature

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Petr Jelinek

On 23/03/17 14:47, Petr Jelinek wrote:
> I am looking into buildfarm failures.
> 
> Looks like the clang issue (mylodon, longfin) is because we are missing
> extern in include/replication/worker_internal.h for ApplyCacheContext.
> 
> Still analyzing the rest.
> 

Ah missing fields in copy/equal function for AlterSubscriptionStmt.

Attached patch adds the above mentioned extern and fixes the copy and
equal functions.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 67c7de6..93d4eb2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4450,7 +4450,10 @@ _copyAlterSubscriptionStmt(const AlterSubscriptionStmt *from)
 {
 	AlterSubscriptionStmt *newnode = makeNode(AlterSubscriptionStmt);
 
+	COPY_SCALAR_FIELD(kind);
 	COPY_STRING_FIELD(subname);
+	COPY_STRING_FIELD(conninfo);
+	COPY_NODE_FIELD(publication);
 	COPY_NODE_FIELD(options);
 
 	return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4d8e7fe..6b40b56 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2199,7 +2199,10 @@ static bool
 _equalAlterSubscriptionStmt(const AlterSubscriptionStmt *a,
 			const AlterSubscriptionStmt *b)
 {
+	COMPARE_SCALAR_FIELD(kind);
 	COMPARE_STRING_FIELD(subname);
+	COMPARE_STRING_FIELD(conninfo);
+	COMPARE_NODE_FIELD(publication);
 	COMPARE_NODE_FIELD(options);
 
 	return true;
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index bf96d34..5bebca3 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -46,7 +46,7 @@ typedef struct LogicalRepWorker
 } LogicalRepWorker;
 
 /* Memory context for cached variables in apply worker. */
-MemoryContext			ApplyCacheContext;
+extern MemoryContextApplyCacheContext;
 
 /* libpqreceiver connection */
 extern struct WalReceiverConn	   *wrconn;

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-03-23 Thread Simon Riggs

Replication lag tracking for walsenders

Adds write_lag, flush_lag and replay_lag cols to pg_stat_replication.

Implements a lag tracker module that reports the lag times based upon
measurements of the time taken for recent WAL to be written, flushed and
replayed and for the sender to hear about it. These times
represent the commit lag that was (or would have been) introduced by each
synchronous commit level, if the remote server was configured as a
synchronous standby.  For an asynchronous standby, the replay_lag column
approximates the delay before recent transactions became visible to queries.
If the standby server has entirely caught up with the sending server and
there is no more WAL activity, the most recently measured lag times will
continue to be displayed for a short time and then show NULL.

Physical replication lag tracking is automatic. Logical replication tracking
is possible but is the responsibility of the logical decoding plugin.
Tracking is a private module operating within each walsender individually,
with values reported to shared memory. Module not used outside of walsender.

Design and code is good enough now to commit - kudos to the author.
In many ways a difficult topic, with important and subtle behaviour so this
shoudl be expected to generate discussion and multiple open items: Test now!

Author: Thomas Munro, following designs by Fujii Masao and Simon Riggs
Review: Simon Riggs, Ian Barwick and Craig Ringer

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/6912acc04f0bbcfdb799a120618507601e862490

Modified Files
--
doc/src/sgml/monitoring.sgml|  69 +++
src/backend/access/transam/xlog.c   |  14 ++
src/backend/catalog/system_views.sql|   3 +
src/backend/replication/walsender.c | 277 +++-
src/include/catalog/pg_proc.h   |   2 +-
src/include/replication/logical.h   |   2 +
src/include/replication/walsender_private.h |   5 +
src/test/regress/expected/rules.out |   5 +-
8 files changed, 370 insertions(+), 7 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Petr Jelinek

I am looking into buildfarm failures.

Looks like the clang issue (mylodon, longfin) is because we are missing
extern in include/replication/worker_internal.h for ApplyCacheContext.

Still analyzing the rest.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Logical replication support for initial data copy

2017-03-23 Thread Peter Eisentraut

Logical replication support for initial data copy

Add functionality for a new subscription to copy the initial data in the
tables and then sync with the ongoing apply process.

For the copying, add a new internal COPY option to have the COPY source
data provided by a callback function.  The initial data copy works on
the subscriber by receiving COPY data from the publisher and then
providing it locally into a COPY that writes to the destination table.

A WAL receiver can now execute full SQL commands.  This is used here to
obtain information about tables and publications.

Several new options were added to CREATE and ALTER SUBSCRIPTION to
control whether and when initial table syncing happens.

Change pg_dump option --no-create-subscription-slots to
--no-subscription-connect and use the new CREATE SUBSCRIPTION
... NOCONNECT option for that.

Author: Petr Jelinek 
Tested-by: Erik Rijkers 

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/7c4f52409a8c7d85ed169bbbc1f6092274d03920

Modified Files
--
contrib/file_fdw/file_fdw.c|   5 +-
doc/src/sgml/catalogs.sgml |  78 ++
doc/src/sgml/config.sgml   |  25 +
doc/src/sgml/logical-replication.sgml  |  55 +-
doc/src/sgml/monitoring.sgml   |   9 +-
doc/src/sgml/protocol.sgml |   9 +-
doc/src/sgml/ref/alter_subscription.sgml   |  50 +-
doc/src/sgml/ref/create_subscription.sgml  |  38 +
doc/src/sgml/ref/pg_dump.sgml  |  15 +-
src/backend/catalog/Makefile   |   1 +
src/backend/catalog/heap.c |   6 +
src/backend/catalog/pg_publication.c   |   4 +-
src/backend/catalog/pg_subscription.c  | 282 +++
src/backend/catalog/system_views.sql   |   1 +
src/backend/commands/copy.c|  23 +-
src/backend/commands/subscriptioncmds.c| 468 ++--
src/backend/parser/gram.y  |  39 +-
src/backend/postmaster/pgstat.c|   6 +
.../libpqwalreceiver/libpqwalreceiver.c| 203 -
src/backend/replication/logical/Makefile   |   2 +-
src/backend/replication/logical/launcher.c | 130 +++-
src/backend/replication/logical/relation.c |   7 +
src/backend/replication/logical/snapbuild.c|  85 ++-
src/backend/replication/logical/tablesync.c| 840 +
src/backend/replication/logical/worker.c   | 203 +++--
src/backend/replication/repl_gram.y|  32 +-
src/backend/replication/repl_scanner.l |   5 +-
src/backend/replication/walsender.c| 104 ++-
src/backend/tcop/postgres.c|   5 +-
src/backend/utils/adt/misc.c   |  20 +
src/backend/utils/cache/syscache.c |  14 +-
src/backend/utils/misc/guc.c   |  12 +
src/bin/pg_dump/pg_backup.h|   2 +-
src/bin/pg_dump/pg_dump.c  |   9 +-
src/bin/pg_dump/t/002_pg_dump.pl   |   2 +-
src/include/catalog/catversion.h   |   2 +-
src/include/catalog/indexing.h |   7 +-
src/include/catalog/pg_proc.h  |   5 +-
src/include/catalog/pg_subscription_rel.h  |  78 ++
src/include/commands/copy.h|   5 +-
src/include/nodes/nodes.h  |   1 +
src/include/nodes/parsenodes.h |  13 +
src/include/nodes/replnodes.h  |   9 +
src/include/parser/kwlist.h|   1 +
src/include/pgstat.h   |   4 +-
src/include/replication/logical.h  |  13 +-
src/include/replication/logicallauncher.h  |   1 +
src/include/replication/snapbuild.h|   1 +
src/include/replication/walreceiver.h  |  67 +-
src/include/replication/walsender.h|  12 +-
src/include/replication/worker_internal.h  |  30 +-
src/include/utils/syscache.h   |   1 +
src/test/regress/expected/object_address.out   |   3 +-
src/test/regress/expected/rules.out|   3 +-
src/test/regress/expected/sanity_check.out |   1 +
src/test/regress/expected/subscription.out |  45 +-
src/test/regress/sql/object_address.sql|   2 +-
src/test/regress/sql/subscription.sql  |  11 +-
src/test/subscription/t/001_rep_changes.pl |  36 +-
src/test/subscription/t/002_types.pl   |   6 +
src/test/subscription/t/003_constraints.pl |   2 +-
src/test/subscription/t/004_sync.pl| 159 
62 files changed, 2966 insertions(+), 341 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes

[COMMITTERS] pgsql: Fix grammar in comment

2017-03-23 Thread Magnus Hagander

Fix grammar in comment

Author: Emil Iggland

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/707576b571f05ec5b89adb65964d55f3bd1b

Modified Files
--
src/backend/commands/publicationcmds.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: ICU support

[COMMITTERS] pgsql: Avoid syntax error on platforms that have neither LOCALE_T nor I

Re: [COMMITTERS] pgsql: ICU support

[COMMITTERS] pgsql: doc: add missing closing 'sect3' tag for ICU patch

[COMMITTERS] pgsql: Add ICU_FLAGS to one more place

[COMMITTERS] pgsql: Fix crash in ICU patch

Re: [COMMITTERS] pgsql: ICU support

[COMMITTERS] pgsql: Fix enum definition.

Re: [COMMITTERS] pgsql: ICU support

[COMMITTERS] pgsql: ICU support

[COMMITTERS] pgsql: Track the oldest XID that can be safely looked up in CLOG.

[COMMITTERS] pgsql: Remove createlang and droplang

[COMMITTERS] pgsql: Allow for parallel execution whenever ExecutorRun() is done only

[COMMITTERS] pgsql: Reduce page locking in GIN vacuum

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

[COMMITTERS] pgsql: Remove trailing comma from enum definition

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

[COMMITTERS] pgsql: Assorted compilation and test fixes

[COMMITTERS] pgsql: Minor spelling correction in comment

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

[COMMITTERS] pgsql: Replication lag tracking for walsenders

Re: [COMMITTERS] pgsql: Logical replication support for initial data copy

[COMMITTERS] pgsql: Logical replication support for initial data copy

[COMMITTERS] pgsql: Fix grammar in comment

26 matches

Site Navigation

Mail list logo

Footer information