[HACKERS] Japanies translation breaks solaris build

[Zdenek Kotala]

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_moth&dt=2010-05-13%2021:06:01


msgfmt -o po/ja.mo po/ja.po
WARNING: the string after closing " is ignored at line number 11.
Error, No space after directive at line number 2008.
ERROR: Exiting...
gmake[2]: *** [po/ja.mo] Error 2


The problem is that it contains mix of DOS/Unix end of lines.


Zdenek


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Magnus Hagander]

On Fri, May 14, 2010 at 5:34 AM, Bruce Momjian  wrote:
> Takahiro Itagaki wrote:
>>
>> Bruce Momjian  wrote:
>>
>> > > >>  2. extern PGDLLIMPORT 
>> > > >> pg_upgrade has own definitions of
>> > > >> extern PGDLLIMPORT Oid binary_upgrade_next_xxx
>> > >
>> > > > The issue here is that you use PGDLLIMPORT where you are importing the
>> > > > variable, not where it is defined.  For example, look at
>> > > > 'seq_page_cost'.  You can see PGDLLIMPORT used where it is imported 
>> > > > with
>> > > > 'extern', but not where is it defined.
>> > >
>> > > Right.  Also we are intentionally not exposing those variables in any
>> > > backend .h file, because they are not meant for general use.  So the
>> > > "extern PGDLLIMPORT" isn't going to be in the main backend and has to
>> > > be in pg_upgrade.  This was discussed awhile ago when we put in those
>> > > variables, I believe.
>> >
>> > Yes, this was discussed.
>>
>> I wonder some compilers or linkers might hide unexported global variables
>> from postgres.lib as if they are declared with 'static' specifiers.
>> I'm especially worried about Windows and MSVC. So, if Windows testers
>> can see it works, there was nothing to worry about.
>
> Yes, none of the variables pg_upgrade is referencing are 'static', and
> Magnus tested MSVC and checked MinGW compiles.

Just to be clear, I only verified that it *built*, didn't have time to
check if it actually *worked*.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Devrim GÜNDÜZ]

On Thu, 2010-05-13 at 17:19 +0200, Magnus Hagander wrote:
> I say remove it. On all accounts.
> 
> There's a fork of postgres for EDB AS, shouldn't there be a fork of
> pg_upgrade the same way, if it requires special code? The code in
> community postgresql certainly shouldn't have any EDB AS code in it.

Agreed.
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz


signature.asc
Description: This is a digitally signed message part

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Anssi Kääriäinen]

On 05/14/2010 03:37 AM, Greg Stark wrote:
> On Thu, May 13, 2010 at 10:25 PM, Florian Pflug  wrote:
>> C1: BEGIN
>> C1: SELECT * FROM t WHERE id = 1 FOR UPDATE
>> C2: BEGIN
>> C2: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
>> C2: SELECT * FROM t -- Take snapshot before C1 commits
>> C1: COMMIT
>> C2: DELETE FROM t WHERE id = 1
>> C2: COMMIT
>>
>
> Can you give an actual realistic example -- ie, not doing a select for
> update and then never updating the row or with an explanation of what
> the programmer is attempting to accomplish with such an unusual
> sequence? The rest of the post talks about FKs but I don't see any
> here...
>

Doing a select for update and then never updating the row is a realistic 
example.


I am currently designing a database where this is an issue. The 
simplified schema to illustrate the problem:


create table object (
   id integer primary key
);

insert into object values(1);

create table attribute (
   object_id integer not null references object,
   attr_type integer not null, -- references attr_types
   value text not null,
   valid_from timestamp not null,
   valid_until timestamp
);

Now, I want to make sure there are no pairs of (object_id, attr_type) 
where the valid_from, valid_until times overlap.


A problematic sequence for this schema, both transactions in isolation 
level serializable:



C1: begin;
C1: select * from object where id = 1 for update;
-- check for conflicting attr_type, realistically where condition should 
have overlapping check, but left out for simplicity...

C1: select * from attribute where object_id = 1 and attr_type = 1;
-- Ok, nothing overlapping, I am able to insert.
C1: insert into attribute values (1, 1, 'Anssi', now(), null);
C2: begin;
-- This blocks.
C2: select * from object where id = 1 for update;
C1: commit;
-- Check for conflicts. This select won't see the insert C1 did.
C2: select * from attribute where object_id = 1 and attr_type = 1;
-- C2 doesn't see anything conflicting
C2: insert into attribute values (1, 1, 'Matti', now(), null);
C2: commit;
-- Inconsistency.

Now, that same sequence does work for read committed isolation level (C2 
sees the insert of C1), and that is my solution for now: require 
applications to use read committed isolation level. This could also be 
solved by issuing "update object set id = id where id = 1" instead of 
using select for update. This would result in serialization error.


I know that for this particular example the upcoming exclusion 
constraints would solve the problem. But if I would want to ensure that 
if attr_value for attr_type 1 is 'Anssi' then attr_value for attr_type 2 
is 'Kääriäinen', then exclusion constraints could not be used.


--
Anssi Kääriäinen

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] quoting and recovery.conf

[Simon Riggs]

On Thu, 2010-05-13 at 21:15 -0400, Tom Lane wrote:
> Greg Stark  writes:
> > On Thu, May 13, 2010 at 11:33 PM, Robert Haas  wrote:
> >> I think we should add a TODO to parse recovery.conf with the same code
> >> we use to parse postgresql.conf, or possibly merge the two files.
> >> This issue was previously alluded to here:
> >> 
> >> http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php
> 
> > And more than alluded to here:
> 
> > http://archives.postgresql.org/message-id/407d949e1002131017u657e4aefo2647c2cbf24fe...@mail.gmail.com
> 
> The main reason for having a separate recovery.conf file is that its
> existence is what drives the setting of InArchiveRecovery.  If we were
> to devise some other trigger for that condition, it'd be possible to
> fold all those settings in as GUC variables.

And the removal of recovery.conf at end of recovery prevents the
re-entry into archive recovery if we crash.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

On Thu, 13 May 2010, Alvaro Herrera wrote:


If most of the questions are badly categorized or cross posted to more
than one list, how useful a label is the X-Mailing-List header?  How
useful is to filter on the "pgsql-general" label?


That is a point, but, IMHO, that is one of our key issues ... we *allow* 
that sort of cross-posting in the first place ... FreeBSD lists allow 
cross-posting to no more then 2 mailing lists, I believe, but there is 
definitely a limit ...


... is there a reason why, other the fact that we don't do now, that we 
can't just put in a restriction against cross posting altogether?



... and, for those that have been here awhile, who "should know better", 
why isn't there any self-management of this sort of stuff in the first 
place?



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] nvarchar notation accepted?

[Peter Eisentraut]

On tor, 2010-05-13 at 23:52 -0400, Tom Lane wrote:
> Takahiro Itagaki  writes:
> > Jaime Casanova  wrote:
> >> i migrate a ms sql server database to postgres and was trying some
> >> queries from the application to find if everything works right...
> >> when i was looking to those queries i found some that has a notation
> >> for nvarchar (ej: campo = N'sometext')
> 
> > Do you have documentation for N'...' literal in SQLServer?
> > Does it mean unicode literal? What is the difference from U& literal?
> > http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html
> 
> > PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only
> > have mutlti-tyte characters. So I think you can remove N and just
> > use "SET client_encoding = UTF8" in the cases.
> 
> Actually, the lexer translates N'foo' to NCHAR 'foo' and then the
> grammar treats that just like CHAR 'foo'.  In short, the N doesn't do
> anything very useful, and it certainly doesn't have any effect on
> encoding behavior.  I think this is something Tom Lockhart put in ten or
> so years back, and never got as far as making it actually do anything
> helpful.

This should maybe changed to just ignoring the N and treating N'' like
''.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Generalized Inverted Generalized Search Tree

[Takahiro Itagaki]

We can index multiple scalar values per row with GIN access method,
and also can index single vector value per row with GiST AM.

Is it worth having a new AM to index multiple vector values per row?
It will be an AM for the missing feature in below:

| scalar | vector |
+++
 single per row | btree  | gist   |
 multi per row  | gin| *HERE* |

We can call the new AM "gigist". Or, there might be another idea
to support expression indexes for SRF functions, like
  =# CREATE TABLE tbl (c circle[]);
  =# CREATE INDEX ON tbl USING gist (unnest(c));

Comments and ideas welcome.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] nvarchar notation accepted?

[Jaime Casanova]

On Thu, May 13, 2010 at 10:13 PM, Takahiro Itagaki
 wrote:
>
> Jaime Casanova  wrote:
>
>> i migrate a ms sql server database to postgres and was trying some
>> queries from the application to find if everything works right...
>> when i was looking to those queries i found some that has a notation
>> for nvarchar (ej: campo = N'sometext')
>
> Do you have documentation for N'...' literal in SQLServer?
> Does it mean unicode literal? What is the difference from U& literal?
> http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html
>

nop, only thing i found is about NVARCHAR:
http://msdn.microsoft.com/en-us/library/ms186939.aspx but it has no
examples about the N'' notation although you can find examples of it
use here: http://msdn.microsoft.com/en-us/library/dd776381.aspx#BasicSyntax

> PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only
> have mutlti-tyte characters. So I think you can remove N and just
> use "SET client_encoding = UTF8" in the cases.
>

i don't want to remove it! i'm trying to understand if this is a bug
that will be removed if no i can safely tell my client to not look for
 those queries so it has less work to do for the migration

-- 
Jaime Casanova www.2ndQuadrant.com
Soporte y capacitación de PostgreSQL

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] nvarchar notation accepted?

[Tom Lane]

Jaime Casanova  writes:
> On Thu, May 13, 2010 at 10:52 PM, Tom Lane  wrote:
>> Actually, the lexer translates N'foo' to NCHAR 'foo' and then the
>> grammar treats that just like CHAR 'foo'.  In short, the N doesn't do
>> anything very useful, and it certainly doesn't have any effect on
>> encoding behavior.  I think this is something Tom Lockhart put in ten or
>> so years back, and never got as far as making it actually do anything
>> helpful.

> so, the N'' syntax is fine and i don't need to hunt them as a migration step?

As long as the implied cast to char(n) doesn't cause you problems, it's
fine.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] nvarchar notation accepted?

[Jaime Casanova]

On Thu, May 13, 2010 at 10:52 PM, Tom Lane  wrote:
>
> Actually, the lexer translates N'foo' to NCHAR 'foo' and then the
> grammar treats that just like CHAR 'foo'.  In short, the N doesn't do
> anything very useful, and it certainly doesn't have any effect on
> encoding behavior.  I think this is something Tom Lockhart put in ten or
> so years back, and never got as far as making it actually do anything
> helpful.
>

so, the N'' syntax is fine and i don't need to hunt them as a migration step?

-- 
Jaime Casanova www.2ndQuadrant.com
Soporte y capacitación de PostgreSQL

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] nvarchar notation accepted?

[Tom Lane]

Takahiro Itagaki  writes:
> Jaime Casanova  wrote:
>> i migrate a ms sql server database to postgres and was trying some
>> queries from the application to find if everything works right...
>> when i was looking to those queries i found some that has a notation
>> for nvarchar (ej: campo = N'sometext')

> Do you have documentation for N'...' literal in SQLServer?
> Does it mean unicode literal? What is the difference from U& literal?
> http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html

> PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only
> have mutlti-tyte characters. So I think you can remove N and just
> use "SET client_encoding = UTF8" in the cases.

Actually, the lexer translates N'foo' to NCHAR 'foo' and then the
grammar treats that just like CHAR 'foo'.  In short, the N doesn't do
anything very useful, and it certainly doesn't have any effect on
encoding behavior.  I think this is something Tom Lockhart put in ten or
so years back, and never got as far as making it actually do anything
helpful.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Alvaro Herrera]

Excerpts from Marc G. Fournier's message of jue may 13 23:11:40 -0400 2010:
> On Thu, 13 May 2010, Joshua D. Drake wrote:
> 
> > Between labels, filters, watch lists and all the other goodies any MUA 
> > will give you, I see no reason to have this all broken out anymore.
> 
> So, if one merges all the lists into one (not arguing for / against that), 
> how do you filter?  Based on what?  Right now, ppl filter based on the 
> X-Mailing-List header, or just the Participant ...

If most of the questions are badly categorized or cross posted to more
than one list, how useful a label is the X-Mailing-List header?  How
useful is to filter on the "pgsql-general" label?
-- 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Bruce Momjian]

Takahiro Itagaki wrote:
> 
> Bruce Momjian  wrote:
> 
> > > >>  2. extern PGDLLIMPORT 
> > > >> pg_upgrade has own definitions of
> > > >> extern PGDLLIMPORT Oid binary_upgrade_next_xxx
> > > 
> > > > The issue here is that you use PGDLLIMPORT where you are importing the
> > > > variable, not where it is defined.  For example, look at
> > > > 'seq_page_cost'.  You can see PGDLLIMPORT used where it is imported with
> > > > 'extern', but not where is it defined.
> > > 
> > > Right.  Also we are intentionally not exposing those variables in any
> > > backend .h file, because they are not meant for general use.  So the
> > > "extern PGDLLIMPORT" isn't going to be in the main backend and has to
> > > be in pg_upgrade.  This was discussed awhile ago when we put in those
> > > variables, I believe.
> > 
> > Yes, this was discussed.
> 
> I wonder some compilers or linkers might hide unexported global variables
> from postgres.lib as if they are declared with 'static' specifiers.
> I'm especially worried about Windows and MSVC. So, if Windows testers
> can see it works, there was nothing to worry about.

Yes, none of the variables pg_upgrade is referencing are 'static', and
Magnus tested MSVC and checked MinGW compiles.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] nvarchar notation accepted?

[Takahiro Itagaki]

Jaime Casanova  wrote:

> i migrate a ms sql server database to postgres and was trying some
> queries from the application to find if everything works right...
> when i was looking to those queries i found some that has a notation
> for nvarchar (ej: campo = N'sometext')

Do you have documentation for N'...' literal in SQLServer?
Does it mean unicode literal? What is the difference from U& literal?
http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html

PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only
have mutlti-tyte characters. So I think you can remove N and just
use "SET client_encoding = UTF8" in the cases.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

On Thu, 13 May 2010, Joshua D. Drake wrote:

Between labels, filters, watch lists and all the other goodies any MUA 
will give you, I see no reason to have this all broken out anymore.


So, if one merges all the lists into one (not arguing for / against that), 
how do you filter?  Based on what?  Right now, ppl filter based on the 
X-Mailing-List header, or just the Participant ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] wal_level in postgresql.conf

[Fujii Masao]

On Fri, May 14, 2010 at 12:22 AM, Bruce Momjian  wrote:
> Bruce Momjian wrote:
>> Why is 'wal_level' listed next to 'fsync' in postgresql.conf?
>>
>>       #wal_level = 'hot_standby'              # minimal, archive, or 
>> hot_standby
>>       #fsync = on                             # turns forced synchronization 
>> on or off
>>       #synchronous_commit = on                # immediate fsync at commit
>>       #wal_sync_method = fsync                # the default is the first 
>> option
>>
>> Seems it should be in the archiving section.

-1 since wal_level affects WAL logging itself, and is to do with
not only archiving but also hot standby.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] quoting and recovery.conf

[Fujii Masao]

On Fri, May 14, 2010 at 9:37 AM, Bruce Momjian  wrote:
> I thought the problem was just quotes inside strings, not the
> requirement of quotes for everything.

You can embed a single quote in a parameter value by writing two quotes.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] wal_level and continuous archiving documentation

[Fujii Masao]

On Fri, May 14, 2010 at 1:42 AM, Joshua Tolley  wrote:
> I was reading through
> http://www.postgresql.org/docs/9.0/static/continuous-archiving.html and
> noticed that wal_level isn't mentioned where I'd expect it to be.
> Specifically, there's a paragraph that starts, "To enable WAL archiving, set
> the archive_mode configuration parameter to on, and specify the shell command
> to use in the archive_command configuration parameter." There follows a long
> discussion of archive_command, but no further discussion of other
> configuration settings for several paragraphs, suggesting that those two
> configuration changes are the only ones required to end up with a useful
> archive. However, further on, it discusses wal_level:
>
>    "When wal_level is minimal some SQL commands are optimized to avoid WAL
>    logging, as described in Section 14.4.7. If archiving or streaming
>    replication were turned on during execution of one of these statements,
>    WAL would not contain enough information for archive recovery."
>
> ISTM wal_archive should make an appearance where the docs bring up
> archive_mode and archive_command, to say "wal_level must be set to 'archive'
> or 'hot_standby'", so all required configuration changes are mentioned close
> together.

Some descriptions about wal_level were added after beta1 release.
http://archives.postgresql.org/pgsql-committers/2010-05/msg00019.php

You can read the latest (development) document from
http://developer.postgresql.org/pgdocs/postgres/continuous-archiving.html

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Takahiro Itagaki]

Bruce Momjian  wrote:

> > >>  2. extern PGDLLIMPORT 
> > >> pg_upgrade has own definitions of
> > >> extern PGDLLIMPORT Oid binary_upgrade_next_xxx
> > 
> > > The issue here is that you use PGDLLIMPORT where you are importing the
> > > variable, not where it is defined.  For example, look at
> > > 'seq_page_cost'.  You can see PGDLLIMPORT used where it is imported with
> > > 'extern', but not where is it defined.
> > 
> > Right.  Also we are intentionally not exposing those variables in any
> > backend .h file, because they are not meant for general use.  So the
> > "extern PGDLLIMPORT" isn't going to be in the main backend and has to
> > be in pg_upgrade.  This was discussed awhile ago when we put in those
> > variables, I believe.
> 
> Yes, this was discussed.

I wonder some compilers or linkers might hide unexported global variables
from postgres.lib as if they are declared with 'static' specifiers.
I'm especially worried about Windows and MSVC. So, if Windows testers
can see it works, there was nothing to worry about.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] JSON manipulation functions

[Joseph Adams]

First off, thank you for allowing me to participate in Google Summer
of Code 2010.  I'm sorry I haven't been active for the past few weeks.
 Today, I added the wiki page for my project, but the project schedule
is highly tentative:
http://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010 .

I'd like to discuss how the functions for type checking and conversion
(the majority of my project) should work.  Below is my first draft for
the JSON manipulation function specs, along with annotations.

== Type checking ==

The following function returns the type of any JSON value.

json_type as enum ('null', 'string', 'number', 'boolean', 'object', 'array')
json_type(json) returns json_type

Would it be a bad idea to give an enum and a function the same name
(which appears to be allowed by PostgreSQL) ?  If so, json_type(json)
could be json_typeof(json) or something instead.

I thought about having predicates like IS STRING and IS NUMBER,
similar to the IS DOCUMENT predicate used for XML.  However, a major
problem with that approach is that it could lead to confusion
involving IS NULL.  By my understanding, the JSON datatype will just
be a specialization of TEXT (it just validates the input).  Like TEXT,
a JSON string can be 'null'.  'null'::JSON is not NULL.  Bear in mind
that json_to_*('null') is NULL, though.

I also thought about having a series of json_is_* functions.  I don't
think it's a bad idea, but I think json_type is a better solution.

== text/number/boolean conversion ==

These functions each convert a non-compound JSON value to its
respective return type.  Run-time type checking is performed; a
conversion will throw an error if the input JSON is not the correct
type.  If the JSON value is 'null', then the return value will be
NULL.

json_to_text(json) returns text
json_to_number(json) returns numeric
json_to_bool(json) returns boolean

These functions convert values to JSON.  Passing NULL to any of the
functions below will return 'null':

text_to_json(text) returns json
number_to_json(numeric) returns json
bool_to_json(boolean) returns json

There could be generic value_to_json(any), but not a
json_to_value(json) function.  See
http://archives.postgresql.org/pgsql-hackers/2010-04/msg00321.php for
more details.

Conversion to/from number or boolean can also be achieved with
casting.  Note well that '"string"'::JSON::TEXT is '"string"', not the
string's actual value.  json_to_text is needed for this conversion.
For this reason, casting JSON might seem like something to recommend
against.  However, IMHO, casting numbers and booleans to/from JSON is
fine and dandy; the paragraphs below give some weight to this.

I originally considered making json_to_number and number_to_json work
with TEXT instead of NUMERIC.  However, as Tom Lane pointed out in the
above link, "Forcing people to insert explicit coercions from text
isn't going to be particularly convenient to use.".  Nevertheless,
NUMERIC introduces a problem.  For instance, if you say:

SELECT '-1e-38'::NUMERIC;

This conversion knocks out the scientific notation and produces a
41-character string.  I seriously doubt that all outside applications
will handle 41-character numbers correctly.

Perhaps there should be individual functions for specific data types,
or maybe just a handful for particular cases.  There might be
json_to_int, json_to_float, and json_to_numeric.  In any case,
converting to/from number types can be achieved quite easily with
casting.

== array/object conversion ==

The json_object function converts a tuple to a JSON object.  If there
are duplicate column names, there will be duplicate keys in the
resulting JSON object.

json_object([content [AS name] [, ...]]) returns json

Likewise, the json_array function converts a tuple to a JSON array.
Column names are ignored.

json_array([content [AS name] [, ...]]) returns json

The json_agg function reduces a set of JSON values to a single array
containing those values.

aggregate json_agg(json) returns json

json_object and json_agg can be used together to convert an entire
result set to one JSON array:

SELECT json_agg(json_object(*)) FROM tablename;

json_keys gets the keys of a JSON object as a set.

json_keys(json) returns setof text

json_values gets the values of a JSON object or the iems of a JSON
array as a set.

json_values(json) returns setof json

Note that all JSON slicing and splicing operations retain the original
formatting of JSON content.

== Miscellaneous ==

The features below would be nice, but will probably not be regarded as
required for this Google Summer of Code project to be considered
complete.

json_cleanup accepts a superset of JSON and, if it can, cleans it up
and returns a valid JSON string.  This superset of JSON supports the
following extra features:

 * Comments:
   - Single-line comments with // and #
   - C-style comments: /* comment */
 * Unquoted object keys: {key: "value"}
 * Single quote strings: 'single quotes; "double quotes" do not need
to be escape

Re: [HACKERS] quoting and recovery.conf

[Tom Lane]

Greg Stark  writes:
> On Thu, May 13, 2010 at 11:33 PM, Robert Haas  wrote:
>> I think we should add a TODO to parse recovery.conf with the same code
>> we use to parse postgresql.conf, or possibly merge the two files.
>> This issue was previously alluded to here:
>> 
>> http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php

> And more than alluded to here:

> http://archives.postgresql.org/message-id/407d949e1002131017u657e4aefo2647c2cbf24fe...@mail.gmail.com

The main reason for having a separate recovery.conf file is that its
existence is what drives the setting of InArchiveRecovery.  If we were
to devise some other trigger for that condition, it'd be possible to
fold all those settings in as GUC variables.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Bruce Momjian]

Tom Lane wrote:
> Bruce Momjian  writes:
> > Takahiro Itagaki wrote:
> >>  2. extern PGDLLIMPORT 
> >> pg_upgrade has own definitions of
> >> extern PGDLLIMPORT Oid binary_upgrade_next_xxx
> >> in pg_upgrade_sysoids.c. But those variables are not declared as
> >> PGDLLIMPORT in the core. Can we access unexported variables here?
> 
> > The issue here is that you use PGDLLIMPORT where you are importing the
> > variable, not where it is defined.  For example, look at
> > 'seq_page_cost'.  You can see PGDLLIMPORT used where it is imported with
> > 'extern', but not where is it defined.
> 
> Right.  Also we are intentionally not exposing those variables in any
> backend .h file, because they are not meant for general use.  So the
> "extern PGDLLIMPORT" isn't going to be in the main backend and has to
> be in pg_upgrade.  This was discussed awhile ago when we put in those
> variables, I believe.

Yes, this was discussed.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] quoting and recovery.conf

[Greg Stark]

On Thu, May 13, 2010 at 11:33 PM, Robert Haas  wrote:
>
> I think we should add a TODO to parse recovery.conf with the same code
> we use to parse postgresql.conf, or possibly merge the two files.
> This issue was previously alluded to here:
>
> http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php

And more than alluded to here:

http://archives.postgresql.org/message-id/407d949e1002131017u657e4aefo2647c2cbf24fe...@mail.gmail.com


-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Tom Lane]

Bruce Momjian  writes:
> Takahiro Itagaki wrote:
>>  2. extern PGDLLIMPORT 
>> pg_upgrade has own definitions of
>> extern PGDLLIMPORT Oid binary_upgrade_next_xxx
>> in pg_upgrade_sysoids.c. But those variables are not declared as
>> PGDLLIMPORT in the core. Can we access unexported variables here?

> The issue here is that you use PGDLLIMPORT where you are importing the
> variable, not where it is defined.  For example, look at
> 'seq_page_cost'.  You can see PGDLLIMPORT used where it is imported with
> 'extern', but not where is it defined.

Right.  Also we are intentionally not exposing those variables in any
backend .h file, because they are not meant for general use.  So the
"extern PGDLLIMPORT" isn't going to be in the main backend and has to
be in pg_upgrade.  This was discussed awhile ago when we put in those
variables, I believe.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Greg Stark]

On Thu, May 13, 2010 at 10:25 PM, Florian Pflug  wrote:
> C1: BEGIN
> C1: SELECT * FROM t WHERE id = 1 FOR UPDATE
> C2: BEGIN
> C2: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
> C2: SELECT * FROM t -- Take snapshot before C1 commits
> C1: COMMIT
> C2: DELETE FROM t WHERE id = 1
> C2: COMMIT
>

Can you give an actual realistic example -- ie, not doing a select for
update and then never updating the row or with an explanation of what
the programmer is attempting to accomplish with such an unusual
sequence? The rest of the post talks about FKs but I don't see any
here...

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] quoting and recovery.conf

[Bruce Momjian]

Robert Haas wrote:
> On Thu, May 13, 2010 at 1:00 PM, Bruce Momjian  wrote:
> > Is there a reason we require single quotes around boolean values in
> > recovery.conf?
> >
> > ? ? ? ?standby_mode = 'off'
> >
> > This does not work:
> >
> > ? ? ? ?standby_mode = off
> >
> > I knew there were inconsistencies between quoting in postgresql.conf and
> > recovery.conf, but I didn't realize it extended to boolean quoting. ?I
> > see this at the top of recovery.conf now:
> >
> > ? ? ? ?# This file consists of lines of the form:
> > ? ? ? ?#
> > ? ? ? ?# ? name = 'value'
> > ? ? ? ?#
> > ? ? ? ?# (The quotes around the value are NOT optional, but the "=" is.)
> > ? ? ? ?#
> >
> > and this issue existed in 8.4 as well. ?Seems I just never noticed it,
> > and it not specifically mentioned in the TODO item we already have.
> 
> I think we should add a TODO to parse recovery.conf with the same code
> we use to parse postgresql.conf, or possibly merge the two files.
> This issue was previously alluded to here:
> 
> http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php

We have a TODO already for this:

Allow recovery.conf to support the same syntax as postgresql.conf,
including quoting

* recovery.conf parsing problems 

I thought the problem was just quotes inside strings, not the
requirement of quotes for everything.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add PGFILEDESC description to Makefiles for all /contrib

[Bruce Momjian]

Magnus Hagander wrote:
> On Thu, May 13, 2010 at 1:47 AM, Bruce Momjian  wrote:
> > bruce wrote:
> >> Tom Lane wrote:
> >> > Bruce Momjian  writes:
> >> > > All other binaries had such a designation, and all /contrib binaries
> >> > > were missing them. ?I assume I was doing cleanup. ?You want the icon
> >> > > removed from the backend makefile?
> >> >
> >> > Yes. ?I'm prepared to believe that not having the icons set on the
> >> > contrib executables was an oversight. ?I'm much less prepared to assume
> >> > that not marking the postgres executable was an oversight. ?Again,
> >> > unless you *know* that this change is needed and appropriate, now is
> >> > not the time to be making it, and especially not without discussion.
> >>
> >> OK, done with attached patch, and I added a comment about why it is not
> >> labeled.
> >
> > I did some research on PGFILEDESC and it does what I thought it does ---
> > in embeds the 'ico' file into the executable in
> > /pg/tools/msvc/Project.pm, and the image looks like the attached JPEG.
> > The image is of a blue elephant head.
> >
> > So, currently, every binary uses that icon, except for the postmaster.
> > Is that what we want? ?You could make the argument that a daemon, like
> > the postmaster, shouldn't have one, which I think is Tom's point.
> 
> It's pretty normal that daemons don't have icons.

Yes, that is the logic I was looking for and documented that in the
postmaster Makefile.

> One could argue the same for binaries that are normally never executed
> by the user, just internally - but I don't think we have any of those
> (we're talking EXE not DLL).
> 
> There is, AFAIK, no rule (hard or of-thumb) for "icons for GUI
> programs only not commandline ones". Commandline tools usually have
> it.
> 
> The argument to be made there is that when you go into the bin
> directory you'll get greeted with a long list of identical elephants.
> It would probably be better if we could have a *different* icon for
> tools that the user is likely to execute himself - which is pretty
> much just psql I think.

All /bin stuff already had icons --- I just added icons to /contrib
binaries.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] How to know killed by pg_terminate_backend

[Tatsuo Ishii]

> > Maybe we could make PostgreSQL a little bit smarter so that it returns
> > a different code than 57P01 when killed by pg_terminate_backend().
> 
> Seems reasonable. Does the victim backend currently know why it has been
> killed?

I don't think so.

One idea is postmaster sets a flag in the shared memory area
indicating it rceived SIGTERM before forwarding the signal to
backends.

Backend check the flag and if it's not set, it knows that the signal
has been sent by pg_terminate_backend(), not postmaster.

What about new error code:

#define ERRCODE_BACKEND_STOP_REQUEST
MAKE_SQLSTATE('5','7', 'P','0','4')
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Bruce Momjian]

Takahiro Itagaki wrote:
> I read pg_upgrade code glance over, and found 4 issues in it.
> Are there any issues to be fixed before 9.0 release?
> 
> 1. NAMEDATASIZE
> 2. extern PGDLLIMPORT
> 3. pathSeparator
> 4. EDB_NATIVE_LANG
> 
>  1. NAMEDATASIZE 
> pg_upgrade has the following definition, but should it be just NAMEDATALEN?
> 
> /* Allocate for null byte */
> #define NAMEDATASIZE  (NAMEDATALEN + 1)
> 
> Table names should be in NAMEDATELEN - 1 bytes. At least 64th bytes in 
> "name" data is always '\0'.
> 
> =# CREATE TABLE "1234567890...(total 70 chars)...1234567890" (i int);
> NOTICE:  identifier "123...890" will be truncated to "123...0123"

Agreed.  I have changed the code to use NAMEDATALEN.

>  2. extern PGDLLIMPORT 
> pg_upgrade has own definitions of
> extern PGDLLIMPORT Oid binary_upgrade_next_xxx
> in pg_upgrade_sysoids.c. But those variables are not declared as
> PGDLLIMPORT in the core. Can we access unexported variables here?

The issue here is that you use PGDLLIMPORT where you are importing the
variable, not where it is defined.  For example, look at
'seq_page_cost'.  You can see PGDLLIMPORT used where it is imported with
'extern', but not where is it defined.

>  3. pathSeparator 
> Path separator for Windows is not only \ but also /. The current code
> ignores /. Also, it might not work if the path string including multi-byte
> characters that have \ (0x5c) in the second byte.

Agreed.  I have modified the code to use only "/" and check for "/" and
"\".  It is used only for checking the last byte so I didn't think it
would affect a multi-byte sequence.  I am actually unclear on that issue
though.  Can you review the new code to see if it is OK.

>  4. EDB_NATIVE_LANG 
> Of course it is commented out with #ifdef, but do we have codes
> for EDB in core?

Yeah, removed.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Florian Pflug]

On May 13, 2010, at 23:51 , Kevin Grittner wrote:

> Florian Pflug  wrote:
> 
>> All in all, I believe that SHARE and UPDATE row-level locks should
>> be changed to cause concurrent UPDATEs to fail with a
>> serialization error. I can come up with a patch that does that,
>> but I wanted to get some feedback on the idea before I put the
>> work in.
> 
> Before you work on that, you might want to wait until you can review
> the work that I and Dan Ports (a Ph.D. candidate from MIT) have been
> doing on support for true serializable transactions.  You don't need
> to use FOR SHARE or FOR UPDATE or any explicit locks as long as the
> concurrent transactions are SERIALIZABLE.  We have it working, but
> have been holding off on discussion or patch submission at Tom's
> request -- he felt it would distract from the process of getting the
> release out.

I'm very exited about the work you're doing there, and believe it'd be a great 
feature to have.

However, I view my proposal as pretty orthogonal to that work. True 
serializable transaction are much more powerful than what I proposed, but at a 
much higher price too, due to the necessity of SIREAD locks. My proposal allows 
for simple FK-like constraints to be implemented at user-level that are correct 
for all isolation levels.

best regards,
Florian Pflug


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Florian Pflug]

On May 13, 2010, at 23:39 , Tom Lane wrote:
> Florian Pflug  writes:
>> All in all, I believe that SHARE and UPDATE row-level locks should be
>> changed to cause concurrent UPDATEs to fail with a serialization
>> error.
> 
> I don't see an argument for doing that for FOR SHARE locks, and it
> already happens for FOR UPDATE (at least if the row actually gets
> updated).

Yes, actually updating the row is a workaround. A prohibitively expensive one, 
though.

The arguments are as stated

a) SHARE or UPDATE locking a concurrently updated row *does* cause as 
serialization error, making the current behavior asymmetric

b) Locking primitives usually ensure that once you obtain the lock you see the 
most recent version of the data. This is currently true for READ COMMITTED 
transactions but not for SERIALIZABLE ones, and pretty undesirable a behavior 
for a locking primitive.

c) I fail to see how the current behavior is useful in the presence of 
SERIALIZABLE transactions. Currently, they could IMHO completely ignore FOR 
SHARE locks, without making any previously correct algorithm incorrect.

plus a weaker one:

d) Oracle does it for FOR UPDATE locks, and actually has an example of a FK 
trigger in PL/SQL in their docs.

> AFAICS this proposal mainly breaks things, in pursuit of
> an unnecessary and probably-impossible-anyway goal of making FK locking
> work with only user-level snapshots.

I don't see the breakage this'd cause. For READ COMMITTED transactions nothing 
changes. For SERIALIZABLE transactions the behavior of FOR UPDATE / FOR SHARE 
becomes much easier to grasp. In both cases a SHARE lock would then say "Only 
update this row if you have seen the locking transaction's changes".

Why do you think that making FK locking work with only user-level snapshots is 
probably-impossible-anyway? With the proposed changes, simply FOR SHARE locking 
the parent row on INSERT/UPDATE of the child, plus checking for child rows on 
UPDATE/DELETE of the parent gives a 100% correct FK trigger.

I do not have a formal proof for that last assertion, but I'm not aware of any 
counter-examples either. Would love to hear of any, though.

best regards,
Florian Pflug


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Bruce Momjian]

Josh Berkus wrote:
> On 5/13/10 10:14 AM, Bruce Momjian wrote:
> > I am trying to think of this as a non-EnterpriseDB employee.  If suppose
> > Greenplum had given us a utility and they wanted it to work with their
> > version of the database, what accommodation would we make for them?  I
> > agree on the documentation, but would we allow #ifdefs that were only
> > used by them if there were only a few of them?  Could we treat it as an
> > operating system that none of us use?  I don't think Greenplum would
> > require us to keep support for their database, but they would prefer it,
> > and it might encourage more contributions from them.  Maybe we would
> > just tell them to keep their own patches, but I figured I would ask
> > specifically so we have a policy for next time.
> 
> My $0.021746:
> 
> If something is going to be included in /contrib, it should only include
> code which relates to standard PostgreSQL.  The independant pg_migrator
> project can be a PG/EDBAS tool; the contrib module needs to be
> vanilla-postgres only.  If the donor of the code wants to keep the
> specific fork support, then it should remain an independant project.
> 
> I'm not just referring to EDB here, or even just proprietary forks; even
> open source forks (like PostgresXC or pgCluster) shouldn't have specific
> code in /contrib.  Within the limits of reasonableness, of course.
> 
> My argument isn't based on purity, but is rather based on:
> (a) avoiding confusing the users, and
> (b) avoiding bulking code with lots of ifdefs if we can avoid it, and
> (c) fork release cycles are often different from pgsql-core, and EDB's
> certainly is.

I was more interested in understanding our policy rather than how to
handle this specific issue.  I have removed all mentions of EnterpriseDB
Advanced Server from pg_upgrade with the attached patch.  I will keep
the patch for submission back to EnterpriseDB when they want it, or they
can just pull it from CVS.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com
Index: contrib/pg_upgrade/check.c
===
RCS file: /cvsroot/pgsql/contrib/pg_upgrade/check.c,v
retrieving revision 1.3
diff -c -c -r1.3 check.c
*** contrib/pg_upgrade/check.c	13 May 2010 15:58:15 -	1.3
--- contrib/pg_upgrade/check.c	13 May 2010 22:48:06 -
***
*** 149,158 
  		{
  			prep_status(ctx, "Adjusting sequences");
  			exec_prog(ctx, true,
! 	SYSTEMQUOTE "\"%s/%s\" --set ON_ERROR_STOP=on --port %d "
  	  "-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
! 	  ctx->new.bindir, ctx->new.psql_exe, ctx->new.port,
! 	  sequence_script_file_name, ctx->logfile);
  			unlink(sequence_script_file_name);
  			pg_free(sequence_script_file_name);
  			check_ok(ctx);
--- 149,158 
  		{
  			prep_status(ctx, "Adjusting sequences");
  			exec_prog(ctx, true,
! 	SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on --port %d "
  	  "-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
! 	  ctx->new.bindir, ctx->new.port, sequence_script_file_name,
! 	  ctx->logfile);
  			unlink(sequence_script_file_name);
  			pg_free(sequence_script_file_name);
  			check_ok(ctx);
Index: contrib/pg_upgrade/controldata.c
===
RCS file: /cvsroot/pgsql/contrib/pg_upgrade/controldata.c,v
retrieving revision 1.1
diff -c -c -r1.1 controldata.c
*** contrib/pg_upgrade/controldata.c	12 May 2010 02:19:10 -	1.1
--- contrib/pg_upgrade/controldata.c	13 May 2010 22:48:06 -
***
*** 9,18 
  #include 
  #include 
  
- #ifdef EDB_NATIVE_LANG
- #include "access/tuptoaster.h"
- #endif
- 
  
  /*
   * get_control_data()
--- 9,14 
***
*** 88,102 
  		got_float8_pass_by_value = true;
  	}
  
- #ifdef EDB_NATIVE_LANG
- 	/* EDB AS 8.3 is an 8.2 code base */
- 	if (cluster->is_edb_as && GET_MAJOR_VERSION(cluster->major_version) <= 803)
- 	{
- 		cluster->controldata.toast = TOAST_MAX_CHUNK_SIZE;
- 		got_toast = true;
- 	}
- #endif
- 
  	/* we have the result of cmd in "output". so parse it line by line now */
  	while (fgets(bufin, sizeof(bufin), output))
  	{
--- 84,89 
***
*** 140,148 
  			p++;/* removing ':' char */
  			cluster->controldata.cat_ver = (uint32) atol(p);
  		}
! 		else if ((p = strstr(bufin, "First log file ID after reset:")) != NULL ||
!  (cluster->is_edb_as && GET_MAJOR_VERSION(cluster->major_version) <= 803 &&
!   (p = strstr(bufin, "Current log file ID:")) != NULL))
  		{
  			p = strchr(p, ':');
  
--- 127,133 
  			p++;/* removing ':' char */
  			cluster->controldata.cat_ver = (uint32) atol(p);
  		}
! 		else if ((p = strstr(bufin, "First log file ID after reset:")) != NULL)
  		{
  			p = strchr(p, ':');
  
***
*** 153,161 
  			cluster->controldata.logid = (uint32) atol(p);
  			got_log_id = true;
  		}
! 		else i

Re: [HACKERS] List traffic

[Tom Lane]

"Joshua D. Drake"  writes:
> On Thu, 2010-05-13 at 19:13 -0300, Marc G. Fournier wrote:
>> But that, IMHO, is the point of the smaller list ... it allows the group 
>> on that list to hash out their ideas, and, hopefully, deal with both 
>> arguments and counter arguments so that when presented to the larger 
>> group, they would then have a more cohesive arg for their ideas ...

> Yes and no. After being on these lists for years, I have kind of been
> moving toward the less is more. E.g; for main list traffic I can see the
> need for two maybe three, that's it:

> hackers
> general
> www

I can see the need for small tightly-focused special lists.  www is a
good example, and perhaps pgsql-cluster-hackers is too (though I'm less
convinced of that than Marc is).  I agree that we've done poorly with
lists with wider charters, mainly because there is so little clarity
about which topics belong where.

I'd keep -bugs and -performance, which seem to be reasonably well
focused, but I can definitely see collapsing most of the other "user"
lists into -general.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Joshua D. Drake]

On Thu, 2010-05-13 at 19:13 -0300, Marc G. Fournier wrote:
> On Thu, 13 May 2010, Tom Lane wrote:
> 
> > "Marc G. Fournier"  writes:
> >> On Thu, 13 May 2010, Magnus Hagander wrote:
> >>> We tried that with pgsql-hackers-win32 and iirc also
> >>> pgsql-hackers-pitr, and it was a big failure...
> >
> >> But, we are doing that now with pgsql-cluster-hackers and it looks to be
> >> working quite well from what I can see ...
> >
> > Is it?  If they want someplace where the majority of hackers won't see
> > the discussion, maybe, but I am not sure that's not counterproductive.
> > Ideas developed by a small group may or may not survive exposure when
> > they reach this list.
> 
> But that, IMHO, is the point of the smaller list ... it allows the group 
> on that list to hash out their ideas, and, hopefully, deal with both 
> arguments and counter arguments so that when presented to the larger 
> group, they would then have a more cohesive arg for their ideas ...

Yes and no. After being on these lists for years, I have kind of been
moving toward the less is more. E.g; for main list traffic I can see the
need for two maybe three, that's it:

hackers
general
www

There is no reason why advocacy can't happen on general. Theoretically
www could be on hackers (although I do see the point of a separate
list).

A good MUA will deal with any overhead you have. I use Evolution and no
its not perfect but I have no problem managing the hordes of email I get
from this community.

Between labels, filters, watch lists and all the other goodies any MUA
will give you, I see no reason to have this all broken out anymore.

Joshua D. Drake


-- 
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] quoting and recovery.conf

[Robert Haas]

On Thu, May 13, 2010 at 1:00 PM, Bruce Momjian  wrote:
> Is there a reason we require single quotes around boolean values in
> recovery.conf?
>
>        standby_mode = 'off'
>
> This does not work:
>
>        standby_mode = off
>
> I knew there were inconsistencies between quoting in postgresql.conf and
> recovery.conf, but I didn't realize it extended to boolean quoting.  I
> see this at the top of recovery.conf now:
>
>        # This file consists of lines of the form:
>        #
>        #   name = 'value'
>        #
>        # (The quotes around the value are NOT optional, but the "=" is.)
>        #
>
> and this issue existed in 8.4 as well.  Seems I just never noticed it,
> and it not specifically mentioned in the TODO item we already have.

I think we should add a TODO to parse recovery.conf with the same code
we use to parse postgresql.conf, or possibly merge the two files.
This issue was previously alluded to here:

http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] nvarchar notation accepted?

[Jaime Casanova]

Hi,

i migrate a ms sql server database to postgres and was trying some
queries from the application to find if everything works right...
when i was looking to those queries i found some that has a notation
for nvarchar (ej: campo = N'sometext')
i was expecting those to fail but this actually works, is that fine? i
know, we can use E'' strings but N'' ones are no where documented, so
can i rely on those or i have to change those strings?

"""
create table t1_nvarchar(col1 text);
insert into t1_nvarchar values (N'texto');
"""

-- 
Jaime Casanova www.2ndQuadrant.com
Soporte y capacitación de PostgreSQL

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

On Thu, 13 May 2010, Tom Lane wrote:


"Marc G. Fournier"  writes:

On Thu, 13 May 2010, Magnus Hagander wrote:

We tried that with pgsql-hackers-win32 and iirc also
pgsql-hackers-pitr, and it was a big failure...



But, we are doing that now with pgsql-cluster-hackers and it looks to be
working quite well from what I can see ...


Is it?  If they want someplace where the majority of hackers won't see
the discussion, maybe, but I am not sure that's not counterproductive.
Ideas developed by a small group may or may not survive exposure when
they reach this list.


But that, IMHO, is the point of the smaller list ... it allows the group 
on that list to hash out their ideas, and, hopefully, deal with both 
arguments and counter arguments so that when presented to the larger 
group, they would then have a more cohesive arg for their ideas ...


Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] HS/SR Assert server crash

[Bruce Momjian]

I was able to easily crash the standby server today just by starting it
and connecting to it via psql.  The master was idle.  The failure was:

LOG:  streaming replication successfully connected to primary
TRAP: FailedAssertion("!(((xmax) >= ((TransactionId) 3)))", File: 
"procarray.c", Line: 1211)
LOG:  server process (PID 12761) was terminated by signal 6: Abort trap
LOG:  terminating any other active server processes

My master postgresql.conf was:

wal_level = hot_standby # minimal, archive, or 
hot_standby
archive_mode = on   # allows archiving to be done
archive_command = 'cp -i %p /u/pg/archive/%f < /dev/null '  # command 
to use to archive a logfile segment
max_wal_senders = 1 # max number of walsender processes

My slave postgresql.conf was:

port = 5433 # (change requires restart)
wal_level = hot_standby # minimal, archive, or 
hot_standby
archive_mode = off  # allows archiving to be done
archive_command = 'cp -i %p /u/pg/archive/%f < /dev/null '  # 
command to use to archive a logfile segment
hot_standby = on# allows queries during recovery
max_wal_senders = 1 # max number of walsender processes

and my slave recovery.conf was:

restore_command = 'cp /u/pg/archive/%f %p'  # e.g. 'cp 
/mnt/server/archivedir/%f %p'
standby_mode = 'on'
primary_conninfo = 'host=localhost port=5432'   # e.g. 
'host=localhost port=5432'

Let me know what additional information I can supply.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Kevin Grittner]

Florian Pflug  wrote:
 
> All in all, I believe that SHARE and UPDATE row-level locks should
> be changed to cause concurrent UPDATEs to fail with a
> serialization error. I can come up with a patch that does that,
> but I wanted to get some feedback on the idea before I put the
> work in.
 
Before you work on that, you might want to wait until you can review
the work that I and Dan Ports (a Ph.D. candidate from MIT) have been
doing on support for true serializable transactions.  You don't need
to use FOR SHARE or FOR UPDATE or any explicit locks as long as the
concurrent transactions are SERIALIZABLE.  We have it working, but
have been holding off on discussion or patch submission at Tom's
request -- he felt it would distract from the process of getting the
release out.
 
Whenever people are ready, I can submit a WIP patch.  All issues
discuss on this thread "Just Work" with the patch applied.  There's
a Wiki page and a public git repository related to this work, for
anyone who is interested and not busy with release work.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Tom Lane]

Florian Pflug  writes:
> All in all, I believe that SHARE and UPDATE row-level locks should be
> changed to cause concurrent UPDATEs to fail with a serialization
> error.

I don't see an argument for doing that for FOR SHARE locks, and it
already happens for FOR UPDATE (at least if the row actually gets
updated).  AFAICS this proposal mainly breaks things, in pursuit of
an unnecessary and probably-impossible-anyway goal of making FK locking
work with only user-level snapshots.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Dimitri Fontaine]

Alvaro Herrera  writes:
> Excerpts from Yeb Havinga's message of jue may 13 15:06:53 -0400 2010:
>
>> Now I made a new gmail account
>
> Yeah, this approach is interesting.  A few days ago I started using Sup
> ( http://sup.rubyforge.org/ ) to manage my email

Feature wise, I think gnus offers more than the two approaches
combined. Speed wise some people use it with some indexing solution, I'm
not finding the need yet.

And yes, to handle our lists traffic you must have a MUA made for
it. That's the reason why I switched, and it's working great here.

Regards,
-- 
dim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

[Florian Pflug]

Hi

After the recent discussion about the impossibility of efficiently implementing 
FK-like constraint triggers in PL/PGSQL that work correctly under SERIALIZABLe 
transactions, I've compared our behavior to that of Oracle. As it turns out, a 
slight difference in Oracle's behavior makes those FK constraint triggers which 
on postgres are only correct in READ COMMITTED mode fully correct in 
SERIALIZABLE mode also.

1. Summary of the previous discussion

The built-in FK constraint trigger looks for rows visible under either the 
transaction's snapshot *or* a freshly taken MVCC snapshot when checking for 
child-table rows that'd prevent an UPDATE or DELETE of a row in the parent 
table. This is necessary even though the parent row is SHARE-locked on 
INSERTs/UPDATEs to the child table, and would also be necessary if it was 
UPDATE-locked. The following series of commands illustrates why

C1: BEGIN
C1: SELECT * FROM t WHERE id = 1 FOR UPDATE
C2: BEGIN
C2: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
C2: SELECT * FROM t -- Take snapshot before C1 commits
C1: COMMIT
C2: DELETE FROM t WHERE id = 1
C2: COMMIT

Since C1 commits before C2 does DELETE, C2 is entirely unaffected by C1's 
UPDATE-lock. C2 has no way of detecting possible dependent rows that C1 might 
have inserted, since C1 is invisible to C2.

Note that if you swap the SELECT .. FOR UPDATE and the DELETE commands, the 
SELECT .. FOR UPDATE will cause a serialization error!

2. The behavior or Oracle

Oracle treats a "FOR UPDATE" lock much like an actual UPDATE when checking for 
serialization conflicts. This causes the DELETE in the example above to raise a 
serialization error, and hence prevents the failure case for FK constraint 
triggers even without a recheck under a current snapshot.

One can think of a FOR UPDATE lock as a kind of read barrier on Oracle - it 
prevents other transactions from messing with the row that don't consider the 
locking transaction to be visible.

3. Conclusio

While it might seem strange at first for a lock to affect other transactions 
even after the locking transaction has ended, it actually makes sense when 
viewed as a kind of write barrier. It is very common for locking primitives to 
use barrier instructions to ensure that one lock holder sees all changes done 
by the previous owner. Raising a serialization error in the example above is 
the transactional equivalent of such a barrier instruction in the case of 
SERIALIZABLE transactions - since updating the transaction's snapshot is 
obviously not an option, the remaining alternative is to restart the whole 
transaction under a current snapshot. This is exactly what raising a 
serialization error accomplishes.

Also, while Oracle's behavior has obvious use-cases (e.g. FK-like constraints), 
I failed to come up with a case where postgres' current behavior is useful. 
When would you want a (SERIALIZABLE) transaction to wait for a lock, but then 
continue as if the lock had never existed? What is the point of waiting then in 
the first place?

All in all, I believe that SHARE and UPDATE row-level locks should be changed 
to cause concurrent UPDATEs to fail with a serialization error. I can come up 
with a patch that does that, but I wanted to get some feedback on the idea 
before I put the work in.

best regards,
Florian Pflug


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Tom Lane]

"Marc G. Fournier"  writes:
> On Thu, 13 May 2010, Magnus Hagander wrote:
>> We tried that with pgsql-hackers-win32 and iirc also
>> pgsql-hackers-pitr, and it was a big failure...

> But, we are doing that now with pgsql-cluster-hackers and it looks to be 
> working quite well from what I can see ...

Is it?  If they want someplace where the majority of hackers won't see
the discussion, maybe, but I am not sure that's not counterproductive.
Ideas developed by a small group may or may not survive exposure when
they reach this list.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

On Thu, 13 May 2010, Alvaro Herrera wrote:


Excerpts from Yeb Havinga's message of jue may 13 15:06:53 -0400 2010:


My $0.02 - I like the whole 'don't sort, search' (or how did they call
it?) just let the inbox fill up, google is fast enough. What would be
really interesting is to have some extra 'tags/headers' added to the
emails (document classification with e.g. self organizing map/kohonen),
so my local filters could make labels based on that, instead of perhaps
badly spelled keywords in subjects or message body.


I missed this when I read it the first time .. all list email does have an 
X-Mailing-List header added so that you can label based on list itself ... 
is that what you mean, or are you thinking of something else entirely?




Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Alvaro Herrera]

Excerpts from Yeb Havinga's message of jue may 13 15:06:53 -0400 2010:

> Now I made a new gmail account, subscribed to all lists with some volume 
> and let it all message per message come into the inbox. Together with 
> thunderbird/imap this works quite nicely. With filters it's possible to 
> tag interesting messages (like does the To: contain my email? -> tag it 
> so it becomes green). Now I only need to view unread mails, (by thread 
> or date), read some messages and then ctrl-shift-c - all read.
> 
> My $0.02 - I like the whole 'don't sort, search' (or how did they call 
> it?) just let the inbox fill up, google is fast enough. What would be 
> really interesting is to have some extra 'tags/headers' added to the 
> emails (document classification with e.g. self organizing map/kohonen), 
> so my local filters could make labels based on that, instead of perhaps 
> badly spelled keywords in subjects or message body.

Yeah, this approach is interesting.  A few days ago I started using Sup
( http://sup.rubyforge.org/ ) to manage my email, and after a rather
lengthy warm-up process, I find it a lot more comfortable than Mutt (or
anything else I've tried earlier, for that matter).  I particularly like
the multiple buffer approach, avoiding the need for switching between
several Mutt instances, one for each mailbox.

So it's almost like gmail: you get fast search, labelling, and a
thread-based approach rather than message-based.  As with gmail, you can
"mute" threads that are not interesting to you, so that if any email
arrives later to that thread, you will not see it unless you actively
look for it.  An old (unmuted) thread receiving a new message jumps back
at the top of the list; and you can dismiss stuff as "archived" with a
single keystroke, and it will stop polluting your immediate environment,
but you can search for it.  And it's pretty *fast* with searches (uses
Xapian as backend).

It's clearly a programmer's MUA -- if you want automatic labelling, you
better be prepared to write some Ruby code.  I have already written some
simple rules that get me the trivial labels for pgsql lists and such; I
have also ported the Perl moderation script I used, and the main
advantage is that it's a tad faster (though I spent a lot more time
writing that function than I'll ever save actually doing moderation --
but hey, I managed to learn some Ruby in the process).

It is rather immature though, so I can't recommend it unless you're
prepared to deal with that.
-- 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

On Thu, 13 May 2010, Magnus Hagander wrote:


On Thu, May 13, 2010 at 8:05 PM, Marc G. Fournier  wrote:


My thought had been a split along the lines of major components of the server 
... for instance, a totally seperate list for HS related issues, so that, if 
nothing else, those 'lurkers' that are only interested in developments on that 
front could be there but not on the main stream -hackers ... almost like 
seperate working groups ...


We tried that with pgsql-hackers-win32 and iirc also
pgsql-hackers-pitr, and it was a big failure...


But, we are doing that now with pgsql-cluster-hackers and it looks to be 
working quite well from what I can see ... guess it depends on if ppl want 
it to fail in the first place or not *shrug*


It also depends if a clear line can be drawn and adhered to ...




Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Magnus Hagander]

On Thu, May 13, 2010 at 8:05 PM, Marc G. Fournier  wrote:
>
> My thought had been a split along the lines of major components of the server 
> ... for instance, a totally seperate list for HS related issues, so that, if 
> nothing else, those 'lurkers' that are only interested in developments on 
> that front could be there but not on the main stream -hackers ... almost like 
> seperate working groups ...

We tried that with pgsql-hackers-win32 and iirc also
pgsql-hackers-pitr, and it was a big failure...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Yeb Havinga]

Greg Stark wrote:

On Wed, May 12, 2010 at 5:24 PM, Robert Haas  wrote:
  

The difference between discussing a patch and discussing an idea that
might lead to a patch is fairly fine.



And importantly -- who would be able to subscribe to one and not the
other? If you have to subscribe to both to get make any sense of
things then there's no point.

Fwiw I'm having trouble keeping up these days too. And I'm quite
accustomed to very heavy traffic email. I've been throwing all
postgres related lists into one folder and skimmed through it looking
for important threads. However this has now broken down. There are
about 45 new threads every day. I've been travelling for a bit and am
now 1,500 threads behind...
  
I've only been actively reading the pg lists for a few months now, after 
several previous attempts that failed mainly because the way I set it up 
did not work nice, mainly because of the volume. I tried digests, didn't 
like it (how to reply?), also didn't like that the pg mails that were so 
many completely swamped the 'main' email I use.


Now I made a new gmail account, subscribed to all lists with some volume 
and let it all message per message come into the inbox. Together with 
thunderbird/imap this works quite nicely. With filters it's possible to 
tag interesting messages (like does the To: contain my email? -> tag it 
so it becomes green). Now I only need to view unread mails, (by thread 
or date), read some messages and then ctrl-shift-c - all read.


My $0.02 - I like the whole 'don't sort, search' (or how did they call 
it?) just let the inbox fill up, google is fast enough. What would be 
really interesting is to have some extra 'tags/headers' added to the 
emails (document classification with e.g. self organizing map/kohonen), 
so my local filters could make labels based on that, instead of perhaps 
badly spelled keywords in subjects or message body.


regards,
Yeb Havinga

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Josh Berkus]

On 5/13/10 10:14 AM, Bruce Momjian wrote:
> I am trying to think of this as a non-EnterpriseDB employee.  If suppose
> Greenplum had given us a utility and they wanted it to work with their
> version of the database, what accommodation would we make for them?  I
> agree on the documentation, but would we allow #ifdefs that were only
> used by them if there were only a few of them?  Could we treat it as an
> operating system that none of us use?  I don't think Greenplum would
> require us to keep support for their database, but they would prefer it,
> and it might encourage more contributions from them.  Maybe we would
> just tell them to keep their own patches, but I figured I would ask
> specifically so we have a policy for next time.

My $0.021746:

If something is going to be included in /contrib, it should only include
code which relates to standard PostgreSQL.  The independant pg_migrator
project can be a PG/EDBAS tool; the contrib module needs to be
vanilla-postgres only.  If the donor of the code wants to keep the
specific fork support, then it should remain an independant project.

I'm not just referring to EDB here, or even just proprietary forks; even
open source forks (like PostgresXC or pgCluster) shouldn't have specific
code in /contrib.  Within the limits of reasonableness, of course.

My argument isn't based on purity, but is rather based on:
(a) avoiding confusing the users, and
(b) avoiding bulking code with lots of ifdefs if we can avoid it, and
(c) fork release cycles are often different from pgsql-core, and EDB's
certainly is.

-- 
  -- Josh Berkus
 PostgreSQL Experts Inc.
 http://www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

My thought had been a split along the lines of major components of the 
server ... for instance, a totally seperate list for HS related issues, so 
that, if nothing else, those 'lurkers' that are only interested in 
developments on that front could be there but not on the main stream 
-hackers ... almost like seperate working groups ...


Twas just a thought ...

On Wed, 12 May 2010, Greg Stark wrote:


On Wed, May 12, 2010 at 5:24 PM, Robert Haas  wrote:

The difference between discussing a patch and discussing an idea that
might lead to a patch is fairly fine.


And importantly -- who would be able to subscribe to one and not the
other? If you have to subscribe to both to get make any sense of
things then there's no point.

Fwiw I'm having trouble keeping up these days too. And I'm quite
accustomed to very heavy traffic email. I've been throwing all
postgres related lists into one folder and skimmed through it looking
for important threads. However this has now broken down. There are
about 45 new threads every day. I've been travelling for a bit and am
now 1,500 threads behind...

If we can find a way to split the content sensibly so I could stop
reading some of it that would be helpful. But cutting splitting it
along subject matter where both sets of subject matter need to be seen
by the same people doesn't really help.

I'm thinking I'll move -general (and the useless -novice) to another
folder. But I'm left wondering what to do with -admin and
-performance. They're a random mix of user content and developer
content. I'll probably move them along with -general but that means I
won't be likely to see any development discussion on them in the
future.




--
greg

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers




Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Andrew Dunstan]

Bruce Momjian wrote:

Indeed.  Given the (presumably large) delta between EDB's code and ours,
having to have some delta in pg_upgrade isn't going to make much
difference for them.  I think the community code and docs should
completely omit any mention of that.



I am trying to think of this as a non-EnterpriseDB employee.  If suppose
Greenplum had given us a utility and they wanted it to work with their
version of the database, what accommodation would we make for them?  I
agree on the documentation, but would we allow #ifdefs that were only
used by them if there were only a few of them?  Could we treat it as an
operating system that none of us use?  I don't think Greenplum would
require us to keep support for their database, but they would prefer it,
and it might encourage more contributions from them.  Maybe we would
just tell them to keep their own patches, but I figured I would ask
specifically so we have a policy for next time.

I guess another question is whether we would accept a patch that was
useful only for a Greenplum build?  And does removing such code use the
same criteria?

I know pgAdmin supports Greenplum, but that is an external tool so it
makes more sense there.

  


What if several vendors want the same thing? The code will quickly 
become spaghetti.


AFAIK the Linux kernel expects distros to keep their patchsets 
separately, and I rather think we should too.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Joshua D. Drake]

On Thu, 2010-05-13 at 17:19 +0200, Magnus Hagander wrote:

> I say remove it. On all accounts.
> 
> There's a fork of postgres for EDB AS, shouldn't there be a fork of
> pg_upgrade the same way, if it requires special code? The code in
> community postgresql certainly shouldn't have any EDB AS code in it.

If the code would be useful for other "projects" then keep it. If it is
only for a closed source product, dump it.

Joshua D. Drake


-- 
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Bruce Momjian]

Tom Lane wrote:
> Magnus Hagander  writes:
> > On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian  wrote:
> >> I have added SGML comments to comment out the text that mentions EDB
> >> Advanced Server. ?Is that enough? ?Should I remove the text from the
> >> SGML? ?Should I move it to the bottom of the SGML? ?Should I remove the
> >> EnterpriseDB Advanced Server checks from the C code too? ?I don't
> >> remember having to deal with anything like this before, so I am unclear
> >> how to proceed.
> 
> > I say remove it. On all accounts.
> 
> > There's a fork of postgres for EDB AS, shouldn't there be a fork of
> > pg_upgrade the same way, if it requires special code? The code in
> > community postgresql certainly shouldn't have any EDB AS code in it.
> 
> Indeed.  Given the (presumably large) delta between EDB's code and ours,
> having to have some delta in pg_upgrade isn't going to make much
> difference for them.  I think the community code and docs should
> completely omit any mention of that.

I am trying to think of this as a non-EnterpriseDB employee.  If suppose
Greenplum had given us a utility and they wanted it to work with their
version of the database, what accommodation would we make for them?  I
agree on the documentation, but would we allow #ifdefs that were only
used by them if there were only a few of them?  Could we treat it as an
operating system that none of us use?  I don't think Greenplum would
require us to keep support for their database, but they would prefer it,
and it might encourage more contributions from them.  Maybe we would
just tell them to keep their own patches, but I figured I would ask
specifically so we have a policy for next time.

I guess another question is whether we would accept a patch that was
useful only for a Greenplum build?  And does removing such code use the
same criteria?

I know pgAdmin supports Greenplum, but that is an external tool so it
makes more sense there.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] max_standby_delay considered harmful

[Josh Berkus]

On 5/12/10 8:07 PM, Robert Haas wrote:
> I think that would be a good thing to check (it'll confirm whether
> this is the same bug), but I'm not convinced we should actually fix it
> that way.  Prior to 8.4, we handled a smart shutdown during recovery
> at the conclusion of recovery, just prior to entering normal running.
> I'm wondering if we shouldn't revert to that behavior in both 8.4 and
> HEAD.

This would be OK as long as we document it well.  We patched the
shutdown the way we did specifically because Fujii thought it would be
an easy fix; if it's complicated, we should revert it and document the
issue for DBAs.

Oh, and to confirm: the same issue exists, and has always existed, with
Warm Standby.

-- 
  -- Josh Berkus
 PostgreSQL Experts Inc.
 http://www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] quoting and recovery.conf

[Bruce Momjian]

Is there a reason we require single quotes around boolean values in
recovery.conf?

standby_mode = 'off'

This does not work:

standby_mode = off

I knew there were inconsistencies between quoting in postgresql.conf and
recovery.conf, but I didn't realize it extended to boolean quoting.  I
see this at the top of recovery.conf now:

# This file consists of lines of the form:
#
#   name = 'value'
#
# (The quotes around the value are NOT optional, but the "=" is.)
#

and this issue existed in 8.4 as well.  Seems I just never noticed it,
and it not specifically mentioned in the TODO item we already have.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Query execution plan from 8.3 -> 8.4

[Brendan Hill]

Thanks for the advice, will do.

Regards,
Brendan Hill
Chief Information Officer
Jims Group Pty Ltd
48 Edinburgh Rd
Mooroolbark VIC 3138
www.jims.net

For all Jims IT enquiries: infot...@jims.net
For emergencies: 1300 130 490 (intl +61 4 3456 5776)


-Original Message-
From: Kevin Grittner [mailto:kevin.gritt...@wicourts.gov] 
Sent: Thursday, 13 May 2010 12:29 AM
To: Brendan Hill; pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Query execution plan from 8.3 -> 8.4

"Brendan Hill"  wrote:
 
> AND Notes.Person_ID IN (SELECT
> ISNULL(Personnel.Person_ID, Businesses.Main_Person_ID)
 
You might try switching this to an EXISTS test.
 
If you post on this topic again, really it should be on the -perform
list, as Stephen mentioned, and review this page for ideas on other
information (like hardware and the postgresql.conf file) which might
help people better understand the problem:
 
http://wiki.postgresql.org/wiki/SlowQueryQuestions
 
-Kevin


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] multibyte charater set in levenshtein function

[Alexander Korotkov]

On Wed, May 12, 2010 at 11:04 PM, Alvaro Herrera wrote:

> On a quick look, I didn't like the way you separated the
> "pg_database_encoding_max_length() > 1" cases.  There seem to be too
> much common code.  Can that be refactored a bit better?
>
I did a little refactoring in order to avoid some similar code.
I'm not quite sure about my CHAR_CMP macro. Is it a good idea?


fuzzystrmatch-0.2.diff.gz
Description: GNU Zip compressed data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Greg Stark]

On Wed, May 12, 2010 at 5:24 PM, Robert Haas  wrote:
> The difference between discussing a patch and discussing an idea that
> might lead to a patch is fairly fine.

And importantly -- who would be able to subscribe to one and not the
other? If you have to subscribe to both to get make any sense of
things then there's no point.

Fwiw I'm having trouble keeping up these days too. And I'm quite
accustomed to very heavy traffic email. I've been throwing all
postgres related lists into one folder and skimmed through it looking
for important threads. However this has now broken down. There are
about 45 new threads every day. I've been travelling for a bit and am
now 1,500 threads behind...

If we can find a way to split the content sensibly so I could stop
reading some of it that would be helpful. But cutting splitting it
along subject matter where both sets of subject matter need to be seen
by the same people doesn't really help.

I'm thinking I'll move -general (and the useless -novice) to another
folder. But I'm left wondering what to do with -admin and
-performance. They're a random mix of user content and developer
content. I'll probably move them along with -general but that means I
won't be likely to see any development discussion on them in the
future.




-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Retiring from the PostgreSQL core team

[David Fetter]

On Thu, May 13, 2010 at 12:24:47AM -0400, Jan Wieck wrote:
> To whom it may concern,
> 
> this is to inform the PostgreSQL community of my retirement from my
> PostgreSQL core team position.
> 
> Over the past years I have not been able to dedicate as much time to
> PostgreSQL as everyone would have liked. The main reason for that was
> that I was swamped with other work and private matters and simply didn't
> have time. I did follow the mailing lists but did not participate much.
> 
> Looking at my publicly visible involvement over the last two years or
> so, there is little that would justify me being on the core team today.
> I was not involved in the release process, in patch reviewing,
> organizing and have contributed little.
> 
> However, in contrast to other previous core team members, I do not plan
> to disappear. Very much to the contrary. I am right now picking up some
> things that have long been on my TODO wish list and Afilias is doubling
> down on the commitment to PostgreSQL and Slony. We can and should talk
> about that stuff next week at PGCon in Ottawa. I will also stay in close
> contact with the remaining core team members, many of whom have become
> very good friends over the past 15 years.
> 
> The entire core team, me included, hoped that it wouldn't come to this
> and that I could have returned to active duty earlier. Things in my
> little sub universe didn't change as fast as we all hoped and we all
> think it is best now that I focus on getting back to speed and do some
> serious hacking.
> 
> I hope to see many of you in Ottawa.

You can run, but you can't hide ;)

Thanks for your deep and broad contributions so far, and I'm sure I
speak for many when I say we're looking forward to upcoming ones.

Cheers,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] wal_level and continuous archiving documentation

[Joshua Tolley]

I was reading through
http://www.postgresql.org/docs/9.0/static/continuous-archiving.html and
noticed that wal_level isn't mentioned where I'd expect it to be.
Specifically, there's a paragraph that starts, "To enable WAL archiving, set
the archive_mode configuration parameter to on, and specify the shell command
to use in the archive_command configuration parameter." There follows a long
discussion of archive_command, but no further discussion of other
configuration settings for several paragraphs, suggesting that those two
configuration changes are the only ones required to end up with a useful
archive. However, further on, it discusses wal_level:

"When wal_level is minimal some SQL commands are optimized to avoid WAL
logging, as described in Section 14.4.7. If archiving or streaming
replication were turned on during execution of one of these statements,
WAL would not contain enough information for archive recovery."

ISTM wal_archive should make an appearance where the docs bring up
archive_mode and archive_command, to say "wal_level must be set to 'archive'
or 'hot_standby'", so all required configuration changes are mentioned close
together.

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com


signature.asc
Description: Digital signature

Re: [HACKERS] multibyte charater set in levenshtein function

[Alexander Korotkov]

 On Thu, May 13, 2010 at 6:03 AM, Alvaro Herrera  wrote:

> Well, since it's only used in one place, why are you defining a macro at
> all?
>
In order to structure code better. My question was about another. Is memcmp
function good choice to compare very short sequences of bytes (from 1 to 4
bytes)?

Re: [HACKERS] List traffic

[damien clochard]

Le 11/05/2010 19:24, Alvaro Herrera a écrit :
> Excerpts from Marc G. Fournier's message of mar may 11 09:58:34 -0400 2010:
> 
>> If list traffic, especially on -hackers, is getting so large, should we 
>> look at maybe splitting it?  I could easily enough split things such that 
>> I duplicate the subscriber list, so nobody would have to subscribe, but it 
>> would make it easier for ppl to filter their incoming ... ?
> 
> Maybe we could create a separate list where people would send patches,
> and keep patchless discussion on -hackers?
> 
> Just a thought ;-)

Here's a simple description of how i use and see the -hackers list. I'm
what you could call a "silent reader", like many other subscribers i
don't participate to the discussions but i'm happy to be able to follow
them. I'm not an end-user and i'm not a developper. Just a guy that
wants to follow the "making-of" this project.

Sure the traffic is huge and sometimes i have thousands of unread
messages. But somewhat i managed to follow the threads i'm interested in
and leave asides others...

If this list is split in two smaller ones, then i guess i'll follow both
and it won't help me in any way. I guess it would even make things more
difficult to understand.

This is my modest experience. Clearly things can be improved, but
speaking for myself i don't think that splitting the list is a good idea.


--
damien clochard
http://www.dalibo.com




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Tom Lane]

Magnus Hagander  writes:
> On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian  wrote:
>> I have added SGML comments to comment out the text that mentions EDB
>> Advanced Server.  Is that enough?  Should I remove the text from the
>> SGML?  Should I move it to the bottom of the SGML?  Should I remove the
>> EnterpriseDB Advanced Server checks from the C code too?  I don't
>> remember having to deal with anything like this before, so I am unclear
>> how to proceed.

> I say remove it. On all accounts.

> There's a fork of postgres for EDB AS, shouldn't there be a fork of
> pg_upgrade the same way, if it requires special code? The code in
> community postgresql certainly shouldn't have any EDB AS code in it.

Indeed.  Given the (presumably large) delta between EDB's code and ours,
having to have some delta in pg_upgrade isn't going to make much
difference for them.  I think the community code and docs should
completely omit any mention of that.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] wal_level in postgresql.conf

[Bruce Momjian]

Bruce Momjian wrote:
> Why is 'wal_level' listed next to 'fsync' in postgresql.conf?
> 
>   #wal_level = 'hot_standby'  # minimal, archive, or 
> hot_standby
>   #fsync = on # turns forced synchronization 
> on or off
>   #synchronous_commit = on# immediate fsync at commit
>   #wal_sync_method = fsync# the default is the first 
> option
> 
> Seems it should be in the archiving section.  

Sorry, subject was wrong.  I am asking about 'wal_level'.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] wal_mode in postgresql.conf

[Bruce Momjian]

Why is 'wal_level' listed next to 'fsync' in postgresql.conf?

#wal_level = 'hot_standby'  # minimal, archive, or 
hot_standby
#fsync = on # turns forced synchronization 
on or off
#synchronous_commit = on# immediate fsync at commit
#wal_sync_method = fsync# the default is the first 
option

Seems it should be in the archiving section.  

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Magnus Hagander]

On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian  wrote:
> Magnus Hagander wrote:
>> On Thu, May 13, 2010 at 8:22 AM, Devrim G?ND?Z  wrote:
>> > On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote:
>> >>  4. EDB_NATIVE_LANG 
>> >> Of course it is commented out with #ifdef, but do we have codes
>> >> for EDB in core?
>> >
>> > I was about to raise similar thing, for the documentation:
>> >
>> > http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html
>> >
>> > This includes some references to EDB AS, which should be removed from
>> > PostgreSQL official documentation, IMHO.
>>
>> +1 on getting rid of those references.
>
> Agreed.  When it was on pgFoundry, I had to mention that because it was
> unclear who would be using it, but in /contrib we know this is for
> community Postgres.  EnterpriseDB did contribute the code so I would
> like to keep the code working for EnterpriseDB Advanced Server if that
> is easy.
>
> I have added SGML comments to comment out the text that mentions EDB
> Advanced Server.  Is that enough?  Should I remove the text from the
> SGML?  Should I move it to the bottom of the SGML?  Should I remove the
> EnterpriseDB Advanced Server checks from the C code too?  I don't
> remember having to deal with anything like this before, so I am unclear
> how to proceed.

I say remove it. On all accounts.

There's a fork of postgres for EDB AS, shouldn't there be a fork of
pg_upgrade the same way, if it requires special code? The code in
community postgresql certainly shouldn't have any EDB AS code in it.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Bruce Momjian]

Magnus Hagander wrote:
> On Thu, May 13, 2010 at 8:22 AM, Devrim G?ND?Z  wrote:
> > On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote:
> >>  4. EDB_NATIVE_LANG 
> >> Of course it is commented out with #ifdef, but do we have codes
> >> for EDB in core?
> >
> > I was about to raise similar thing, for the documentation:
> >
> > http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html
> >
> > This includes some references to EDB AS, which should be removed from
> > PostgreSQL official documentation, IMHO.
> 
> +1 on getting rid of those references.

Agreed.  When it was on pgFoundry, I had to mention that because it was
unclear who would be using it, but in /contrib we know this is for
community Postgres.  EnterpriseDB did contribute the code so I would
like to keep the code working for EnterpriseDB Advanced Server if that
is easy.

I have added SGML comments to comment out the text that mentions EDB
Advanced Server.  Is that enough?  Should I remove the text from the
SGML?  Should I move it to the bottom of the SGML?  Should I remove the
EnterpriseDB Advanced Server checks from the C code too?  I don't
remember having to deal with anything like this before, so I am unclear
how to proceed.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] make install fails due to "/bin/mkdir: missing operand"

[Peter Eisentraut]

On mån, 2010-05-10 at 11:40 -0400, Alvaro Herrera wrote:
> Excerpts from Kenichiro Tanaka's message of lun may 10 07:07:27 -0400 2010:
> 
> > Reproduce case:
> > #prefix and with-pgport are not important
> > ./configure --enable-nls='UFT_JP' --prefix=/home/p900/posgrehome
> > --with-pgport=1900
> > make && make install
> 
> I think this is pilot error, in the sense that it doesn't fail if you
> don't pass an invalid language name.  Maybe the bug is that we allow
> --enable-nls to pass down junk down to the install Makefile, instead of
> erroring out right there.

The language name in his example is obviously wrong, but in general this
case if valid.  Even if you use something "correct" like
--enable-nls='de es fr', but one subdirectory doesn't have any of those,
the mkdir call would fail.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] make install fails due to "/bin/mkdir: missing operand"

[Peter Eisentraut]

On mån, 2010-05-10 at 20:07 +0900, Kenichiro Tanaka wrote:
> Reproduce case:
> #prefix and with-pgport are not important
> ./configure --enable-nls='UFT_JP' --prefix=/home/p900/posgrehome
> --with-pgport=1900
> make && make install
> 
> make install
> log---
> :
> :
> make[3]: Leaving directory `/home/p900/postgresql-9.0beta1/src/timezone'
> /bin/mkdir: missing operand
> Try `/bin/mkdir --help' for more information.
> make[2]: *** [installdirs-po] Error 1
> make[2]: Leaving directory `/home/p900/postgresql-9.0beta1/src/backend'
> make[1]: *** [install] Error 2
> make[1]: Leaving directory `/home/p900/postgresql-9.0beta1/src'
> make: *** [install] Error 2
> --
> 

Fixed.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Magnus Hagander]

On Thu, May 13, 2010 at 3:27 PM, Marc G. Fournier  wrote:
> On Thu, 13 May 2010, Magnus Hagander wrote:
>
>> On Thu, May 13, 2010 at 2:04 AM, Marc G. Fournier  wrote:
>>>
>>> On Wed, 12 May 2010, Greg Stark wrote:
>>>
 I'm thinking I'll move -general (and the useless -novice) to another 
 folder. But I'm left wondering what to do with -admin and -performance. 
 They're a random mix of user content and developer content. I'll probably 
 move them along with -general but that means I won't be likely to see any 
 development discussion on them in the future
>>>
>>> There shouldn't be any dev discussions on them as it is ... that isn't 
>>> their mandate ... those are/were meant to be end-user lists, not developer 
>>> ones ...
>>
>> We know from experience that doesn't work. People just end up
>> crossposting, because they're not sure people are on both lists. And
>> then you want to move a discussion, which just means you have to CC in
>> both lists, leading to even more duplication.
>>
>> If there was a clear distinction between end-user and dev it might
>> make sense. That how commercial software companies tend to work -
>> don't let devs talk to end users. That's not how we work. Forcing
>> people to look in different places just throws hurdles in front of
>> those trying to help out.
>
> What *are* you talking about?  This doesn't seem to have anything related to 
> what I said :)
>
> All I was saying was that -performance and -admin are not development 
> discusion lists, not that developers aren't subscribed / talking on them ... 
> that doesn't make them any less end-user lists ...

Yes, and I'm saying there is no real difference between end-user,
development, admin and performance. The amount of crossover is so
large the distinction rapidly becomes pointless.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Marc G. Fournier]

On Thu, 13 May 2010, Magnus Hagander wrote:


On Thu, May 13, 2010 at 2:04 AM, Marc G. Fournier  wrote:

On Wed, 12 May 2010, Greg Stark wrote:


I'm thinking I'll move -general (and the useless -novice) to another folder. 
But I'm left wondering what to do with -admin and -performance. They're a 
random mix of user content and developer content. I'll probably move them along 
with -general but that means I won't be likely to see any development 
discussion on them in the future


There shouldn't be any dev discussions on them as it is ... that isn't their 
mandate ... those are/were meant to be end-user lists, not developer ones ...


We know from experience that doesn't work. People just end up
crossposting, because they're not sure people are on both lists. And
then you want to move a discussion, which just means you have to CC in
both lists, leading to even more duplication.

If there was a clear distinction between end-user and dev it might
make sense. That how commercial software companies tend to work -
don't let devs talk to end users. That's not how we work. Forcing
people to look in different places just throws hurdles in front of
those trying to help out.


What *are* you talking about?  This doesn't seem to have anything related 
to what I said :)


All I was saying was that -performance and -admin are not development 
discusion lists, not that developers aren't subscribed / talking on them 
... that doesn't make them any less end-user lists ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Tags missing from GIT mirror?

[Andrew Dunstan]

Peter Eisentraut wrote:

On tor, 2010-05-13 at 04:41 -0400, Andrew Dunstan wrote:
  

Right, and omitting tags was in fact one of the "features" of fromcvs
that made us use it, because any tool that tries to convert tags will
explode on our CVS tree, for reasons explained in the above paragraph.

We have also discussed this in more detail about three times before.
  
Well, yes, but I have been wondering if this has to be an all or nothing 
deal. How many tags can we not tie to a known tree in git? My suspicion 
is we can probably identify most of them quite well. If we can that 
would be nice.



http://archives.postgresql.org/pgsql-hackers/2008-04/msg00036.php

  


Quite so.  All the tags apparently causing problems are of no more than 
historical interest to us. But more recent tags, especially for 
currently maintained branches, are of interest.



cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Tags missing from GIT mirror?

[Peter Eisentraut]

On tor, 2010-05-13 at 04:41 -0400, Andrew Dunstan wrote:
> > Right, and omitting tags was in fact one of the "features" of fromcvs
> > that made us use it, because any tool that tries to convert tags will
> > explode on our CVS tree, for reasons explained in the above paragraph.
> >
> > We have also discussed this in more detail about three times before.
> 
> Well, yes, but I have been wondering if this has to be an all or nothing 
> deal. How many tags can we not tie to a known tree in git? My suspicion 
> is we can probably identify most of them quite well. If we can that 
> would be nice.

http://archives.postgresql.org/pgsql-hackers/2008-04/msg00036.php



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Simon Riggs]

On Thu, 2010-05-13 at 20:13 +0900, Fujii Masao wrote:
> On Thu, May 13, 2010 at 8:05 PM, Simon Riggs  wrote:
> > On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote:
> >> On Thu, May 13, 2010 at 7:22 PM, Simon Riggs  wrote:
> >> > On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote:
> >> >
> >> >> I was able to reproduce such a hang by not executing another
> >> >> transaction after rollback. In this case, walsender cannot replicate
> >> >> the rollback since it's not in the disk.
> >> >
> >> > WALWriter is not active?
> >>
> >> WALWriter is active, but unfortunately it doesn't flush all of the WAL
> >> records in wal_buffers. Please see XLogBackgroundFlush().
> >
> > Yes, I wrote it. It flushes after at most 3 cycles, stated in comments.
> 
> Yeah, what is worse is that RecordTransactionAbort() doesn't update the
> XLogCtl->asyncCommitLSN, so rollback might not be flushed even after at
> 3 cycles.

Well spotted. Fix applied, thanks both.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Simon Riggs]

On Thu, 2010-05-13 at 20:13 +0900, Fujii Masao wrote:
> On Thu, May 13, 2010 at 8:05 PM, Simon Riggs  wrote:
> > On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote:
> >> On Thu, May 13, 2010 at 7:22 PM, Simon Riggs  wrote:
> >> > On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote:
> >> >
> >> >> I was able to reproduce such a hang by not executing another
> >> >> transaction after rollback. In this case, walsender cannot replicate
> >> >> the rollback since it's not in the disk.
> >> >
> >> > WALWriter is not active?
> >>
> >> WALWriter is active, but unfortunately it doesn't flush all of the WAL
> >> records in wal_buffers. Please see XLogBackgroundFlush().
> >
> > Yes, I wrote it. It flushes after at most 3 cycles, stated in comments.
> 
> Yeah, what is worse is that RecordTransactionAbort() doesn't update the
> XLogCtl->asyncCommitLSN, so rollback might not be flushed even after at
> 3 cycles.

What you mean then is that there is a bug, not that it should work this
way. Will look.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Fujii Masao]

On Thu, May 13, 2010 at 8:05 PM, Simon Riggs  wrote:
> On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote:
>> On Thu, May 13, 2010 at 7:22 PM, Simon Riggs  wrote:
>> > On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote:
>> >
>> >> I was able to reproduce such a hang by not executing another
>> >> transaction after rollback. In this case, walsender cannot replicate
>> >> the rollback since it's not in the disk.
>> >
>> > WALWriter is not active?
>>
>> WALWriter is active, but unfortunately it doesn't flush all of the WAL
>> records in wal_buffers. Please see XLogBackgroundFlush().
>
> Yes, I wrote it. It flushes after at most 3 cycles, stated in comments.

Yeah, what is worse is that RecordTransactionAbort() doesn't update the
XLogCtl->asyncCommitLSN, so rollback might not be flushed even after at
3 cycles.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Simon Riggs]

On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote:
> On Thu, May 13, 2010 at 7:22 PM, Simon Riggs  wrote:
> > On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote:
> >
> >> I was able to reproduce such a hang by not executing another
> >> transaction after rollback. In this case, walsender cannot replicate
> >> the rollback since it's not in the disk.
> >
> > WALWriter is not active?
> 
> WALWriter is active, but unfortunately it doesn't flush all of the WAL
> records in wal_buffers. Please see XLogBackgroundFlush().

Yes, I wrote it. It flushes after at most 3 cycles, stated in comments.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Fujii Masao]

On Thu, May 13, 2010 at 7:22 PM, Simon Riggs  wrote:
> On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote:
>
>> I was able to reproduce such a hang by not executing another
>> transaction after rollback. In this case, walsender cannot replicate
>> the rollback since it's not in the disk.
>
> WALWriter is not active?

WALWriter is active, but unfortunately it doesn't flush all of the WAL
records in wal_buffers. Please see XLogBackgroundFlush().

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Simon Riggs]

On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote:

> I was able to reproduce such a hang by not executing another
> transaction after rollback. In this case, walsender cannot replicate
> the rollback since it's not in the disk.

WALWriter is not active?

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List traffic

[Magnus Hagander]

On Thu, May 13, 2010 at 2:04 AM, Marc G. Fournier  wrote:
> On Wed, 12 May 2010, Greg Stark wrote:
>
>> I'm thinking I'll move -general (and the useless -novice) to another folder. 
>> But I'm left wondering what to do with -admin and -performance. They're a 
>> random mix of user content and developer content. I'll probably move them 
>> along with -general but that means I won't be likely to see any development 
>> discussion on them in the future
>
> There shouldn't be any dev discussions on them as it is ... that isn't their 
> mandate ... those are/were meant to be end-user lists, not developer ones ...

We know from experience that doesn't work. People just end up
crossposting, because they're not sure people are on both lists. And
then you want to move a discussion, which just means you have to CC in
both lists, leading to even more duplication.

If there was a clear distinction between end-user and dev it might
make sense. That how commercial software companies tend to work -
don't let devs talk to end users. That's not how we work. Forcing
people to look in different places just throws hurdles in front of
those trying to help out.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add PGFILEDESC description to Makefiles for all /contrib

[Magnus Hagander]

On Thu, May 13, 2010 at 1:47 AM, Bruce Momjian  wrote:
> bruce wrote:
>> Tom Lane wrote:
>> > Bruce Momjian  writes:
>> > > All other binaries had such a designation, and all /contrib binaries
>> > > were missing them.  I assume I was doing cleanup.  You want the icon
>> > > removed from the backend makefile?
>> >
>> > Yes.  I'm prepared to believe that not having the icons set on the
>> > contrib executables was an oversight.  I'm much less prepared to assume
>> > that not marking the postgres executable was an oversight.  Again,
>> > unless you *know* that this change is needed and appropriate, now is
>> > not the time to be making it, and especially not without discussion.
>>
>> OK, done with attached patch, and I added a comment about why it is not
>> labeled.
>
> I did some research on PGFILEDESC and it does what I thought it does ---
> in embeds the 'ico' file into the executable in
> /pg/tools/msvc/Project.pm, and the image looks like the attached JPEG.
> The image is of a blue elephant head.
>
> So, currently, every binary uses that icon, except for the postmaster.
> Is that what we want?  You could make the argument that a daemon, like
> the postmaster, shouldn't have one, which I think is Tom's point.

It's pretty normal that daemons don't have icons.

One could argue the same for binaries that are normally never executed
by the user, just internally - but I don't think we have any of those
(we're talking EXE not DLL).

There is, AFAIK, no rule (hard or of-thumb) for "icons for GUI
programs only not commandline ones". Commandline tools usually have
it.

The argument to be made there is that when you go into the bin
directory you'll get greeted with a long list of identical elephants.
It would probably be better if we could have a *different* icon for
tools that the user is likely to execute himself - which is pretty
much just psql I think.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade code questions

[Magnus Hagander]

On Thu, May 13, 2010 at 8:22 AM, Devrim GÜNDÜZ  wrote:
> On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote:
>>  4. EDB_NATIVE_LANG 
>> Of course it is commented out with #ifdef, but do we have codes
>> for EDB in core?
>
> I was about to raise similar thing, for the documentation:
>
> http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html
>
> This includes some references to EDB AS, which should be removed from
> PostgreSQL official documentation, IMHO.

+1 on getting rid of those references.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Fujii Masao]

On Thu, May 13, 2010 at 6:47 PM, Simon Riggs  wrote:
> Rollbacks are always flushed to disk, so this explanation doesn't work.
> Even if it were it would take no longer than ~1 sec if everything were
> working correctly on the test system.

Yeah, rollbacks are always flushed sooner or later, but not *immediately*,
since RecordTransactionAbort() calls only XLogInsert() but not XLogFlush().
Until XLogFlush() is executed by another process, the WAL record of rollback
would stay in wal_buffers.

On the other hand, RecordTransactionCommit() calls XLogFlush(),
so commits are always flushed to the disk immediately.

> The "weird hang" is a lock wait and is perfectly normal in database
> systems. Robert says he hasn't checked whether it is reproduceable, so
> there is no evidence to show there is anything other than pilot error,
> at this point.

I was able to reproduce such a hang by not executing another transaction
after rollback. In this case, walsender cannot replicate the rollback
since it's not in the disk.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] weird hang while running in HS mode

[Simon Riggs]

On Thu, 2010-05-13 at 10:49 +0900, Fujii Masao wrote:
> On Thu, May 13, 2010 at 3:50 AM, Robert Haas  wrote:
> > rhaas=# rollback;
> > ROLLBACK
> >
> > So at this point, one would think that there are no locks hanging
> > around anywhere.  Back to the standby:
> >
> > rhaas=# select * from pgbench_accounts;
> > 
> 
> I think that this problem happens because the WAL record of ROLLBACK
> is not flushed to the disk immediately (i.e., until another transaction
> flushes the WAL records in wal_buffers) when we execute the ROLLBACK
> command. The walsender sends only the WAL records in the disk, so the
> standby server would not be able to replay the ROLLBACK and not release
> the lock.

Rollbacks are always flushed to disk, so this explanation doesn't work.
Even if it were it would take no longer than ~1 sec if everything were
working correctly on the test system.

The "weird hang" is a lock wait and is perfectly normal in database
systems. Robert says he hasn't checked whether it is reproduceable, so
there is no evidence to show there is anything other than pilot error,
at this point.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Retiring from the PostgreSQL core team

[Simon Riggs]

On Thu, 2010-05-13 at 00:24 -0400, Jan Wieck wrote:

> Over the past years I have not been able to dedicate as much time to
> PostgreSQL as everyone would have liked. The main reason for that was
> that I was swamped with other work and private matters and simply
> didn't have time. I did follow the mailing lists but did not
> participate much.
> 
> Looking at my publicly visible involvement over the last two years or
> so, there is little that would justify me being on the core team
> today.
> I was not involved in the release process, in patch reviewing,
> organizing and have contributed little.

My feeling, which I'm sure would be supported by a great many people, is
that you have made huge contributions to PostgreSQL, all very much
appreciated.

Yes, that may not have been visible to all because much of that was via
the Slony project, though I know of your quieter contributions to other
major projects. Slony alone has been the backbone of PostgreSQL across
many releases and will continue to be important in the future also.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Retiring from the PostgreSQL core team

[Andrew Dunstan]

Jan Wieck wrote:

To whom it may concern,

this is to inform the PostgreSQL community of my retirement from my
PostgreSQL core team position.

Over the past years I have not been able to dedicate as much time to
PostgreSQL as everyone would have liked. The main reason for that was
that I was swamped with other work and private matters and simply didn't
have time. I did follow the mailing lists but did not participate much.



Your good humor and technical brilliance have been sorely missed.



Looking at my publicly visible involvement over the last two years or
so, there is little that would justify me being on the core team today.
I was not involved in the release process, in patch reviewing,
organizing and have contributed little.

However, in contrast to other previous core team members, I do not plan
to disappear. Very much to the contrary. I am right now picking up some
things that have long been on my TODO wish list and Afilias is doubling
down on the commitment to PostgreSQL and Slony. We can and should talk
about that stuff next week at PGCon in Ottawa. I will also stay in close
contact with the remaining core team members, many of whom have become
very good friends over the past 15 years.

The entire core team, me included, hoped that it wouldn't come to this
and that I could have returned to active duty earlier. Things in my
little sub universe didn't change as fast as we all hoped and we all
think it is best now that I focus on getting back to speed and do some
serious hacking.



We hope for many more good things from you yet!



I hope to see many of you in Ottawa.





You'll certainly see me!

Best wishes and many thanks for your good work over so many years.

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Tags missing from GIT mirror?

[Andrew Dunstan]

Peter Eisentraut wrote:

On ons, 2010-05-12 at 16:11 -0400, Andrew Dunstan wrote:
  
Of course, we might also find some other brokenness if we try to import 
all the tags. Also, be aware of this (from 
):


Differences between CVS and git branch/tag models: CVS allows a
branch or tag to be created from arbitrary combinations of source
revisions from multiple source branches. It even allows file
revisions that were never contemporaneous to be added to a single
branch/tag. Git, on the other hand, only allows the full source
tree, as it existed at some instant in the history, to be branched
or tagged as a unit. Moreover, the ancestry of a git revision makes
implications about the contents of that revision. This difference
means that it is fundamentally impossible to represent an arbitrary
CVS history in a git repository 100% faithfully. 



Right, and omitting tags was in fact one of the "features" of fromcvs
that made us use it, because any tool that tries to convert tags will
explode on our CVS tree, for reasons explained in the above paragraph.

We have also discussed this in more detail about three times before.

  


Well, yes, but I have been wondering if this has to be an all or nothing 
deal. How many tags can we not tie to a known tree in git? My suspicion 
is we can probably identify most of them quite well. If we can that 
would be nice.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Tags missing from GIT mirror?

[Peter Eisentraut]

On ons, 2010-05-12 at 16:11 -0400, Andrew Dunstan wrote:
> Of course, we might also find some other brokenness if we try to import 
> all the tags. Also, be aware of this (from 
> ):
> 
> Differences between CVS and git branch/tag models: CVS allows a
> branch or tag to be created from arbitrary combinations of source
> revisions from multiple source branches. It even allows file
> revisions that were never contemporaneous to be added to a single
> branch/tag. Git, on the other hand, only allows the full source
> tree, as it existed at some instant in the history, to be branched
> or tagged as a unit. Moreover, the ancestry of a git revision makes
> implications about the contents of that revision. This difference
> means that it is fundamentally impossible to represent an arbitrary
> CVS history in a git repository 100% faithfully. 

Right, and omitting tags was in fact one of the "features" of fromcvs
that made us use it, because any tool that tries to convert tags will
explode on our CVS tree, for reasons explained in the above paragraph.

We have also discussed this in more detail about three times before.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers