Re: [HACKERS] 8.5 development schedule

2009-07-02 Thread Guillaume Smet
On Wed, Jul 1, 2009 at 11:42 PM, Andrew Dunstanand...@dunslane.net wrote:
 You have correctly identified the requirement in the sentence quoted above.
 I for one am quite prepared to support core or some person designated by
 core having such authority. I agree with you that without something like
 that not much will improve.

That's the role of a release manager. Perhaps it's time to think about
designating one for each release to endorse this responsability.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] HEAD is open for 8.5 development

2009-07-02 Thread Peter Eisentraut
On Thursday 02 July 2009 05:29:58 Robert Haas wrote:
 If you are taking a look at
 any of the patches submitted for this CommitFest, please put your name
 next to them on the wiki.  Please ALSO add a comment saying something
 like tgl says: I am reviewing this, no need to assign a round-robin
 reviewer.

 During the last CommitFest, we had occasional shenanigans where
 someone would stick the name of a committer or other community member
 next to a patch just because they had made some comments on it.  But
 the person whose name got stuck there didn't necessarily know about it
 or feel any responsibility to conduct a complete review.  So it would
 be helpful if everyone could make sure to add a comment when claiming
 patches.

Um, how about no one sticks no one's name to any patch unless it is themselves 
or they are the commit fest management team and the reviewer has explicitly 
OK'ed it.

Let's not begin to use several levels of Reviewer: X, X says: I will really 
review it., RH says: This X is legit., Reconfirmed-by: X.  The wiki is 
weird enough to use that we shouldn't need to edit it in more than one place 
for each logical step.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Dave Page
On Thu, Jul 2, 2009 at 1:21 AM, Josh Berkusj...@agliodbs.com wrote:

 Robert, I have not reviewed your software.  Do I need a login or something?

It's not in productions yet as It's not been documented or given a
public hostname/servicename, both of which are requirements for any
postgresql.org services. As far as I'm aware, there's been no code
review yet either, which would probably be a good idea.

Which reminds me - that pentabarf installation is getting dangerously
close to being shut down...

-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Robert Haas

On Jul 2, 2009, at 3:41 AM, Dave Page dp...@pgadmin.org wrote:


On Thu, Jul 2, 2009 at 1:21 AM, Josh Berkusj...@agliodbs.com wrote:

Robert, I have not reviewed your software.  Do I need a login or  
something?


It's not in productions yet as It's not been documented or given a
public hostname/servicename, both of which are requirements for any
postgresql.org services. As far as I'm aware, there's been no code
review yet either, which would probably be a good idea.

Which reminds me - that pentabarf installation is getting dangerously
close to being shut down...


How do you recommend that we try to address these issues?

...Robert

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Dave Page
On Thu, Jul 2, 2009 at 12:04 PM, Robert Haasrobertmh...@gmail.com wrote:
 On Jul 2, 2009, at 3:41 AM, Dave Page dp...@pgadmin.org wrote:

 On Thu, Jul 2, 2009 at 1:21 AM, Josh Berkusj...@agliodbs.com wrote:

 Robert, I have not reviewed your software.  Do I need a login or
 something?

 It's not in productions yet as It's not been documented or given a
 public hostname/servicename, both of which are requirements for any
 postgresql.org services. As far as I'm aware, there's been no code
 review yet either, which would probably be a good idea.

 Which reminds me - that pentabarf installation is getting dangerously
 close to being shut down...

 How do you recommend that we try to address these issues?

Suggest a suitable name by which we can address the service (we don't
use the internal names like coridan because things can get moved
around), and I can set that up in a few minutes.

The documentation is the important bit as we don't deploy any service
without proper documentation any more (see pgFoundry for reasons why
not). We don't have a fixed format for that - what we need is a
description of what software is installed, how it's configured, and so
on. Enough that any of the sysadmin team can figure out in a couple of
minutes where the database is and how to access it, or what webserver
is being used and how to restart it etc. It should be enough that in a
pinch we can rebuild the server without lots of head-scratching.

We also need a list of key config files, which will be added to our
autobackup system to ensure we have copies and some change tracking.


-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 7:11 AM, Dave Pagedp...@pgadmin.org wrote:
 On Thu, Jul 2, 2009 at 12:04 PM, Robert Haasrobertmh...@gmail.com wrote:
 On Jul 2, 2009, at 3:41 AM, Dave Page dp...@pgadmin.org wrote:

 On Thu, Jul 2, 2009 at 1:21 AM, Josh Berkusj...@agliodbs.com wrote:

 Robert, I have not reviewed your software.  Do I need a login or
 something?

 It's not in productions yet as It's not been documented or given a
 public hostname/servicename, both of which are requirements for any
 postgresql.org services. As far as I'm aware, there's been no code
 review yet either, which would probably be a good idea.

 Which reminds me - that pentabarf installation is getting dangerously
 close to being shut down...

 How do you recommend that we try to address these issues?

 Suggest a suitable name by which we can address the service (we don't
 use the internal names like coridan because things can get moved
 around), and I can set that up in a few minutes.

pgcommitfest?  or just commitfest?

 The documentation is the important bit as we don't deploy any service
 without proper documentation any more (see pgFoundry for reasons why
 not). We don't have a fixed format for that - what we need is a
 description of what software is installed, how it's configured, and so
 on. Enough that any of the sysadmin team can figure out in a couple of
 minutes where the database is and how to access it, or what webserver
 is being used and how to restart it etc. It should be enough that in a
 pinch we can rebuild the server without lots of head-scratching.

OK.  Unfortunately, it's been a while said I did it, but it was mostly
a matter of installing the right set of ports, mostly Perl packages
like Template and Date::Calc.  I will try to write something up.

 We also need a list of key config files, which will be added to our
 autobackup system to ensure we have copies and some change tracking.

OK.  That should be easy to document.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query progress indication - an implementation

2009-07-02 Thread Greg Stark
On Thu, Jul 2, 2009 at 2:32 AM, Bruce Momjianbr...@momjian.us wrote:
 I think the only resonable solution would be to consider the estimated
 cost of each node and then compute what percentage complete each node
 is.


Well you can do better for some nodes. A sequential scan for example
can tell you exactly what percentage of the way through its scan it
is. A sort node that's fnished the sort can produce an value based on
both the estimate of the relative costs of the sort vs reading the
results and the actual percentage progress reading the results.

So I think it has to come down to another ExecProcNode method the way
I had it arranged in my patch that actually implemented this.

I was partly waiting for the other patch which multiplexed signals
onto fewer actual unix signals to go through. And for XML explain
plans to go through. Once we have those then I think my patch is
actually nearly there, it just needs some additional tweaking of the
heuristics for more plan types.

Then comes the fun part of figuring out a useful UI for psql and
pgadmin. Personally I'm happy for psql to just print the plan whenever
the user hits siginfo. I think an apt-style curses progress bar would
be unecessarily heavyweight for the lightweight vision I have for
psql. But I know others have more ambitious visions for psql.



-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] HEAD is open for 8.5 development

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 2:42 AM, Peter Eisentrautpete...@gmx.net wrote:
 On Thursday 02 July 2009 05:29:58 Robert Haas wrote:
 If you are taking a look at
 any of the patches submitted for this CommitFest, please put your name
 next to them on the wiki.  Please ALSO add a comment saying something
 like tgl says: I am reviewing this, no need to assign a round-robin
 reviewer.

 During the last CommitFest, we had occasional shenanigans where
 someone would stick the name of a committer or other community member
 next to a patch just because they had made some comments on it.  But
 the person whose name got stuck there didn't necessarily know about it
 or feel any responsibility to conduct a complete review.  So it would
 be helpful if everyone could make sure to add a comment when claiming
 patches.

 Um, how about no one sticks no one's name to any patch unless it is themselves
 or they are the commit fest management team and the reviewer has explicitly
 OK'ed it.

I'm all in favor of that.

 Let's not begin to use several levels of Reviewer: X, X says: I will really
 review it., RH says: This X is legit., Reconfirmed-by: X.  The wiki is
 weird enough to use that we shouldn't need to edit it in more than one place
 for each logical step.

Well, IIRC, Tom was very insistent when we first set this up that the
last column of the chart should be labelled Reviewers rather than
Reviewer, so as not to project the idea that if there already is a
reviewer, then no other reviewers need apply.  And in fact we do
sometimes assign multiple reviewers, if the first one gets busy and
can't continue reviewing or if the first one flakes out and never
posts a review at all or if the patch needs review from somebody else
with a different skillset or if someone who has not volunteered to be
a rrreviewer takes an interest in a patch and volunteers to review
that specific thing or if (ha ha ha) we have enough rrreviewers to
assign extras to certain patches.

So I think it's important to clarify the distinction between when a
committer is planning to look at the patch but maybe somebody else
should too, and when the committer is planning to take exclusive
responsibility for the patch and nobody else should bother touching
it.  Unless, of course, we want to make the rule that if a committer's
name is next to a patch, then that automatically means that no help
from anyone else is needed.  But that seems like a policy that would
inevitably give rise to exceptions.

Also, in general, I have found that when trying to recall the status
of a patch, the comment trail is a lot more useful than the main
summary line.  It's a lot easier to look at the comment trail and try
to remember whether anything has happened since then than it is to
look at the at the summary line and try to remember whether it matches
current reality.  Also, if people date-stamp their comments (which
will happen automatically if we get over to my new system), it makes
it possible to get a sense of how long it's been since a certain state
change took place.  Right now, I can look at the Wiki page and say,
Oh, Peter is looking at a bunch of patches, great.  And I know that
it's recent because your name wasn't there a couple of days ago, and I
don't really care whether you're going to commit them soon because the
CommitFest hasn't even started yet.  But by the first of August I
suspect there will be so much activity that without a comment trail it
will be impossible to reconstruct what is going on.  So I'd like to
encourage not just committers claiming patches but ANYONE who updates
ANYTHING to leave a comment with the details.

Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Joshua Tolley
On Thu, Jul 02, 2009 at 08:41:27AM +0100, Dave Page wrote:
 As far as I'm aware, there's been no code
 review yet either, which would probably be a good idea.

I don't have loads of time in the coming days, but IIRC I've taken a glance at
a past version of the code, and would be willing to do so again, if it would
be useful.

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com


signature.asc
Description: Digital signature


[HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
I'd like to return to the project I suggested here:
http://archives.postgresql.org/message-id/18653.1239741...@sss.pgh.pa.us
of getting rid of plpgsql's private lexer and having it use the core
lexer instead.  This will require making the core lexer re-entrant,
which is not possible with our oldest supported version of flex
(2.5.4a).  A look at the flex change history indicates that %reentrant
was added in 2.5.6 but nasty bugs were being fixed up to 2.5.30;
so the new minimum supported version would probably be 2.5.33
(2.5.31 was kinda broken for other reasons, and there was no 2.5.32).

Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] tsvector extraction patch

2009-07-02 Thread Hans-Juergen Schoenig -- PostgreSQL

hello,

i made a small patch which i found useful for my personal tasks.
it would be nice to see this in 8.5. if not core then maybe contrib.
it transforms a tsvector to table format which is really nice for text 
processing and comparison.


test=# SELECT * FROM tsvcontent(to_tsvector('english', 'i am pretty sure 
this is a good patch'));

 lex   | rank
+--
good   |8
patch  |9
pretti |3
sure   |4
(4 rows)

   many thanks,

  hans

--
Cybertec Schoenig  Schoenig GmbH
Reyergasse 9 / 2
A-2700 Wiener Neustadt
Web: www.postgresql-support.de

diff -dcrpN postgresql-8.4.0.old/contrib/Makefile postgresql-8.4.0/contrib/Makefile
*** postgresql-8.4.0.old/contrib/Makefile	2009-03-26 00:20:01.0 +0100
--- postgresql-8.4.0/contrib/Makefile	2009-06-29 11:03:04.0 +0200
*** WANTED_DIRS = \
*** 39,44 
--- 39,45 
  		tablefunc	\
  		test_parser	\
  		tsearch2	\
+ 		tsvcontent	\
  		vacuumlo
  
  ifeq ($(with_openssl),yes)
diff -dcrpN postgresql-8.4.0.old/contrib/tsvcontent/Makefile postgresql-8.4.0/contrib/tsvcontent/Makefile
*** postgresql-8.4.0.old/contrib/tsvcontent/Makefile	1970-01-01 01:00:00.0 +0100
--- postgresql-8.4.0/contrib/tsvcontent/Makefile	2009-06-29 11:20:21.0 +0200
***
*** 0 
--- 1,19 
+ # $PostgreSQL: pgsql/contrib/tablefunc/Makefile,v 1.9 2007/11/10 23:59:51 momjian Exp $
+ 
+ MODULES = tsvcontent
+ DATA_built = tsvcontent.sql
+ DATA = uninstall_tsvcontent.sql
+ 
+ 
+ SHLIB_LINK += $(filter -lm, $(LIBS))
+ 
+ ifdef USE_PGXS
+ PG_CONFIG = pg_config
+ PGXS := $(shell $(PG_CONFIG) --pgxs)
+ include $(PGXS)
+ else
+ subdir = contrib/tsvcontent
+ top_builddir = ../..
+ include $(top_builddir)/src/Makefile.global
+ include $(top_srcdir)/contrib/contrib-global.mk
+ endif
diff -dcrpN postgresql-8.4.0.old/contrib/tsvcontent/tsvcontent.c postgresql-8.4.0/contrib/tsvcontent/tsvcontent.c
*** postgresql-8.4.0.old/contrib/tsvcontent/tsvcontent.c	1970-01-01 01:00:00.0 +0100
--- postgresql-8.4.0/contrib/tsvcontent/tsvcontent.c	2009-06-29 11:18:35.0 +0200
***
*** 0 
--- 1,169 
+ #include postgres.h
+ 
+ #include fmgr.h
+ #include funcapi.h
+ #include miscadmin.h
+ #include executor/spi.h
+ #include lib/stringinfo.h
+ #include nodes/nodes.h
+ #include utils/builtins.h
+ #include utils/lsyscache.h
+ #include utils/syscache.h
+ #include utils/memutils.h
+ #include tsearch/ts_type.h
+ #include tsearch/ts_utils.h
+ #include catalog/pg_type.h
+ 
+ #include tsvcontent.h
+ 
+ PG_MODULE_MAGIC;
+ 
+ PG_FUNCTION_INFO_V1(tsvcontent);
+ 
+ Datum
+ tsvcontent(PG_FUNCTION_ARGS)
+ {
+ 	FuncCallContext 	*funcctx;
+ 	TupleDesc		ret_tupdesc;
+ 	AttInMetadata		*attinmeta;
+ 	int			call_cntr;
+ 	int			max_calls;
+ 	ts_to_txt_fctx		*fctx;
+ 	Datum			result[2];
+ 	bool			isnull[2] = { false, false };
+ 	MemoryContext 		oldcontext;
+ 
+ 	/* input value containing the TS vector */
+ 	TSVector	in = PG_GETARG_TSVECTOR(0);
+ 
+ 	/* stuff done only on the first call of the function */
+ 	if (SRF_IS_FIRSTCALL())
+ 	{
+ 		TupleDesc	tupdesc;
+ 		int		i, j;
+ 		char		*wepv_base;
+ 
+ 		/* create a function context for cross-call persistence */
+ 		funcctx = SRF_FIRSTCALL_INIT();
+ 
+ 		/*
+ 		 * switch to memory context appropriate for multiple function calls
+ 		 */
+ 		oldcontext = MemoryContextSwitchTo(funcctx-multi_call_memory_ctx);
+ 
+ 		switch (get_call_result_type(fcinfo, NULL, tupdesc))
+ 		{
+ 			case TYPEFUNC_COMPOSITE:
+ /* success */
+ break;
+ 			case TYPEFUNC_RECORD:
+ /* failed to determine actual type of RECORD */
+ ereport(ERROR,
+ 		(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ 		errmsg(function returning record called in context 
+ that cannot accept type record)));
+ break;
+ 			default:
+ /* result type isn't composite */
+ elog(ERROR, return type must be a row type);
+ break;
+ 		}
+ 
+ 		/* make sure we have a persistent copy of the tupdesc */
+ 		tupdesc = CreateTupleDescCopy(tupdesc);
+ 
+ 		/*
+ 		 * Generate attribute metadata needed later to produce tuples from raw
+ 		 * C strings
+ 		 */
+ 		attinmeta = TupleDescGetAttInMetadata(tupdesc);
+ 		funcctx-attinmeta = attinmeta;
+ 
+ 		/* allocate memory */
+ 		fctx = (ts_to_txt_fctx *) palloc(sizeof(ts_to_txt_fctx));
+ 
+ 		wepv_base = (char *)in + offsetof(TSVectorData, entries) + in-size * sizeof(WordEntry);
+ 		
+ 		fctx-n_tsvt = 0;
+ 		for (i = 0; i  in-size; i++)
+ 		{
+ 			if (in-entries[i].haspos)
+ 			{
+ WordEntryPosVector *wepv = (WordEntryPosVector *)
+ (wepv_base + in-entries[i].pos + SHORTALIGN(in-entries[i].len));
+ 
+ fctx-n_tsvt += wepv-npos;
+ 			}
+ 			else
+ fctx-n_tsvt++;
+ 		}
+ 
+ 		fctx-tsvt = palloc(fctx-n_tsvt * sizeof(tsvec_tuple));
+ 
+ 		for (i = 0, j = 0; i  in-size; i++)
+ 		{
+ 			int pos = in-entries[i].pos;
+ 			int len = in-entries[i].len;
+ 
+ 			if (in-entries[i].haspos)
+ 			{
+ WordEntryPosVector *wepv = (WordEntryPosVector *)
+ 

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Andrew Dunstan



Tom Lane wrote:

I'd like to return to the project I suggested here:
http://archives.postgresql.org/message-id/18653.1239741...@sss.pgh.pa.us
of getting rid of plpgsql's private lexer and having it use the core
lexer instead.  This will require making the core lexer re-entrant,
which is not possible with our oldest supported version of flex
(2.5.4a).  A look at the flex change history indicates that %reentrant
was added in 2.5.6 but nasty bugs were being fixed up to 2.5.30;
so the new minimum supported version would probably be 2.5.33
(2.5.31 was kinda broken for other reasons, and there was no 2.5.32).

Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?


  




I think it would need to be benchmarked. My faint recollection is that 
the re-entrant lexers are slower.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote:
 
 so the new minimum supported version would probably be 2.5.33
 (2.5.31 was kinda broken for other reasons, and there was no
 2.5.32).
 
 Since 2.5.33 is now over three years old, this does not seem like an
 onerous requirement, but I thought I'd better ask if anyone has an
 objection?
 
You'd be causing problems for SuSE Enterprise users, like us:
 
kgri...@ccdev-db:/etc/init.d cat /etc/SuSE-release
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 2
kgri...@ccdev-db:/etc/init.d flex --version
flex 2.5.31
 
Just how broken is the version we're using?
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Dave Page
On Thu, Jul 2, 2009 at 3:22 PM, Joshua Tolleyeggyk...@gmail.com wrote:
 On Thu, Jul 02, 2009 at 08:41:27AM +0100, Dave Page wrote:
 As far as I'm aware, there's been no code
 review yet either, which would probably be a good idea.

 I don't have loads of time in the coming days, but IIRC I've taken a glance at
 a past version of the code, and would be willing to do so again, if it would
 be useful.

If you can look over it, that would be great. i'm not really qualified
to review perl code, and we always prefer to have at least two sets of
eyeballs on anything that we put into production for obvious reasons.

-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Dickson S. Guedes
Em Thu, 02 Jul 2009 00:26:14 -0300, Brendan Jurd dire...@gmail.com  
escreveu:

I imagine that migration would basically be converting the wiki data
into SQL, so I would need the database schema underlying the new CF
app.


How about parsing wiki content and create a migration script based on
pgcommitfest tables sctructure [1]?

[1]  
http://git.postgresql.org/gitweb?p=pgcommitfest.git;a=blob;f=etc/table.sql;h=c60a298c863ef3e88dcfd16572781d2b435ca629;hb=HEAD


--
Dickson S. Guedes
mail/xmpp: gue...@guedesoft.net - skype: guediz
http://guedesoft.net - http://www.postgresql.org.br

http://www.rnp.br/keyserver/pks/lookup?search=0x8F3E3C06D428D10A

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] tsvector extraction patch

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 10:13 AM, Hans-Juergen Schoenig --
PostgreSQLpostg...@cybertec.at wrote:
 hello,

 i made a small patch which i found useful for my personal tasks.
 it would be nice to see this in 8.5. if not core then maybe contrib.
 it transforms a tsvector to table format which is really nice for text
 processing and comparison.

 test=# SELECT * FROM tsvcontent(to_tsvector('english', 'i am pretty sure
 this is a good patch'));
  lex   | rank
 +--
 good   |    8
 patch  |    9
 pretti |    3
 sure   |    4
 (4 rows)

   many thanks,

      hans

If you'd like this reviewed for the next CommitFest, please add it to
the wiki here:

http://wiki.postgresql.org/wiki/CommitFestOpen

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] tsvector extraction patch

2009-07-02 Thread Oleg Bartunov
I have simple solution 
http://www.sai.msu.su/~megera/wiki/2008-12-17



On Thu, 2 Jul 2009, Hans-Juergen Schoenig -- PostgreSQL wrote:


hello,

i made a small patch which i found useful for my personal tasks.
it would be nice to see this in 8.5. if not core then maybe contrib.
it transforms a tsvector to table format which is really nice for text 
processing and comparison.


test=# SELECT * FROM tsvcontent(to_tsvector('english', 'i am pretty sure this 
is a good patch'));

lex   | rank
+--
good   |8
patch  |9
pretti |3
sure   |4
(4 rows)

  many thanks,

 hans




Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: 
 Tom Lane t...@sss.pgh.pa.us wrote:
  
 so the new minimum supported version would probably be 2.5.33
 
We still have five SuSE Enterprise 9 boxes in use as database servers;
while we've got someone tasked with replacing them, I wonder if that
version is so dead that we want to abandon support for it.
 
kgri...@inhouseapps:~ cat /etc/SuSE-release
SUSE LINUX Enterprise Server 9 (i586)
VERSION = 9
PATCHLEVEL = 3
kgri...@inhouseapps:~ flex --version
flex version 2.5.4
 
Of course, one could always download source for a more recent version
and build and install flex that way.  (We do that for PostgreSQL
itself and libpoppler.)  If someone's willing to build PostgreSQL,
perhaps it's reasonable to require them to build flex first?
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes:
 I think it would need to be benchmarked. My faint recollection is that 
 the re-entrant lexers are slower.

The flex documentation states in so many words:

The option `--reentrant' does not affect the performance of the scanner.

Do you feel a need to verify their claim?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Dickson S. Guedes
Em Thu, 02 Jul 2009 00:15:22 -0300, Robert Haas robertmh...@gmail.com  
escreveu:

The main problem is that my fine software only contains test data at
this point.  If we have any volunteers who are available to migrate
the information from the Wiki to my app (which will involve a fair
amount of legwork), then I recommend that we accept their help as it
will make things easier for the CommitFest management team...  which I
am all in favor of, especially now that I'm on that team.  On the
other hand, if we don't have any volunteers, then I recommend that we
continue to use the Wiki for this CommitFest but make sure that all
patches for the next CommitFest, and any that follow, get added via
the app.

So, any volunteers?


I don't know if this tool will be approved yet, but I'm working on
copying the Wiki entries of CommitFest to pgcommitfest.

Until now i created the following CommitFest Topics based on topics in
the Wiki:

* Contrib modules
* EXPLAIN
* Error Reporting
* Instrumentation
* Miscellaneous 
* My New Topic
* Performance   
* Replication
* SQL language features
* Security

All the patches on Miscellaneous topic in Wiki was copied to coridan, but i
couldn't copy comments of patches thath have one.

Would be nice if a theres is a way to order by some column like Patch Name,
Topic, Status, Author and Last Activity for example. Some descriptions was
truncated because de maxsize of textbox.

regards,
--
Dickson S. Guedes
mail/xmpp: gue...@guedesoft.net - skype: guediz
http://guedesoft.net - http://www.postgresql.org.br

http://www.rnp.br/keyserver/pks/lookup?search=0x8F3E3C06D428D10A

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Andrew Dunstan



Tom Lane wrote:

Andrew Dunstan and...@dunslane.net writes:
  
I think it would need to be benchmarked. My faint recollection is that 
the re-entrant lexers are slower.



The flex documentation states in so many words:

The option `--reentrant' does not affect the performance of the scanner.

Do you feel a need to verify their claim?


  


No, I'll take their word for it. I must have been thinking of something 
else.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Dave Page
On Thu, Jul 2, 2009 at 12:48 PM, Robert Haasrobertmh...@gmail.com wrote:
 On Thu, Jul 2, 2009 at 7:11 AM, Dave Pagedp...@pgadmin.org wrote:
 Suggest a suitable name by which we can address the service (we don't
 use the internal names like coridan because things can get moved
 around), and I can set that up in a few minutes.

 pgcommitfest?  or just commitfest?

 commitfest.postgresql.org
Server: mx3.hub.org
Address:206.223.169.73#53

commitfest.postgresql.org   canonical name = coridan.postgresql.org.
Name:   coridan.postgresql.org
Address: 98.129.198.114

 OK.  Unfortunately, it's been a while said I did it, but it was mostly
 a matter of installing the right set of ports, mostly Perl packages
 like Template and Date::Calc.  I will try to write something up.

Thanks.


-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Alvaro Herrera
Kevin Grittner wrote:
 Tom Lane t...@sss.pgh.pa.us wrote:
  
  so the new minimum supported version would probably be 2.5.33
  (2.5.31 was kinda broken for other reasons, and there was no
  2.5.32).
  
  Since 2.5.33 is now over three years old, this does not seem like an
  onerous requirement, but I thought I'd better ask if anyone has an
  objection?
  
 You'd be causing problems for SuSE Enterprise users, like us:
  
 kgri...@ccdev-db:/etc/init.d cat /etc/SuSE-release
 SUSE Linux Enterprise Server 10 (x86_64)
 VERSION = 10
 PATCHLEVEL = 2
 kgri...@ccdev-db:/etc/init.d flex --version
 flex 2.5.31

Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex.  Since you do some
experimental playing from CVS I guess you could install flex from source
in your environment only ...

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 kgri...@ccdev-db:/etc/init.d flex --version
 flex 2.5.31
 
 Just how broken is the version we're using?

You might want to take that up with SuSE.  The flex NEWS file just cites
numerous bug and security fixes from .31 to .33.

Our CVS logs show half a dozen changes we made some years ago to make
our code work with 2.5.31, but I can't tell from the log entries whether
those were workarounds for bugs in flex or fixes for obsolete coding on
our side.

We could try setting the min version at 2.5.31, though, until we find
some specific reason it doesn't work for us.  2.5.31 is significantly
older (release date 2003-04-01, vs 2006-02-20), so there's definitely
more chance of people having it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Kevin Grittner wrote:
 Tom Lane t...@sss.pgh.pa.us wrote:
 Since 2.5.33 is now over three years old, this does not seem like an
 onerous requirement, but I thought I'd better ask if anyone has an
 objection?
 
 You'd be causing problems for SuSE Enterprise users, like us:

 Keep in mind that if you build from a tarball, the lexer has already
 been run elsewhere and you don't need flex.

Right, this will only affect people doing development or otherwise
building from a CVS pull.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Andrew Dunstan



Tom Lane wrote:

Alvaro Herrera alvhe...@commandprompt.com writes:
  

Kevin Grittner wrote:


Tom Lane t...@sss.pgh.pa.us wrote:
  

Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?


You'd be causing problems for SuSE Enterprise users, like us:
  


  

Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex.



Right, this will only affect people doing development or otherwise
building from a CVS pull.


  


Of course, that includes the whole buildfarm. We might need to ask some 
people to upgrade there.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: 
 
 Keep in mind that if you build from a tarball, the lexer has already
 been run elsewhere and you don't need flex.
 
Does that hold for the daily snapshots?  If so, I should be good.  My
workstation is OK on flex version.  (I run kubuntu on my desktop.)
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes:
 Tom Lane wrote:
 Right, this will only affect people doing development or otherwise
 building from a CVS pull.

 Of course, that includes the whole buildfarm. We might need to ask some 
 people to upgrade there.

Yes.  What I was thinking of doing was committing a configure change to
reject flex  2.5.31, and waiting to see how much of the buildfarm goes
red.

One point here is that if buildfarm owners update to more recent flex,
that will also affect their testing of back branches.  I think this
should be okay --- AFAICT from the CVS logs, all of the stuff we did to
be compatible with 2.5.31 was pre-7.4 and so all supported versions
should still work.  But we'd find out for sure ;-)

regards, tom lane

PS: speaking of buildfarm, I don't see any REL8_4_STABLE entries yet.
Need to ping the owners anyway, apparently.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Alvaro Herrera
Kevin Grittner wrote:
 Alvaro Herrera alvhe...@commandprompt.com wrote: 
  
  Keep in mind that if you build from a tarball, the lexer has already
  been run elsewhere and you don't need flex.
  
 Does that hold for the daily snapshots?  If so, I should be good.

Yes, the snapshots have the derived files too.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 11:05 AM, Dickson S. Guedeslis...@guedesoft.net wrote:
 Em Thu, 02 Jul 2009 00:15:22 -0300, Robert Haas robertmh...@gmail.com
 escreveu:

 The main problem is that my fine software only contains test data at
 this point.  If we have any volunteers who are available to migrate
 the information from the Wiki to my app (which will involve a fair
 amount of legwork), then I recommend that we accept their help as it
 will make things easier for the CommitFest management team...  which I
 am all in favor of, especially now that I'm on that team.  On the
 other hand, if we don't have any volunteers, then I recommend that we
 continue to use the Wiki for this CommitFest but make sure that all
 patches for the next CommitFest, and any that follow, get added via
 the app.

 So, any volunteers?

 I don't know if this tool will be approved yet, but I'm working on
 copying the Wiki entries of CommitFest to pgcommitfest.

 Until now i created the following CommitFest Topics based on topics in
 the Wiki:

 * Contrib modules
 * EXPLAIN
 * Error Reporting
 * Instrumentation
 * Miscellaneous
 * My New Topic
 * Performance
 * Replication
 * SQL language features
 * Security

 All the patches on Miscellaneous topic in Wiki was copied to coridan, but i
 couldn't copy comments of patches thath have one.

 Would be nice if a theres is a way to order by some column like Patch Name,
 Topic, Status, Author and Last Activity for example. Some descriptions was
 truncated because de maxsize of textbox.

Brendan Jurd I think is working on an awk script - you probably want
to coordinate with him...

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Greg Stark
 Tom Lane wrote:
 Right, this will only affect people doing development or otherwise
 building from a CVS pull.

Someone doing a CVS pull already needs a specific recent version of
autoconf anyways. How old is this version of flex compared to the
version of autoconf we require?

-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 11:39 AM, Greg Starkgsst...@mit.edu wrote:
 Tom Lane wrote:
 Right, this will only affect people doing development or otherwise
 building from a CVS pull.

 Someone doing a CVS pull already needs a specific recent version of
 autoconf anyways. How old is this version of flex compared to the
 version of autoconf we require?

Really?  configure is in CVS, so I don't think you need autoconf
unless you want to modify configure.in.  Which people may want to do,
but certainly not for every patch.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Jul 2, 2009 at 11:39 AM, Greg Starkgsst...@mit.edu wrote:
 Someone doing a CVS pull already needs a specific recent version of
 autoconf anyways. How old is this version of flex compared to the
 version of autoconf we require?

 Really?  configure is in CVS, so I don't think you need autoconf
 unless you want to modify configure.in.

Right.  That's been debated on occasion in the past, but right now
only those who commit changes to configure.in (and related files)
need to have autoconf at all.

But to answer Greg's question: 8.4 and HEAD require autoconf 2.61
which was released 2006-11-17, or rather later than flex 2.5.33.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: 
 
 Yes, the snapshots have the derived files too.
 
Then I take it back -- the new flex versions would have little or no
impact on me.  Worst case, I might need to download a snapshot to
apply my patch for testing on the big machines.  If I understood
what make options I could use on my machine to create derived files
for other machines, even that would go away.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] single bit integer (TINYINT) revisited for 8.5

2009-07-02 Thread Simon Riggs

On Wed, 2009-07-01 at 11:19 -0400, Caleb Cushing wrote:
 I'd like to see this topic revisited since as far as I can see it
 hasn't been seriously discussed in years. I believe the main arguments
 against are why do we need more more numeric datatypes and increased
 maintenance. It would seem to me that a tinyint datatype maintenance
 wise would get all the same updates as the other int types, making it
 only a slight increase in maintenance. I think there was 1 more reason
 but I can't find the original thread now.
 
 most (if not all?) of posgresql's major competitor's (mysql, sql
 server, db2, etc) support a single bit integer datatype. it would
 bring increased compatibility with existing mysql apps esp, making
 them easier to port.
 
 It (in theory?) should also bring a speed enhancement where usable
 since it would take less disk space.
 
 A couple of times I've been told you don't need tinyint, use boolean
 which is not true, several projects I've worked on I've needed and
 integer field that supports number within a small range 0-5 1-10 1-100
 or something similar. I end up using smallint but it's range is huge
 for the actual requirements.

Completely agree.

I'm most or the way through working on this as an add-on module, rather
than a new datatype in core. I don't see much reason to include it in
core: its not an SQL standard datatype, it complicates catalog entries
and most people don't need or want it.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 Then I take it back -- the new flex versions would have little or no
 impact on me.  Worst case, I might need to download a snapshot to
 apply my patch for testing on the big machines.  If I understood
 what make options I could use on my machine to create derived files
 for other machines, even that would go away.

Peter or Marc could clue you in a bit better, but I think it's as simple
as saying make dist at the top level of a modified source tree.  This
gets you a source tarball the same way the release tarballs are made.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Joshua Tolley
On Thu, Jul 02, 2009 at 03:42:56PM +0100, Dave Page wrote:
 On Thu, Jul 2, 2009 at 3:22 PM, Joshua Tolleyeggyk...@gmail.com wrote:
  On Thu, Jul 02, 2009 at 08:41:27AM +0100, Dave Page wrote:
  As far as I'm aware, there's been no code
  review yet either, which would probably be a good idea.
 
  I don't have loads of time in the coming days, but IIRC I've taken a glance 
  at
  a past version of the code, and would be willing to do so again, if it would
  be useful.
 
 If you can look over it, that would be great. i'm not really qualified
 to review perl code, and we always prefer to have at least two sets of
 eyeballs on anything that we put into production for obvious reasons.

Is git://git.postgresql.org/git/pgcommitfest.git still the right place to get
the source?

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com


signature.asc
Description: Digital signature


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
I wrote:
 Yes.  What I was thinking of doing was committing a configure change to
 reject flex  2.5.31, and waiting to see how much of the buildfarm goes
 red.

Actually, most of the buildfarm members show which flex version they are
running in the configure output.  A quick look shows that of the 45
members that have reported on HEAD in the past 2 days, 22 are running
2.5.4, which is a lot higher than I was expecting.  Most of these are
the Solaris boxen, which I imagine can be updated fairly painlessly
since there are some of them that are already running something newer.
However I'm a bit worried about the situation for Windows --- does
anyone know whether a newer flex is readily available for Windows?

In any case it seems like it'd be prudent to prod the buildfarm owners
to update their flex *before* we pull the trigger...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: 
 
 Peter or Marc could clue you in a bit better, but I think it's as
 simple as saying make dist at the top level of a modified source
 tree.  This gets you a source tarball the same way the release
 tarballs are made.
 
I seem to be able to sneak up on it from behind by doing a regular
make and then make distclean and copying the results.  Perhaps
someone knows off-hand what I'm missing that prevents make dist from
working.  The attempt ends with:
 
make[3]: Entering directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/contrib/xml2'
make[3]: Nothing to be done for `distprep'.
make[3]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/contrib/xml2'
make[2]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/contrib'
make[1]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel'
make -C postgresql-8.5devel/doc/src/sgml/ HISTORY INSTALL
regress_README
make[1]: Entering directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/doc/src/sgml'
/usr/bin/perl generate_history.pl . release.sgml
tempfile_HISTORY.sgml
jade  -D .  -d stylesheet.dsl -i output-text -t sgml -V nochunks
tempfile_HISTORY.sgml HISTORY.html
/bin/sh: jade: not found
make[1]: *** [HISTORY.html] Error 127
make[1]: *** Deleting file `HISTORY.html'
make[1]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/doc/src/sgml'
make: *** [distdir] Error 2
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 I seem to be able to sneak up on it from behind by doing a regular
 make and then make distclean and copying the results.  Perhaps
 someone knows off-hand what I'm missing that prevents make dist from
 working.  The attempt ends with:

 jade  -D .  -d stylesheet.dsl -i output-text -t sgml -V nochunks
 tempfile_HISTORY.sgml HISTORY.html
 /bin/sh: jade: not found

Looks like you're missing openjade.  However, that's only used to
produce the HISTORY file which is hardly critical, so copying your
source tree after building is probably a perfectly good answer.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: 
 Kevin Grittner kevin.gritt...@wicourts.gov writes:
 
 jade  -D .  -d stylesheet.dsl -i output-text -t sgml -V nochunks
 tempfile_HISTORY.sgml HISTORY.html
 /bin/sh: jade: not found
 
 Looks like you're missing openjade.  However, that's only used to
 produce the HISTORY file which is hardly critical, so copying your
 source tree after building is probably a perfectly good answer.
 
Thanks.  I was able to install jade, but it throws pages of errors.
Rather than try to get that working, I'll just count on the other
approach.  I'd want to make it and check it on my machine first,
anyway, so the distclean is no big deal at that point.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Peter Eisentraut
On Thursday 02 July 2009 19:46:04 Tom Lane wrote:
 Kevin Grittner kevin.gritt...@wicourts.gov writes:
  Then I take it back -- the new flex versions would have little or no
  impact on me.  Worst case, I might need to download a snapshot to
  apply my patch for testing on the big machines.  If I understood
  what make options I could use on my machine to create derived files
  for other machines, even that would go away.

 Peter or Marc could clue you in a bit better, but I think it's as simple
 as saying make dist at the top level of a modified source tree.  This
 gets you a source tarball the same way the release tarballs are made.

make distprep is what prepares those files.  make dist depends on make 
distprep and then builds a tarball out of everything.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Jaime Casanova
On Thu, Jul 2, 2009 at 12:13 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 However I'm a bit worried about the situation for Windows --- does
 anyone know whether a newer flex is readily available for Windows?


MSYS Suplementary Tools (for mingw) includes flex-2.5.33
http://sourceforge.net/projects/mingw/files/

-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Peter Eisentraut pete...@gmx.net wrote: 
 
 make distprep is what prepares those files.
 
Perfect!  Flex files built.  No errors.  Thanks much!
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 1:10 PM, Joshua Tolleyeggyk...@gmail.com wrote:
 On Thu, Jul 02, 2009 at 03:42:56PM +0100, Dave Page wrote:
 On Thu, Jul 2, 2009 at 3:22 PM, Joshua Tolleyeggyk...@gmail.com wrote:
  On Thu, Jul 02, 2009 at 08:41:27AM +0100, Dave Page wrote:
  As far as I'm aware, there's been no code
  review yet either, which would probably be a good idea.
 
  I don't have loads of time in the coming days, but IIRC I've taken a 
  glance at
  a past version of the code, and would be willing to do so again, if it 
  would
  be useful.

 If you can look over it, that would be great. i'm not really qualified
 to review perl code, and we always prefer to have at least two sets of
 eyeballs on anything that we put into production for obvious reasons.

 Is git://git.postgresql.org/git/pgcommitfest.git still the right place to get
 the source?

Yes.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_migrator mention in documentation

2009-07-02 Thread Peter Eisentraut
On Friday 19 June 2009 00:56:42 Bruce Momjian wrote:

  The makefile for pg_migrator currently assumes by default that it is
  located under contrib/.  Which confuses me.

 You can compile pg_migrator by copying it to /contrib, or using PGXS;
 both work.  Read the 15-step install instructions for details:

   (7)  Build pg_migrator

   For pg_migrator source installs, keep in mind the compile must use the
   _new_ PostgreSQL source directory and be installed in the new Postgres
   install directory.

   The simplest build option is to point to the top of the PostgreSQL
   source tree by running something like:

   gmake top_builddir=/usr/src/pgsql install

   Replace '/usr/src/pgsql' with your source directory.  pg_migrator also
   understands the 'prefix=' specification if you installed Postgres in a
   custom location.  Another build option is to copy the pg_migrator
   directory into contrib/pg_migrator in the new PostgreSQL source tree and
   run a simple 'gmake install'.  A third install method is to use PGXS
   (assuming the new 'pg_config' is in your $PATH):

   USE_PGXS=1 gmake prefix=/usr/local/pgsql.new install

Maybe the latter method should be the default, as it matches better with how 
we encourage other extension modules to be built?

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Query progress indication - an implementation

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 12:48 PM, Euler Taveira de
Oliveiraeu...@timbira.com wrote:
 I know that it didn't solve the estimation problem but ... IMHO the
 [under|over]estimation should be treated by an external tool (autoexplain?).
 So when we enable the query progress and some node reports a difference
 between estimated and real more than x%, log the plan. Doing it, we will be
 helping DBAs to investigate the bad plans.

Keep in mind that it is frequently the case that the estimates are
substantially off but the plan still works OK.  I just put a dirty
hack into one of my apps to improve the selectivity estimates by a
factor of 200, but they're still off by a factor of 5.  Even when they
were off by 1000x the bad plan happened only intermittently.  You
notice the cases where the estimates are off and it makes for a bad
plan, but there are lots of other cases where the estimates are off
but the plan is still OK.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_migrator mention in documentation

2009-07-02 Thread Alvaro Herrera
Peter Eisentraut wrote:
 On Friday 19 June 2009 00:56:42 Bruce Momjian wrote:
 
   The makefile for pg_migrator currently assumes by default that it is
   located under contrib/.  Which confuses me.
 
  You can compile pg_migrator by copying it to /contrib, or using PGXS;
  both work.  Read the 15-step install instructions for details:
 
  (7)  Build pg_migrator
 
  For pg_migrator source installs, keep in mind the compile must use the
  _new_ PostgreSQL source directory and be installed in the new Postgres
  install directory.
 
  The simplest build option is to point to the top of the PostgreSQL
  source tree by running something like:
 
  gmake top_builddir=/usr/src/pgsql install
 
  Replace '/usr/src/pgsql' with your source directory.  pg_migrator also
  understands the 'prefix=' specification if you installed Postgres in a
  custom location.  Another build option is to copy the pg_migrator
  directory into contrib/pg_migrator in the new PostgreSQL source tree and
  run a simple 'gmake install'.  A third install method is to use PGXS
  (assuming the new 'pg_config' is in your $PATH):
 
  USE_PGXS=1 gmake prefix=/usr/local/pgsql.new install
 
 Maybe the latter method should be the default, as it matches better with how 
 we encourage other extension modules to be built?

I wonder why we have two ways at all (I'm not counting the stuff about
copying it to contrib because it seems pointless).  The other day I was
looking at orafce code in pgfoundry, and at clearxlogtail too IIRC, and
they both had the ifdef USE_PGXS stuff in the Makefile.  I wonder why.
Why not just rip the ifdef line and the non-PGXS code, and just use
PGXS always?

It seems to me like module authors are just copying the contrib
Makefiles without stopping to think that the Makefiles could be a lot
shorter if they got rid of the non-PGXS part, which provides no extra
value.  More concretely,

*** Makefile.orig   2009-07-02 14:35:03.0 -0400
--- Makefile2009-07-02 14:35:08.0 -0400
***
*** 15,26 
  PG_CPPFLAGS  += -g -O0 -Wall -Wmissing-prototypes -Wmissing-declarations 
-Wpointer-arith -Wcast-align
  endif
  
- ifdef USE_PGXS
  PGXS := $(shell pg_config --pgxs)
  include $(PGXS)
- else
- subdir = contrib/pg_migrator/src
- top_builddir = ../../..
- include $(top_builddir)/src/Makefile.global
- include $(top_srcdir)/contrib/contrib-global.mk
- endif
--- 15,19 


-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_migrator mention in documentation

2009-07-02 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: 
 
 I wonder why we have two ways at all (I'm not counting the stuff
 about copying it to contrib because it seems pointless).  The other
 day I was looking at orafce code in pgfoundry, and at clearxlogtail
 too IIRC, and they both had the ifdef USE_PGXS stuff in the
 Makefile.  I wonder why.  Why not just rip the ifdef line and the
 non-PGXS code, and just use PGXS always?
 
Well, most of our database servers wind up with multiple builds of
PostgreSQL, and we find it is less error-prone for the non-programmer
DBAs to expand a tarball under such a directory from the normal build
(for the appropriate version) and make it there.  There also seemed
some possibility that clearxlogtail would be accepted into the
distribution as a contrib module, so I was trying to have it ready to
go, should that happen.
 
As an aside, our function for extracting text from a PDF in a bytea
didn't work right when I tried to build it using the PGXS the other
day.  I moved the directory into our PostgreSQL build location and
built it the other way and it worked.  I haven't tracked down why, but
it leaves me leery of carving out the form which worked for me.  (When
you don't really understand something, you resort to superstitious
ritual)
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Autoconf 2.63

2009-07-02 Thread Peter Eisentraut
... is now the required version to rebuild configure.

Other than that, nothing changes.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 8.5 development schedule

2009-07-02 Thread Chris Browne
gsst...@mit.edu (Greg Stark) writes:
 I would like to propose a different strategy. Instead of always
 tackling all the smaller patches and leaving the big patches for last,
 I would suggest we start with Hot Standby.

 In fact I would suggest as Hot Standby has already gotten a first pass
 review that we consider applying it on day 1. That gets it into
 everyone's development trees so they can see any suspicious code or
 effects it has in their peculiar environments. It may not be perfect
 but if we apply it now there's plenty of time to make improvements.

 Then we can have a regular commitfest a month or so later. Hopefully
 any followon changes to Hot Standby would actually get into that
 commitfest if they're relatively minor.

I could see going either way on this, either:

a) Doing an as-early-as-possible CommitFest to knock off easy items
that have been waiting a while, and having the *second* Fest be the
one where we expect all the large, controversial items to get added
(e.g. - stuff like hot standby, SEPostgreSQL), or

b) Focusing on the likely-hard ones (hot standby, SE PostgreSQL)
first, and deferring others to Fest #2.
-- 
select 'cbbrowne' || '@' || 'acm.org';
http://cbbrowne.com/info/linuxdistributions.html
Everything should be built top-down, except the first time.
-- Alan J. Perlis

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Psql List Languages

2009-07-02 Thread Peter Eisentraut
On Monday 02 February 2009 15:29:51 Fernando Ike wrote:
 Hi,

 On Fri, Jan 30, 2009 at 3:03 PM, Fernando Ike f...@midstorm.org wrote:
  Hi,,
 
 My job, I maintainer some postgres server for clients. We have
  many PL/(Java, Perl, Ruby, Python, R)  and to more easy
  administration, I worked new little psql attribute to list languages
  com shorcurt/function \dL.

 [..]

I know that this moment is inappropriate to submit patch, with the
  discussions about features for 8.4. But, if can added for commitfest
  to 8.5 version. I'm appreciate.

I update patch for added gettext fields and change spaces/tab to 4
 spaces. :)

Looks good, but needs documentation and tab-complete.c updates, it seems.  And 
you should revisit this patch to make it consistent with the S flag that was 
added to most \d commands.  For example, \dL would show only user-added 
languages, but \dLS would include c, internal, and sql.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Psql List Languages

2009-07-02 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 Looks good, but needs documentation and tab-complete.c updates, it seems.  
 And 
 you should revisit this patch to make it consistent with the S flag that was 
 added to most \d commands.  For example, \dL would show only user-added 
 languages, but \dLS would include c, internal, and sql.

Another recent change in the \d support is that commands are expected to
work (as best they can) for all server versions back to 7.4.  I didn't
look to see if this patch needs any fixes for that, but it's something
to keep in mind while reviewing.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Zdenek Kotala

Tom Lane píše v čt 02. 07. 2009 v 11:32 -0400:
 Andrew Dunstan and...@dunslane.net writes:
  Tom Lane wrote:
  Right, this will only affect people doing development or otherwise
  building from a CVS pull.
 
  Of course, that includes the whole buildfarm. We might need to ask some 
  people to upgrade there.
 
 Yes.  What I was thinking of doing was committing a configure change to
 reject flex  2.5.31, and waiting to see how much of the buildfarm goes
 red.

Solaris 10 has 2.5.4 version. OpenSolaris already has 2.5.33.

Zdenek


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Zdenek Kotala

Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400:
 I wrote:
  Yes.  What I was thinking of doing was committing a configure change to
  reject flex  2.5.31, and waiting to see how much of the buildfarm goes
  red.
 
 Actually, most of the buildfarm members show which flex version they are
 running in the configure output.  A quick look shows that of the 45
 members that have reported on HEAD in the past 2 days, 22 are running
 2.5.4, which is a lot higher than I was expecting.  Most of these are
 the Solaris boxen, which I imagine can be updated fairly painlessly
 since there are some of them that are already running something newer.

I hope, need to check/compile, but I don't expect any problems.

Zdenek


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_migrator mention in documentation

2009-07-02 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On Friday 19 June 2009 00:56:42 Bruce Momjian wrote:
 A third install method is to use PGXS
 (assuming the new 'pg_config' is in your $PATH):
 
 USE_PGXS=1 gmake prefix=/usr/local/pgsql.new install

 Maybe the latter method should be the default, as it matches better with how 
 we encourage other extension modules to be built?

Also, the recommendation to specify prefix here is redundant and
error-prone.  It can get the correct prefix from pg_config.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Zdenek Kotala

Josh Berkus píše v st 01. 07. 2009 v 17:21 -0700:
 Folks,
 
 There's been a lot of discussion/argument around how to handle the last 
 commitfest, but there seems to be a total consensus that we want to have 
 the first CF on July 15th.

Can we add flags like bump catalog version, bump page layout version,
modify AM for each patch? It should help to track pg_upgrade changes.

Zdenek



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] PGXS problem with pdftotext

2009-07-02 Thread Kevin Grittner
I've been wondering whether anyone else would want to use the
functions we wrote to extract text from PDF documents stored in bytea
columns.  If so, I would need to sort out the problems I've been
having with builds through the PGXS techniques.  Here's the directory,
after a successful build under contrib:
 
kgri...@project-db:~/postgresql-8.3.7/contrib/pdftotext ll
total 108
-rw-r--r-- 1 kgrittn dbas 22990 2009-04-14 17:14 libpdftotext.a
lrwxrwxrwx 1 kgrittn dbas19 2009-04-14 17:14 libpdftotext.so -
libpdftotext.so.0.0
lrwxrwxrwx 1 kgrittn dbas19 2009-04-14 17:14 libpdftotext.so.0 -
libpdftotext.so.0.0
-rwxr-xr-x 1 kgrittn dbas 21666 2009-04-14 17:14 libpdftotext.so.0.0
-rw-r--r-- 1 kgrittn dbas   443 2009-04-14 17:14 Makefile
-rw-r--r-- 1 kgrittn dbas  2980 2008-07-22 13:00 pdftotext.c
-rw-r--r-- 1 kgrittn dbas 14184 2009-04-14 17:14 pdftotext.o
-rw-r--r-- 1 kgrittn dbas   285 2009-04-14 17:14 pdftotext.sql
-rw-r--r-- 1 kgrittn dbas   285 2008-07-22 13:00 pdftotext.sql.in
-rw-r--r-- 1 kgrittn dbas  4658 2009-04-13 17:02 poppler_compat.cc
-rw-r--r-- 1 kgrittn dbas   355 2008-07-22 13:00 poppler_compat.h
-rw-r--r-- 1 kgrittn dbas  8208 2009-04-14 17:14 poppler_compat.o
-rw-r--r-- 1 kgrittn dbas   733 2008-07-22 13:00 README.pdftotext
 
Here's the Makefile contents:
 
MODULE_big = pdftotext
OBJS = pdftotext.o poppler_compat.o
DATA_built = pdftotext.sql
DOCS = README.pdftotext

PG_CPPFLAGS =-I/usr/include/poppler -shared -fpic
SHLIB_LINK = -lpoppler -L/usr/local/lib

ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
else
subdir = contrib/pdftotext
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
 
If we export PGXS=1 and make ; sudo make install outside the
PostgreSQL build tree, it seems to build and deploy OK, but it can't
find the poppler implementation at run time.  If we do it in the build
tree, all is good.  Where's the problem?  Is the SHLIB_LINK setting
proper?  What's the right way to do this?
 
BTW, libpoppler is GPL licensed, and always reminds me of what
Churchill said about democracy, if that affects anyone's interest in
the code.  You're likely to need to tweak the code based on the
particular version of libpoppler you're using.  If you use an older
version of libpoppler, it can crash the whole PostgreSQL environment
if you try to use it with a PDF using newer features.  :-(
 
If anyone's still interested, and I can fix the build problem, I'll
throw the source code onto pgfoundry.
 
-Kevin
 
It has been said that democracy is the worst form of government
except all the others that have been tried.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] bug in Google translate snippet

2009-07-02 Thread Alvaro Herrera
Hi,

I was having a look at this snippet:
http://wiki.postgresql.org/wiki/Google_Translate
and it turns out that it doesn't work if the result contains non-ASCII
chars.  Does anybody know how to fix it?

alvherre=# select gtranslate('en', 'es', 'he');
ERROR:  plpython: function gtranslate could not create return value
DETALLE:  type 'exceptions.UnicodeEncodeError': 'ascii' codec can't encode 
character u'\xe9' in position 0: ordinal not in range(128)

By adding a plpy.log() call you can see that the answer is él:
LOG:  (u'\xe9l',)

I guess it needs some treatment similar to the one in this function:
http://wiki.postgresql.org/wiki/Strip_accents_from_strings


For completeness, here is the code:

CREATE OR REPLACE FUNCTION gtranslate(src text, target text, phrase text) 
RETURNS text
LANGUAGE plpythonu
AS $$
import re
import urllib
 
import simplejson as json
 
class UrlOpener(urllib.FancyURLopener):
version = py-gtranslate/1.0
 
base_uri = http://ajax.googleapis.com/ajax/services/language/translate;
default_params = {'v': '1.0'}
 
def translate(src, to, phrase):
args = default_params.copy()
args.update({
'langpair': '%s%%7C%s' % (src, to),
'q': urllib.quote_plus(phrase),
})
argstring = '%s' % (''.join(['%s=%s' % (k,v) for (k,v) in 
args.iteritems()]))
resp = json.load(UrlOpener().open('%s?%s' % (base_uri, argstring)))
try:
return resp['responseData']['translatedText']
except:
# should probably warn about failed translation
return phrase
 
return translate(src, target, phrase)
$$;

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PGXS problem with pdftotext

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 PG_CPPFLAGS =-I/usr/include/poppler -shared -fpic
 SHLIB_LINK = -lpoppler -L/usr/local/lib

It doesn't seem appropriate to put -shared or -fpic into PG_CPPFLAGS.
If you need those, the makefiles should add them automatically.

The other thing that seems peculiar is looking for the include files
in /usr/include and the library in /usr/local/lib.  I've never
seen any package install itself like that --- either everything goes
under /usr/local or nothing does.  I suspect you might have two
incompatible poppler installations on the machine and you're picking
up the wrong combination of files.

Running ldd or local equivalent on pdftotext.so might help you determine
what's going on as far as finding the library goes.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PGXS problem with pdftotext

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: 
 Kevin Grittner kevin.gritt...@wicourts.gov writes:
 PG_CPPFLAGS =-I/usr/include/poppler -shared -fpic
 SHLIB_LINK = -lpoppler -L/usr/local/lib
 
 It doesn't seem appropriate to put -shared or -fpic into
 PG_CPPFLAGS. If you need those, the makefiles should add them
 automatically.
 
 The other thing that seems peculiar is looking for the include files
 in /usr/include and the library in /usr/local/lib.  I've never
 seen any package install itself like that --- either everything goes
 under /usr/local or nothing does.  I suspect you might have two
 incompatible poppler installations on the machine and you're picking
 up the wrong combination of files.
 
 Running ldd or local equivalent on pdftotext.so might help you
 determine what's going on as far as finding the library goes.
 
Thanks.  Let's just say that the poppler build from source has not
ever gone as smoothly as the most eventful PostgreSQL build from
source.  We've had to do much ad hoc hacking to get anything usable,
and I'm sure we've made some bad choices in the process.  I'll take a
close look at where everything has landed in light of your advice, and
see if I can arrange things more sensibly.
 
Does it seem likely that fixing these issues will allow PGXS to work?
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] TODO items

2009-07-02 Thread Jaime Casanova
Hi,

I guess todo items marked as [D] (Done) should be removed now... right?

-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Alvaro Herrera
Jaime Casanova wrote:
 Hi,
 
 I guess todo items marked as [D] (Done) should be removed now... right?

Would there be some value in creating a new page TodoDone84 and moving
those items to it?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bug in Google translate snippet

2009-07-02 Thread Andrew Dunstan



Alvaro Herrera wrote:

Hi,

I was having a look at this snippet:
http://wiki.postgresql.org/wiki/Google_Translate
and it turns out that it doesn't work if the result contains non-ASCII
chars.  Does anybody know how to fix it?

alvherre=# select gtranslate('en', 'es', 'he');
ERROR:  plpython: function gtranslate could not create return value
DETALLE:  type 'exceptions.UnicodeEncodeError': 'ascii' codec can't encode 
character u'\xe9' in position 0: ordinal not in range(128)
  


This looks like a python issue rather than a Postgres issue. The problem 
is probably in python-simplejson.


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Joshua D. Drake
On Thu, 2009-07-02 at 17:29 -0400, Alvaro Herrera wrote:
 Jaime Casanova wrote:
  Hi,
  
  I guess todo items marked as [D] (Done) should be removed now... right?
 
 Would there be some value in creating a new page TodoDone84 and moving
 those items to it?

For historical record I could see benefit.

Joshua D. Drake


 
 -- 
 Alvaro Herrerahttp://www.CommandPrompt.com/
 The PostgreSQL Company - Command Prompt, Inc.
 
-- 
PostgreSQL - XMPP: jdr...@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PGXS problem with pdftotext

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes:
 Does it seem likely that fixing these issues will allow PGXS to work?

Couldn't say.  It would be useful to compare ldd output for
pdftotext.so built both ways.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bug in Google translate snippet

2009-07-02 Thread Alvaro Herrera
Andrew Dunstan wrote:


 Alvaro Herrera wrote:
 Hi,

 I was having a look at this snippet:
 http://wiki.postgresql.org/wiki/Google_Translate
 and it turns out that it doesn't work if the result contains non-ASCII
 chars.  Does anybody know how to fix it?

 alvherre=# select gtranslate('en', 'es', 'he');
 ERROR:  plpython: function gtranslate could not create return value
 DETALLE:  type 'exceptions.UnicodeEncodeError': 'ascii' codec can't encode 
 character u'\xe9' in position 0: ordinal not in range(128)

 This looks like a python issue rather than a Postgres issue. The problem  
 is probably in python-simplejson.

I think the problem happens when the PL tries to create the output
value.  Otherwise I wouldn't be able to see the value in plpy.log.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Andrew Dunstan



Joshua D. Drake wrote:

On Thu, 2009-07-02 at 17:29 -0400, Alvaro Herrera wrote:
  

Jaime Casanova wrote:


Hi,

I guess todo items marked as [D] (Done) should be removed now... right?
  

Would there be some value in creating a new page TodoDone84 and moving
those items to it?



For historical record I could see benefit.

  


Doesn't the wiki keep a page history? I don't see much point in keeping 
the old items in a current page.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Jaime Casanova wrote:
 I guess todo items marked as [D] (Done) should be removed now... right?

 Would there be some value in creating a new page TodoDone84 and moving
 those items to it?

In the past we've figured that old TODO versions could be retrieved
from CVS history.  While that's still possible in the wiki environment,
it might be worth saving the old entries in a more easily accessible
fashion.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Greg Stark
So I currently have some free time as I'm currently between jobs. I
can start going through the queued patches in the next few weeks.
Should I be looking at the wiki currently? Or is your tool ready to
go?

-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Alvaro Herrera
Andrew Dunstan wrote:


 Joshua D. Drake wrote:
 On Thu, 2009-07-02 at 17:29 -0400, Alvaro Herrera wrote:
   
 Jaime Casanova wrote:
 
 I guess todo items marked as [D] (Done) should be removed now... right?
   
 Would there be some value in creating a new page TodoDone84 and moving
 those items to it?

 For historical record I could see benefit.

 Doesn't the wiki keep a page history?

Yes, but it is not as convenient.


 I don't see much point in keeping  the old items in a current page.

It is useful to point out to people: look at all the things we did in 8.4!

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Jaime Casanova
On Thu, Jul 2, 2009 at 4:50 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
 Jaime Casanova wrote:
 I guess todo items marked as [D] (Done) should be removed now... right?

 Would there be some value in creating a new page TodoDone84 and moving
 those items to it?

 In the past we've figured that old TODO versions could be retrieved
 from CVS history.  While that's still possible in the wiki environment,
 it might be worth saving the old entries in a more easily accessible
 fashion.


We want to use the same format/sections like in the TODO?


-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TODO items

2009-07-02 Thread Tom Lane
Jaime Casanova jcasa...@systemguards.com.ec writes:
 We want to use the same format/sections like in the TODO?

Sure, why not?  Just copy the page and strip out the not-done items.
No reason to think hard here.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_migrator mention in documentation

2009-07-02 Thread Bruce Momjian
Peter Eisentraut wrote:
 On Friday 19 June 2009 00:56:42 Bruce Momjian wrote:
 
   The makefile for pg_migrator currently assumes by default that it is
   located under contrib/.  Which confuses me.
 
  You can compile pg_migrator by copying it to /contrib, or using PGXS;
  both work.  Read the 15-step install instructions for details:
 
  (7)  Build pg_migrator
 
  For pg_migrator source installs, keep in mind the compile must use the
  _new_ PostgreSQL source directory and be installed in the new Postgres
  install directory.
 
  The simplest build option is to point to the top of the PostgreSQL
  source tree by running something like:
 
  gmake top_builddir=/usr/src/pgsql install
 
  Replace '/usr/src/pgsql' with your source directory.  pg_migrator also
  understands the 'prefix=' specification if you installed Postgres in a
  custom location.  Another build option is to copy the pg_migrator
  directory into contrib/pg_migrator in the new PostgreSQL source tree and
  run a simple 'gmake install'.  A third install method is to use PGXS
  (assuming the new 'pg_config' is in your $PATH):
 
  USE_PGXS=1 gmake prefix=/usr/local/pgsql.new install
 
 Maybe the latter method should be the default, as it matches better with how 
 we encourage other extension modules to be built?

I looked at that and the problem is that pg_migrator must be built
against the _new_ source tree, and will issue an error and exit if it
isn't.  The problem with PGXS is it silently chooses the source tree to
use based on which pg_config it finds in its path first;  that seems
error-prone.  Any ideas for a clearer way to specify pg_config, and is
that really helping things if the user has to specify it?  As you can
see, pg_migrator has the requirement of running in a multi-pg_config
binary environment, so it has extra complexity that might make pg_config
an undesirable option to be promoted first.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_migrator mention in documentation

2009-07-02 Thread Bruce Momjian
Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  On Friday 19 June 2009 00:56:42 Bruce Momjian wrote:
  A third install method is to use PGXS
  (assuming the new 'pg_config' is in your $PATH):
  
  USE_PGXS=1 gmake prefix=/usr/local/pgsql.new install
 
  Maybe the latter method should be the default, as it matches better with 
  how 
  we encourage other extension modules to be built?
 
 Also, the recommendation to specify prefix here is redundant and
 error-prone.  It can get the correct prefix from pg_config.

Again, see my email just posted about using pg_migrator in a
multi-pg_config-binary environment.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 5:50 PM, Greg Starkgsst...@mit.edu wrote:
 So I currently have some free time as I'm currently between jobs. I
 can start going through the queued patches in the next few weeks.
 Should I be looking at the wiki currently? Or is your tool ready to
 go?

Wiki for now.  Brendan Jurd is working on a script to bulk-import the
data to the tool, and he (and I) will make sure to update -hackers and
redirect the wiki page when he gets that done.  I'm hoping that it
will happen in the next few days.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] SE-PostgreSQL Updates rev.2096

2009-07-02 Thread Robert Haas
2009/6/30 KaiGai Kohei kai...@ak.jp.nec.com:
 The SE-PostgreSQL patches are updated as follows:

 01) http://sepgsql.googlecode.com/files/sepgsql-01-sysatt-8.4-r2096.patch
 02) http://sepgsql.googlecode.com/files/sepgsql-02-core-8.4-r2096.patch
 03) http://sepgsql.googlecode.com/files/sepgsql-03-gram-8.4-r2096.patch
 04) http://sepgsql.googlecode.com/files/sepgsql-04-writable-8.4-r2096.patch
 05) http://sepgsql.googlecode.com/files/sepgsql-05-rowlevel-8.4-r2096.patch
 06) http://sepgsql.googlecode.com/files/sepgsql-06-perms-8.4-r2096.patch
 07) http://sepgsql.googlecode.com/files/sepgsql-07-extra-8.4-r2096.patch
 08) http://sepgsql.googlecode.com/files/sepgsql-08-utils-8.4-r2096.patch
 09) http://sepgsql.googlecode.com/files/sepgsql-09-tests-8.4-r2096.patch
 10) http://sepgsql.googlecode.com/files/sepgsql-10-docs-8.4-r2096.patch

KaiGai,

It appears that some things that you were previously asked to remove
for the first version, like row-level security, have crept back in
here.  I think that there is a clear consensus that what you have here
is too ambitious for a single commit.

http://archives.postgresql.org/message-id/20090311135413.gg4...@alvh.no-ip.org
http://archives.postgresql.org/message-id/336.1236792...@sss.pgh.pa.us

To put some numbers around this:

$ cat *.patch | diffstat | tail -1
 219 files changed, 11819 insertions(+), 29 deletions(-), 857 modifications(!)

For comparison:

Infrastructure Changes for Recovery
 8 files changed, 912 insertions(+), 183 deletions(-)
Window Functions
 92 files changed, 6631 insertions(+), 232 deletions(-)
WITH/WITH RECURSIVE
 77 files changed, 5826 insertions(+), 246 deletions(-)

Based on that, it looks to me like you should really aim to have a
patch set that changes AT MOST 5-6K lines, and less would be improve
your odds of having something accepted.  You can always submit
follow-on patches once the basic implementation is in, but I think I'm
reflecting the shared view of those who have looked at this in the
past when I say that it's never going to get committed in this form.

Just to be clear, the fact that you have split this up into multiple,
separate patch FILES is of no value at all, because they are
interdependent.  For example, I just took a quick look at the 01
patch and it obviously includes code that is only necessary for
row-level security.  Nothing is going to get committed here unless it
is a self-contained change that does not presume that there will be
further commits in the future.

Unless this is resubmitted in a form that complies with the changes
that were requested the last time it was reviewed, there is really no
point in reviewing it again.

Thanks,

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Robert Haas
On Tue, Jun 16, 2009 at 2:13 AM, Fujii Masaomasao.fu...@gmail.com wrote:
 The attached latest patch provides this capability. You can easily set up the
 synch rep according to the following procedure.
 http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects#How_to_set_up_Synch_Rep

This patch no longer applies cleanly.  Can you rebase and resubmit it
for the upcoming CommitFest?  It might also be good to go through and
clean up the various places where you have trailing whitespace and/or
spaces preceding tabs.

It seems this will be one of the big patches for the upcoming
CommitFest.  Hot Standby seems to be off the table, because Simon has
indicated that he thinks Synch Rep should go first, and Heikki has
indicated that he's willing to review and commit, but not also play
lead developer.

http://archives.postgresql.org/pgsql-hackers/2009-07/msg5.php
http://archives.postgresql.org/pgsql-hackers/2009-06/msg01534.php

Given that this is a substantial patch, I have a couple of questions
about strategy.  First, I am wondering whether this patch should be
reviewed (and committed) as a whole, or whether there are distinct
chunks of it that should be reviewed and committed separately -
particularly the signal handling piece, which AIUI is independently
useful.  I note that it seems to be included in the tarball as a
separate patch file, which is very useful.

Second, I am wondering whether Heikki feels that it would be useful to
assign round-robin reviewers for this patch, or whether he's going to
be the principal reviewer himself.  We could assign either a reviewer
(or reviewers) to the whole patch, or we could assign reviewers to
particular chunks of the patch, such as the signal handling piece.

Thanks,

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 10:02 PM, Robert Haasrobertmh...@gmail.com wrote:
 Second, I am wondering whether Heikki feels that it would be useful to
 assign round-robin reviewers for this patch, or whether he's going to
 be the principal reviewer himself.  We could assign either a reviewer
 (or reviewers) to the whole patch, or we could assign reviewers to
 particular chunks of the patch, such as the signal handling piece.

Hmm, taking a look at the wiki, I see that Simon's name is listed for
this patch as a reviewer already.  Assuming that's a point of view
that Simon agrees with and not the result of his name having been
added by someone else, I guess the question is whether we need
additional reviewers here beyond Heikki and Simon.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] First CommitFest: July 15th

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 3:42 PM, Zdenek Kotalazdenek.kot...@sun.com wrote:

 Josh Berkus píše v st 01. 07. 2009 v 17:21 -0700:
 Folks,

 There's been a lot of discussion/argument around how to handle the last
 commitfest, but there seems to be a total consensus that we want to have
 the first CF on July 15th.

 Can we add flags like bump catalog version, bump page layout version,
 modify AM for each patch? It should help to track pg_upgrade changes.

That's not a bad idea, and it wouldn't be hard to add various flags
and things to the CommitFest app I wrote, but how would we maintain
the information and keep it correct?  It seems like there might be a
danger that patch authors wouldn't know whether or not they were doing
those things.  Also, how would we handle changes by committers, who
don't always go through the CommitFest process?

Not sure of the answers here, just thinking out loud.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Alvaro Herrera
Robert Haas escribió:

 Second, I am wondering whether Heikki feels that it would be useful to
 assign round-robin reviewers for this patch, or whether he's going to
 be the principal reviewer himself.  We could assign either a reviewer
 (or reviewers) to the whole patch, or we could assign reviewers to
 particular chunks of the patch, such as the signal handling piece.

WRT the signal handling piece, I remember something in that area being
committed and then reverted because it had issues.  Does this version
fix those issues?  (Assuming it's the same patch)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] SE-PostgreSQL Updates rev.2096

2009-07-02 Thread KaiGai Kohei

Robert Haas wrote:

2009/6/30 KaiGai Kohei kai...@ak.jp.nec.com:

The SE-PostgreSQL patches are updated as follows:

01) http://sepgsql.googlecode.com/files/sepgsql-01-sysatt-8.4-r2096.patch
02) http://sepgsql.googlecode.com/files/sepgsql-02-core-8.4-r2096.patch
03) http://sepgsql.googlecode.com/files/sepgsql-03-gram-8.4-r2096.patch
04) http://sepgsql.googlecode.com/files/sepgsql-04-writable-8.4-r2096.patch
05) http://sepgsql.googlecode.com/files/sepgsql-05-rowlevel-8.4-r2096.patch
06) http://sepgsql.googlecode.com/files/sepgsql-06-perms-8.4-r2096.patch
07) http://sepgsql.googlecode.com/files/sepgsql-07-extra-8.4-r2096.patch
08) http://sepgsql.googlecode.com/files/sepgsql-08-utils-8.4-r2096.patch
09) http://sepgsql.googlecode.com/files/sepgsql-09-tests-8.4-r2096.patch
10) http://sepgsql.googlecode.com/files/sepgsql-10-docs-8.4-r2096.patch


KaiGai,

It appears that some things that you were previously asked to remove
for the first version, like row-level security, have crept back in
here.  I think that there is a clear consensus that what you have here
is too ambitious for a single commit.


Robert,

The row-level security is added at the fifth patch.
If you apply only the first three patches, it performs without row-level
security version.

I don't believe all the 10 patches get commited at the first commit fest.
If preferable, I can agree to forcus on the first three patches on the first
commit fest, according to the step-by-step approach, previously suggested.

 01 ... It provides a facility to manage security labels of database obejcts.
 02 ... It provides a core access control stuff including communication with 
kernel.
 03 ... It provides SECURITY_LABEL option in some of CREATE/ALTER statement.


http://archives.postgresql.org/message-id/20090311135413.gg4...@alvh.no-ip.org
http://archives.postgresql.org/message-id/336.1236792...@sss.pgh.pa.us

To put some numbers around this:

$ cat *.patch | diffstat | tail -1
 219 files changed, 11819 insertions(+), 29 deletions(-), 857 modifications(!)


Some of files are parched by multiple patches.

It is more correct estimation.
  $ diffstat sepgsql-00-full-8.4.0-r2096.patch.gz
  165 files changed, 11788 insertions(+), 2 deletions(-), 657 modifications(!)


For comparison:

Infrastructure Changes for Recovery
 8 files changed, 912 insertions(+), 183 deletions(-)
Window Functions
 92 files changed, 6631 insertions(+), 232 deletions(-)
WITH/WITH RECURSIVE
 77 files changed, 5826 insertions(+), 246 deletions(-)

Based on that, it looks to me like you should really aim to have a
patch set that changes AT MOST 5-6K lines, and less would be improve
your odds of having something accepted.  You can always submit
follow-on patches once the basic implementation is in, but I think I'm
reflecting the shared view of those who have looked at this in the
past when I say that it's never going to get committed in this form.


Scale of the first three patches is as follows:
$ diffstat sepgsql-01-sysatt-8.4.0-r2095.patch \
   sepgsql-02-core-8.4.0-r2095.patch \
   sepgsql-03-gram-8.4.0-r2095.patch
110 files changed, 5162 insertions(+), 2 deletions(-), 308 modifications(!)

The SE-PostgreSQL without row-level security version changes about 5-6K lines.
I can agree it may be a maximum scale in a single commit fest.


Just to be clear, the fact that you have split this up into multiple,
separate patch FILES is of no value at all, because they are
interdependent.  For example, I just took a quick look at the 01
patch and it obviously includes code that is only necessary for
row-level security.  Nothing is going to get committed here unless it
is a self-contained change that does not presume that there will be
further commits in the future.


I considered the separated patches reflects the step-by-step approach
previously suggested. At least, the minimum SE-PostgreSQL can works
with the first two patches.
The purpose of 01 patch is to provides a facility to manage security
labels of any database objects, including tables, columns and so on.
However, indeed, I could find a part only necessary for row-level stuff,
such as definitions of security_labels and security_acl.
I can agree to move them to the later patch.


Unless this is resubmitted in a form that complies with the changes
that were requested the last time it was reviewed, there is really no
point in reviewing it again.


Right now, I think it is preferable to focus on the first three patches
at the first commit fest.

What's your opinion?
--
KaiGai Kohei kai...@kaigai.gr.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Fujii Masao
Hi,

On Fri, Jul 3, 2009 at 11:02 AM, Robert Haasrobertmh...@gmail.com wrote:
 On Tue, Jun 16, 2009 at 2:13 AM, Fujii Masaomasao.fu...@gmail.com wrote:
 The attached latest patch provides this capability. You can easily set up the
 synch rep according to the following procedure.
 http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects#How_to_set_up_Synch_Rep

 This patch no longer applies cleanly.  Can you rebase and resubmit it
 for the upcoming CommitFest?  It might also be good to go through and
 clean up the various places where you have trailing whitespace and/or
 spaces preceding tabs.

Sure. I'll resubmit the patch after fixing some bugs and finishing
the documents.

 Given that this is a substantial patch, I have a couple of questions
 about strategy.  First, I am wondering whether this patch should be
 reviewed (and committed) as a whole, or whether there are distinct
 chunks of it that should be reviewed and committed separately -
 particularly the signal handling piece, which AIUI is independently
 useful.  I note that it seems to be included in the tarball as a
 separate patch file, which is very useful.

I think that the latter strategy makes more sense. At least, the signal
handling piece and non-blocking pqcomm (communication between
a frontend and a backend) can be reviewed independently of synch rep
itself.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Fujii Masao
Hi,

On Fri, Jul 3, 2009 at 11:59 AM, Alvaro
Herreraalvhe...@commandprompt.com wrote:
 WRT the signal handling piece, I remember something in that area being
 committed and then reverted because it had issues.  Does this version
 fix those issues?  (Assuming it's the same patch)

Yes. After the patch was reverted, Heikki and I fixed the problems.

The problem which was pointed out is:
http://archives.postgresql.org/message-id/14969.1228835...@sss.pgh.pa.us

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] commitfest.postgresql.org

2009-07-02 Thread Robert Haas
Per Dave Page's request of this morning, my CommitFest management
application now has a real hostname (see subject line).  I have also
sent Dave an email with details of the install process and location of
files, per his request (let me know if there's somewhere else those
details should be posted).

Brendan Jurd has graciously migrated all of the data from the
CommitFest wiki page to the app by writing a script to parse the wiki
markup and inserting the resulting data directly into the database.
There are a few loose ends.  The application stamps comments with the
community login of the person who left them, but the import stamped
them with names instead.  This is actually of some significance, since
the app will allow you to edit your own comments but not those of
other people.  We could probably fix this if someone can give us
access to (or a dump of) the realname to username mappings from the
community login DB.  Also, we're currently missing the reviewer names
due to limitations of the import script; Brendan is fixing this.

Filling the DB with live data revealed a few warts.  In particular,
the original ordering of patches was alpha by topic and then alpha by
name, which I thought would be OK, but upon seeing how it really
looked, I hated it.  So the topic manager now lets you set a sort
order for commitfest topics, and the order is now numeric by topic
sortorder, then alpha by topic, then by ascending patch ID (so the
oldest patch comes up first within each topic section).  Also, I
originally had the topics displayed as a table column, but that didn't
really work for me once I saw it either, so it's been reorganized to
do it the same way the wiki does.

We're still hacking on a few other details of the formatting and
interface, but you might want to cruise over and have a look.  Please
however continue to make ALL CHANGES on the wiki, and not in the app.
Brendan is going to manually move changes over to the new system
incrementally until we get the kinks worked out.  I think we're close,
but we're not quite there yet.  As a reminder, if you'd like to review
or submit patches against the source, you can find it here:

http://git.postgresql.org/gitweb?p=pgcommitfest.git;a=summary
git://git.postgresql.org/git/pgcommitfest.git

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Robert Haas
On Fri, Jul 3, 2009 at 12:32 AM, Fujii Masaomasao.fu...@gmail.com wrote:
 Hi,

 On Fri, Jul 3, 2009 at 11:02 AM, Robert Haasrobertmh...@gmail.com wrote:
 On Tue, Jun 16, 2009 at 2:13 AM, Fujii Masaomasao.fu...@gmail.com wrote:
 The attached latest patch provides this capability. You can easily set up 
 the
 synch rep according to the following procedure.
 http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects#How_to_set_up_Synch_Rep

 This patch no longer applies cleanly.  Can you rebase and resubmit it
 for the upcoming CommitFest?  It might also be good to go through and
 clean up the various places where you have trailing whitespace and/or
 spaces preceding tabs.

 Sure. I'll resubmit the patch after fixing some bugs and finishing
 the documents.

 Given that this is a substantial patch, I have a couple of questions
 about strategy.  First, I am wondering whether this patch should be
 reviewed (and committed) as a whole, or whether there are distinct
 chunks of it that should be reviewed and committed separately -
 particularly the signal handling piece, which AIUI is independently
 useful.  I note that it seems to be included in the tarball as a
 separate patch file, which is very useful.

 I think that the latter strategy makes more sense. At least, the signal
 handling piece and non-blocking pqcomm (communication between
 a frontend and a backend) can be reviewed independently of synch rep
 itself.

My preference for ease of CommitFest management would be one thread on
-hackers for each chunk that can be separately reviewed and committed.
 So if there are three severable chunks here, send a patch for each
one with a descriptive subject line, and mention the dependencies in
the body of the email (before applying this patch, you must first
apply blah blah link to archives).  That way, we can keep the
discussion of each topic separate, have separate entries on the
CommitFest page with subjects that match the email thread, etc.

Thanks,

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] SE-PostgreSQL Updates rev.2096

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 10:45 PM, KaiGai Koheikai...@kaigai.gr.jp wrote:
 Right now, I think it is preferable to focus on the first three patches
 at the first commit fest.

 What's your opinion?

If the first three patches are the least amount of code that makes a
complete feature, then that's the right way to go.  Please remove
everything from those patches that isn't part of the basic feature set
(e.g. row-level security) and repost the updated versions of just
those patches, so that it is clear which code we are reviewing.  Maybe
call it SE-pgsql lite, or whatever.

Thanks,

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synch Rep: direct transfer of WAL file from the primary to the standby

2009-07-02 Thread Fujii Masao
Hi,

On Fri, Jul 3, 2009 at 2:01 PM, Robert Haasrobertmh...@gmail.com wrote:
 My preference for ease of CommitFest management would be one thread on
 -hackers for each chunk that can be separately reviewed and committed.
  So if there are three severable chunks here, send a patch for each
 one with a descriptive subject line, and mention the dependencies in
 the body of the email (before applying this patch, you must first
 apply blah blah link to archives).  That way, we can keep the
 discussion of each topic separate, have separate entries on the
 CommitFest page with subjects that match the email thread, etc.

That sounds good. I'll submit the patches separately.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] HEAD is open for 8.5 development

2009-07-02 Thread Greg Smith

On Thu, 2 Jul 2009, Robert Haas wrote:

I think it's important to clarify the distinction between when a 
committer is planning to look at the patch but maybe somebody else 
should too, and when the committer is planning to take exclusive 
responsibility for the patch and nobody else should bother touching it.


Suggesting additional work for the committers is never a good idea, and 
the tagging scheme you suggested seems pretty heavy for something that's 
not particularly common.  How about you add an additional Committer 
field to the table, to be filled in when it's appropriate to do so and 
defaulting to nobody as usual.  That would be handy in a lot of cases to 
track who is working on that part of patch application anyway.  And in the 
special case you're trying to handle, I'd think listing their name under 
both reviewer/committer fields would be sufficient to deflect wasted 
review effort.


Also, if people date-stamp their comments (which will happen 
automatically if we get over to my new system), it makes it possible to 
get a sense of how long it's been since a certain state change took 
place.


The Mediawiki standard here is that you can use the various tilde macros 
to fill these in:  http://www.mediawiki.org/wiki/Help:Signatures


I wonder if it's possible to include that expansion in the CommitFest 
templates?  Even if it's not, we could strongly encourage its use.  When 
you're using the page editor, one of the little buttons at the top is 
Your signature with timestamp, and it expands to --; that makes it 
easy to remember this particular bit.


--
* Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] SE-PostgreSQL Updates rev.2096

2009-07-02 Thread KaiGai Kohei

Robert Haas wrote:

On Thu, Jul 2, 2009 at 10:45 PM, KaiGai Koheikai...@kaigai.gr.jp wrote:

Right now, I think it is preferable to focus on the first three patches
at the first commit fest.

What's your opinion?


If the first three patches are the least amount of code that makes a
complete feature, then that's the right way to go.  Please remove
everything from those patches that isn't part of the basic feature set
(e.g. row-level security) and repost the updated versions of just
those patches, so that it is clear which code we are reviewing.  Maybe
call it SE-pgsql lite, or whatever.


OK, I'll re-organize my patch set.
Please wait for a few days.

--
KaiGai Kohei kai...@kaigai.gr.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-02 Thread Brendan Jurd
2009/7/3 Robert Haas robertmh...@gmail.com:
 Also, we're currently missing the reviewer names
 due to limitations of the import script; Brendan is fixing this.

Update: The reviewer names have now been populated.

Cheers,
BJ

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-02 Thread Jaime Casanova
On Fri, Jul 3, 2009 at 12:34 AM, Brendan Jurddire...@gmail.com wrote:
 2009/7/3 Robert Haas robertmh...@gmail.com:
 Also, we're currently missing the reviewer names
 due to limitations of the import script; Brendan is fixing this.

 Update: The reviewer names have now been populated.


looks good... now, how can i mark a patch as being reviewed? or i have
to told you in order to you mark it?

-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [pgsql-www] commitfest.postgresql.org

2009-07-02 Thread Brendan Jurd
2009/7/3 Jaime Casanova jcasa...@systemguards.com.ec:
 On Fri, Jul 3, 2009 at 12:34 AM, Brendan Jurddire...@gmail.com wrote:
 2009/7/3 Robert Haas robertmh...@gmail.com:
 Also, we're currently missing the reviewer names
 due to limitations of the import script; Brendan is fixing this.

 Update: The reviewer names have now been populated.


 looks good... now, how can i mark a patch as being reviewed? or i have
 to told you in order to you mark it?

Until we have a solid consensus to switch to using the new app, please
continue to make your changes on the wiki.  I'll replicate the changes
by hand as we go until switchover.

If you'd like to make your changes on the app yourself and save me the
trouble, please do!  Just select the patch you're interested in, click
edit patch on the upper right and change the status and/or reviewer
fields as required.

Please let us know if you encounter any problems with withe app, or
have suggestions for improvement.

Cheers,
BJ

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers