Re: [HACKERS] Query progress indication - an implementation
+1. Especially if I run it a few times and I can see which counters are still moving. Per-node percentage is easy to do (given the perfect estimates, of course). The problem comes when you want to give an overall percentage. I wouldn't know where to put that explain-like output though: in a column in pg_stat_get_activity??? (and it would be available only if the proper variable was on before sending the query) -1. A counter that slowly goes from 99% to 99.5% done is much worse than a counter that takes the same much time going from 1000% of estimated rows done to 2000% of estimated rows done. It's not just about estimates. Even with 100% correct estimates, IMHO there's no way to get the perfect amount of work done so far. And this is even without considering multiple queries running at the same time... If someone has some time to read those papers let me know what he thinks about them... because I think their methods couldn't give them those results... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH 0/2 v3] SIGPIPE masking in local socket connections
A new approach to avioding manipulating the signal mask during for every
send - this time round, use SO_NOSIGPIPE and MSG_NOSIGNAL if available.
The patches have been tested on Linux and OSX, and I've confirmed that
'struct foo { };' will compile with a MSVC compiler. I'd still like a
little more testing though, is there a machine that allows both
SO_NOSIGPIPE and MSG_NOSIGNAL?
v3: respin as context diffs
Again, comments most welcome,
Jeremy
---
Jeremy Kerr (2):
[libpq] rework sigpipe-handling macros
[libpq] Try to avoid manually masking SIGPIPEs on every send()
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH 2/2 v3] [libpq] Try to avoid manually masking SIGPIPEs on every send()
Currently, libpq will wrap each send() call on the connection with
two system calls to mask SIGPIPEs. This results in 3 syscalls instead
of one, and (on Linux) can lead to high contention on the signal
mask locks in threaded apps.
We have a couple of other methods to avoid SIGPIPEs:
sockopt(SO_NOSIGPIPE) and the MSG_NOSIGNAL flag to send().
This change attempts to use these if they're available at compile-
and run-time. If not, we drop back to manipulating the signal mask as
before.
Signed-off-by: Jeremy Kerr j...@ozlabs.org
---
src/interfaces/libpq/fe-connect.c | 40 ++
src/interfaces/libpq/fe-secure.c | 83 +-
src/interfaces/libpq/libpq-int.h |2
3 files changed, 107 insertions(+), 18 deletions(-)
*** a/src/interfaces/libpq/fe-connect.c
--- b/src/interfaces/libpq/fe-connect.c
***
*** 1085,1090 keep_going: /* We
will come back to here until there is
--- 1085,1091
while (conn-addr_cur != NULL)
{
struct addrinfo *addr_cur =
conn-addr_cur;
+ int optval;
/* Remember current address for
possible error msg */
memcpy(conn-raddr.addr,
addr_cur-ai_addr,
***
*** 1149,1154 keep_going: /* We
will come back to here until there is
--- 1150,1194
}
#endif /* F_SETFD */
+ /* We have three methods of blocking
sigpipe during
+* send() calls to this socket:
+*
+* - setsockopt(sock, SO_NOSIGPIPE)
+* - send(sock, ..., MSG_NOSIGNAL)
+* - setting the signal mask to
SIG_IGN during send()
+*
+* The first two reduce the number of
syscalls (for the
+* third, we require three syscalls to
implement a send()),
+* so use them if they're available.
Their availability is
+* flagged in the following members of
PGconn:
+*
+* conn-sigpipe_so - we
have set up SO_NOSIGPIPE
+* conn-sigpipe_flag - we're
specifying MSG_NOSIGNAL
+*
+* If we can use SO_NOSIGPIPE, then set
sigpipe_so here and
+* we don't need to care about anything
else. Otherwise,
+* try MSG_NOSIGNAL by setting
sigpipe_flag. If we get an
+* error with MSG_NOSIGNAL, we clear
the flag and revert
+* to manual masking.
+*/
+ conn-sigpipe_so = false;
+ #ifdef MSG_NOSIGNAL
+ conn-sigpipe_flag = true;
+ #else /* !MSG_NOSIGNAL */
+ conn-sigpipe_flag = false;
+ #endif /* MSG_NOSIGNAL */
+
+ #ifdef SO_NOSIGPIPE
+ optval = 1;
+ if (!setsockopt(conn-sock, SOL_SOCKET,
SO_NOSIGPIPE,
+ (char *)optval,
sizeof(optval)))
+ {
+ conn-sigpipe_so = true;
+ conn-sigpipe_flag = false;
+ }
+ #endif /* SO_NOSIGPIPE */
+
+
/*
* Start/make connection. This should
not block, since we
* are in nonblock mode. If it does,
well, too bad.
*** a/src/interfaces/libpq/fe-secure.c
--- b/src/interfaces/libpq/fe-secure.c
***
*** 122,127 static long win32_ssl_create_mutex = 0;
--- 122,139
*/
#ifndef WIN32
+
+ static inline int sigpipe_masked(PGconn *conn)
+ {
+ /* If we're on an SSL connection, we can only use SO_NOSIGPIPE masking.
+* Otherwise, we can handle SO_NOSIGPIPE or the MSG_NOSIGNAL flag */
+ #ifdef USE_SSL
+ if (conn-ssl)
+ return conn-sigpipe_so;
+ #endif
+ return conn-sigpipe_so || conn-sigpipe_flag;
+ }
+
#ifdef ENABLE_THREAD_SAFETY
struct sigpipe_info {
***
*** 130,137
[HACKERS] [PATCH 1/2 v3] [libpq] rework sigpipe-handling macros
Currently, the sigpipe-masking code in libpq is implemented as
a set of macros, which depend on declaring local variables.
This change adds a (private) struct sigpipe_info to contain the
compile-dependent data required for sigpipe masking and restoring.
The caller can then declare a struct sigpipe info explicitly, and
pass this to the subsequent sigpipe-masking code.
This allows us to separate the variable declarations from the code,
and gives the caller more flexibility for controlling the scope of
these variables.
Also, since we don't need to declare variables in the macros, we
can change the code to be implemented as static inlines.
Signed-off-by: Jeremy Kerr j...@ozlabs.org
---
src/interfaces/libpq/fe-secure.c | 88 ---
1 file changed, 55 insertions(+), 33 deletions(-)
*** a/src/interfaces/libpq/fe-secure.c
--- b/src/interfaces/libpq/fe-secure.c
***
*** 119,163 static long win32_ssl_create_mutex = 0;
/*
* Macros to handle disabling and then restoring the state of SIGPIPE
handling.
- * Note that DISABLE_SIGPIPE() must appear at the start of a block.
*/
#ifndef WIN32
#ifdef ENABLE_THREAD_SAFETY
! #define DISABLE_SIGPIPE(failaction) \
! sigset_tosigmask; \
! boolsigpipe_pending; \
! boolgot_epipe = false; \
! \
! if (pq_block_sigpipe(osigmask, sigpipe_pending) 0) \
! failaction
! #define REMEMBER_EPIPE(cond) \
! do { \
! if (cond) \
! got_epipe = true; \
! } while (0)
! #define RESTORE_SIGPIPE() \
! pq_reset_sigpipe(osigmask, sigpipe_pending, got_epipe)
#else /* !ENABLE_THREAD_SAFETY */
! #define DISABLE_SIGPIPE(failaction) \
! pqsigfunc oldsighandler = pqsignal(SIGPIPE, SIG_IGN)
! #define REMEMBER_EPIPE(cond)
! #define RESTORE_SIGPIPE() \
! pqsignal(SIGPIPE, oldsighandler)
#endif/* ENABLE_THREAD_SAFETY */
#else /* WIN32 */
! #define DISABLE_SIGPIPE(failaction)
! #define REMEMBER_EPIPE(cond)
! #define RESTORE_SIGPIPE()
#endif/* WIN32 */
--- 119,180
/*
* Macros to handle disabling and then restoring the state of SIGPIPE
handling.
*/
#ifndef WIN32
#ifdef ENABLE_THREAD_SAFETY
! struct sigpipe_info {
! sigset_toldsigmask;
! boolsigpipe_pending;
! boolgot_epipe;
! };
! static inline int disable_sigpipe(struct sigpipe_info *info)
! {
! info-got_epipe = false;
! return pq_block_sigpipe(info-oldsigmask, info-sigpipe_pending) 0;
! }
! static inline void remember_epipe(struct sigpipe_info *info, bool cond)
! {
! if (cond)
! info-got_epipe = true;
! }
!
! static inline void restore_sigpipe(struct sigpipe_info *info)
! {
! pq_reset_sigpipe(info-oldsigmask, info-sigpipe_pending,
info-got_epipe);
! }
#else /* !ENABLE_THREAD_SAFETY */
! struct sigpipe_info {
! pqsigfunc oldhandler;
! };
! static inline int disable_sigpipe(struct sigpipe_info *info)
! {
! info-oldhandler = pqsignal(SIGPIPE, SIG_IGN);
! return 0;
! }
!
! static inline void remember_epipe(struct sigpipe_info *info, bool cond)
! {
! }
! static inline void restore_sigpipe(struct sigpipe_info *info)
! {
! pqsignal(SIGPIPE, info-oldhandler);
! }
#endif/* ENABLE_THREAD_SAFETY */
#else /* WIN32 */
! struct sigpipe_info { };
! static inline int disable_sigpipe(struct sigpipe_info *info) { return 0; }
! static inline void remember_epipe(struct sigpipe_info *info, bool cond) { }
! static inline void restore_sigpipe(struct sigpipe_info *info) { }
#endif/* WIN32 */
***
*** 286,294 pqsecure_read(PGconn *conn, void *ptr, size_t len)
if (conn-ssl)
{
int err;
/* SSL_read can write to the socket, so we need to disable
SIGPIPE */
! DISABLE_SIGPIPE(return -1);
rloop:
n = SSL_read(conn-ssl, ptr, len);
--- 303,313
if (conn-ssl)
{
int err;
+ struct sigpipe_info info;
/* SSL_read can write to the socket, so we need to disable
SIGPIPE */
! if (disable_sigpipe(info))
! return -1;
rloop:
n = SSL_read(conn-ssl, ptr, len);
***
*** 315,321 rloop:
if (n == -1)
{
! REMEMBER_EPIPE(SOCK_ERRNO ==
EPIPE);
printfPQExpBuffer(conn-errorMessage,
libpq_gettext(SSL SYSCALL error: %s\n),
SOCK_STRERROR(SOCK_ERRNO, sebuf,
Re: [HACKERS] foreign.h is not installed
On Tuesday 30 June 2009 07:36:36 Itagaki Takahiro wrote: Header file include/foreign/foreign.h is not installed in 8.4.0. Did we forget to add subdir foreign to Makefile? So it seems. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hello to all postgresql developers :)
Hello, I am Oracle developer for 2 years, and I have a magister work - to realize TPR index on RDBMS :) I desided to realize TPR index on PostgreSql RDBMS. I am really new programmer in C language. what I done 1 Installed Xubuntu on my computer. 2 build and installed PostgreSql. 3 Installed Anjuta on my computer. Can anybody advise what I need to do next. Grateful, Sergej
[HACKERS] [PATCH v3] Avoid manual shift-and-test logic in AllocSetFreeIndex
Move the shift-and-test login into a separate fls() function, which
can use __builtin_clz() if it's available.
This requires a new check for __builtin_clz in the configure script.
Results in a ~2% performance increase on PowerPC.
Signed-off-by: Jeremy Kerr j...@ozlabs.org
---
v3: respin as context diff
---
configure.in | 13 +
src/backend/utils/mmgr/aset.c | 34 +++---
2 files changed, 40 insertions(+), 7 deletions(-)
*** a/configure.in
--- b/configure.in
***
*** 1361,1366 case $host_os in
--- 1361,1379
AC_FUNC_FSEEKO;;
esac
+ # GCC builtins
+ #
+ # We need AC_TRY_LINK here, as the prototype generated by AC_CHECK_FUNC
+ # will cause gcc to try to reference a non-builtin symbol.
+
+ AC_MSG_CHECKING([for __builtin_clz])
+ AC_TRY_LINK([],
+ [__builtin_clz(0);],
+ [AC_DEFINE(HAVE_BUILTIN_CLZ, 1,
+ [Define to 1 if you have __builtin_clz().])
+ AC_MSG_RESULT(yes)],
+ [AC_MSG_RESULT(no)])
+
#
# Pthreads
*** a/src/backend/utils/mmgr/aset.c
--- b/src/backend/utils/mmgr/aset.c
***
*** 255,260 static MemoryContextMethods AllocSetMethods = {
--- 255,285
#define AllocAllocInfo(_cxt, _chunk)
#endif
+ /*
+ * fls: find last set bit.
+ *
+ * Returns the 1-based index of the most-significant bit in x. The MSB
+ * is bit number 32, the LSB is bit number 1. If x is zero, the result is
+ * undefined.
+ */
+ static inline int
+ fls(unsigned int x)
+ {
+ #ifdef HAVE_BUILTIN_CLZ
+ return 32 - __builtin_clz(x);
+ #else
+ int ls = 0;
+
+ while (x != 0)
+ {
+ ls++;
+ x = 1;
+ }
+
+ return ls;
+ #endif
+ }
+
/* --
* AllocSetFreeIndex -
*
***
*** 268,281 AllocSetFreeIndex(Size size)
{
int idx = 0;
! if (size 0)
{
! size = (size - 1) ALLOC_MINBITS;
! while (size != 0)
! {
! idx++;
! size = 1;
! }
Assert(idx ALLOCSET_NUM_FREELISTS);
}
--- 293,301
{
int idx = 0;
! if (size (1 ALLOC_MINBITS))
{
! idx = fls((size - 1) ALLOC_MINBITS);
Assert(idx ALLOCSET_NUM_FREELISTS);
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] [v8.5] Security checks on largeobjects
I concluded that the following issues should be solved when we apply
largeobject-like interfaces on the big toasted data within general
relations, not only pg_largeobject system catalog.
At first, we need to add a new strategy to store the given varlena data
on the external toast relation.
If we try to seek and fetch a certain data chunk, it is necessary to be
computable what chunk stores the required data specified by offset and
length. So, the external chunks should be uncompressed always. It is a
common requirement for both of read and write operations.
If we try to update a part of the toasted data chunks, it should not be
inlined independent from length of the datum, because we need to update
whole the tuple which contains inlined toasted chunks in this case.
If we open the toasted varlena with read-only mode, inlined one does not
prevent anything. It is an issue for only write operation.
I would like to add a new strategy on pg_type.typstorage with the following
characteristics:
1. It always stores the given varlena data without any compression.
So, the given data is stored as a set of fixed-length chunks.
2. It always stores the given varlena data on external toast relation.
I suggest a new built-in type named BLOB which has an identical definition
to BYTEA type, expect for its attstorage.
Next, a different version of lo_open() should be provided to accept
BLOB type as follows:
SELECT pictname, lo_open(pictdata, x'2'::int) FROM my_picture;
It will allocate a largeobject descriptor for the given BLOB data,
and user can read and write using loread() and lowrite() interfaces.
issue:
In this case, should it hold the relation handler and locks on the
my_picture relation, not only its toast relation?
issue:
Should the lo_open() with read-only mode be available on the existing
TEXT or BYTEA types? I could not find any reason to deny them.
Next, pg_largeobject system catalog can be redefined using the BLOB
type as follows:
CATALOG(pg_largeobject,2613)
{
Oid loowner;/* OID of the largeobject owner */
Oid lonsp; /* OID of the largeobject namespace */
aclitem loacl[1]; /* access permissions */
bloblodata; /* contents of the largeobject */
} FormData_pg_largeobject;
The existing largeobject interfaces perform on pg_largeobject.lodata
specified by largeobject identifier.
Rest of metadata can be used for access control purpose.
Thanks,
KaiGai Kohei wrote:
Tom Lane wrote:
Bernd Helmle maili...@oopsware.de writes:
It might be interesting to dig into your proposal deeper in conjunction
with TOAST (you've already mentioned this TODO). Having serial access with
a nice interface into TOAST would be eliminating the need for
pg_largeobject completely (i'm not a big fan of this one-big-system-table
approach the old LO interface currently is).
Yeah, it would be more useful probably to fix that than to add
decoration to the LO facility. Making LO more usable is just going to
encourage people to bump into its other limitations (32-bit OIDs,
32-bit object size, finite maximum size of pg_largeobject, lack of
dead-object cleanup, etc etc).
The reason why I tried to mention the named largeobject feature is
that dac security checks on largeobject require them to belong to
a certain schema, so I thought it is quite natural to have a string
name. However, obviously, it is not a significant theme for me.
I can also agree your opinion that largeobject interfaces should be
redefined to access partial stuff of TOAST'ed verlena data structure,
not only pg_largeobject.
In this case, we will need a new pg_type.typstorage option which
force to put the given verlena data on external relation without
compression, because we cannot estimate the data offset in inlined
or compressed external verlena data.
I'll try to submit a design within a few days.
Thanks,
--
OSS Platform Development Division, NEC
KaiGai Kohei kai...@ak.jp.nec.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hello to all postgresql developers :)
On Tuesday 30 June 2009 10:06:49 Sergej Galkin wrote: Hello, I am Oracle developer for 2 years, and I have a magister work - to realize TPR index on RDBMS :) I desided to realize TPR index on PostgreSql RDBMS. I am really new programmer in C language. what I done 1 Installed Xubuntu on my computer. 2 build and installed PostgreSql. 3 Installed Anjuta on my computer. Can anybody advise what I need to do next. I suggest you start perusing the information here: http://wiki.postgresql.org/wiki/Development_information -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Query progress indication - an implementation
On Tue, 2009-06-30 at 07:04 +0200, Dimitri Fontaine wrote: Le 30 juin 2009 à 01:34, Greg Stark gsst...@mit.edu a écrit : Basically I disagree that imperfect progress reports annoy users. I think we can do better than reporting 250% done or having a percentage that goes backward though. It would be quite tolerable (though perhaps for no logical reason) to have a progress indicator which slows done as it gets closer to 100% and never seems to make it to 100%. I guess bad stats are such an important problem in planning queries that a 250% progress is doing more good than harm in showing users how badly they need to review their analyze related settings. Yeh, I agree. We can define it as planned work, rather than actual. So if the progress bar says 250% and query is still going at least you know it is doing more work, rather than just being slow at doing the planned work. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Query progress indication - an implementation
On Mon, 2009-06-29 at 18:49 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Mon, 2009-06-29 at 14:07 -0400, Tom Lane wrote: I think this is pretty much nonsense --- most queries run all their plan nodes concurrently to some extent. You can't usefully say that a query is on some node, nor measure progress by whether some node is done. The requirement is not nonsense, even if the detail was slightly off. I was applying the word nonsense to the proposed implementation, not the desire to have query progress indications ... Understood, just trying to limit the blast radius. We can regard plans as acting in phases with each blocking node separating the plan. We know which nodes those are, so we can report that. [ shrug... ] You can regard them that way, but you won't get particularly helpful results for a large fraction of real queries. The system is generally set up to prefer streaming evaluation as much as it can. Even in nominally blocking nodes like Sort and Hash, there are operational modes that look more like streaming, or at least chunking. It's not always useful, though many large queries do have multiple phases. The concept and the name come from ETL tools and it is of real practical use in those environments. We can put the phase number on the EXPLAIN easily, and it is very simple to calculate the total number of phases and the current phase - e.g. 2 of 5 phases complete. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] user mapping extension to pg_ident.conf
Am Montag, 29. Juni 2009 18:12:13 schrieben Sie: Lars Kanis ka...@comcard.de writes: The problem I have, is that I want to use an ordinary windows application, which connects to an arbitrary ODBC data source. This application stores a fixed username und password for the connection within it's own binary data file. It doesn't know anything about TLS-connection nor smartcard based authentication. All this is done in the libpg.dll. This seems to boil down to I'm willing to damage PG's authentication mechanisms to an unlimited extent to work around one broken proprietary windows app. I'm not excited about it. Oh no! You totally misunderstood me. So please give me another try to explain the addition: It's NOT about one stupid or broken (it isn't) application that has to be corrected. It's NOT about a specific problem with ODBC connections. It's a much more general thing, regardless which application/programming language/connection-library you use. I wouldn't post a quick an dirty hack to work around one broken proprietary windows app! The situation with any external single sign-on authentication method (GSSAPI, SSPI, Kerberos, Certificate authentication) is that you always have to provide a username. It is differentiated between the SYSTEM-USERNAME and the PG-USERNAME. The SYSTEM-USERNAME is the name the user is identified to by the external authentication. It is usually done with cryptographic validation of some tickets or certificates. So this value is the important one in terms of security. The PG-USERNAME is what the application tells the server the user wants to act as, in the database. It has no value at all in terms of security - it's just a wish. It is verified through the mapping of pg_ident.conf whether the wish can be granted with the given SYSTEM-USERNAME. The problem arises when you want to work with 1:1 mapping of SYSTEM-USERNAME to PG-USERNAME. You don't want to let the user type in the PG-USERNAME he wants to act as (I have never seen an kerberos-enabled application to do so). So the application has to take into account which authentication method you actually use, to extract the exact username from credentials/certificate. That means you have to open a second channel to your SSO-System to provide the correct username to the PG-server. And it means you have to do some sort of duplicated code in the application, which is already realised within libpq. Currently you can not say: Dear libpq, please identify the current user and authenticate to the PG-server (by whatever authentication methods you're allowed to by pg_hba.conf), and log in to the own account of the user. My addition allows to abstract the application from the used authentication method. It adds an level of indirection between PG-USERNAME (given by the application) and the EFFECTIVE-USERNAME (for actual db-role) based on the SYSTEM-USERNAME (given by the SSO-system). And it adds it in a way that fits nicely into the already existend mapping-file pg_ident.conf. Let's show the following example-table in pg_ident.conf: # MAPNAME SYSTEM-USERNAMEPG-USERNAME EFFECTIVE-USERNAME gssapi-user/^(.*)@domain\.com$simple-role \1 gssapi-user/^use...@domain\.com$ super-roleuser_a gssapi-user/^use...@domain\.com$ super-roleuser_c The first line says: Any user with an account on the PG-server and valid GSSAPI-credentials can log in to it's own PG-account. The application doesn't need to care about which user is actually working with it. It simply needs to say to server to use the users 'simple-role'. You can write the application completely independent to the authentication method to use. For whatever security relevant action in this or another application, you can log in to the 'super-role' of the user. But only user_a and user_c are able to do so. Likewise the application don't need to care about the actually used SSO-method or system. Also to mention: The addition is fully backward compatible. Have you even considered the potential for security problems arising from this? Yes, of course. The primary goal of the addition is to get more security. Some questions I considered: Can I go around permission checks which are carried out by PG? That's the most important question, I think. As I stated above, the PG-USERNAME always has an informative nature. It's just a wish with no value in terms of security. So it is safe to change the name, as long as all usual permission checks are done on the changed one. In fact they are done. The identification and validation of users credentials are completed, so we can trust the SYSTEM-USERNAME. Then the pg_ident.conf comes in action and determines the EFFECTIVE-USERNAME. Any further permission checks are done afterwards. So the EFFECTIVE-USERNAME is checked against login-permission and so on. Does it get more security to give every application user it's own account within the database? Yes.
Re: [HACKERS] [PATCH] user mapping extension to pg_ident.conf
# MAPNAME SYSTEM-USERNAMEPG-USERNAME EFFECTIVE-USERNAME gssapi-user/^(.*)@domain\.com$simple-role \1 gssapi-user/^use...@domain\.com$ super-roleuser_a gssapi-user/^use...@domain\.com$ super-roleuser_c My fault. The lower lines should be: gssapi-user/^use...@domain\.com$ super-rolesuper_user_a gssapi-user/^use...@domain\.com$ super-rolesuper_user_c regards Lars Kanis signature.asc Description: This is a digitally signed message part.
[HACKERS] use of pg_stat_database
Hellow, I like to track the number of committed transaction for my database So I use the following command Select * from pg_stat_database; The column xact_commit gives me the number of transaction committed But if I execute the same command another time, the column xact_commit gives me an other number. Between the first select and the second select I don’t execute any transaction on my database Please, can any one help me to understanding the cause of the change? Thanks very much
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 1:33 AM, Peter Eisentrautpete...@gmx.net wrote: Now that 8.4.0 is out the door, development for 8.5devel will be opened any day now. But we haven't discussed the development timeline so far. The core team has several proposals: CommitFest Alpha Aug. 1 Sept. 1 Oct. 1 Nov. 1 Dec. 1 Jan ~~ 5 Feb. 1 March 4 Release ~ May 2010 This puts us on track for a release at the same time next year, maybe a little earlier. (Alpha is a semiformal snapshot release at the end of the commitfest, for those who haven't heard yet. Details later.) If we want to avoid a commitfest in December, then this: CommitFest Alpha Sept. 1 Oct. 1 Nov. 1 Dec. 1 Jan. 1 Feb 1 March 1 April 2 Release ~ June 2010 But this has the drawback of waiting an extra month for the first commit fest, for no particularly good reason. (Check the current list, if you are curious.) Or, one more commitfest: CommitFest Alpha Aug. 1 Sept. 1 Oct. 1 Nov. 1 Dec. 1 Jan ~~ 5 Feb. 1 March 3 April 3 May 3 Release ~ July 2010 But that gets 8.5 out even later than this year, and past PGCon. Comments? Waiting until September for the first CommitFest seems like a really bad idea. We already have almost 40 patches on the wiki page, and there are some that haven't been added yet: I suspect we will have over 50 in another week, and maybe closer to 60. If we wait two months, we're likely to have 100 patches or more, which will be a reviewing effort that I don't like to think about. It will also increase the number of patches that collide in mid-air. So at the very latest, the first CommitFest should start August 1. However, if anything, I think if anything we should go the other way and start the first CommitFest July 15th. That may give people a little less time than they were expecting to finish up WIP, but I think it's worth it to give people who have already submitted patches feedback that much sooner, as well as to maintain reviewer and committer sanity. By the way, are going to switch over to the commitfest management tool I wrote (http://coridan.postgresql.org/)? There's room for improvement, but it's a solid starting point, and all the comments I have received so far have been basically positive. Also by the way, I'd be willing to be a commitfest manager, co-commitfest manager, or some other supporting role of that type, if that would be helpful. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] use of pg_stat_database
In response to abdelhak benmohamed : Hellow, I like to track the number of committed transaction for my database So I use the following command Select * from pg_stat_database; The column xact_commit gives me the number of transaction committed But if I execute the same command another time, the column xact_commit gives me an other number. I can't reproduce that: test=*# Select sum(xact_commit) from pg_stat_database; sum --- 73470 (1 row) -- 5 minutes later: test=*# Select sum(xact_commit) from pg_stat_database; sum --- 73470 (1 row) Between the first select and the second select I don?t execute any transaction on my database Autovacuum? Andreas -- Andreas Kretschmer Kontakt: Heynitz: 035242/47150, D1: 0160/7141639 (mehr: - Header) GnuPG-ID: 0x3FFF606C, privat 0x7F4584DA http://wwwkeys.de.pgp.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 12:22 PM, Robert Haasrobertmh...@gmail.com wrote: Waiting until September for the first CommitFest seems like a really bad idea. We already have almost 40 patches on the wiki page, and there are some that haven't been added yet: I suspect we will have over 50 in another week, and maybe closer to 60. I would like to propose a different strategy. Instead of always tackling all the smaller patches and leaving the big patches for last, I would suggest we start with Hot Standby. In fact I would suggest as Hot Standby has already gotten a first pass review that we consider applying it on day 1. That gets it into everyone's development trees so they can see any suspicious code or effects it has in their peculiar environments. It may not be perfect but if we apply it now there's plenty of time to make improvements. Then we can have a regular commitfest a month or so later. Hopefully any followon changes to Hot Standby would actually get into that commitfest if they're relatively minor. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] question about meaning of character varying without length
Konstantin Izmailov wrote: Delivery to the following recipient failed permanently: pgsql-gene...@postgresql.com mailto:pgsql-gene...@postgresql.com should be pgsql-gene...@postgresql.org, not pgsql-gene...@postgresql.com. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 7:53 AM, Greg Starkgsst...@mit.edu wrote: On Tue, Jun 30, 2009 at 12:22 PM, Robert Haasrobertmh...@gmail.com wrote: Waiting until September for the first CommitFest seems like a really bad idea. We already have almost 40 patches on the wiki page, and there are some that haven't been added yet: I suspect we will have over 50 in another week, and maybe closer to 60. I would like to propose a different strategy. Instead of always tackling all the smaller patches and leaving the big patches for last, I would suggest we start with Hot Standby. In fact I would suggest as Hot Standby has already gotten a first pass review that we consider applying it on day 1. That gets it into everyone's development trees so they can see any suspicious code or effects it has in their peculiar environments. It may not be perfect but if we apply it now there's plenty of time to make improvements. Then we can have a regular commitfest a month or so later. Hopefully any followon changes to Hot Standby would actually get into that commitfest if they're relatively minor. If Hot Standby were ready to be applied, I would be all in favor of that, but in fact I don't believe that's the case. There's been no movement on Hot Standby since February, or at least nothing on the mailing list and no changes to http://git.postgresql.org/gitweb?p=simon.git;a=summary My recollection is there was some discussion of whether Simon was even prepared to put any more work into that patch or leave it others (in particular, Heikki) to finish. I think we had better resolve the question of who is going to finish that patch and when they plan to do it before we start planning our CommitFest schedule around it. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 1:04 PM, Robert Haasrobertmh...@gmail.com wrote: If Hot Standby were ready to be applied, I would be all in favor of that, but in fact I don't believe that's the case. There's been no movement on Hot Standby since February Well Simon was happy with it as submitted so unless people are reading the patch and giving feedback or using it and running into problems I wouldn't really expect him to make changes. That's part of the problem with leaving patches outside the source tree while they're being developed. It's part of what led us to suddenly have a massive reviewing job and making big changes in the final commitfest. If we apply things earlier in the cycle we can be a lot less conservative. We don't have to be 100% sure everything was dealt with in a single commit. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hello to all postgresql developers :)
Sergej Galkin sergej.gal...@gmail.com writes: I am Oracle developer for 2 years, and I have a magister work - to realize TPR index on RDBMS :) I desided to realize TPR index on PostgreSql RDBMS. I am really new programmer in C language. I don't know what a TPR index is, but it could be that the following material could help you: http://wiki.postgresql.org/wiki/Image:Prato_2008_prefix.pdf Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to register my function into backend?
Oh. Thanks for your instruction! Best regards. Bruce 2009/6/30 Robert Haas robertmh...@gmail.com On Mon, Jun 29, 2009 at 4:53 AM, Bruce YUANsua...@gmail.com wrote:rsd My function is to collect some backend information for user anlysis. How to register my function into backend? It make that we can called it via libpg/PQfn(). Thanks! I'm not sure this is -hackers question; seems like -general might be more appropriate. I think you are looking for CREATE OR REPLACE FUNCTION. http://www.postgresql.org/docs/current/static/sql-createfunction.html http://www.postgresql.org/docs/current/static/xfunc.html ...Robert []
Re: [HACKERS] pre-proposal: permissions made easier
* Greg Stark gsst...@mit.edu [090630 00:18]: Perhaps tieing it to the schema is wrong and we should actually require the user to specify the template they want explicitly which would be even better for that. So it would be something like WITH GRANTS LIKE sensitive_table. And, not having any experience with the current permissions code, or the code required to do that (;-]), I would *love* something like that... *especially* if those grants remain by reference, i.e. If I change the GRANTS/REVOKES on sensitive_table, those are automatically apply to all tables created with the WITH GRANTS LIKE sensitive_table... It would simplify all the work I have to do in: make_$PERMISSION_ROLE(table) and make it much more elegant, and save me having to re-run them all if I want to change some permissions. But I realize that since I'm as anal about my database schemas as I am about my code, I'm probably not your typical DB dev shop people like Josh are used to dealing with... a. -- Aidan Van Dyk Create like a god, ai...@highrise.ca command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] pre-proposal: permissions made easier
Aidan Van Dyk wrote: *especially* if those grants remain by reference, i.e. If I change the GRANTS/REVOKES on sensitive_table, those are automatically apply to all tables created with the WITH GRANTS LIKE sensitive_table... Isn't that exactly what Tom is objecting to, namely that the permissions of an object would not be contained entirely in catalog entry for the object itself? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 8:12 AM, Greg Starkgsst...@mit.edu wrote: On Tue, Jun 30, 2009 at 1:04 PM, Robert Haasrobertmh...@gmail.com wrote: If Hot Standby were ready to be applied, I would be all in favor of that, but in fact I don't believe that's the case. There's been no movement on Hot Standby since February Well Simon was happy with it as submitted so unless people are reading the patch and giving feedback or using it and running into problems I wouldn't really expect him to make changes. That's part of the problem with leaving patches outside the source tree while they're being developed. It's part of what led us to suddenly have a massive reviewing job and making big changes in the final commitfest. If we apply things earlier in the cycle we can be a lot less conservative. We don't have to be 100% sure everything was dealt with in a single commit. +1 (I'm all for getting HS in people's hands ASAP) Given that there is also a lot of work on synchronous replication, is it better to get the HS in so the SR stuff can use that as a baseline, or to triage in both patches together? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pre-proposal: permissions made easier
* Andrew Dunstan and...@dunslane.net [090630 09:08]: Aidan Van Dyk wrote: *especially* if those grants remain by reference, i.e. If I change the GRANTS/REVOKES on sensitive_table, those are automatically apply to all tables created with the WITH GRANTS LIKE sensitive_table... Isn't that exactly what Tom is objecting to, namely that the permissions of an object would not be contained entirely in catalog entry for the object itself? Well, it depends on how it's done... If one of the permissions on an object you can assign is look at $X, the you don't get the hidden permissions problem. The object itself still contains everything you need to trace the permissions of an object... I have no idea if it's something that even half-aligns with the internal permission model/code... a. -- Aidan Van Dyk Create like a god, ai...@highrise.ca command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] use of pg_stat_database
On Tue, Jun 30, 2009 at 7:13 AM, abdelhak benmohamedabdelhak.benmoha...@yahoo.fr wrote: Hellow, I like to track the number of committed transaction for my database So I use the following command Select * from pg_stat_database; The column xact_commit gives me the number of transaction committed But if I execute the same command another time, the column xact_commit gives me an other number. Between the first select and the second select I don’t execute any transaction on my database Please, can any one help me to understanding the cause of the change? try turning on statement logging (log_statement='all'). please post qa time questions to -general. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 8:12 AM, Greg Starkgsst...@mit.edu wrote: On Tue, Jun 30, 2009 at 1:04 PM, Robert Haasrobertmh...@gmail.com wrote: If Hot Standby were ready to be applied, I would be all in favor of that, but in fact I don't believe that's the case. There's been no movement on Hot Standby since February Well Simon was happy with it as submitted so unless people are reading the patch and giving feedback or using it and running into problems I wouldn't really expect him to make changes. The last substantive email I can find on this topic is: http://archives.postgresql.org/message-id/1235644369.16176.480.ca...@ebony.2ndquadrant I would summarize that as saying that Simon was happy with the patch, but Heikki was not. Since I think it will be Heikki who ultimately commits this, the fact that he doesn't feel that it's ready for prime-time is a pretty important fact that we shouldn't overlook. Now, it's possible that Heikki has changed his mind, or it's possible that given where we are in the development cycle he'd be OK comitting it as-is to 8.5, or it's possible that some work has been done in the background and there's a committable version now, in which case - great! Or, alternatively, if Heikki wants to sit out the next CommitFest so that he can work on (and hopefully commit) Hot Standby, also great! But I don't see why other patches can't be committed in the meantime, assuming for the moment that they're not things which are likely to create massive merge problems (then again, the pgindent run has probably done that already). In any case, we probably need some weigh-in from Heikki and Simon on their plans for Hot Standby before we make any decisions... That's part of the problem wmith leaving patches outside the source tree while they're being developed. It's part of what led us to suddenly have a massive reviewing job and making big changes in the final commitfest. Yep. Figuring out what to do about that is a hard problem. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Robert Haas robertmh...@gmail.com wrote: Waiting until September for the first CommitFest seems like a really bad idea. We already have almost 40 patches on the wiki page, and there are some that haven't been added yet: I suspect we will have over 50 in another week, and maybe closer to 60. If we wait two months, we're likely to have 100 patches or more, which will be a reviewing effort that I don't like to think about. It will also increase the number of patches that collide in mid-air. So at the very latest, the first CommitFest should start August 1. However, if anything, I think if anything we should go the other way and start the first CommitFest July 15th. I'm curious what the counter-arguments to this are. Is it review-fatigue from getting the release out, or is there an economy of scale to building up a 100 patches before starting to review? Would reviewing these get some contributors moving again, thus boosting the total work hours available for the 8.5 release? Would it pull people off of WIP? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tuesday 30 June 2009 16:50:55 Kevin Grittner wrote: However, if anything, I think if anything we should go the other way and start the first CommitFest July 15th. I'm curious what the counter-arguments to this are. Is it review-fatigue from getting the release out, or is there an economy of scale to building up a 100 patches before starting to review? Would reviewing these get some contributors moving again, thus boosting the total work hours available for the 8.5 release? Would it pull people off of WIP? Well, think about what could happen if we go this way. What you basically have here are people who have essentially ignored the commitfest and beta mandates and worked on new patches. And they now get to say, because we already have enough patches, let's start the commit fest early. And then the same people might ignore the commitfest mandate again and produce another 100 patches by the time this commit fest ends. So let's start the next commit fest right after this one. So, I think, the schedule should be balanced, reflective of our desired development method, independent of momentary circumstances, and certainly not unduly influenced by those who chose to ignore the very same schedule. These points are debatable, but then you are almost debating the point of having a schedule at all. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Hi, However, if anything, I think if anything we should go the other way and start the first CommitFest July 15th. I'm curious what the counter-arguments to this are. Is it review-fatigue from getting the release out, or is there an economy of scale to building up a 100 patches before starting to review? Would reviewing these get some contributors moving again, thus boosting the total work hours available for the 8.5 release? Would it pull people off of WIP? Agreed, especially for some of the largish WIP (like the partitioning work for example) patches, a little effort being spent by our reviewers on agreeing (or disagreeing right now!) on the direction-implementation being pursued will go a long way in pulling those efforts off from the WIP mode. ISTM that identifying and quantifying a certain effort as small, medium, large and having an appropriate review mechanism in place might help too. Regards, Nikhils -- http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Peter Eisentraut pete...@gmx.net wrote: Well, think about what could happen if we go this way. What you basically have here are people who have essentially ignored the commitfest and beta mandates and worked on new patches. And they now get to say, because we already have enough patches, let's start the commit fest early. Well, patches started going onto the wiki page for this commit fest over seven months ago. Early review takes on a different meaning in this context. I think the basic problem with the schedule at this point is that we're releasing six months past the planned date; kinda throws things off. The question is how to get back on track and avoid that for 8.5. You probably have a point in that some of the patches came from people who might have been able to help more in the review and commit process, but I think some people lack the confidence to take on that role; and with the features that dragged out the release half-way to what would have been our next release date, there probably aren't many who could have made useful contributions to the review process. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 10:11 AM, Peter Eisentrautpete...@gmx.net wrote: On Tuesday 30 June 2009 16:50:55 Kevin Grittner wrote: However, if anything, I think if anything we should go the other way and start the first CommitFest July 15th. I'm curious what the counter-arguments to this are. Is it review-fatigue from getting the release out, or is there an economy of scale to building up a 100 patches before starting to review? Would reviewing these get some contributors moving again, thus boosting the total work hours available for the 8.5 release? Would it pull people off of WIP? Well, think about what could happen if we go this way. What you basically have here are people who have essentially ignored the commitfest and beta mandates and worked on new patches. Well, the only person who has proposed this so far is me, and I don't think there can be more than three or four people who are not committers who put in as much work into the November CommitFest as I did, and I've already volunteered to do more work for the next CommitFest. I'm not exactly sure what the beta mandate is, and I admit that I haven't done much beta-testing, but even just in the course of developing the patches I've submitted recently I've found several bugs which were fixed for 8.4 (try searching your -committers email for Robert Haas). I probably would not have found those bugs if I had just set out to test 8.4, because I wouldn't have thought of those things, so I really feel that I have done as well as I can. If you disagree, we should discuss, perhaps off-list. At any rate, the idea that nobody is should do any development during the seven months for which the tree has been in feature freeze doesn't seem like a very good one. If we accept that proposition, then presumably nobody should also do any development during August, October, or December, since those months are set aside for CommitFests. Therefore, during calendar year 2009, there will be a total of 91 days during which people are allowed to work on their own patches, specifically July 1-July 31, September 1-September 30, and November 1-30. How are we going to move this project forward by telling people that they're only allowed to do development 25% of the days out of the year? And even if we do accept that proposition, my proposal to back everything up 15 days wouldn't change the total number of development days: it would add the second half of December at the expense of the second half of July. I'm of the opinion that the way that we should be striving to maximize the amount of useful development that gets done, and I think the way to do that is to give people prompt feedback on their patches. A lot of the people who have submitted patches for the next CommitFest are first-time or occasional contributors who may already have lost interest in the project; waiting longer to review those patches is not going to increase the chances that those people will eventually get more involved, either as patch authors or as patch reviewers. Others are people like Fujii Masao, Kevin Grittner, and Pavan Deolasee who, I venture to say, have done enough work on this project to deserve having their contributions reviewed in a timely fashion, regardless of exactly when they choose to do their development. There may be a few people who aren't carrying the burden of contributing back to the community, but I don't think it's anything like a majority. And they now get to say, because we already have enough patches, let's start the commit fest early. And then the same people might ignore the commitfest mandate again and produce another 100 patches by the time this commit fest ends. So let's start the next commit fest right after this one. I don't think we have enough patches; I'm not sure what that means. Enough for what? It would be great if we had more patches, assuming that they were of good quality and did useful things to advance PostgreSQL. What I think we have is a lot of people who are waiting for feedback, and we should try to give them some. I also know that reviewing 60 patches for the November CommitFest was a ton of work, and the reviewers (including the committers) ran out of steam well before we got done. That, and not any desire to jump the queue, is the reason why I would like to get the reviewing process started before the patch list grows unmanageably large. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Robert Haas robertmh...@gmail.com writes: What I think we have is a lot of people who are waiting for feedback, and we should try to give them some. I also know that reviewing 60 patches for the November CommitFest was a ton of work, and the reviewers (including the committers) ran out of steam well before we got done. That, and not any desire to jump the queue, is the reason why I would like to get the reviewing process started before the patch list grows unmanageably large. Yeah. In core's private discussion of this, I too was arguing for running a CommitFest ASAP, in order to have some motion on the existing patch backlog. I don't know that we'd actually end up committing many, but we need to provide feedback so people can take the next steps. People who *were* following the project calendar (like me for instance) have been largely ignoring the 8.5 queue, so many of those patches are just sitting out there without any substantive comment. Right at the moment I imagine a large fraction of those patches are broken anyway by the recent pgindent run --- has anyone checked? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Greg Stark gsst...@mit.edu writes: I would like to propose a different strategy. Instead of always tackling all the smaller patches and leaving the big patches for last, I would suggest we start with Hot Standby. In fact I would suggest as Hot Standby has already gotten a first pass review that we consider applying it on day 1. Hot Standby wasn't ready for 8.4, and it's not any more ready now, because nothing has been done on it since then. What Simon told us at the developers' meeting is that he needs to find someone who will bankroll further work on it. I hope that will happen, but we can't design the 8.5 schedule around the assumption that it will. I'm also not prepared to push a large and unstable feature into the tree on the hope that it will get fixed. The general consensus among -core, and I think most of -hackers as well, is that we want to try to keep CVS HEAD pretty stable, so that developers aren't fighting each others' bugs. This also ties into the alpha releases concept that Peter mentioned --- if HEAD isn't stable then we can hardly put out a testable alpha release. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Peter Eisentraut pete...@gmx.net writes: Now that 8.4.0 is out the door, development for 8.5devel will be opened any day now. But we haven't discussed the development timeline so far. The core team has several proposals: [ details snipped ] ISTM there are two critical decisions here: when's the first commitfest, and when's the target release date? There's already been considerable chatter about the first decision, but not much about the second. I would like to propose aiming for a release around April/May 2010 ... in time for PGCon if you like, but the main point is to have it out before people start disappearing for summer break. We've already run into problems with scheduling the 8.4 release because of that. Or we could slide the target release date into the fall, but it seemed to me that the spring release timeframe worked better (or would have if we'd been able to meet it fully). Of the schedules Peter mentioned, the only one that has a realistic chance of releasing before June is the one with the final commitfest starting Feb 1. Even then, we need to do something to prevent that fest from expanding the way the last 8.4 fest did. The core committee speculated a bit about instituting a rule like major patches must be submitted into a CF before the last one; the last one will only accept resubmissions and small patches. But then you have to draw the line between major and minor patches. Actually, we did have a rule in the 8.4 cycle specifying that we reserved the right to reject large patches during the final CF. The problem was that in practice we failed to get up the gumption to say no and make it stick. This has been a persistent project management failing for many years, and I'm not sure how we change that dynamic. There's always somebody cheerleading for the latest-and-greatest patch... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, 2009-06-30 at 12:11 -0400, Tom Lane wrote: I'm also not prepared to push a large and unstable feature into the tree on the hope that it will get fixed. The general consensus among -core, and I think most of -hackers as well, is that we want to try to keep CVS HEAD pretty stable, so that developers aren't fighting each others' bugs. This also ties into the alpha releases concept that Peter mentioned --- if HEAD isn't stable then we can hardly put out a testable alpha release. +1 Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, 2009-06-30 at 12:29 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Now that 8.4.0 is out the door, development for 8.5devel will be opened any day now. But we haven't discussed the development timeline so far. The core team has several proposals: [ details snipped ] ISTM there are two critical decisions here: when's the first commitfest, and when's the target release date? There's already been considerable chatter about the first decision, but not much about the second. I would like to propose aiming for a release around April/May 2010 ... in time for PGCon if you like, but the main point is to have it out before people start disappearing for summer break. We've already run into problems with scheduling the 8.4 release because of that. I generally agree with this however why not just have a When it is done?. Let's hit some commitfests and some time near the end of the year start discussing Beta and release. We are not a company. We don't have a deadline. Why can't we just develop and say, Yeah, this looks like it would make a substantive release.? Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Joshua D. Drake j...@commandprompt.com writes: On Tue, 2009-06-30 at 12:29 -0400, Tom Lane wrote: I would like to propose aiming for a release around April/May 2010 ... in time for PGCon if you like, but the main point is to have it out before people start disappearing for summer break. We've already run into problems with scheduling the 8.4 release because of that. I generally agree with this however why not just have a When it is done?. Let's hit some commitfests and some time near the end of the year start discussing Beta and release. We are not a company. We don't have a deadline. Why can't we just develop and say, Yeah, this looks like it would make a substantive release.? Well, then you might as well not have a schedule at all. The point of setting up a schedule is not to have a deadline that we must meet or die trying (and certainly not to ship whether it's ready or not, as a certain other OS database has been accused of doing). Rather, the point of this exercise is to give individual developers a framework to plan in. Without a target date it's tough to decide what is reasonable to work on for 8.5. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] dependencies for generated header files
Robert Haas escribió: Woops. It seems that patch generates some warnings on a vpath build which I failed to notice. Corrected version that guards against same is attached. Interestingly, this patch causes diffstat (at least my version of it) to attribute the line additions to src/backend/catalog/dependency.c instead of the new file ... -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, 2009-06-30 at 12:37 -0400, Tom Lane wrote: We are not a company. We don't have a deadline. Why can't we just develop and say, Yeah, this looks like it would make a substantive release.? Well, then you might as well not have a schedule at all. The point of setting up a schedule is not to have a deadline that we must meet or die trying (and certainly not to ship whether it's ready or not, as a certain other OS database has been accused of doing). Rather, the point of this exercise is to give individual developers a framework to plan in. Without a target date it's tough to decide what is reasonable to work on for 8.5. Right, I get that. That is why I mentioned the start discussing at the end of the year. The idea being, we really don't know what's going to hit. So we set a review date of work being done. Say December 1st. On December 1st we look at what is in, what appears to be coming and then determine a potential release date. We already push and pull our release dates based are what in the queue, we just do so informally. Why not just make it part of the process? That way we are being up front and saying, Yeah, we have no idea. We will review in 6 months and that is when we decide our target. Shrug... Sincerely, Joshua D. Drake regards, tom lane -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 12:37 PM, Tom Lanet...@sss.pgh.pa.us wrote: Joshua D. Drake j...@commandprompt.com writes: On Tue, 2009-06-30 at 12:29 -0400, Tom Lane wrote: I would like to propose aiming for a release around April/May 2010 ... in time for PGCon if you like, but the main point is to have it out before people start disappearing for summer break. We've already run into problems with scheduling the 8.4 release because of that. I generally agree with this however why not just have a When it is done?. Let's hit some commitfests and some time near the end of the year start discussing Beta and release. We are not a company. We don't have a deadline. Why can't we just develop and say, Yeah, this looks like it would make a substantive release.? Well, then you might as well not have a schedule at all. The point of setting up a schedule is not to have a deadline that we must meet or die trying (and certainly not to ship whether it's ready or not, as a certain other OS database has been accused of doing). Rather, the point of this exercise is to give individual developers a framework to plan in. Without a target date it's tough to decide what is reasonable to work on for 8.5. I agree. On the other hand, I think all of the proposed schedules are somewhat optimistic about how long the final release will take. We started the final CommitFest for 8.4 on November 1st and are set to release July 1st. The proposed schedule for next time involves starting the final CommitFest three months later and releasing two months earlier. I'd like to think that with a little more discipline around CommitFests we can tighten things up a little, but it seems excessively optimistic to think that we're going to cut down from seven months to two. I would propose to start CommitFests July 15th, September 15th, November 15th, and January 15th, planning all but the last to be one month long. The last CommitFest I would plan on closing up by March 1st, with release hopefully by June 1st. As for thresholds, I'd propose that we measure the size of patches using diff -u | diffstat. If the number of insertions plus the number of deletions is = 1000, then the patch is not eligible for the final CommitFest unless it was submitted for the penultimate CommitFest. This obvious discriminates against patches with a large footprint that are not very invasive and in favor of those with a small footprint that are more destabilizing, but it's a clean line in the sand, and I think having such a line is better than trying to apply human judgment to every case. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Joshua D. Drake j...@commandprompt.com writes: We already push and pull our release dates based are what in the queue, we just do so informally. Why not just make it part of the process? That way we are being up front and saying, Yeah, we have no idea. We will review in 6 months and that is when we decide our target. I think we used to do it more or less like that, but people didn't like it because they couldn't do any long-range planning. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Inconsistent Errors on Row Comparisons
Howdy, I'm working on functions to compare result sets for pgTAP. In the process, I found what appears to be an inconsistency in error handling when comparing incomparable results. I'm testing in 8.4RC2, but the issue may go back for all I know. Perhaps it's intentional? This is what I see. This query: VALUES (1, 2), (3, 4) EXCEPT VALUES (1, 'foo'), (3, 'bar'); Throws 42804 DATATYPE MISMATCH. Meanwhile, this query: VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); Throws 42601 SYNTAX ERROR. It'd be nice if the error was a bit more specific (maybe tell me that there are different numbers of columns, perhaps 54011?), but at least it's distinct from the data type mismatch. However, when I do a row-by-row comparison of rows in cursors, I get a different behavior. The attached test case has the details, but assuming a function `restults_eq(refcursor, refcursor)` that does the row-by-row comparison, this code: DECLARE cwant CURSOR FOR VALUES (1, 2), (3, 4); DECLARE chave CURSOR FOR VALUES (1, 'foo'), (3, 'bar'); SELECT results_eq( 'cwant'::refcursor, 'chave'::refcursor ); Throws 42804 DATATYPE MISMATCH, as expected. On the other hand, this code: DECLARE cwant2 CURSOR FOR VALUES (1, 2), (3, 4); DECLARE chave2 CURSOR FOR VALUES (1), (3); SELECT results_eq( 'cwant2'::refcursor, 'chave2'::refcursor ); Also throws Throws 42804 DATATYPE MISMATCH. For consistency with the row comparisons done by EXCEPT and friends, should it not throw 42601 SYNTAX ERROR? Thanks, David try.sql Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Robert Haas robertmh...@gmail.com writes: I agree. On the other hand, I think all of the proposed schedules are somewhat optimistic about how long the final release will take. We started the final CommitFest for 8.4 on November 1st and are set to release July 1st. The proposed schedule for next time involves starting the final CommitFest three months later and releasing two months earlier. I'd like to think that with a little more discipline around CommitFests we can tighten things up a little, but it seems excessively optimistic to think that we're going to cut down from seven months to two. Yeah. What I think the 8.4 cycle taught us is that commitfests are a good thing but they won't magically eliminate the need to say no at the end. If we are willing to be sufficiently hardnosed about saying no, we can make the final commitfest short. Otherwise it's going to drag on just like this one did. Keeping the final-CF-to-release period short is desirable for the reasons already mentioned --- we don't want to shut down development for a long period. So even if it's optimistic, I think we should write the schedule that way. We can be certain that the period won't last *less* time than scheduled :-( I would propose to start CommitFests July 15th, September 15th, November 15th, and January 15th, planning all but the last to be one month long. The last CommitFest I would plan on closing up by March 1st, with release hopefully by June 1st. Hmm. If we drop the notion that CFs have to start on month boundaries, that's actually not a bad schedule --- in particular, it keeps us away from expecting much of anything to get done in the last half of December. I'd suggest setting the target release date to May 15 (pre-PGCon), as long as we're working with ides-of-whichever dates. As for thresholds, I'd propose that we measure the size of patches using diff -u | diffstat. If the number of insertions plus the number of deletions is = 1000, then the patch is not eligible for the final CommitFest unless it was submitted for the penultimate CommitFest. This obvious discriminates against patches with a large footprint that are not very invasive and in favor of those with a small footprint that are more destabilizing, but it's a clean line in the sand, and I think having such a line is better than trying to apply human judgment to every case. Well, in the end it will come down to committers' judgements anyway; if someone thinks a patch is ready it will probably go in, regardless of size. But this gives us another tool for saying no, so I'm agreeable ;-) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
David E. Wheeler da...@kineticode.com writes: This is what I see. This query: VALUES (1, 2), (3, 4) EXCEPT VALUES (1, 'foo'), (3, 'bar'); Throws 42804 DATATYPE MISMATCH. Yeah ... Meanwhile, this query: VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); Throws 42601 SYNTAX ERROR. Not for me: regression=# VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); ERROR: each EXCEPT query must have the same number of columns regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 10:28 AM, Tom Lane wrote: VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); Throws 42601 SYNTAX ERROR. Not for me: regression=# VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); ERROR: each EXCEPT query must have the same number of columns Turn on verbosity: try=# \set VERBOSITY verbose try=# VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); ERROR: 42601: each EXCEPT query must have the same number of columns LOCATION: transformSetOperationTree, analyze.c:1502 42601 is a SYNTAX ERROR. The inconsistency when comparing records from cursors stands, too. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign.h is not installed
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: Header file include/foreign/foreign.h is not installed in 8.4.0. Did we forget to add subdir foreign to Makefile? Applied, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Tom Lane t...@sss.pgh.pa.us wrote: I think we used to do it more or less like that, but people didn't like it because they couldn't do any long-range planning. Well, obviously the 8.4 release cycle did little to help them. As has already been observed, there is a crying need to say no at some point to get a release out. It might actually help to do that on big patches if we don't let too many tiny ones accumulate. I seem to remember the argument being tossed about that we might as well keep working on this one because there's all these others to wrap up. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Robert Haas wrote: In any case, we probably need some weigh-in from Heikki and Simon on their plans for Hot Standby before we make any decisions... I'm planning to spend considerable amount of time reviewing and helping with hot standby and synchronous replication, but someone else will have to take the lead. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
David E. Wheeler da...@kineticode.com writes: On Jun 30, 2009, at 10:28 AM, Tom Lane wrote: Not for me: regression=# VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); ERROR: each EXCEPT query must have the same number of columns Turn on verbosity: try=# \set VERBOSITY verbose try=# VALUES (1, 2), (3, 4) EXCEPT VALUES (1), (3); ERROR: 42601: each EXCEPT query must have the same number of columns LOCATION: transformSetOperationTree, analyze.c:1502 42601 is a SYNTAX ERROR. Oh, you're complaining about the SQLSTATE not the error text. I guess that to the extent that any actual thought went into it (which may not have been much) the reasoning was that you'd have to change the syntax of your query in order to fix this. But I guess a case could be made for ERRCODE_DATATYPE_MISMATCH there. I definitely do not agree with your suggestion of ERRCODE_TOO_MANY_COLUMNS --- that's from Program Limit Exceeded category which is 100% the wrong thing. The other errors are coming from within record_eq(), where what it's got is two composite values that don't match as to structure. It seems fairly clear that DATATYPE_MISMATCH is the right thing there. So if we feel that these errors should match, I'd vote for changing to DATATYPE_MISMATCH, not changing to SYNTAX_ERROR. But I'm not entirely convinced that there's a reason to make them match. I'm not sure that they really have the same cause when you look at it concretely. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Merlin Moncure wrote: Given that there is also a lot of work on synchronous replication, is it better to get the HS in so the SR stuff can use that as a baseline, or to triage in both patches together? Whichever finishes first. Although they're very useful together, there is little if any code-level dependency between them. It would be dangerous to have one wait for the other, as one or the other could well be delayed for some reason. We can't even be sure that both are finished in time for 8.5. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
]On Jun 30, 2009, at 11:00 AM, Tom Lane wrote: Oh, you're complaining about the SQLSTATE not the error text. I guess that to the extent that any actual thought went into it (which may not have been much) the reasoning was that you'd have to change the syntax of your query in order to fix this. But I guess a case could be made for ERRCODE_DATATYPE_MISMATCH there. I definitely do not agree with your suggestion of ERRCODE_TOO_MANY_COLUMNS --- that's from Program Limit Exceeded category which is 100% the wrong thing. Yeah, that was just an aside. I liked that I got different errors when there were different numbers of columns than when the data types of the columns disagreed. I'm not sure that SYNTAX ERROR is a great code for when the count disagrees, but at least it's distinct from the column data type error. And I'm going on SQLSTATE here because I'm doing exception handling in pl/PgSQL and want to handle the two errors differently. The other errors are coming from within record_eq(), where what it's got is two composite values that don't match as to structure. It seems fairly clear that DATATYPE_MISMATCH is the right thing there. I see, it's thinking of the two row objects as distinct types, rather than complaining about different numbers of columns. So if we feel that these errors should match, I'd vote for changing to DATATYPE_MISMATCH, not changing to SYNTAX_ERROR. But I'm not entirely convinced that there's a reason to make them match. I'm not sure that they really have the same cause when you look at it concretely. Okay. I'll have to see what I can do with SQLERRM then. But isn't it localized? Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
David E. Wheeler da...@kineticode.com writes: Yeah, that was just an aside. I liked that I got different errors when there were different numbers of columns than when the data types of the columns disagreed. I'm not sure that SYNTAX ERROR is a great code for when the count disagrees, but at least it's distinct from the column data type error. And I'm going on SQLSTATE here because I'm doing exception handling in pl/PgSQL and want to handle the two errors differently. So really what you're wishing for is that we treat different-numbers-of- columns as a whole new SQLSTATE inside category 42. What's the argument for needing to handle this differently from DATATYPE_MISMATCH? Okay. I'll have to see what I can do with SQLERRM then. But isn't it localized? Yeah, it is. You don't really want code looking at that to decide what to do, if you can possibly avoid it. It's intended for human consumption. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 11:18 AM, Tom Lane wrote: So really what you're wishing for is that we treat different-numbers- of- columns as a whole new SQLSTATE inside category 42. What's the argument for needing to handle this differently from DATATYPE_MISMATCH? For my results_eq() in pgTAP, it could output different diagnostics. I'm already doing this for the set_eq() function I wrote, which uses EXCEPT. For that function, if you pass two statements with different numbers of columns, pgTAP says: # Failed test 148 # Number of columns differs between queries While for a call with the same numbers of columns but different data types (say int,text and inet,text), pgTAP says: # Failed test 149 # Column types differ between queries Essentially, while on a row object-level, they are different types, the caller of my function doesn't know that it's comparing rows, just that it's comparing result sets. So I like to give as much information as possible about the difference in the result sets of the queries. Hell, ideally it'd actually say something like: # Failed test 148 # Number of columns differs between queries # have: 4 columns # want: 3 columns # Failed test 149 # Column types differ between queries # have: (integer,text) # want: (inet,text) This gives the tester a lot of information to help diagnose the test failure. I don't know that I can gather that kind of information, though. Okay. I'll have to see what I can do with SQLERRM then. But isn't it localized? Yeah, it is. You don't really want code looking at that to decide what to do, if you can possibly avoid it. It's intended for human consumption. As I thought, thanks. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Kevin Grittner wrote: It might actually help to do that on big patches if we don't let too many tiny ones accumulate. I seem to remember the argument being tossed about that we might as well keep working on this one because there's all these others to wrap up. Yeah, and the people who was able to work on the small patches was too busy helping on the bigger items. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 11:27 AM, David E. Wheeler wrote: # Failed test 148 # Number of columns differs between queries # have: 4 columns # want: 3 columns # Failed test 149 # Column types differ between queries # have: (integer,text) # want: (inet,text) This gives the tester a lot of information to help diagnose the test failure. I don't know that I can gather that kind of information, though. Actually, I can for `set_eq()`, since it creates a temporary table, I can just get the list of types from the system catalog. Is there a way to get a RECORD object to tell me what data types it contains? Then I could use the same error for both situations, since the difference in the number of columns is implicit in the list of data types: # Failed test 148 # Column types differ between queries # have: (integer,text,integer) # want: (inet,text) # Failed test 149 # Column types differ between queries # have: (integer,text) # want: (inet,text) Thanks, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
I would propose to start CommitFests July 15th, September 15th, November 15th, and January 15th, planning all but the last to be one month long. The last CommitFest I would plan on closing up by March 1st, with release hopefully by June 1st. This sounds good to me. Anyone object? As for thresholds, I'd propose that we measure the size of patches using diff -u | diffstat. If the number of insertions plus the number of deletions is= 1000, then the patch is not eligible for the final CommitFest unless it was submitted for the penultimate CommitFest. This obvious discriminates against patches with a large footprint that are not very invasive and in favor of those with a small footprint that are more destabilizing, but it's a clean line in the sand, and I think having such a line is better than trying to apply human judgment to every case. I think we need human judgement. You could easily make a 4-line change to a macro or one of the planner files which would require endless debugging. The deciding people on big patches for the final commitfest should be a combination of the commitfest managers and the core team. And we should weed out the disqualified before January 15. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On 6/29/09 10:33 PM, Peter Eisentraut wrote: Now that 8.4.0 is out the door, development for 8.5devel will be opened any day now. But we haven't discussed the development timeline so far. The core team has several proposals: CommitFest Alpha Aug. 1 Sept. 1 Oct. 1 Nov. 1 Dec. 1 Jan ~~ 5 Feb. 1 March 4 Release ~ May 2010 This puts us on track for a release at the same time next year, maybe a little earlier. One thing Peter forgot to mention here is that the next-to-last commitfest is the *last* commitfest for new major features. For the *last* commitfest, any patch introduced either has to be a resubmission of something we've seen at a prior CF, or has to be very small (i.e. not many lines/files, no side effects, no API or defined standard API). This makes the next-to-last CF our biggest CF. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
David E. Wheeler da...@kineticode.com writes: On Jun 30, 2009, at 11:18 AM, Tom Lane wrote: What's the argument for needing to handle this differently from DATATYPE_MISMATCH? For my results_eq() in pgTAP, it could output different diagnostics. Well, that's not terribly compelling ;-). I wouldn't have any big objection to splitting out ERRCODE_COLUMN_COUNT_MISMATCH as a separate SQLSTATE for 8.5 and beyond, but I doubt we'd consider back-patching such a change. It's not clear to me whether you need a solution that works in back branches. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
David E. Wheeler da...@kineticode.com writes: Is there a way to get a RECORD object to tell me what data types it contains? Not at the SQL level. Of course, if you're writing C, you can do something similar to what record_eq and friends do. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Tue, Jun 30, 2009 at 11:27:20AM -0700, David Wheeler wrote: On Jun 30, 2009, at 11:18 AM, Tom Lane wrote: So really what you're wishing for is that we treat different-numbers- of- columns as a whole new SQLSTATE inside category 42. What's the argument for needing to handle this differently from DATATYPE_MISMATCH? For my results_eq() in pgTAP, it could output different diagnostics. I'm already doing this for the set_eq() function I wrote, which uses EXCEPT. For that function, if you pass two statements with different numbers of columns, pgTAP says: # Failed test 148 # Number of columns differs between queries While for a call with the same numbers of columns but different data types (say int,text and inet,text), pgTAP says: # Failed test 149 # Column types differ between queries Essentially, while on a row object-level, they are different types, the caller of my function doesn't know that it's comparing rows, just that it's comparing result sets. So I like to give as much information as possible about the difference in the result sets of the queries. Hell, ideally it'd actually say something like: # Failed test 148 # Number of columns differs between queries # have: 4 columns # want: 3 columns Shouldn't that just read: have: (int, int, text, point) want: (int, int, text) Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Josh Berkus j...@agliodbs.com writes: I would propose to start CommitFests July 15th, September 15th, November 15th, and January 15th, planning all but the last to be one month long. The last CommitFest I would plan on closing up by March 1st, with release hopefully by June 1st. This sounds good to me. Anyone object? I'd like to set the target before PGCon not after; so May 1 or May 15. Otherwise +1. The deciding people on big patches for the final commitfest should be a combination of the commitfest managers and the core team. And we should weed out the disqualified before January 15. As you say, we should reject outright (at the start of the final CF) anything that's clearly too big. But in the final 8.4 CF it didn't really become clear until later on that some things were not ready or were too big to deal with. We need to be more willing to swing the axe later in the fest, rather than allowing it to drag on just a bit more in the hopes of getting big feature X in. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Josh Berkus j...@agliodbs.com writes: One thing Peter forgot to mention here is that the next-to-last commitfest is the *last* commitfest for new major features. For the *last* commitfest, any patch introduced either has to be a resubmission of something we've seen at a prior CF, or has to be very small (i.e. not many lines/files, no side effects, no API or defined standard API). We've been kicking around variations of that idea already. As I mentioned in the core discussion, I'm a bit concerned that this would have the effect of choking off development too soon. We could have a situation where nothing major is supposed to be getting worked on from Nov 15 to mid-May, which seems much too long. So very small seems too strict. Robert's suggestion of a 1000-line cutoff might be workable though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: I think we used to do it more or less like that, but people didn't like it because they couldn't do any long-range planning. Well, obviously the 8.4 release cycle did little to help them. As has already been observed, there is a crying need to say no at some point to get a release out. It might actually help to do that on big patches if we don't let too many tiny ones accumulate. I seem to remember the argument being tossed about that we might as well keep working on this one because there's all these others to wrap up. Have you chaps considered a simple points system? Every patch would need five minutes attention to triage it into one of: small (1 point), medium (2), large (10), huge (50 points - Sync Repl etc). First CF gets (say) 200 points, next 150, next 100, next 75. First-come, first-served - if your patch goes over the limit it goes in the next commit-fest. -- Richard Huxton Archonet Ltd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi-Dimensional Histograms
I'm finding myself unable to follow all the terminology on this thead. What's dimension reduction? What's PCA? ( Sorry for the jargon - thanks Josh ) It feels like what you might need is statistics for colB (MCVs and/or a histogram) for certain particular values of colA. Certainly - this is exactly what a multi-dimensional histogram would store. Unfortunately, in the general case the set of values of colA for which you need these statistics might be inconveniently large. Which is why, in the general case, we need some sort of dimension reduction. If we could say that, for instance, the distribution of colB is roughly the same for all values of colA less than 100, and it is roughly the same for all values of colA = 100, then we would have effectively reduced the dimension from ndistinct(colB)*ndistinct(colA) to 2*ndistinct(colB). The multidimensional histogram in the above paper is somewhat akin to what you suggested and I just described - it attempts to select contiguous regions that are the same. PCA is a much different approach, but it is still, broadly, a dimension reduction technique. At the end of the day, we cant do any dimension reduction unless the ordering encodes some sort of useful information, and the data type being in R^n is certainly no guarantee. Consider, for instance, the cross correlation of zip-codes and area codes - you would really want to order those by some geographic relation. I think that is why cross-column stats is so hard in the general case. To clarify my response to Tom, directly above, if as in Robert's example, all of the values in colA 100 were different than all of the values 100 with respect to colB, then it is easy to represent the structure in something manageable. Ie, we store two histograms for colB: 1 for colA 100, one for colA 100. However, if there are the same two classes in colB ( lets say C1 and C2 ) but rather than C1 being associated with values in colA 100, it is associated with 100 completely random values in colA ( ie 1, 22, 47, 89, 91, 92, 107, ... ) there is not a whole lot we can do besides storing a list of all of those values. We could maybe use the ratio of classes to improve the average plan choice, but you would still get a lot of bad plans. Could anyone provide a concrete example ( or better yet a data set ) where this occurs? -Nathan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 11:46 AM, Tom Lane wrote: For my results_eq() in pgTAP, it could output different diagnostics. Well, that's not terribly compelling ;-). Pllt. I wouldn't have any big objection to splitting out ERRCODE_COLUMN_COUNT_MISMATCH as a separate SQLSTATE for 8.5 and beyond, but I doubt we'd consider back-patching such a change. It's not clear to me whether you need a solution that works in back branches. Makes sense. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 11:48 AM, Tom Lane wrote: Is there a way to get a RECORD object to tell me what data types it contains? Not at the SQL level. Of course, if you're writing C, you can do something similar to what record_eq and friends do. Pity. I'm trying to keep C out of pgTAP (for the most part) so that folks can just distribute a copy of it with their modules. But I can at least include that information in the diagnostics from set_eq(). Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 11:54 AM, David Fetter wrote: # Failed test 148 # Number of columns differs between queries # have: 4 columns # want: 3 columns Shouldn't that just read: have: (int, int, text, point) want: (int, int, text) Yes, that's my ideal, but Tom says I need to write C code to get that information from RECORD objects, alas. :-( Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Tue, Jun 30, 2009 at 01:10:01PM -0700, David Wheeler wrote: On Jun 30, 2009, at 11:54 AM, David Fetter wrote: # Failed test 148 # Number of columns differs between queries # have: 4 columns # want: 3 columns Shouldn't that just read: have: (int, int, text, point) want: (int, int, text) Yes, that's my ideal, but Tom says I need to write C code to get that information from RECORD objects, alas. :-( Would this be the first C piece? If not, it might be worth doing. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 1:40 PM, David Fetter wrote: Yes, that's my ideal, but Tom says I need to write C code to get that information from RECORD objects, alas. :-( Would this be the first C piece? If not, it might be worth doing. I don't understand the question. But yes, I think it'd be worth doing. I'd like to have functions like: pg_record_attrs(RECORD) RETURNS SETOF regtype[] And maybe another function to return attribute names. If it could returns both names and types, that'd be cool, but I'm not sure what kind of data type that would be. Does this sound interesting to other folks? Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Tue, Jun 30, 2009 at 02:01:26PM -0700, David Wheeler wrote: On Jun 30, 2009, at 1:40 PM, David Fetter wrote: Yes, that's my ideal, but Tom says I need to write C code to get that information from RECORD objects, alas. :-( Would this be the first C piece? If not, it might be worth doing. I don't understand the question. I was thinking of this as part of PgTAP. But yes, I think it'd be worth doing. I'd like to have functions like: pg_record_attrs(RECORD) RETURNS SETOF regtype[] And maybe another function to return attribute names. If it could returns both names and types, that'd be cool, but I'm not sure what kind of data type that would be. It's possible to have it return SETOF RECORD with OUT parameters, I think. Does this sound interesting to other folks? Sure. Maybe that should go in pg_catalog in 8.5... Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Jun 30, 2009, at 3:05 PM, David Fetter wrote: Would this be the first C piece? If not, it might be worth doing. I don't understand the question. I was thinking of this as part of PgTAP. Oh. There is a piece of C, but it's just an implementation of pg_typeof() so that pgTAP can use it in versions of PostgreSQL 8.4. In 8.4 and later, no C code is built. Have been trying to keep it that way for now. But yes, I think it'd be worth doing. I'd like to have functions like: pg_record_attrs(RECORD) RETURNS SETOF regtype[] And maybe another function to return attribute names. If it could returns both names and types, that'd be cool, but I'm not sure what kind of data type that would be. It's possible to have it return SETOF RECORD with OUT parameters, I think. Out parameters? I was thinking of SETOF RECORD with two attributes in each record: name and type. Are there other attributes of RECORD attributes that might be of interest? Does this sound interesting to other folks? Sure. Maybe that should go in pg_catalog in 8.5... Right, that was my thought. I could then throw it in pgTAP when building on 8.5. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inconsistent Errors on Row Comparisons
On Tue, Jun 30, 2009 at 03:16:09PM -0700, David Wheeler wrote: On Jun 30, 2009, at 3:05 PM, David Fetter wrote: Would this be the first C piece? If not, it might be worth doing. I don't understand the question. I was thinking of this as part of PgTAP. Oh. There is a piece of C, but it's just an implementation of pg_typeof() so that pgTAP can use it in versions of PostgreSQL 8.4. In 8.4 and later, no C code is built. Have been trying to keep it that way for now. OK, so the can is already open, but you're trying to discourage the worms from escaping. :) But yes, I think it'd be worth doing. I'd like to have functions like: pg_record_attrs(RECORD) RETURNS SETOF regtype[] And maybe another function to return attribute names. If it could returns both names and types, that'd be cool, but I'm not sure what kind of data type that would be. It's possible to have it return SETOF RECORD with OUT parameters, I think. Out parameters? CREATE OR REPLACE FUNCTION pg_record_info( IN r RECORD, OUT regtype RETYPE, OUT name NAME, ) RETURNS SETOF RECORD LANGUAGE C AS 'filename', 'pg_record_info'; I was thinking of SETOF RECORD with two attributes in each record: name and type. Are there other attributes of RECORD attributes that might be of interest? Does this sound interesting to other folks? Sure. Maybe that should go in pg_catalog in 8.5... Right, that was my thought. I could then throw it in pgTAP when building on 8.5. Right :) Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
* Tom Lane (t...@sss.pgh.pa.us) wrote: Robert Haas robertmh...@gmail.com writes: What I think we have is a lot of people who are waiting for feedback, and we should try to give them some. I also know that reviewing 60 patches for the November CommitFest was a ton of work, and the reviewers (including the committers) ran out of steam well before we got done. That, and not any desire to jump the queue, is the reason why I would like to get the reviewing process started before the patch list grows unmanageably large. Yeah. In core's private discussion of this, I too was arguing for running a CommitFest ASAP, in order to have some motion on the existing patch backlog. I don't know that we'd actually end up committing many, but we need to provide feedback so people can take the next steps. I'm in agreement that we should try to provide feedback (in general, to be honest) on patches/suggestions/ideas/designs/etc as quickly as possible. The commitfest approach is good for this when it's in motion, but it's been stale for months. +1 from me for starting early. To flip it around a bit, I don't know that there's actually a moratorium on providing feedback? If people aren't busy working on 8.4 (I hope not at this point, except testing.. :), have patches that they've submitted and are waiting for feedback on, or aren't otherwise busy.. I say feel free to pull patches and review/comment on. As I like to tell the people who work with me- being proactive and self-starting is almost always looked on favorably. I'm like to encourage the above for the entire development cycle, for that matter. Perhaps it won't change much (I'm confident we'll still need a commitfest mom, etc) but I don't see why we should actively prevent it. People who *were* following the project calendar (like me for instance) have been largely ignoring the 8.5 queue, so many of those patches are just sitting out there without any substantive comment. Certainly following the calendar is a good thing, it's just that we're about to pass a milestone on the project calendar and need to decide what we're gonna do next and when. Right at the moment I imagine a large fraction of those patches are broken anyway by the recent pgindent run --- has anyone checked? I havn't yet, but it certainly sounds like a great idea. Maybe we could even have a mini-fix-pgindent commitfest in the last 2 weeks of July.. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] 8.5 development schedule
* Tom Lane (t...@sss.pgh.pa.us) wrote: I would like to propose aiming for a release around April/May 2010 ... in time for PGCon if you like, but the main point is to have it out before people start disappearing for summer break. We've already run into problems with scheduling the 8.4 release because of that. Big +1 from me for having it in time for PGCon. Or we could slide the target release date into the fall, but it seemed to me that the spring release timeframe worked better (or would have if we'd been able to meet it fully). Agreed. Of the schedules Peter mentioned, the only one that has a realistic chance of releasing before June is the one with the final commitfest starting Feb 1. Even then, we need to do something to prevent that fest from expanding the way the last 8.4 fest did. The core committee speculated a bit about instituting a rule like major patches must be submitted into a CF before the last one; the last one will only accept resubmissions and small patches. But then you have to draw the line between major and minor patches. I liked the general idea of only accepting already been seen patches or bug-fix only patches in the last commitfest. Thanks, Stephen signature.asc Description: Digital signature
[HACKERS] [PATCH] SE-PostgreSQL Updates rev.2096
The SE-PostgreSQL patches are updated as follows: 01) http://sepgsql.googlecode.com/files/sepgsql-01-sysatt-8.4-r2096.patch 02) http://sepgsql.googlecode.com/files/sepgsql-02-core-8.4-r2096.patch 03) http://sepgsql.googlecode.com/files/sepgsql-03-gram-8.4-r2096.patch 04) http://sepgsql.googlecode.com/files/sepgsql-04-writable-8.4-r2096.patch 05) http://sepgsql.googlecode.com/files/sepgsql-05-rowlevel-8.4-r2096.patch 06) http://sepgsql.googlecode.com/files/sepgsql-06-perms-8.4-r2096.patch 07) http://sepgsql.googlecode.com/files/sepgsql-07-extra-8.4-r2096.patch 08) http://sepgsql.googlecode.com/files/sepgsql-08-utils-8.4-r2096.patch 09) http://sepgsql.googlecode.com/files/sepgsql-09-tests-8.4-r2096.patch 10) http://sepgsql.googlecode.com/files/sepgsql-10-docs-8.4-r2096.patch The SE-PostgreSQL online documentation: http://wiki.postgresql.org/wiki/SEPostgreSQL The purpose of every patches are introduced at: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development#Patches List of updates: * Its base version was updated to the latest CVS HEAD. * The third gram patch is a separated part from the second core patch to reduce the scale of the patch. It provides extensions in SQL grammer, such as SECURITY_LABEL = 'xxx'. * bugfix: it didn't prevent accesses when user tries to select pg_toast.pg_toast_%u by hand. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
* Tom Lane (t...@sss.pgh.pa.us) wrote: As I mentioned in the core discussion, I'm a bit concerned that this would have the effect of choking off development too soon. We could have a situation where nothing major is supposed to be getting worked on from Nov 15 to mid-May, which seems much too long. So very small seems too strict. Robert's suggestion of a 1000-line cutoff might be workable though. Maybe we should have a hard rule now, but then leave ourselves the option to relax it (perhaps not publically) once we actually get to the last CF? I dunno, just a thought. I do share your concern about not choking off development for too long, but it sure seems like we've always got a big patch or two that we're trying to get into the next release near the end.. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 9:17 PM, Stephen Frostsfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: As I mentioned in the core discussion, I'm a bit concerned that this would have the effect of choking off development too soon. We could have a situation where nothing major is supposed to be getting worked on from Nov 15 to mid-May, which seems much too long. So very small seems too strict. Robert's suggestion of a 1000-line cutoff might be workable though. Maybe we should have a hard rule now, but then leave ourselves the option to relax it (perhaps not publically) once we actually get to the last CF? I dunno, just a thought. I do share your concern about not choking off development for too long, but it sure seems like we've always got a big patch or two that we're trying to get into the next release near the end.. With all due respect, you guys are worrying about the wrong problem. I suspect it's much more likely that we're going to want to postpone 900-line patches than it is that you're going to want to not postpone 1100-line patches. If by some chance Tom wants to commit an 1100-line patch submitted three weeks after the start of the last CommitFest, he'll explain why he wants to do it and in all likelihood the rest of us will shrug our shoulders and say OK, do it. But I seriously doubt that's gonna happen. What's a lot more likely is that Tom will look at a 900-line patch submitted three weeks BEFORE the start of the last CommitFest and say this is going to break a lot of stuff, we should bounce this to 8.6, and everyone will say no, it's a great feature, commit the broken code now! now, we say!. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Yeah. In core's private discussion of this, I too was arguing for running a CommitFest ASAP, in order to have some motion on the existing patch backlog. I don't know that we'd actually end up committing many, but we need to provide feedback so people can take the next steps. I'm in agreement that we should try to provide feedback (in general, to be honest) on patches/suggestions/ideas/designs/etc as quickly as possible. The commitfest approach is good for this when it's in motion, but it's been stale for months. +1 from me for starting early. To flip it around a bit, I don't know that there's actually a moratorium on providing feedback? Well, I wouldn't put it as strongly as moratorium, but ... the point of having a release cycle is that at certain times people are supposed to focus on stabilizing and testing a release, not on working on new development. If, at any time in the past six months, I were to have gone off and reviewed a patch that's clearly 8.5 material, that's man-hours taken directly away from getting a solid 8.4 release out the door. Which means that much longer until 8.4 does get out, which hurts everybody including the submitter of the premature patch. So in general I agree with the viewpoint Peter outlined that working on new development during beta period is not really playing by the rules, and that people who expect feedback for new development during beta period simply don't understand how the project is supposed to work. If you find yourself with nothing else to do except review a new patch during beta, then sure, go do it. But is there *really* nothing you could be doing to help expedite the beta? Anyway, as of now that's all moot until next spring. But it's still true that people want time to work on their own patches, which is why we came up with the commitfests. It's so you can get some time to work without feeling guilty about not reviewing other people's work instead. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Josh Berkus wrote: One thing Peter forgot to mention here is that the next-to-last commitfest is the *last* commitfest for new major features. For the *last* commitfest, any patch introduced either has to be a resubmission of something we've seen at a prior CF, or has to be very small (i.e. not many lines/files, no side effects, no API or defined standard API). This makes the next-to-last CF our biggest CF. I thought we discussed that at pgCon in May and rejected it. I have very, very serious reservations about it. I think we need to get better about proper triage, especially on the final commitfest, rather than moving the effective feature freeze back a whole commitfest. ISTM we're in danger of becoming dominated by procedures. Let's keep it light and loose. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: I'm in agreement that we should try to provide feedback (in general, to be honest) on patches/suggestions/ideas/designs/etc as quickly as possible. The commitfest approach is good for this when it's in motion, but it's been stale for months. +1 from me for starting early. To flip it around a bit, I don't know that there's actually a moratorium on providing feedback? Well, I wouldn't put it as strongly as moratorium, but ... the point of having a release cycle is that at certain times people are supposed to focus on stabilizing and testing a release, not on working on new development. I certainly agree with this. I tried to qualify my comment above in the sentence which followed it, but apparently I didn't do a good job. - This only applies before feature freeze (eg: right after 8.4 is out) - No work being done on a current patch (eg: waiting for feedback) - Have cycles to spare Our current arrangment is based on a premise that a given person will have X cycles in a month to work on PG, and that they have =X amount of work to do on developing their patch(s) that month. I feel like there are a number of patch submitters who have X cycles to work on PG, and X work they'll be able to do on their patch in a given month (in part because at some point they'll submit it and then wait for feedback). I see minimial drawback to encouraging those people to review other patches with any extra cycles they have, even if we're not actually in a CF-mode at that point. If, at any time in the past six months, I were to have gone off and reviewed a patch that's clearly 8.5 material, that's man-hours taken directly away from getting a solid 8.4 release out the door. Which means that much longer until 8.4 does get out, which hurts everybody including the submitter of the premature patch. So in general I agree with the viewpoint Peter outlined that working on new development during beta period is not really playing by the rules, and that people who expect feedback for new development during beta period simply don't understand how the project is supposed to work. It wasn't my intent to imply this would be done during feature-freeze, or beta, but rather something to be done during development. Perhaps I should have phrased it as the moratorium on looking at patches can end when 8.4 is released, and not be fully reinstated until 8.5 beta. Anyway, as of now that's all moot until next spring. But it's still true that people want time to work on their own patches, which is why we came up with the commitfests. It's so you can get some time to work without feeling guilty about not reviewing other people's work instead. I definitely think that's good, and to be honest I don't think my suggestion would apply to core much at all (they've always got =X work to do on patches :), but as we saw during the commitfests in 8.4, there's a number of non-core folks out there volunteering to review. If they're willing and able to spend cycles reviewing other patches rather than twiddling their thumbs waiting for feedback, let's encourage that. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] 8.5 development schedule
Tom Lane wrote: As for thresholds, I'd propose that we measure the size of patches using diff -u | diffstat. If the number of insertions plus the number of deletions is = 1000, then the patch is not eligible for the final CommitFest unless it was submitted for the penultimate CommitFest. This obvious discriminates against patches with a large footprint that are not very invasive and in favor of those with a small footprint that are more destabilizing, but it's a clean line in the sand, and I think having such a line is better than trying to apply human judgment to every case. Well, in the end it will come down to committers' judgements anyway; if someone thinks a patch is ready it will probably go in, regardless of size. But this gives us another tool for saying no, so I'm agreeable ;-) I realize there is the perception that the large patches that were eventually rejected held up the release, but for all the patches I can think of, they were not rejected immediately _because_ we had other valid patches to work on. Once all valid patches were applied, we were quickly able to reject the large unready patches. So, rejecting the large patches earily would not have significantly moved the release date earlier. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Tue, Jun 30, 2009 at 11:18 PM, Bruce Momjianbr...@momjian.us wrote: Tom Lane wrote: As for thresholds, I'd propose that we measure the size of patches using diff -u | diffstat. If the number of insertions plus the number of deletions is = 1000, then the patch is not eligible for the final CommitFest unless it was submitted for the penultimate CommitFest. This obvious discriminates against patches with a large footprint that are not very invasive and in favor of those with a small footprint that are more destabilizing, but it's a clean line in the sand, and I think having such a line is better than trying to apply human judgment to every case. Well, in the end it will come down to committers' judgements anyway; if someone thinks a patch is ready it will probably go in, regardless of size. But this gives us another tool for saying no, so I'm agreeable ;-) I realize there is the perception that the large patches that were eventually rejected held up the release, but for all the patches I can think of, they were not rejected immediately _because_ we had other valid patches to work on. Once all valid patches were applied, we were quickly able to reject the large unready patches. So, rejecting the large patches earily would not have significantly moved the release date earlier. I don't believe it. :-) If the large patches had been rejected earlier, the reviewers and committers who were spending time on them could have started spending time on other things sooner. I also believe, although I cannot prove it, that it would have increased the pressure to get the remaining items dealt with. When there are 100 patches, everyone can say oh, well it doesn't matter whether I get this taken care of today, because there will still be 99 other patches. When there are 3 patches, that argument doesn't hold water. Another thing I'd like to improve for the next CommitFest is the communication between reviewers and committers. In the November CommitFest, I, and I think others, assumed that the committers were reading all of the review threads in detail throughout the process and that they would jump in when things got close to being done, or maybe when things got marked as Ready for Committer on the Wiki. That worked OK, but there were rough patches where things bogged down. As much as we may express opinions on the merits of certain patches, I think all of us non-committers are (certainly I am) loathe to avoid the perception that we're telling the committers what to do. But, I think it would be helpful to have periodic updates on what the different committers are currently focused on, how long they intend to be focused on that, and what they intend to focus on next; and I think it would be helpful for the CF mgmt team to make suggestions as to what patches we think are the closest to being ready to go or most in need of committer-level input. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
Andrew, I thought we discussed that at pgCon in May and rejected it. That's not what we have in my notes: http://wiki.postgresql.org/wiki/PgCon_2009_Developer_Meeting#CommitFests Of course, I may have missed some options, a lot of people were talking. I have very, very serious reservations about it. I think we need to get better about proper triage, especially on the final commitfest, rather than moving the effective feature freeze back a whole commitfest. Thing is, if we're getting 15,000 lines of code we've never seen before (collectively) at the final commitfest, there's simply no way we're getting a release out within 3 months of that. ISTM we're in danger of becoming dominated by procedures. Let's keep it light and loose. Hmmm ... actually, I think Andrew's right that we don't need a specific last commitfest rule which is special. Really, any patch which gets submitted to any commitfest gets returned if it's not ready to be committed immediately after review. The only way the last CF is special is that anything bounced goes to the next version. We forgot that with the November CF, which is why it dragged on for 3.5 months and burned a lot of people out. Let's just stick to that this time and we don't need any new rules. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
