Re: [HACKERS] Is this really really as designed or defined in some standard
"Pavel Stehule" <[EMAIL PROTECTED]> writes: > 2008/9/1 Tom Lane <[EMAIL PROTECTED]>: >> However, since this is a behavioral change that could break code that >> works now, I think it should be a HEAD-only change; no backpatch. > I agree - it's could break only 100% wrong code, but it could problems > in minor update. > Could you backpach only warning? I'm not for that. I dislike back-patching changes that are not the same as what goes into CVS HEAD, because that means those changes will go out in minor releases of stable branches without any detectable amount of developer testing. If we thought this was a change that really deserved incremental warnings, then the right thing would be to make it a warning in 8.4 and an error in some later release. And maybe even make a GUC variable to control that behavior. But I don't think it's worth that. An error starting in 8.4 seems entirely sufficient from where I sit. BTW, there are actually two separate issues here: input parameters and output parameters. After brief thought it seems like we should enforce uniqueness of non-omitted parameter names for IN parameters (including INOUT), and separately enforce uniqueness of non-omitted parameter names for OUT parameters (including INOUT). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
David Fetter <[EMAIL PROTECTED]> writes: > On Tue, Sep 02, 2008 at 02:42:25AM -0400, Tom Lane wrote: >> It's not like we haven't seen a SQL draft go down in flames before. > Do you think that anything in the windowing functions section will > disappear? Who's to say? I have no objection to looking at the 2003 and 200n documents in parallel, especially if there are places where 200n clarifies the intent of 2003. But I'd be suspicious of designing around entirely-new features presented by 200n. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is this really really as designed or defined in some standard
2008/9/2 Tom Lane <[EMAIL PROTECTED]>: > "Pavel Stehule" <[EMAIL PROTECTED]> writes: >> 2008/9/1 Tom Lane <[EMAIL PROTECTED]>: >>> However, since this is a behavioral change that could break code that >>> works now, I think it should be a HEAD-only change; no backpatch. > >> I agree - it's could break only 100% wrong code, but it could problems >> in minor update. > >> Could you backpach only warning? > > I'm not for that. I dislike back-patching changes that are not the same > as what goes into CVS HEAD, because that means those changes will go out > in minor releases of stable branches without any detectable amount of > developer testing. > > If we thought this was a change that really deserved incremental > warnings, then the right thing would be to make it a warning in 8.4 and > an error in some later release. And maybe even make a GUC variable to > control that behavior. But I don't think it's worth that. > +1 > An error starting in 8.4 seems entirely sufficient from where I sit. > > BTW, there are actually two separate issues here: input parameters and > output parameters. After brief thought it seems like we should enforce > uniqueness of non-omitted parameter names for IN parameters (including > INOUT), and separately enforce uniqueness of non-omitted parameter names > for OUT parameters (including INOUT). > It's well thought, but I afraid so this can hide some bug, and it's little bit dangerous. I thing, so we can simply duplicate values in result then allowing duplicate params in function. postgres=# create function foo(out a int, out b int) returns record as $$ select 1,2 $$ language sql; CREATE FUNCTION postgres=# select a, a, b from foo(); a | a | b ---+---+--- 1 | 1 | 2 (1 row) Duplicate arguments are more like copy-paste bug then good style. We check it in column definition too: postgres=# create function foo1() returns record as $$ select 1,1,2 $$ language sql; CREATE FUNCTION postgres=# select * from foo1() as (c int,c int,c int); ERROR: column name "c" specified more than once Pavel >regards, tom lane > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] COPY statement cannot take binding parameters
Trying to parse and bind the following: COPY (SELECT $1::INT) TO STDOUT gives a correct parsing-done, but then in the parameterdescription tells me that there are no parameters. Is this intended? Is this a limitation of the COPY statement that will not change? In that case, might I suggest the server generates an appropriate error message when someone attempts to do this? -- Sincerely, Stephen R. van den Berg. "Having a non-smoking section in a restaurant is like having a non-peeing section in a pool." -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
Tom Lane wrote: David Fetter <[EMAIL PROTECTED]> writes: On Tue, Sep 02, 2008 at 02:42:25AM -0400, Tom Lane wrote: It's not like we haven't seen a SQL draft go down in flames before. Do you think that anything in the windowing functions section will disappear? Who's to say? I have no objection to looking at the 2003 and 200n documents in parallel, especially if there are places where 200n clarifies the intent of 2003. But I'd be suspicious of designing around entirely-new features presented by 200n. well http://www.wiscorp.com/SQLStandards.html states: "This points to the documents which wlll likely be the documents that represent the SQL 2008 Standard. These documents are out for International Standard ballot at this time. The vote is an Up/Down vote. No changes allowed." Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
2008/9/2 Heikki Linnakangas <[EMAIL PROTECTED]>: > Gregory Stark wrote: >> What would the executor do for a query like >> >> SELECT lead(x,1),lead(y,2),lead(y,3) >> >> It would not only have to keep a tuplestore to buffer the output but it >> would >> have to deal with receiving data from different SRFs at different times. >> The >> best approach I can think of would be to keep a tuplestore for each SRF >> using >> themas queues, reading old values from the head as soon as they all have >> at >> least one new value in them. > > Hitoshi solved that creating a separate Window node for each window > function. So the plan tree for that would look something like: > > Window (lead(x,1)) > Window (lead(y,2)) >Window (lead(y,3)) > Seq Scan ... > > That keeps the Window node implementation quite simple because it only needs > to handle one window function at a time. To say more accurately, one Window node can handle more than one window function. If it is thought to be the same using equalFuncs comparing targetlists, some functions are put into one node. Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Sat, 2008-08-30 at 02:04 +0900, Hitoshi Harada wrote: > Here's the latest window functions patch against HEAD. It seems to be > ready for the September commit fest, as added documents, WINDOW clause > feature and misc tests. I guess this would be the window functions > feature freeze for 8.4. The remaining feature will be implemented for > the later release. > > This patch consists of: > - windowed aggregates > - cooperate with GROUP BY aggregates > - some optimizations with multiple windows > - ranking functions > - WINDOW clause > - windowing SQL regression tests > - sgml documents > > Looking up the total road map, the dropped features are: > > - sliding window (window framing) > - lead(), lag(), etc. that reach for random rows > - user defined window functions > > The first and second topics are difficult to implement currently. > Because these features require random row access, it seems that > tuplestore would be able to save multiple positions to mark/restore. > This is fundamental change that is over my capability. Also, user > defined window functions seem to have much more to decide. I think I > can't put into shape the general needs of user's window functions now. > Lacking these feature, this stage looks compatible to SQLServer 2005, > while Oracle and DB2 have almost full of the specification. If you've done all of that, then I'm impressed. Well done. Few general comments * The docs talk about "windowing functions", yet you talk about "window functions" here. I think the latter is correct, but whichever we choose we should be consistent (and hopefully matching SQL Standard). * You don't use duplicate the examples from the docs into the tests, which is always a good way to get conflicting reports from users. :-) * The tests seem very light for such a huge range of new functionality. (8 tests is hardly sufficient). I'd like to see a wide range of tests - probably 5-10 times as many individual test statements. I would also like to see test failures that illustrate the as-yet unimplemented features and the warning messages that are thrown - this will help us understand exactly what is missing also. It would also be useful to see other common coding mistakes/misconceptions and the corresponding error messages. > Also, current implementation has only a type of plan which uses sort > operation. It should be optimized by re-position the windows and/or > using hashtable. I would like to see some performance test results also. It would be good to know whether they are fast/slow etc.. It will definitely help the case for inclusion if they are faster than alternative multi-statement approaches to solving the basic data access problems. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New FSM patch
On Fri, 2008-08-29 at 10:47 +0300, Heikki Linnakangas wrote: > - Per comments and discussion with Simon, I've changed the "bubble up" > behavior so that when a bottom-level page is updated, if the amount of > free space was decreased, the change is not immediately bubbled up to > upper page. Instead, searchers that traverse down the tree will update > the upper pages when they see that they're out of sync. This should > alleviate the worry that we need to keep a bottom-level page exclusively > locked across I/O. Thanks for taking time with the new FSM. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New FSM allocation policy
On Fri, 2008-08-29 at 18:55 +0300, Heikki Linnakangas wrote: > Tom Lane wrote: > > Gregory Stark <[EMAIL PROTECTED]> writes: > >> One idea, we could scan the rest of the current page and use the first > >> match. > > > >> Another, given the way your tree structure works you can also descend the > >> tree > >> with a "target" page. You can find the first page with enough free space > >> after > >> the target page if there are any. (Take left branch if it's > target and > >> has > >> enough free space else take right branch if there's enough free space else > >> take left branch). > > > > I think the trick here is how to also preserve the property that > > different backends tend to be inserting into different pages. > > Yep. If we just always look at the next page, there's the danger of > having multiple backends compete for the same pages again. Can the FSM hand out page ranges? That way we would be able to use the next page logic without fear of competition. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] statement_cost_limit for regression testing.
On Thu, 2008-08-28 at 19:57 -0700, Ryan Bradetich wrote: > I am not sure of the status of the patch, but I did read through the > thread at: >http://archives.postgresql.org/pgsql-hackers/2008-08/msg00054.php > > > I just wanted to throw out another possible use for this GUC. There > maybe a better way to > solve this problem, but I believe this patch would be useful for > regression testing. It's possible to do this using planner hooks, so no patch needed. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Hello all,
On Mon, Aug 11, 2008 at 2:24 AM, Gregory Stark <[EMAIL PROTECTED]> wrote:
>
> "Ryan Bradetich" <[EMAIL PROTECTED]> writes:
>
> > After a cursory glance at the HeapTupleHeaderData structure, it appears it
> > could be aligned with INTALIGN instead of MAXALIGN. The one structure I was
> > worried about was the 6 byte t_ctid structure. The comments in
> > src/include/storage/itemptr.h file indicate the ItemPointerData structure is
> > composed of 3 int16 fields. So everthing in the HeapTupleHeaderData
> > structure is 32-bits or less.
>
> Sure, but the tuple itself could contain something with double alignment. If
> you have a bigint or double in the tuple then heap_form_tuple needs to know
> where to put it so it ends up at right alignment.
For fun, I looked around in heap_form_tuple() today to see how big of a job this
change would be. It did not seem very hard to implement. I know there are
probably several other places I missed with this patch, but this patch does pass
the regression tests and simple testing I threw at it (including ALTER TABLE
ADD COLUMN, etc).
The patch concept is fairly simple.
1. Add a new boolean local variable: require_max_align
(initialized to false).
2. When we loop through the attributes, check to see if any
attributes require
double alignment. If they do, set require_max_align = true.
3. If the tuple has OIDS (double alignment requirement), set
require_max_align = true.
4. If require_max_align = true, use the MAXALIGN macro; otherwise
use the INTALIGN macro.
A check of the physical storage for my previous test case does show
the tuple length to be 44 bytes
instead of the previous 48 bytes, so the patch does appear to work. I
have not run any performance
tests using this patch, etc.
Attached is my proof-of-concept patch for this idea in case someone
wants to inform me of other
problems this patch will introduce :)
Thanks,
- Ryan
*** a/src/backend/access/common/heaptuple.c
--- b/src/backend/access/common/heaptuple.c
***
*** 863,868 heap_form_tuple(TupleDesc tupleDescriptor,
--- 863,869
data_len;
int hoff;
bool hasnull = false;
+ bool require_max_align = false;
Form_pg_attribute *att = tupleDescriptor->attrs;
int numberOfAttributes = tupleDescriptor->natts;
int i;
***
*** 886,891 heap_form_tuple(TupleDesc tupleDescriptor,
--- 887,895
*/
for (i = 0; i < numberOfAttributes; i++)
{
+ if (att[i]->attalign == 'd')
+ require_max_align = true;
+
if (isnull[i])
hasnull = true;
else if (att[i]->attlen == -1 &&
***
*** 907,916 heap_form_tuple(TupleDesc tupleDescriptor,
if (hasnull)
len += BITMAPLEN(numberOfAttributes);
! if (tupleDescriptor->tdhasoid)
len += sizeof(Oid);
! hoff = len = MAXALIGN(len); /* align user data safely */
data_len = heap_compute_data_size(tupleDescriptor, values, isnull);
--- 911,925
if (hasnull)
len += BITMAPLEN(numberOfAttributes);
! if (tupleDescriptor->tdhasoid) {
len += sizeof(Oid);
+ require_max_align = true;
+ }
! if (require_max_align == true)
! hoff = len = MAXALIGN(len); /* align user data safely */
! else
! hoff = len = INTALIGN(len); /* align user data safely */
data_len = heap_compute_data_size(tupleDescriptor, values, isnull);
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] statement_cost_limit for regression testing.
Hello Simon, On Mon, Sep 1, 2008 at 9:35 AM, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2008-08-28 at 19:57 -0700, Ryan Bradetich wrote: >> I just wanted to throw out another possible use for this GUC. There >> maybe a better way to >> solve this problem, but I believe this patch would be useful for >> regression testing. > It's possible to do this using planner hooks, so no patch needed. Excellent! Time for me to read up on using these planner hooks. Thanks! - Ryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
Simon Riggs <[EMAIL PROTECTED]> wrote: > > Why do we need to set rmgr_hook in _PG_init(), and add or mofify rmgrs > > in our hook functions? > > If we modify RmgrTable in _PG_init() then we would have to have that > structure available in all backends, which was a stated objective to > avoid. We would still need a fast access data structure for the > XLogInsert() check, so the RmgrTable would just be wasted space in all > normal backends. In the patch, plugin is only called when we call > RmgrInitialize(), so the memory is malloc'd only when required. Um? AFAICS RmgrTable is not accessed in XLogInsert unless we use WAL_DEBUG. I see that RmgrTable should be malloc'd when required, but there is another issue; when to load rmgr libraries. Rmgr objects are needed only in startup process during recovery. If we want to reduce resource consumption by rmgrs, I think it is better not to load rmgr libraries through shared_preload_libraries. We don't have to load rmgr libs if recovery is not needed or after recovery. How about adding a new variable "recovery_preload_libaries" like as shared_preload_libraries? Rmgr libs in it are loaded only in startup process and only if recovery is needed. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] statement_cost_limit for regression testing.
2008/9/2 Ryan Bradetich <[EMAIL PROTECTED]>: > Hello Simon, > > On Mon, Sep 1, 2008 at 9:35 AM, Simon Riggs <[EMAIL PROTECTED]> wrote: >> On Thu, 2008-08-28 at 19:57 -0700, Ryan Bradetich wrote: >>> I just wanted to throw out another possible use for this GUC. There >>> maybe a better way to >>> solve this problem, but I believe this patch would be useful for >>> regression testing. >> It's possible to do this using planner hooks, so no patch needed. > > Excellent! > > Time for me to read up on using these planner hooks. > > Thanks! it should be nice contrib module :) Pavel > > - Ryan > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Mon, 2008-09-01 at 21:00 +0300, Heikki Linnakangas wrote: > 1. It's important that what gets committed now can be extended to handle > all of the window function stuff in SQL2003 in the future, as well as > user-defined-window-functions in the spirit of PostgreSQL extensibility. > Even if we don't implement all of it in this release. I think whatever public APIs get published now must be sufficient to support user-defined-window functions across all future releases, so on that point I agree completely. (One reason why I argued earlier in favour of avoiding an API for now). We shouldn't restrict the implementation of the internals to be upward compatible though because I foresee some aspect of complexity stalling and thus killing the patch in the short term if we do that. We don't have much time left for this release. If we only have the combined (brain * time) to get a partial implementation in for this release then I would urge we go for that, rather than wait for perfection - as long as there are no other negative effects. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
On Tue, 2008-09-02 at 18:30 +0900, ITAGAKI Takahiro wrote: > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > Why do we need to set rmgr_hook in _PG_init(), and add or mofify rmgrs > > > in our hook functions? > > > > If we modify RmgrTable in _PG_init() then we would have to have that > > structure available in all backends, which was a stated objective to > > avoid. We would still need a fast access data structure for the > > XLogInsert() check, so the RmgrTable would just be wasted space in all > > normal backends. In the patch, plugin is only called when we call > > RmgrInitialize(), so the memory is malloc'd only when required. > > Um? AFAICS RmgrTable is not accessed in XLogInsert unless we use WAL_DEBUG. Exactly why I want to malloc it. > I see that RmgrTable should be malloc'd when required, > but there is another issue; when to load rmgr libraries. > Rmgr objects are needed only in startup process during recovery. > If we want to reduce resource consumption by rmgrs, I think it is > better not to load rmgr libraries through shared_preload_libraries. > We don't have to load rmgr libs if recovery is not needed or after recovery. > > How about adding a new variable "recovery_preload_libaries" like as > shared_preload_libraries? Rmgr libs in it are loaded only in startup > process and only if recovery is needed. Good point. If others agree, I will re-implement this way. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
On Tue, Sep 02, 2008 at 01:49:43AM -0700, Ryan Bradetich wrote: > For fun, I looked around in heap_form_tuple() today to see how big of a job > this > change would be. It did not seem very hard to implement. I know there are > probably several other places I missed with this patch, but this patch does > pass > the regression tests and simple testing I threw at it (including ALTER TABLE > ADD COLUMN, etc). You need to arrange testing on an architechture that has strict alignment reuiqrements. For example i386 doesn't care about alignment at all and will anything from anywhere, with performance degradation. Other architechtures will simply throw exceptions, that's the smoke test. Also, what's the performance cost? Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Tue, Sep 02, 2008 at 10:44:31AM +0100, Simon Riggs wrote: > If we only have the combined (brain * time) to get a partial > implementation in for this release then I would urge we go for that, > rather than wait for perfection - as long as there are no other negative > effects. "premature optimization is the root of all evil." (Knuth, Donald.) "make it work, then make it better". Getting a partial implementation that works out now is far better than waiting until its perfect. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] [PATCH] Make gram.y use palloc/pfree for memory management
On 9/1/08, Marko Kreen <[EMAIL PROTECTED]> wrote: > First a correction, overriding malloc/free seems dangerous they > seems to leak out, so correct would be to use YYMALLOC/YYFREE. > This leaves 1.875 potentially leaking, but danger seems small. Here is the safer patch. As the chance for the leak is rare, leaving 1.875 vulnerable should not be problem. -- marko diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0af6142..62c4b63 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -62,6 +62,11 @@ #include "utils/numeric.h" #include "utils/xml.h" +/* + * bison does not re-use buffers, so we can order it to use palloc/pfree. + */ +#define YYMALLOC palloc +#define YYFREE pfree /* Location tracking support --- simpler than bison's default */ #define YYLLOC_DEFAULT(Current, Rhs, N) \ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
ITAGAKI Takahiro wrote: I see that RmgrTable should be malloc'd when required, but there is another issue; when to load rmgr libraries. Rmgr objects are needed only in startup process during recovery. If we want to reduce resource consumption by rmgrs, I think it is better not to load rmgr libraries through shared_preload_libraries. We don't have to load rmgr libs if recovery is not needed or after recovery. How about adding a new variable "recovery_preload_libaries" like as shared_preload_libraries? Rmgr libs in it are loaded only in startup process and only if recovery is needed. It doesn't seem worth it to introduce a new GUC like that, just to reduce the memory usage a tiny bit in the rare case that a rmgr plugin is present. How much memory will loading an extra library consume anyway? Depends on the library of course, but I believe we're talking about something in the ballpark of a few hundred kb. Besides, a decent OS should swap that to disk, if it's not used, and the system is tight on memory. Also, presumably the library containing the recovery functions, also contains the functions that generate those WAL records. So, it will be needed after startup anyway, if the plugin is used at all. There's one more reason to use shared_preload_libraries. It provides a sanity check that the library required for recovery is present and can be loaded, even when no recovery is required. If you have misconfigured your system so that it can't recover, you want to find out sooner rather than later when recovery is needed. So IMHO, just use shared_preload_libraries. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Auto-explain patch
On 8/28/08, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: > Here is a contrib version of auto-explain. > You can use shared_preload_libraries or local_preload_libraries to > load the module automatically. If you do so, you also need to add > "explain" in custom_variable_classes and define explain.* variables > in your postgresql.conf. Is it possible to use LOAD command to load the module? -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Auto-explain patch
"Marko Kreen" <[EMAIL PROTECTED]> wrote: > On 8/28/08, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: > > You can use shared_preload_libraries or local_preload_libraries to > > load the module automatically. If you do so, you also need to add > > "explain" in custom_variable_classes and define explain.* variables > > in your postgresql.conf. > > Is it possible to use LOAD command to load the module? Yes, but disabled in default. You need to enable it like: LOAD 'auto_explain'; SET explain.log_min_duration = '100ms'; SET explain.log_analyze = true; SELECT ... In that case, you don't need to define custom_variable_classes. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Auto-explain patch
On 9/2/08, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: > "Marko Kreen" <[EMAIL PROTECTED]> wrote: > > On 8/28/08, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: > > > You can use shared_preload_libraries or local_preload_libraries to > > > load the module automatically. If you do so, you also need to add > > > "explain" in custom_variable_classes and define explain.* variables > > > in your postgresql.conf. > > > > Is it possible to use LOAD command to load the module? > > Yes, but disabled in default. > You need to enable it like: > > LOAD 'auto_explain'; > SET explain.log_min_duration = '100ms'; > > SET explain.log_analyze = true; > > SELECT ... > > In that case, you don't need to define custom_variable_classes. I was interested if it is possible to enable it for single session. Seems it is. Good. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Out-of-tree compilation seems broken in HEAD (plpgsql)
$ mkdir build $ cd build $ ../PostgreSQL.dev/configure $ make [...] gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv -fpic -I/home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src -I../../../../src/include -I/home/marko/src/build/../PostgreSQL.dev/src/include -D_GNU_SOURCE -c -o pl_comp.o /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c:20:21: error: pl_gram.h: No such file or directory -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
On Tue, 2008-09-02 at 13:38 +0300, Heikki Linnakangas wrote: > There's one more reason to use shared_preload_libraries. It provides a > sanity check that the library required for recovery is present and > can > be loaded, even when no recovery is required. If you have > misconfigured > your system so that it can't recover, you want to find out sooner > rather > than later when recovery is needed. Great reason. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Out-of-tree compilation seems broken in HEAD (plpgsql)
Marko Kreen wrote: $ mkdir build $ cd build $ ../PostgreSQL.dev/configure $ make [...] gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv -fpic -I/home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src -I../../../../src/include -I/home/marko/src/build/../PostgreSQL.dev/src/include -D_GNU_SOURCE -c -o pl_comp.o /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c:20:21: error: pl_gram.h: No such file or directory I got the same error yesterday, but I was building in-tree. "make maintainer-clean" worked for me. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Tue, 2008-09-02 at 03:14 -0400, Tom Lane wrote: > David Fetter <[EMAIL PROTECTED]> writes: > > On Tue, Sep 02, 2008 at 02:42:25AM -0400, Tom Lane wrote: > >> It's not like we haven't seen a SQL draft go down in flames before. > > > Do you think that anything in the windowing functions section will > > disappear? > > Who's to say? > > I have no objection to looking at the 2003 and 200n documents in > parallel, especially if there are places where 200n clarifies the > intent of 2003. But I'd be suspicious of designing around > entirely-new features presented by 200n. I have confirmation from Michael Gorman, Wiscorp, that > The new standard was approved in early Summer. SQL 2008 is finished. So as of now, SQL2008 exists, all hail. SQL2003 and earlier versions have been superseded and can be ignored. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION name> FOR FROM [STRCOLFN ] [ ] [ ] [ LCCOLLATE ] [ LCCTYPE ] How do you plan to make a collation case sensitive or accent sensitive? I have previously commented that this is not a realistic view on how collations work. Since you are apparently planning to use the system locales, I don't see how you can make this work. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO item: Implement Boyer-Moore searching (First time hacker)
David Rowley wrote: Reference: Bruce Momjian writes: -> http://archives.postgresql.org/pgsql-committers/2007-09/msg00402.php Other references: Boyer Moore?? -> http://www.cs.utexas.edu/users/moore/best-ideas/string-searching/fstrpos-example.html I look forward to receiving feedback on this. Please send a patch in diff -c format. And don't put a single patch file in a tar file. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: > Radek Strnad wrote: > >- new collations can be defined with command CREATE COLLATION >name> FOR FROM > >[STRCOLFN ] > >[ ] [ ] [ LCCOLLATE ] > >[ LCCTYPE ] > > How do you plan to make a collation case sensitive or accent sensitive? > I have previously commented that this is not a realistic view on how > collations work. Since you are apparently planning to use the system > locales, I don't see how you can make this work. While it's true POSIX locales don't handle this, other collation libraries do and we should support them if the user wants. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] [PATCH] Cleanup of GUC units code
Marko Kreen wrote: In the meantime, here is simple patch for case-insensivity. You might be able to talk me into accepting various unambiguous, common alternative spellings of various units. But for instance allowing MB and Mb to mean the same thing is insane. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
I think at least case sensitivity can be done by comparing two strings converted to upper case with toupper() function. Regards Radek Strnad > > On Tue, Sep 2, 2008 at 2:00 PM, Martijn van Oosterhout <[EMAIL > PROTECTED]>wrote: > >> On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: >> > Radek Strnad wrote: >> > >- new collations can be defined with command CREATE COLLATION >> > > >name> FOR FROM >> > >[STRCOLFN ] >> > >[ ] [ ] [ LCCOLLATE >> ] >> > >[ LCCTYPE ] >> > >> > How do you plan to make a collation case sensitive or accent sensitive? >> > I have previously commented that this is not a realistic view on how >> > collations work. Since you are apparently planning to use the system >> > locales, I don't see how you can make this work. >> >> While it's true POSIX locales don't handle this, other collation >> libraries do and we should support them if the user wants. >> >> Have a nice day, >> -- >> Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ >> > Please line up in a tree and maintain the heap invariant while >> > boarding. Thank you for flying nlogn airlines. >> >> -BEGIN PGP SIGNATURE- >> Version: GnuPG v1.4.6 (GNU/Linux) >> >> iD8DBQFIvSrIIB7bNG8LQkwRAnkWAJ9FaiR9cOHFN2vkVmQaK5y7N9OJoQCbB+Ks >> e0E4722hY/Q+Cz8tpzA0CGs= >> =2Svh >> -END PGP SIGNATURE- >> >> >
Re: [HACKERS] Window functions patch v04 for the September commit fest
2008/9/2 Simon Riggs <[EMAIL PROTECTED]>: > If you've done all of that, then I'm impressed. Well done. > > Few general comments > > * The docs talk about "windowing functions", yet you talk about "window > functions" here. I think the latter is correct, but whichever we choose > we should be consistent (and hopefully matching SQL Standard). That's what I'm embarrassed. Now we have "window functions" meaning two, one for generic name of window expressions and the other for non-window-aggregates. It is a word play, which is difficult problem for non-native people, but... let's use "window functions". I'll revise sgml docs. > * You don't use duplicate the examples from the docs into the tests, > which is always a good way to get conflicting reports from users. :-) > > * The tests seem very light for such a huge range of new functionality. > (8 tests is hardly sufficient). I'd like to see a wide range of tests - > probably 5-10 times as many individual test statements. I would also > like to see test failures that illustrate the as-yet unimplemented > features and the warning messages that are thrown - this will help us > understand exactly what is missing also. It would also be useful to see > other common coding mistakes/misconceptions and the corresponding error > messages. > >> Also, current implementation has only a type of plan which uses sort >> operation. It should be optimized by re-position the windows and/or >> using hashtable. > > I would like to see some performance test results also. It would be good > to know whether they are fast/slow etc.. It will definitely help the > case for inclusion if they are faster than alternative multi-statement > approaches to solving the basic data access problems. > OK, thanks for your advices. I'll work on tests, docs and benchmarks, then send in another patch in a week or so. Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] What is d2mdir?
$ grep -i D2MDIR * -R doc/src/sgml/Makefile:D2MSCRIPT= $(D2MDIR)/docbook2man-spec.pl I could not find anything in the code related to this. I am trying to create man pages in -HEAD, and getting an error: [EMAIL PROTECTED] sgml]$ make man onsgmls -D . postgres.sgml | sgmlspl /docbook2man-spec.pl --lowercase --section l --date "`date '+%Y-%m-%d'`" FATAL: /docbook2man-spec.pl does not exist. make: *** [man] Error 2 Ok, but I need to specify a path for that script. How should I fix this? Regards, -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org signature.asc Description: This is a digitally signed message part
Re: [HACKERS] [PATCH] Cleanup of GUC units code
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Marko Kreen wrote: >> In the meantime, here is simple patch for case-insensivity. > > You might be able to talk me into accepting various unambiguous, common > alternative spellings of various units. But for instance allowing MB and Mb > to > mean the same thing is insane. Because you think some user will be trying to specify their shared_buffers in bits? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] posix advises ...
On Sun, 31 Aug 2008, Alvaro Herrera wrote: Greg Smith wrote: This patch does need a bit of general care in a couple of areas. The reviewing game plan I'm working through goes like this: Did this review effort go anywhere? Haven't made much progress--all my spare time for work like this lately has been stuck fighting broken hardware and OS issues on my test systems. I have a lot more time set aside for PostgreSQL hacking this month, and I'll look at that and the checkpoint block sorting patch I've also been tinkering with soon and kick back an initial review of each. But if anybody else is keen on working on either patch, let me know. Be glad to assist instead of lead. It takes fairly serious hardware to work on either of these usefully though. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Page layout footprint
Hi Heikki, I'm sorry for lack of explanation. It is my fault. Heikki says (on commit fest wiki): I believe I debunked this patch enough already. Apparently there's some compatibility issue between 32-bit and 64-bit Sparcs, but this patch didn't catch that. It doesn't seem like this provides any extra safeness or better error messages. If I'm missing something, please provide more details on what scenario we currently have a problem, and how this helps with it. The original proposal (http://archives.postgresql.org/message-id/[EMAIL PROTECTED]) contains two parts. First part is implementation of --footprint cmd line switch which shows you page layout structures footprint. It is useful for development (mostly for in-place upgrade) and also for manual data recovery when you need to know exact structures. Second part was add this information also into pg_control.file, but how you correctly mentioned there is not real use case to do it at this moment. However, there is still --footprint switch which is useful and it is reason why I put it on wiki for review and feedback. The switch could also use in build farm for collecting footprints from build farm members. 32/64 bit issue is little bit different story and it is general (not only SPARC but on SPARC has bigger impact). Problem is that CRC32 gives probably different result when it is compiled 32bit or 64bit. I'm going to examine it more. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
Hitoshi Harada wrote: 2008/9/2 Heikki Linnakangas <[EMAIL PROTECTED]>: In my understanding, the "Window Frame" is defined by clauses such like "ROWS BETWEEN ... ", "RANGE BETWEEN ... " or so, contrast to "Window Partition" defined by "PARTITION BY" clause. A frame slides within a partition or there's only one frame if those clauses are not specified. The current patch has not implemented this yet. I'll update the docs. Yes, that's how I read it as well. Another way to think of it is that there's always a window frame, but if no ROWS BETWEEN or similar clause is given, the window frame spans the whole partition (or from the 1st row of the partition, up to the current row, if ORDER BY is given). I don't like to call the second type "ranking aggregates" because it may refer to only ranking functions though there are more types of function like ntile(), lead() and lag(). But "window functions" doesn't seem appropriate either since it's ambiguous with the general name of window expressions. Yep, confusing :-(. The SQL standard says that "a window function is one of: a rank function, a distribution function, a row number function, a window aggregate function, the ntile function, the lead function, the lag function, the first-value function, the last-value function, the nth-value function". So, window aggregate functions are a special class of window functions, and there's no term to refer to all the rest of the window functions excluding window aggregate functions. Your docSQL spec Window expression Window function Window function Any window function other than a window aggregate function Window aggregateWindow aggregate function I tried to coin the term "ranking aggregate" for the SQL2008 term "Any window function other than a window aggregate function", but you're right that that's still confusing, because the SQL2008 term "rank function" includes only RANK() and DENSE_RANK(). The spec calls them "group aggregate functions", when they're used with GROUP BY, rather than as a window function. I think we could use that term. Your proposal is smarter than the current implementation. But it doesn't seem complete in some way. From logical point of view, the window functions should be allowed to access whole of rows in the frame the current row belongs to (c.f. inverse distribution functions). By the whole of rows, do you mean a) the chosen value or expression of all rows, or b) all columns of the input rows? Different window functions have different needs. RANK() for example does need to see all columns, to compare them, but it only needs access to the previous and current row. CUME_DIST on the other hand needs access to all columns of all rows, and LAG needs access to a specific column of a fixed number of rows. And row_number needs nothing. The needs of access to the rows are so different that it seems best to me to delegate the buffering to the window function. Actually, looking closer to the ranking functions, they don't really need access to all columns. They just need to be able to compare them, according to the window ordering, so we should probably only provide access to the arguments of the aggregate, evaluated for any row in the frame, and a comparator function, that can determine if any given rows in the frame are <, = or >. From implementing and performance point of view, there need to consider such case like mixing window aggregates and ranking aggregates in the same window, which means it is better that the two types of functions are processed in the similar flow. Also logically, SQL spec doesn't forbid ranking aggregates in sliding window frames. It doesn't? Oh, bummer. It seems we need a grand unified theory of ranking and other aggregates then :-(. How does something like RANK work with a window frame? What does it return, if the current row is excluded from the window frame, e.g with EXCLUDE CURRENT ROW? Let's look at the trivial, generic, and slow implementation first, and then figure out what tricks we can do to make it faster. I gather that that generic algorithm, following how the SQL spec defines the window frame, looks like this: 0. Construct the ordered set of tuples, containing all the tuples in the partition. For each row, start with the ordered set constructed in step 0, and: 1. Remove all tuples from the set that are before the start of the sliding frame 2. Remove all tuples from the set that are after the end of the sliding frame 3. Remove all tuples specified by the window-frame-exclusion clause (EXCLUDE CURRENT ROW/GROUP/TIES/NO OTHERS). 4. Run the aggregate over all tuples still in the set. Now, the first obvious optimization is that we don't want to construct the window frame from scratch for each row. Instead: 0. Start with an empty ordered set S. For each row: 1. Read forward until we hit the end of the new sliding frame, and add those rows to S. 2. Remove all tuples
Re: [HACKERS] Window functions patch v04 for the September commit fest
Martijn van Oosterhout wrote: On Tue, Sep 02, 2008 at 10:44:31AM +0100, Simon Riggs wrote: If we only have the combined (brain * time) to get a partial implementation in for this release then I would urge we go for that, rather than wait for perfection - as long as there are no other negative effects. "premature optimization is the root of all evil." (Knuth, Donald.) "make it work, then make it better". Getting a partial implementation that works out now is far better than waiting until its perfect. Sure. Just have to watch out that we don't follow a dead-end, making it harder to add missing functionality later on. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Page layout footprint
Zdenek Kotala wrote: Hi Heikki, I'm sorry for lack of explanation. It is my fault. Heikki says (on commit fest wiki): I believe I debunked this patch enough already. Apparently there's some compatibility issue between 32-bit and 64-bit Sparcs, but this patch didn't catch that. It doesn't seem like this provides any extra safeness or better error messages. If I'm missing something, please provide more details on what scenario we currently have a problem, and how this helps with it. The original proposal (http://archives.postgresql.org/message-id/[EMAIL PROTECTED]) contains two parts. First part is implementation of --footprint cmd line switch which shows you page layout structures footprint. It is useful for development (mostly for in-place upgrade) and also for manual data recovery when you need to know exact structures. Second part was add this information also into pg_control.file, but how you correctly mentioned there is not real use case to do it at this moment. However, there is still --footprint switch which is useful and it is reason why I put it on wiki for review and feedback. The switch could also use in build farm for collecting footprints from build farm members. Ok, understood. I'll take another look from that point of view. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
Martijn van Oosterhout wrote: On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION name> FOR FROM [STRCOLFN ] [ ] [ ] [ LCCOLLATE ] [ LCCTYPE ] How do you plan to make a collation case sensitive or accent sensitive? I have previously commented that this is not a realistic view on how collations work. Since you are apparently planning to use the system locales, I don't see how you can make this work. While it's true POSIX locales don't handle this, other collation libraries do and we should support them if the user wants. Do they handle exactly those two attributes specifically? Can you point out references? Or do you mean, other collation libraries allow their collations to be configured/customized? I think linguistically it is a very narrow view of the world to hardcode those two attributes. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
Gregory Stark wrote: Peter Eisentraut <[EMAIL PROTECTED]> writes: Marko Kreen wrote: In the meantime, here is simple patch for case-insensivity. You might be able to talk me into accepting various unambiguous, common alternative spellings of various units. But for instance allowing MB and Mb to mean the same thing is insane. Because you think some user will be trying to specify their shared_buffers in bits? My concern is that this information does not stay in the configuration files. It will invariably leak out into whitepapers, presentations, product documentation, and before long there will be confusion about why you can't stuff N Mb over an N Mb connection. I am not making this up. Mb does not add any typing ease (as "KB" might) or readability (as "sec" might), and there is no respectable source that will claim it is an acceptable alias for MB. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
On 9/2/08, Peter Eisentraut <[EMAIL PROTECTED]> wrote: > Marko Kreen wrote: > > In the meantime, here is simple patch for case-insensivity. > > You might be able to talk me into accepting various unambiguous, common > alternative spellings of various units. But for instance allowing MB and Mb > to mean the same thing is insane. How would the docs for that look like? And anyway, what is wrong with Mb for megabytes? mB may be slightly weird but if some user likes it, I see no reason to reject it. You do realize that misspelling unit name can cause downtime of several minutes instead of couple seconds? We can easily do restart in couple of seconds but the restart, look logs, launch editor, find value, change, save, restart cycle will take quite a lot more. Why should we increase the chance that any config edit causes problems? Secondly, humans don't have byte-exact memory, instead they generalize and deduce (eg. from nearby parameters). Thus remembering "KB, MB, GB" or "kb, mb, gb" is easier than remembering "kB, MB, GB". Also remembering "ms, s, m, h, d" is easier than "ms, s, min, h, d". (I'm not proposing 'm' for minutes, just noting, that as we shouldn't ever introduce 'month' unit for core config values, the 'm' as minutes is preferable for 'min'.) Thirdly, please don't use "standard units" argument, unless you plan to propose use of "KiB, MiB, GiB" at the same moment. The units will be remembered as units-for-postgresql.conf. It is good if they conform to user expectations, but it's even more important they are easy to remember. Fourth, for the same reason, it is preferable the amount of units accepted stays small. Again, to be easily rememberable. This also kills any chance "mb" can be confused with "milli-bits". If any extension/module wants to use any other units or full-SI, it can but that should not extend to core config values. Again, to reduce chance for confusion. So, are there any other arguments for keeping current behaviour? -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
On 9/2/08, Peter Eisentraut <[EMAIL PROTECTED]> wrote: > Gregory Stark wrote: > > Peter Eisentraut <[EMAIL PROTECTED]> writes: > > > Marko Kreen wrote: > > > > In the meantime, here is simple patch for case-insensivity. > > > > > > > You might be able to talk me into accepting various unambiguous, common > > > alternative spellings of various units. But for instance allowing MB > and Mb to > > > mean the same thing is insane. > > > > > > > Because you think some user will be trying to specify their shared_buffers > in > > bits? > > > > My concern is that this information does not stay in the configuration > files. It will invariably leak out into whitepapers, presentations, product > documentation, and before long there will be confusion about why you can't > stuff N Mb over an N Mb connection. I am not making this up. Uh. So you want force "proper" units in presentations at the price of everyday admin operations? Does not seem like a sensible tradeoff. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Out-of-tree compilation seems broken in HEAD (plpgsql)
Marko Kreen escribió: > $ mkdir build > $ cd build > $ ../PostgreSQL.dev/configure > $ make > [...] > gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith > -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing > -fwrapv -fpic -I/home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src > -I../../../../src/include > -I/home/marko/src/build/../PostgreSQL.dev/src/include -D_GNU_SOURCE > -c -o pl_comp.o > /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c > /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c:20:21: > error: pl_gram.h: No such file or directory Try running "make maintainer-clean" -- see http://archives.postgresql.org/message-id/20080829162252.GG3983%40alvh.no-ip.org -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Out-of-tree compilation seems broken in HEAD (plpgsql)
On 9/2/08, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > Marko Kreen escribió: > > $ mkdir build > > $ cd build > > $ ../PostgreSQL.dev/configure > > $ make > > [...] > > gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith > > -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing > > -fwrapv -fpic -I/home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src > > -I../../../../src/include > > -I/home/marko/src/build/../PostgreSQL.dev/src/include -D_GNU_SOURCE > > -c -o pl_comp.o > > /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c > > /home/marko/src/build/../PostgreSQL.dev/src/pl/plpgsql/src/pl_comp.c:20:21: > > error: pl_gram.h: No such file or directory > > > Try running "make maintainer-clean" -- see > > > http://archives.postgresql.org/message-id/20080829162252.GG3983%40alvh.no-ip.org Note I started with empty tree.. I'll try with new cvs checkout. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Out-of-tree compilation seems broken in HEAD (plpgsql)
Marko Kreen escribió: > On 9/2/08, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > > Marko Kreen escribió: > > > error: pl_gram.h: No such file or directory > > > > Try running "make maintainer-clean" -- see > > > > > > http://archives.postgresql.org/message-id/20080829162252.GG3983%40alvh.no-ip.org > > Note I started with empty tree.. The build tree may be empty, but the source tree contains the derived files. I urge you to read the whole thread I linked. > I'll try with new cvs checkout. That'll have the same effect as make maintainer-clean, and should work equally well. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
At 2008-09-02 15:10:23 +0300, [EMAIL PROTECTED] wrote: > > [EMAIL PROTECTED] sgml]$ make man As Alvaro noted recently, you need to use "make man D2MDIR=/some/path". -- ams -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
On Tue, 2008-09-02 at 19:47 +0530, Abhijit Menon-Sen wrote: > > [EMAIL PROTECTED] sgml]$ make man > > As Alvaro noted recently, I probably missed that. > you need to use "make man D2MDIR=/some/path". Thanks :) Cheers. -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Out-of-tree compilation seems broken in HEAD (plpgsql)
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Marko Kreen escribió: >> I'll try with new cvs checkout. > That'll have the same effect as make maintainer-clean, and should work > equally well. No, it'll work better. The real problem here is that in the CVS-HEAD makefiles, "make maintainer-clean" fails to remove the *old* derived files. Perhaps we should have left the old filenames listed in the clean targets... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
Abhijit Menon-Sen <[EMAIL PROTECTED]> writes: > At 2008-09-02 15:10:23 +0300, [EMAIL PROTECTED] wrote: >> >> [EMAIL PROTECTED] sgml]$ make man > As Alvaro noted recently, you need to use "make man D2MDIR=/some/path". I see it's been like that for quite some time, but still it seems pretty bogus. Why isn't configure handling this? If there's some good reason not to automate it, why isn't it documented in the "Building The Documentation" appendix? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is this really really as designed or defined in some standard
"Pavel Stehule" <[EMAIL PROTECTED]> writes: > 2008/9/2 Tom Lane <[EMAIL PROTECTED]>: >> BTW, there are actually two separate issues here: input parameters and >> output parameters. After brief thought it seems like we should enforce >> uniqueness of non-omitted parameter names for IN parameters (including >> INOUT), and separately enforce uniqueness of non-omitted parameter names >> for OUT parameters (including INOUT). > It's well thought, but I afraid so this can hide some bug, and it's > little bit dangerous. > I thing, so we can simply duplicate values in result then allowing > duplicate params in function. Um ... what? I'm not sure what behavior you're proposing here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is this really really as designed or defined in some standard
2008/9/2 Tom Lane <[EMAIL PROTECTED]>: > "Pavel Stehule" <[EMAIL PROTECTED]> writes: >> 2008/9/2 Tom Lane <[EMAIL PROTECTED]>: >>> BTW, there are actually two separate issues here: input parameters and >>> output parameters. After brief thought it seems like we should enforce >>> uniqueness of non-omitted parameter names for IN parameters (including >>> INOUT), and separately enforce uniqueness of non-omitted parameter names >>> for OUT parameters (including INOUT). > >> It's well thought, but I afraid so this can hide some bug, and it's >> little bit dangerous. > >> I thing, so we can simply duplicate values in result then allowing >> duplicate params in function. > > Um ... what? I'm not sure what behavior you're proposing here. > >regards, tom lane > I am sorry - I really have to learn english. Simply I don't thing, so duplicit OUT parameters is good idea, but I am haven't strong objections - some programmer's bugs are visible in this case. regards Pavel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
"Ryan Bradetich" <[EMAIL PROTECTED]> writes: > The patch concept is fairly simple. >1. Add a new boolean local variable: require_max_align > (initialized to false). This really can't possibly work, because you'd need to propagate knowledge of the tuple's alignment requirement all over the place. In particular, how would code *reading* the tuple know where the data starts? Also, I don't think you get (very much of) the actual benefit unless the code that inserts tuples into disk pages knows to do something different in the int-align case. It's conceivable that we could make this work if we wanted to dedicate an infomask bit to showing whether the tuple needs int or double alignment. I don't really think it's worth the trouble though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
"Marko Kreen" <[EMAIL PROTECTED]> writes: > Uh. So you want force "proper" units in presentations at the price > of everyday admin operations? Does not seem like a sensible tradeoff. It didn't to anyone else when Peter wrote the current version either, but as the person willing to actually do the work and write the code Peter got to make the decision. Nobody else stepped up to do the work to change it and we can't exactly force Peter to do work he doesn't agree with. The current version is unquestionably better than what came before where you had to specify some things in units of Postgres blocks, for example. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
"Ryan Bradetich" <[EMAIL PROTECTED]> writes: >4. If require_max_align = true, use the MAXALIGN macro; otherwise > use the INTALIGN macro. Huh, I didn't think of doing it like that. But I'm confused. You seem to be tweaking the alignment of the data inside the tuple? After the tuple header? I thought we had only one byte of wasted space in there and that's used by the null bitmap. So unless you have more than 8 columns and some of them are null I wouldn't expect you to save any space. If you do then I guess you could save 4 bytes if the null bitmap is 2-5 bytes (mod 8) long. I thought the goal was to save space by aligning the tuples on the page more densely. That seems to me to be more fruitful as about half the tuples will save four bytes even on tables with small or missing null bitmaps. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Tom Lane <[EMAIL PROTECTED]> writes: > "Ryan Bradetich" <[EMAIL PROTECTED]> writes: >> The patch concept is fairly simple. > >>1. Add a new boolean local variable: require_max_align >> (initialized to false). > > This really can't possibly work, because you'd need to propagate > knowledge of the tuple's alignment requirement all over the place. > In particular, how would code *reading* the tuple know where the > data starts? Uh, at t_hoff, no? > Also, I don't think you get (very much of) the actual benefit unless the > code that inserts tuples into disk pages knows to do something different in > the int-align case. +1 -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2008-09-02 at 18:30 +0900, ITAGAKI Takahiro wrote: >> How about adding a new variable "recovery_preload_libaries" like as >> shared_preload_libraries? Rmgr libs in it are loaded only in startup >> process and only if recovery is needed. > Good point. If others agree, I will re-implement this way. Aside from the objections raised by Heikki, I'm not seeing the use-case for an rmgr that only executes during recovery; in fact I'm not entirely sure that I see a use-case for this entire patch. Where are the WAL records that the "loadable rmgr" processes going to come from? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
Tom Lane wrote: Abhijit Menon-Sen <[EMAIL PROTECTED]> writes: At 2008-09-02 15:10:23 +0300, [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] sgml]$ make man As Alvaro noted recently, you need to use "make man D2MDIR=/some/path". I see it's been like that for quite some time, but still it seems pretty bogus. Why isn't configure handling this? If there's some good reason not to automate it, why isn't it documented in the "Building The Documentation" appendix? regards, tom lane The documentation says that """ To generate quality man pages, it might be necessary to use a hacked version of the conversion utility or do some manual postprocessing. which is why no standard path exists. Something about that needs to be cleaned up, I admit. I'll make a note about it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: > >While it's true POSIX locales don't handle this, other collation > >libraries do and we should support them if the user wants. > > Do they handle exactly those two attributes specifically? Can you point > out references? Or do you mean, other collation libraries allow their > collations to be configured/customized? I think linguistically it is a > very narrow view of the world to hardcode those two attributes. Well, yes. Accents and case are attributes of a character. (I'm using the unicode model here). So, to do a case insensetive match you take the characters, strip the attributes and then do the comparison. There are specialised routines which handle the denormalisation of the string for you so in theory you could even get specific about which accents you ignore. In practice I don't think people do that. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] Mysterious Bus Error with get_fn_expr_argtype()
On Sep 1, 2008, at 22:31, Brendan Jurd wrote: Oh, another thing: it shouldn't be STRICT. Nulls have perfectly good types. Agreed. Barring any further comments/objections, I'll go ahead and prepare a patch to add this to core. So it will return a text representation or an Oid? Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Hello Martijn, > You need to arrange testing on an architechture that has strict > alignment reuiqrements. For example i386 doesn't care about alignment > at all and will anything from anywhere, with performance degradation. > > Other architechtures will simply throw exceptions, that's the smoke > test. >From other comments in this thread, I still have quite a bit of work ahead of me :) If I can accomplish this, then I will run it on a different architecture. I have access to a parisc-linux and a powerpc system. > Also, what's the performance cost? Not sure yet. My goal would be 0 impact on 32-bit system and something small on 64-bit systems that would hopefully be overcome by a performance gain in IO. Thanks for your feedback. - Ryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
On Tue, 2008-09-02 at 18:40 +0300, Peter Eisentraut wrote: > The documentation says that Even though I could not find it, here is an error: $ make man.tar.gz D2MDIR=/usr/share/sgml/docbook/utils-0.6.14/helpers/ make -C sgml man make[1]: Entering directory `/home/devrim/PostgreSQL/pgsql/doc/src/sgml' onsgmls -D . postgres.sgml | sgmlspl /usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl --lowercase --section l --date "`date '+%Y-%m-%d'`" Unknown SDATA: [mdash ] at /usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl line 1241, line 11975. make[1]: *** [man] Error 9 make[1]: Leaving directory `/home/devrim/PostgreSQL/pgsql/doc/src/sgml' make: *** [man.tar] Error 2 Google pointed me to your e-mail that you sent at 2006: http://archives.postgresql.org/pgsql-docs/2006-10/msg00017.php I believe I'm using a recent version: docbook-utils-0.6.14 @ Fedora-9. Any comments? -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org signature.asc Description: This is a digitally signed message part
[HACKERS] Index scan troubles
Hi, I'm having a hard time using an index scan. So far, I've done quite well with ScanKeyInit for equality searches. But now I need to scan an index from a given starting point. Something like: (x, y, z,...) > (const, const, const,...) For the equality operatior, I've used get_sort_group_operators() in combination with get_opcode() to pass that on as the sk_func of the scan key. I slowly begin to doubt if that's correct at all. While it works for equality scans, it does somehow not work for for BTGreaterStrategy. What am I missing? I do have the following: an indexStruct on the index (primary key) I want to use, a TupleDescriptor for the relation I want tuples from and of course the list of constants (datums) to compare against. I want to do an index scan to retrieve tuples from that given lower bound upwards (or forwards). I don't quite grok all the opfamily and opclass things, yet. Hints and pointers on where to read on greatly appreciated. A virtual beer for sample code. ;-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
On Tue, 2008-09-02 at 19:07 +0300, Devrim GÜNDÜZ wrote: > > On Tue, 2008-09-02 at 18:40 +0300, Peter Eisentraut wrote: > > The documentation says that Oh, sorry -- I had missed the remaining part of your e-mail. -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org signature.asc Description: This is a digitally signed message part
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
On Tue, 2008-09-02 at 11:39 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Tue, 2008-09-02 at 18:30 +0900, ITAGAKI Takahiro wrote: > >> How about adding a new variable "recovery_preload_libaries" like as > >> shared_preload_libraries? Rmgr libs in it are loaded only in startup > >> process and only if recovery is needed. > > > Good point. If others agree, I will re-implement this way. > > Aside from the objections raised by Heikki Heikki hasn't raised any. He was objecting to an additional thought from Itagaki. There haven't been any other objections to this concept. > , I'm not seeing the use-case > for an rmgr that only executes during recovery; in fact I'm not entirely > sure that I see a use-case for this entire patch. Where are the WAL > records that the "loadable rmgr" processes going to come from? There is ample reason to do this. I covered this in my first post, please re-read up thread. You have commented on this post already, so I'm suprised by your comments. Rmgr functions only execute during recovery, that is their role in life. Except when we have WAL_DEBUG enabled they are never called elsewhere. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Hello Tom, On Tue, Sep 2, 2008 at 8:07 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Ryan Bradetich" <[EMAIL PROTECTED]> writes: >> The patch concept is fairly simple. > >>1. Add a new boolean local variable: require_max_align >> (initialized to false). > > This really can't possibly work, because you'd need to propagate > knowledge of the tuple's alignment requirement all over the place. > In particular, how would code *reading* the tuple know where the > data starts? Also, I don't think you get (very much of) the actual > benefit unless the code that inserts tuples into disk pages knows > to do something different in the int-align case. I figured there was something wrong with my proof-of-concept patch because it was too easy and it passed regression tests on the first attempt. I just didn't know what was wrong. After reading your comment, I went back and looked at my test data and I am embarrassed to admit that I misread the data :( Sure the tuple length is 44 bytes, but the data is still aligned on a 8 byte boundary. So this poc patch effectively does nothing :( As Greg mentioned, the code reading the user data would know where to start because of the t_hoff field. In my first email in this thread, I had looked at the requirements for the HeapTupleHeaderData to be MAXALIGN. I could not find any reason the HeapTupleHeaderData needed to be MAXALIGN, but Greg pointed out that the t_hoff filed is required to be MAXALIGN. I can only see two reasons why the t_hoff field is required to be MAXALIGN: 1. If the tuple has oids (OIDS require MAXALIGN). 2. If the tuple has any column requiring MAXALIGN. If neither of the above conditions are true, then it seems to me that t_hoff could safely be INTALIGN. I was working under the mistaken assumption that if I addressed this issue in heap_form_tuple, that it would be written down on disk this way. Obviously I was wrong and still have a lot of code reading and experimentation to do :) > It's conceivable that we could make this work if we wanted to dedicate > an infomask bit to showing whether the tuple needs int or double > alignment. I don't really think it's worth the trouble though. I am not sure what an infomask bit is yet, but is it safe to assume (in theory) that if I get the page written correctly, that the tuple reading code would work because of the t_hoff field? Thanks for your feedback! - Ryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Hello Greg, On Tue, Sep 2, 2008 at 8:30 AM, Gregory Stark <[EMAIL PROTECTED]> wrote: > "Ryan Bradetich" <[EMAIL PROTECTED]> writes: > >>4. If require_max_align = true, use the MAXALIGN macro; otherwise >> use the INTALIGN macro. > > Huh, I didn't think of doing it like that. > > But I'm confused. You seem to be tweaking the alignment of the data inside the > tuple? After the tuple header? I thought we had only one byte of wasted space > in there and that's used by the null bitmap. So unless you have more than 8 > columns and some of them are null I wouldn't expect you to save any space. If > you do then I guess you could save 4 bytes if the null bitmap is 2-5 bytes > (mod 8) long. I was not trying to tweak the alignment of the data inside the tuple header, I was trying to adjust the alignment of t_hoff so it would not have the requirement of MAXALIGN. I believe my proof-of-concept patch was bad and I need to spend some more time on it tonight with the new knowledge I gained from this email thread. > I thought the goal was to save space by aligning the tuples on the page more > densely. That seems to me to be more fruitful as about half the tuples will > save four bytes even on tables with small or missing null bitmaps. That is the goal. Basically my test data is 44 bytes in length for each tuple. I have no data in the tuple that is required to be MAXALIGN, but since t_hoff has the MAXALIGN requirement I throw away 4 bytes for each tuple (on a 64-bit machine). This proof-of-concept patch is to help me understand the PostgreSQL code and to see if I can recover those 4 bytes per tuple. Thanks again for your feedback! - Ryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Tue, Sep 02, 2008 at 12:42:45PM +0100, Simon Riggs wrote: > > On Tue, 2008-09-02 at 03:14 -0400, Tom Lane wrote: > > David Fetter <[EMAIL PROTECTED]> writes: > > > On Tue, Sep 02, 2008 at 02:42:25AM -0400, Tom Lane wrote: > > >> It's not like we haven't seen a SQL draft go down in flames > > >> before. > > > > > Do you think that anything in the windowing functions section > > > will disappear? > > > > Who's to say? > > > > I have no objection to looking at the 2003 and 200n documents in > > parallel, especially if there are places where 200n clarifies the > > intent of 2003. But I'd be suspicious of designing around > > entirely-new features presented by 200n. > > I have confirmation from Michael Gorman, Wiscorp, that > > > The new standard was approved in early Summer. SQL 2008 is > > finished. > > So as of now, SQL2008 exists, all hail. SQL2003 and earlier versions > have been superseded and can be ignored. Any chance we can buy a few copies of the official one for use on the project? Cheers, David. -- David Fetter <[EMAIL PROTECTED]> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Gregory Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> In particular, how would code *reading* the tuple know where the >> data starts? > Uh, at t_hoff, no? Doh, right. Obviously need more caffeine. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
Devrim GÜNDÜZ wrote: > $ make man.tar.gz D2MDIR=/usr/share/sgml/docbook/utils-0.6.14/helpers/ > make -C sgml man > make[1]: Entering directory `/home/devrim/PostgreSQL/pgsql/doc/src/sgml' > onsgmls -D . postgres.sgml | sgmlspl > /usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl --lowercase > --section l --date "`date '+%Y-%m-%d'`" > Unknown SDATA: [mdash ] at > /usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl line 1241, > line 11975. Please see here: http://archives.postgresql.org/message-id/20080821130203.GN4169%40alvh.no-ip.org -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: >> >While it's true POSIX locales don't handle this, other collation >> >libraries do and we should support them if the user wants. I think that's backwards. We have to go with the lowest common denominator functionality of those libraries if we're going to be portable. As long as it's a superset of the SQL standard functionality. If we support features of some of them that can't be emulated with others then users end up with SQL code that will only work on some builds and not others. That might be worth it for some features but I'm not sure this is one. > Well, yes. Accents and case are attributes of a character. (I'm using > the unicode model here). So, to do a case insensetive match you take > the characters, strip the attributes and then do the comparison. There > are specialised routines which handle the denormalisation of the string > for you so in theory you could even get specific about which accents > you ignore. In practice I don't think people do that. I don't think composable unicode characters are really about collations. I think it had more to do with representing glyphs in UTF32 before they gave up on that. Does anyone still use composable characters? Note that we don't currently support composable characters at all. I'm not sure if that's a "nobody really cares" issue or a bug we should aim to fix with real collation support. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
"Ryan Bradetich" <[EMAIL PROTECTED]> writes: > Hello Greg, > > On Tue, Sep 2, 2008 at 8:30 AM, Gregory Stark <[EMAIL PROTECTED]> wrote: >> >> But I'm confused. You seem to be tweaking the alignment of the data inside >> the >> tuple? After the tuple header? I thought we had only one byte of wasted space >> in there and that's used by the null bitmap. So unless you have more than 8 >> columns and some of them are null I wouldn't expect you to save any space. If >> you do then I guess you could save 4 bytes if the null bitmap is 2-5 bytes >> (mod 8) long. > > I was not trying to tweak the alignment of the data inside the tuple header, > I was trying to adjust the alignment of t_hoff so it would not have the > requirement of MAXALIGN. I believe my proof-of-concept patch was bad and I > need to spend some more time on it tonight with the new knowledge I gained > from this email thread. The structure on the page is to have a bunch of tuples stored one after the other. Each tuple is maxaligned after the previous (actually before the previous since they start from the end but we'll ignore that -- the point is that they all *start* at a maxaligned position on the page). Within each tuple there's a header which is 23 bytes long. Then an optional null bitmap. In the header is t_hoff which points to the first byte of actual data -- effectively specifying the length of the null bitmap. t_hoff is maxaligned as well. So by intaligning t_hoff you're adjusting where the data within the tuple fits by making the null bitmap 4 bytes shorter. That seems worth a few cycles but isn't going to affect every table. It'll only affect tables that have a null bitmap between 2-5 bytes (mod 8), ie, with 9-40 fields at least one of which is null. Tables with 9 fields are likely to be relatively wide which means saving 4 bytes isn't that much of a gain percentagewise. (Though they might be mostly null) The other 4 bytes you could save is by packing the whole tuples themselves more closely on the page. That happens when the item pointer is added and pointed to the tuple. To do that heap_form_tuple would have to return data back to the caller about the alignment needed to whomever is going to copy it into the page. AFAICS that could be done in the HeapTuple structure itself rather than in the tuple though which doesn't seem so bad. Tom may be seeing something I'm not though. Incidentally, can't you set the alignment during the compute_data_size() call instead of doing an extra pass through the attributes? Probably doesn't change the cost though. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
"Ryan Bradetich" <[EMAIL PROTECTED]> writes: > On Tue, Sep 2, 2008 at 8:07 AM, Tom Lane <[EMAIL PROTECTED]> wrote: >> It's conceivable that we could make this work if we wanted to dedicate >> an infomask bit to showing whether the tuple needs int or double >> alignment. I don't really think it's worth the trouble though. > I am not sure what an infomask bit is yet, but is it safe to assume > (in theory) that if I get the > page written correctly, that the tuple reading code would work because > of the t_hoff field? Well, as Greg pointed out, setting t_hoff correctly should be sufficient for squeezing out useless padding between the tuple header and the tuple data. The real objection here is that that's leaving most of the possible gain still on the table. The tuple *as a whole* (header and data both) is still going to get maxaligned when it's plopped onto a disk page. I think that in real-world cases that's often where the main padding cost comes in: with 23-byte tuple headers and no OIDs, you aren't saving a thing by int-aligning t_hoff unless you have a nulls bitmap and it's the wrong number of bytes long. So IMHO the real problem is over near PageAddItem. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Tue, 2008-09-02 at 09:35 -0700, David Fetter wrote: > On Tue, Sep 02, 2008 at 12:42:45PM +0100, Simon Riggs wrote: > > > > On Tue, 2008-09-02 at 03:14 -0400, Tom Lane wrote: > > > David Fetter <[EMAIL PROTECTED]> writes: > > > > On Tue, Sep 02, 2008 at 02:42:25AM -0400, Tom Lane wrote: > > > >> It's not like we haven't seen a SQL draft go down in flames > > > >> before. > > > > > > > Do you think that anything in the windowing functions section > > > > will disappear? > > > > > > Who's to say? > > > > > > I have no objection to looking at the 2003 and 200n documents in > > > parallel, especially if there are places where 200n clarifies the > > > intent of 2003. But I'd be suspicious of designing around > > > entirely-new features presented by 200n. > > > > I have confirmation from Michael Gorman, Wiscorp, that > > > > > The new standard was approved in early Summer. SQL 2008 is > > > finished. > > > > So as of now, SQL2008 exists, all hail. SQL2003 and earlier versions > > have been superseded and can be ignored. > > Any chance we can buy a few copies of the official one for use on the > project? You have to buy them from ISO website I think, but it was $00s when I looked some years back - we'd probably need a dozen copies at least since we hardly ever meet. Michael's .pdf was the final version, so we do actually have the final form even if the page formatting somewhat different. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What is d2mdir?
On Tue, 2008-09-02 at 12:47 -0400, Alvaro Herrera wrote:
> > Unknown SDATA: [mdash ]
> at /usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl
> line 1241, line 11975.
>
> Please see here:
>
> http://archives.postgresql.org/message-id/20080821130203.GN4169%
> 40alvh.no-ip.org
Thanks, it fixed some part of the problem (and I'm now submitting it as
a patch to Fedora)
Now:
make man.tar.gz D2MDIR=/usr/share/sgml/docbook/utils-0.6.14/helpers/
make -C sgml man
make[1]: Entering directory `/home/devrim/PostgreSQL/pgsql/doc/src/sgml'
onsgmls -D . postgres.sgml | sgmlspl
/usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl --lowercase
--section l --date "`date '+%Y-%m-%d'`"
Warning: output contains unresolved XRefs
onsgmls -D . postgres.sgml | sgmlspl
/usr/share/sgml/docbook/utils-0.6.14/helpers//docbook2man-spec.pl --lowercase
--section l --date "`date '+%Y-%m-%d'`"
Warning: output contains unresolved XRefs
"/usr/bin/perl" -npi -e 's{\[XRef to GUC-([A-Z0-9-]*)\]}{($l = $1) =~
tr/A-Z-/a-z_/, $l}ge || s{\[XRef to [A-Z0-9-]*\]}{in the documentation}g' *.1
*.l
Can't open *.l: No such file or directory, <> line 6943.
/bin/sh ../../../config/mkinstalldirs man1 manl
mkdir -p -- man1 manl
mv *.1 man1/
mv *.l manl/
mv: cannot stat `*.l': No such file or directory
make[1]: *** [man] Error 1
make[1]: Leaving directory `/home/devrim/PostgreSQL/pgsql/doc/src/sgml'
make: *** [man.tar] Error 2
Marc, do you use a patched version of this script? If so, could you please put
it somewhere public so that we can push it to upstream?
Thanks.
--
Devrim GÜNDÜZ, RHCE
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org
signature.asc
Description: This is a digitally signed message part
Re: [HACKERS] Index scan troubles
Markus Wanner <[EMAIL PROTECTED]> writes: > Hi, > > I'm having a hard time using an index scan. So far, I've done quite well with > ScanKeyInit for equality searches. But now I need to scan an index from a > given > starting point. Something like: > >(x, y, z,...) > (const, const, const,...) > > For the equality operatior, I've used get_sort_group_operators() in > combination > with get_opcode() to pass that on as the sk_func of the scan key. I slowly > begin to doubt if that's correct at all. It's right for your equality case which is effectively x=const, y=const, z=const. It's not for row comparisons case for which you need a funny "header" ScanKey. See the comments in access/skey.h, search for "row comparisons". I'm not sure if there's a function to create this for you or if you have to do it yourself. Search for other places where SK_ROW_HEADER appears. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
On Tue, Sep 02, 2008 at 05:42:13PM +0100, Gregory Stark wrote: > Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > > On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: > >> >While it's true POSIX locales don't handle this, other collation > >> >libraries do and we should support them if the user wants. > > I think that's backwards. We have to go with the lowest common denominator > functionality of those libraries if we're going to be portable. And I think that's backwards. Why can we only use a feature once every OS out there implements it? We still run on systems that don't have SSL support. LC_TYPE settings are not portable between systems, yet that doesn't bother anyone. Why should we have a problem with collate settings not being portable? > I don't think composable unicode characters are really about collations. I > think it had more to do with representing glyphs in UTF32 before they gave up > on that. Does anyone still use composable characters? Lookup the various normalisations forms: http://en.wikipedia.org/wiki/Unicode_normalization In particular Normal Form D. Sure, composable characters have nothing to do with collation, but they provide a uniform way of doing accent insensetive collation. > Note that we don't currently support composable characters at all. Any character which is an accent on a latin character is a decomposable character. And last I checked we supported those. Have a niceday, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
[HACKERS] What is the use of the CommitFestBlank template?
The description of the CommitFestBlank template suggests that it sets up an editable page for you, but AFAICT it does no such thing. I had to manually create all the right sections: http://wiki.postgresql.org/index.php?title=CommitFest:2008-11&diff=2260&oldid=2258 in order to have a page that people could actually put patches into. Unless templates have power to replace themselves in the page source, it seems like this can't possibly work. It would be more helpful to have a blank page that we could clone (if MediaWiki has such a concept). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mysterious Bus Error with get_fn_expr_argtype()
On Tue, Sep 02, 2008 at 08:58:04AM -0700, David E. Wheeler wrote: > >Barring any further comments/objections, I'll go ahead and prepare a > >patch to add this to core. > > So it will return a text representation or an Oid? Hopefully regtype. The function doesn't need changing, but then users can get a text representation of the type easily then. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] Mysterious Bus Error with get_fn_expr_argtype()
On Sep 2, 2008, at 08:58, David E. Wheeler wrote:
On Sep 1, 2008, at 22:31, Brendan Jurd wrote:
Oh, another thing: it shouldn't be STRICT. Nulls have perfectly
good
types.
Agreed.
Barring any further comments/objections, I'll go ahead and prepare a
patch to add this to core.
So it will return a text representation or an Oid?
Looks like regtype displays as an integer. So how about pg_regtypeof()
and pg_typeof()?
PG_FUNCTION_INFO_V1(pg_regtypeof);
Datum
pg_regtypeof(PG_FUNCTION_ARGS)
{
PG_RETURN_OID(get_fn_expr_argtype(fcinfo->flinfo, 0));
}
PG_FUNCTION_INFO_V1(pg_typeof);
Datum
pg_typeof(PG_FUNCTION_ARGS)
{
Oidtypeoid;
typeoid = get_fn_expr_argtype(fcinfo->flinfo, 0);
if (typeoid == InvalidOid) {
ereport(
ERROR, (
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("could not determine data type of argument to
pg_typeof()")
)
);
}
PG_RETURN_DATUM(CStringGetDatum(format_type_be(typeoid)));
}
Best,
David
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mysterious Bus Error with get_fn_expr_argtype()
On Sep 2, 2008, at 10:43, David E. Wheeler wrote:
Looks like regtype displays as an integer. So how about
pg_regtypeof() and pg_typeof()?
Sorry, make that:
PG_FUNCTION_INFO_V1(pg_regtypeof);
Datum
pg_regtypeof(PG_FUNCTION_ARGS)
{
PG_RETURN_OID(get_fn_expr_argtype(fcinfo->flinfo, 0));
}
PG_FUNCTION_INFO_V1(pg_typeof);
Datum
pg_typeof(PG_FUNCTION_ARGS)
{
Oidtypeoid;
typeoid = get_fn_expr_argtype(fcinfo->flinfo, 0);
if (typeoid == InvalidOid) {
ereport(
ERROR, (
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("could not determine data type of argument to
pg_typeof()")
)
);
}
PG_RETURN_DATUM(DirectFunctionCall1(textin,
CStringGetDatum(format_type_be(typeoid;
}
Best,
David
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Index scan troubles
Markus Wanner <[EMAIL PROTECTED]> writes: > I'm having a hard time using an index scan. So far, I've done quite well > with ScanKeyInit for equality searches. But now I need to scan an index > from a given starting point. Something like: > (x, y, z,...) > (const, const, const,...) > For the equality operatior, I've used get_sort_group_operators() in > combination with get_opcode() to pass that on as the sk_func of the scan > key. I slowly begin to doubt if that's correct at all. > While it works for equality scans, it does somehow not work for for > BTGreaterStrategy. What am I missing? A row comparison (a,b,c) > (x,y,z) means something entirely different from a>x AND b>y AND c>z; but it sounds like the keys you are creating define the latter condition. Look at the RowCompareExpr stuff in nodeIndexscan.c to see how to build a scankey that means the former. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mysterious Bus Error with get_fn_expr_argtype()
"David E. Wheeler" <[EMAIL PROTECTED]> writes: > Looks like regtype displays as an integer. Better try that again. regression=# select 1043::regtype; regtype --- character varying (1 row) regression=# I see no need for two functions here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Gregory Stark <[EMAIL PROTECTED]> writes: > The other 4 bytes you could save is by packing the whole tuples themselves > more closely on the page. That happens when the item pointer is added and > pointed to the tuple. To do that heap_form_tuple would have to return data > back to the caller about the alignment needed to whomever is going to copy it > into the page. AFAICS that could be done in the HeapTuple structure itself > rather than in the tuple though which doesn't seem so bad. Tom may be seeing > something I'm not though. You'd need to keep the info available for subsequent re-compactions of the page, so I think an infomask bit is really the only solution that would work. But on reflection, that's really a separable issue from whether we minimize t_hoff padding. And as Ryan's patch shows, it's not that hard to do the latter. We should probably grab the low-hanging fruit here. > Incidentally, can't you set the alignment during the compute_data_size() call > instead of doing an extra pass through the attributes? +1. This code is executed often enough that every little bit helps. BTW, there are at least two copies of that code to be changed. I'd suggest grepping for assignments to t_hoff to be sure there aren't more. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mysterious Bus Error with get_fn_expr_argtype()
On Sep 2, 2008, at 11:06, Tom Lane wrote: Better try that again. regression=# select 1043::regtype; regtype --- character varying (1 row) regression=# I see no need for two functions here. Oh. I tried: try=# select 1::regtype; regtype - 1 I had assumed that 1 would be some type, but apparently not. Oops. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
Radek Strnad escribió: > Ok, so do you suggest to leave it with a notice "reindex database" or start > to solve it somehow? I don't know. If there are two tasks that need the same treatment, it seems a safe conclusion that they need a common solution. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Index scan troubles
Hi, Gregory Stark wrote: It's right for your equality case which is effectively x=const, y=const, z=const. It's not for row comparisons case for which you need a funny "header" ScanKey. See the comments in access/skey.h, search for "row comparisons". I'm not sure if there's a function to create this for you or if you have to do it yourself. Search for other places where SK_ROW_HEADER appears. Ah, so the default for multiple ScanKeys is 'x > const AND y > const' and not the row comparison. That explains my troubles. Thanks a lot. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: Collation support
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > And I think that's backwards. Why can we only use a feature once every > OS out there implements it? We still run on systems that don't have SSL > support. LC_TYPE settings are not portable between systems, yet that > doesn't bother anyone. Why should we have a problem with collate > settings not being portable? If we're going to approach it that way, we need a syntax for CREATE COLLATION that doesn't hard-wire what the possible collation modifiers are. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Page layout footprint
Zdenek Kotala napsal(a): 32/64 bit issue is little bit different story and it is general (not only SPARC but on SPARC has bigger impact). Problem is that CRC32 gives probably different result when it is compiled 32bit or 64bit. I'm going to examine it more. I'm sorry about noise. Everything works as expected. I tested 8.3 and looked into 8.4 code :(. The problem with 8.3 is that control file uses time_t which is defined as long and it is not portable between 32bits and 64bits server version. Version 8.4 is OK, because it uses pg_time_t which is alway 64bit long. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] make dist does not include man.tar.gz and postgres.tar.gz
Hi, Devrim has been trying to set up RPM files for 8.4devel. However, the tarballs he is generating are not alike those found in our FTP site; official ones include manpages and HTML docs as man.tar.gz and postgres.tar.gz, but a simple "make dist" does not seem to include them. I'm wondering how are these things supposed to be generated. Does Marc create the regular tarballs using "make dist", and then unpack them and include the other tarballs inside? Does he use a different make target? One idea I suggested was running "make man.tar.gz postgres.tar.gz" before "make dist", but that doesn't really work apparently. Here's why: [EMAIL PROTECTED] pgsql]$ make dist rm -rf postgresql-8.4devel* =install= for x in `cd . && find . -name CVS -prune -o -print`; do \ file=`expr X$x : 'X\./\(.*\)'`; \ if test -d "./$file" ; then \ mkdir "postgresql-8.4devel/$file" && chmod 777 "postgresql-8.4devel/$file"; \ else \ ln "./$file" "postgresql-8.4devel/$file" >/dev/null 2>&1 \ || cp "./$file" "postgresql-8.4devel/$file"; \ fi || exit; \ done cp: cannot stat `./doc/src/sgml/CREATE': No such file or directory -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Make gram.y use palloc/pfree for memory management
"Marko Kreen" <[EMAIL PROTECTED]> writes: > On 9/1/08, Marko Kreen <[EMAIL PROTECTED]> wrote: >> First a correction, overriding malloc/free seems dangerous they >> seems to leak out, so correct would be to use YYMALLOC/YYFREE. >> This leaves 1.875 potentially leaking, but danger seems small. > Here is the safer patch. As the chance for the leak is rare, > leaving 1.875 vulnerable should not be problem. Actually, 1.875 appears to default to using alloca() when available, so the issue doesn't even arise if you're building with gcc. (This is probably why I'd never noticed a problem in this area.) I thought that we might have an issue with flex as well, but so far as I can tell there's no real problem given our current usage style for the lexers. Applied to all the backend .l files ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make dist does not include man.tar.gz and postgres.tar.gz
Alvaro Herrera <[EMAIL PROTECTED]> writes: > I'm wondering how are these things supposed to be generated. Does Marc > create the regular tarballs using "make dist", and then unpack them and > include the other tarballs inside? Does he use a different make target? I think you need "make distprep" first. I've never seen Marc's actual build script, however. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
Gregory Stark <[EMAIL PROTECTED]> writes: > "Marko Kreen" <[EMAIL PROTECTED]> writes: >> Uh. So you want force "proper" units in presentations at the price >> of everyday admin operations? Does not seem like a sensible tradeoff. > It didn't to anyone else when Peter wrote the current version either, but as > the person willing to actually do the work and write the code Peter got to > make the decision. Nobody else stepped up to do the work to change it and we > can't exactly force Peter to do work he doesn't agree with. It's not that, in my mind: it's that Peter feels more strongly about it than the rest of us. This proposal has come up before and he's successfully argued it down each time. He does have a point about there being some potential for confusion; and the arguments on the other side are not much better than "I'm lazy". Being lazy myself, I'd prefer a case insensitive implementation; but I don't feel strongly enough about it to want to override Peter's opinion. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
Anything that will reduce potential downtime should be way to go. To me it seems that Peter uses the loudest voice and others just don't care enough. Using kB for kilobyte seems quite alien and confusing. I have not noticed that to be used in software i use in my everyday work and could not find any either with quick search. On Wed, Sep 3, 2008 at 12:15 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > Gregory Stark <[EMAIL PROTECTED]> writes: > > "Marko Kreen" <[EMAIL PROTECTED]> writes: > >> Uh. So you want force "proper" units in presentations at the price > >> of everyday admin operations? Does not seem like a sensible tradeoff. > > > It didn't to anyone else when Peter wrote the current version either, but > as > > the person willing to actually do the work and write the code Peter got > to > > make the decision. Nobody else stepped up to do the work to change it and > we > > can't exactly force Peter to do work he doesn't agree with. > > It's not that, in my mind: it's that Peter feels more strongly about it > than the rest of us. This proposal has come up before and he's > successfully argued it down each time. He does have a point about there > being some potential for confusion; and the arguments on the other side > are not much better than "I'm lazy". Being lazy myself, I'd prefer a > case insensitive implementation; but I don't feel strongly enough about > it to want to override Peter's opinion. > >regards, tom lane > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] Question regarding the database page layout.
Hello Greg, On Tue, Sep 2, 2008 at 9:54 AM, Gregory Stark <[EMAIL PROTECTED]> wrote: > "Ryan Bradetich" <[EMAIL PROTECTED]> writes: > The structure on the page is to have a bunch of tuples stored one after the > other. Each tuple is maxaligned after the previous (actually before the > previous since they start from the end but we'll ignore that -- the point is > that they all *start* at a maxaligned position on the page). > > Within each tuple there's a header which is 23 bytes long. Then an optional > null bitmap. In the header is t_hoff which points to the first byte of actual > data -- effectively specifying the length of the null bitmap. t_hoff is > maxaligned as well. Thanks for your persistence in explaining this issue to me. I think I am starting to understand the issues now and I appreciate the help. I think I see the error in my thinking. You and Tom are saying the tuple is MAXALIGN on the page as well. My assumption was that since t_hoff had to be MAXALIGN, the tuple on the page was being MAXALIGN by default. If I was able to INTALIGNt_hoff then the tuple on the page would be free to be INTALIGN as well. Tom gave me the next place to go look later tonight (PageAddItem). > So by intaligning t_hoff you're adjusting where the data within the tuple fits > by making the null bitmap 4 bytes shorter. That seems worth a few cycles but > isn't going to affect every table. It'll only affect tables that have a null > bitmap between 2-5 bytes (mod 8), ie, with 9-40 fields at least one of which > is null. Tables with 9 fields are likely to be relatively wide which means > saving 4 bytes isn't that much of a gain percentagewise. (Though they might be > mostly null) > > The other 4 bytes you could save is by packing the whole tuples themselves > more closely on the page. That happens when the item pointer is added and > pointed to the tuple. To do that heap_form_tuple would have to return data > back to the caller about the alignment needed to whomever is going to copy it > into the page. AFAICS that could be done in the HeapTuple structure itself > rather than in the tuple though which doesn't seem so bad. Tom may be seeing > something I'm not though. > > Incidentally, can't you set the alignment during the compute_data_size() call > instead of doing an extra pass through the attributes? Probably doesn't change > the cost though. Actually I did not do an extra pass through the attributes. I added the check in an existing pass through the attributes. I did look at adding this check to the heap_compute_data_size(), but took the easy path to see if my patch worked. I will update my patch to do the alignment check in heap_compute_data_size(). Thanks! - Ryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question regarding the database page layout.
Hello Tom, > Well, as Greg pointed out, setting t_hoff correctly should be sufficient > for squeezing out useless padding between the tuple header and the tuple > data. The real objection here is that that's leaving most of the > possible gain still on the table. The tuple *as a whole* (header and > data both) is still going to get maxaligned when it's plopped onto a > disk page. I think that in real-world cases that's often where the > main padding cost comes in: with 23-byte tuple headers and no OIDs, > you aren't saving a thing by int-aligning t_hoff unless you have a nulls > bitmap and it's the wrong number of bytes long. > > So IMHO the real problem is over near PageAddItem. Thanks for your feed back as well. Between you and Greg I think I finally understand the error in my thinking. I will investigate the PageAddItem() later tonight. - Ryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Window functions patch v04 for the September commit fest
On Tue, Sep 2, 2008 at 9:35 AM, David Fetter <[EMAIL PROTECTED]> wrote: > Any chance we can buy a few copies of the official one for use on the > project? AFAIK there is no significant difference between the "official" standard and the draft version available online, so I don't see the point. Neil -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature request: better debug messages
Brian Hurt wrote: > So I thought upping the debug level of postgres would help, by adding > a -o '-d 5' to the pg_ctl command line. The log file that gets spit > out in this case is attached. What is noticeable is how unhelpfull > it is. I think this is just the stderr. The rest of the error messages are going somewhere else -- syslog maybe? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gsoc, oprrest function for text search take 2
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= <[EMAIL PROTECTED]> writes: > Pre-sorting introduced one problem (see XXX in code): it's not easy > anymore to get the minimal frequency of MCELEM values. I was using it to > assert that the selectivity of a tsquery node containing a lexeme not in > MCELEM is no more that min(MCELEM freqs) / 2. That's only significant > when the minimum frequency is less than DEFAULT_TS_SEL * 2, so I'm kind > of inclined to ignore it and maybe drop a comment in the code that this > may be a potential problem. This is easily fixed: there is nothing saying that a pg_statistic slot's contents must contain the same numbers of Values and Numbers. Make the numbers array have one extra element and store the min frequency there. Maybe it'd be worth having 2 extra elements and dropping the max in, as well. I don't immediately have a use for it, but it'll be a lot harder to add it later if we don't put it in now. > If nothing is fundamentally broken with this, I'll repeat my profiling > tests to see if anything has been gained. I don't have much except minor stylistic gripes (like the ordering of the functions in ts_selfuncs.c seeming a bit random). One possibly performance-relevant point is to use DatumGetTextPP for detoasting; you've already paid the costs by using VARDATA_ANY etc, so you might as well get the benefit. Please fix the above and do the performance testing ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
