Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Tom Lane wrote: I hadn't really gotten as far as working out a reasonable API for the module. The $64 question seems to be what is the input: a textual query string, a raw parse analysis tree, or what? It should be easy enough to accept either, and then convert from the query string into a raw

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Qingqing Zhou wrote: I don't quite understand the difference between a SQL function and a PL/PgSQL function here - since there is a overlapped functionality that we could implement by SQL function or by PL/PgSQL function. The difference is between an inlined function (which is integrated directly

[HACKERS] invalidating cached plans

2005-03-13 Thread Neil Conway
PostgreSQL should invalidate a cached query plan when one of the objects the plan depends upon is modified. This is the common case of a more general problem: a query plan depends on various parts of the environment at plan-creation time. That environment includes the definitions of database

Re: [HACKERS] invalidating cached plans

2005-03-13 Thread Neil Conway
Tom Lane wrote: I would like to see this folded together with creation of a centralized plan caching module. Interesting. Can you elaborate on how you'd envision call sites making use of this module? The difficulty with this after-the-fact approach is that the finished plan tree may contain no

[HACKERS] cvsup binaries?

2005-03-11 Thread Neil Conway
The PostgreSQL docs suggest that ftp.postgresql.org holds binary builds of CVSup: http://developer.postgresql.org/docs/postgres/cvsup.html Is this still the case? (I couldn't see any cvsup binaries, but perhaps they are well-hidden). -Neil ---(end of

Re: [HACKERS] fool-toleranced optimizer

2005-03-09 Thread Neil Conway
Simon Riggs wrote: Oleg is saying that the optimizer doesn't protect against foolish SQL requests. His query is an example of a foolishly written query. IMHO calling this a foolishly written query is completely arbitrary. I can imagine plenty of applications for which a cartesian join makes

Re: [HACKERS] fool-toleranced optimizer

2005-03-08 Thread Neil Conway
Oleg Bartunov wrote: I just noticed a little optimizer problem - in second query there is unused 'tycho t2' table alias which gets backend buried. It's not an unused table alias, it is specifying the cartesian product of `tycho' with itself. I don't see how this is an optimizer problem: it's a

Re: [HACKERS] postgreSQL-8.0.1 configure --enable-thread-safety with

2005-03-08 Thread Neil Conway
Bruce Momjian wrote: The attached patch should remove the warnings but I am not applying it because a non-static/extern global variable should be fine in C code. What's the harm in applying it? Variables and functions not used outside the compilation unit in which they are defined _should_ be

Re: [HACKERS] refactoring fork() and EXEC_BACKEND

2005-03-05 Thread Neil Conway
Magnus Hagander wrote: This is a lot like what I was planning to work towards with the refactoring of the forkexec code I promised to do for 8.1. Cool. BTW, have we accepted that EXEC_BACKEND is the way we're going to workaround the lack of fork() on Win32 for the foreseeable future? I mean, it

Re: [HACKERS] bitmap AM design

2005-03-04 Thread Neil Conway
Pailloncy Jean-Gerard wrote: You should have a look to this thread http://archives.postgresql.org/pgsql-hackers/2005-02/msg00263.php Take a look at this paper about lock-free parallel hash table http://www.cs.rug.nl/~wim/mechver/hashtable/ Is this relevant? Hash indexes are on-disk data

Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch

2005-03-03 Thread Neil Conway
Thomas F.O'Connell wrote: I have a feeling Bruce was referring to item 1.4: http://developer.postgresql.org/readtext.php?src/FAQ/ FAQ_DEV.html+Developers-FAQ#1.4 It has never been standard practice to ask for comments before the development of small features, such as this one. The recently

[HACKERS] refactoring fork() and EXEC_BACKEND

2005-03-03 Thread Neil Conway
While going through the usual motions needed to fork a child process of the postmaster, it occurred to me that there's a fair bit of duplicated code involved. There are also #ifdef for various situations (BeOS, LINUX_PROFILE, and EXEC_BACKEND), which makes the code yet more ugly. I think we

Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch

2005-03-01 Thread Neil Conway
Bruce Momjian wrote: Have you read the developer's FAQ. It is a requirement to get involved. How does the developer's FAQ address this concern? -Neil ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] idea for concurrent seqscans

2005-02-27 Thread Neil Conway
Jeff Davis wrote: The only reason I did that was because the original source was difficult to read and work on. Is there a reason that code and comments wrap around to the next line throughout the file? I'm not sure what you mean. Assuming your editor expand tabs to 4 spaces (which is the

Re: [pgsql-advocacy] [HACKERS] Development Plans

2005-02-27 Thread Neil Conway
Tom Lane wrote: I wouldn't mind seeing people be a little more vocal on the hackers list about what they plan to be doing, just so that there's not duplication of effort. Stuff I have done in some form that I need to finish up and submit: - GiST improvements: sane memory management, 10% scan perf.

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Neil Conway
Jeff Davis wrote: I have a newer version of my Synchronized Scanning patch which hopefully makes it closer to a real patch, the first one was more of a proof of concept. A few minor comments: - context diffs (diff -c) are the preferred format. Also, folks usually send patches as a single diff; an

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Neil Conway
William Volkman wrote: The patch is for *concurrent* seqscans, would the regression tests even be affected by this (IIRC they are single user, single process)? No; `make installcheck' is serial, but `make check' executes tests in parallel in multiple backends concurrently. -Neil

Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Neil Conway
Bruce Momjian wrote: Is there a TODO here? Probably -- I think there is definitely room for improving psql's handling of multi-line queries. However, \e works well enough for me, and I don't think I'll get around to looking at this for 8.1. So feel free to add a TODO item. -Neil

Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Neil Conway
Bruce Momjian wrote: And what would the TODO item be? Improve psql's handling of multi-line queries is too vague. If you can include a link to the archives or the text of the relevant mails, it seems fine to me. I'm not sure specifically _how_ we want to improve the handling of multi-line

Re: [HACKERS] Where are we on stored procedures?

2005-02-24 Thread Neil Conway
Tom Lane wrote: Essentially I'm thinking about the JDBC solution, but automated a bit better. So would your proposal invent a new stored procedure construct, or just add some sugar to the existing function stuff? i.e. will you be able to issue a CREATE FUNCTION that specifies OUT parameters?

Re: [HACKERS] left-deep plans?

2005-02-22 Thread Neil Conway
Kenneth Marshall wrote: GEQO is an attempt to provide a near-optimal join order without using an exhaustive search. An exhaustive, deterministic search of a subset of the search space has a non-zero probability of finding only a local minimum in execution time. I'm not sure what you mean. By an

[HACKERS] left-deep plans?

2005-02-21 Thread Neil Conway
Presently the planner considers left-deep, right-deep, and bushy plans (i.e. it will consider plans in which the outer operand of a join is a join, the inner operand is a join, or both operands are joins). It is a fairly standard heuristic in the literature to restrict the search to left-deep

[HACKERS] psql: recall previous command?

2005-02-21 Thread Neil Conway
Is there a way to recall the previous command in psql? Obviously, up arrow or Ctrl-P using readline and the default readline bindings is close, but it recalls the previous _line_ of input. That is not at all the same thing in the case of a multiple line command, for example. If there is no way

Re: [HACKERS] left-deep plans?

2005-02-21 Thread Neil Conway
Tom Lane wrote: Yes, and it's been rejected. The notion is obviously bogus; it amounts to assuming that every database is a star schema with only one core table. Interesting; yes, I suppose that's true. Once we get into GEQO territory, we are using the left-deep-only heuristic because that's the

Re: [HACKERS] postgres crashing on a seemingly good query

2005-02-20 Thread Neil Conway
On Sat, 2005-02-19 at 14:35 -0500, Tom Lane wrote: Good catch. I've applied the attached patch (this is against 8.0/CVS tip but applies with some fuzz to 7.4). Is there a way to repro this via SQL? (It would be nice to have a regression test...) -Neil ---(end of

Re: [HACKERS] win32 performance - fsync question

2005-02-18 Thread Neil Conway
Tom Lane wrote: Portability, or rather the complete lack of it. Stuff that isn't in the Single Unix Spec is a hard sell. O_DIRECT is reasonably common among modern Unixen (it is supported by Linux, FreeBSD, and probably a couple of the commercial variants like AIX or IRIX); it should also be

Re: [HACKERS] enforcing a plan (in brief)

2005-02-15 Thread Neil Conway
On Tue, 2005-02-15 at 02:38 -0500, Greg Stark wrote: I don't know what software you work with but the Postgres source is far and away the best documented source I've had the pleasure to read. I agree the PostgreSQL source is very nice (for the most part), but I think there could be more

Re: [HACKERS] getting oid of function

2005-02-14 Thread Neil Conway
On Mon, 2005-02-14 at 17:02 +0500, Sibtay Abbas wrote: thank you for the detailed reply But what i wanted to know is that how can we actually get a function's oid from its name from within postgresql code itself You'll want to query the syscache. Note that due to function overloading, there

Re: [HACKERS] enforcing a plan (in brief)

2005-02-14 Thread Neil Conway
On Mon, 2005-02-14 at 22:56 -0500, Bruce Momjian wrote: And the user maintenance of updating those hints for every release of PostgreSQL as we improve the database engine. ... and maintaining those hints as the data changes over time. But I think this thread has been hijacked toward a subject

Re: [HACKERS] enforcing a plan (in brief)

2005-02-10 Thread Neil Conway
On Thu, 2005-02-10 at 14:37 -0500, Bruce Momjian wrote: No, we feel that is of limited value. If the optimizer isn't doing things properly, we will fix it. I agree that improving the optimizer is the right answer for normal usage, so I can't get excited about query-level plan hints, but I can

Re: [HACKERS] Thinking about breaking up the BufMgrLock

2005-02-06 Thread Neil Conway
On Sun, 2005-02-06 at 19:30 -0500, Tom Lane wrote: This would be pretty good from a locking point of view, except that update the LRU state seems to require taking an exclusive lock on a global data structure, which puts us about back where we were. We're only concerned with a buffer's access

Re: [HACKERS] Connect By for 8.0

2005-02-04 Thread Neil Conway
Robert Treat wrote: Actually i believe people want both syntax's as the former is used by oracle and the latter by db2 (iirc) I think the past consensus has been to adopt the SQL standard syntax. Is there any reason to also support the Oracle syntax other than for compatibility? (And if that is

Re: [HACKERS] LWLockRelease

2005-02-03 Thread Neil Conway
Simon Riggs wrote: Setting MAX_SIMUL_LWLOCKS to this fairly high number doesn't seem to match the optimistic use of the O(N) algorithm. How so? The algorithm is O(n) for the number of locks _currently held_, not the maximum number of locks we might be able to hold. In other words, in

Re: [HACKERS] LWLock cache line alignment

2005-02-03 Thread Neil Conway
Simon Riggs wrote: ...and BTW, what is MMCacheLock?? is that an attempt at padding already? One would hope not, as it would be a totally braindead attempt :) It appears to have been formerly used by smgr/mm.c; when that was removed, the MMCacheLock should have been removed but was not. Barring

[HACKERS] subselects in the target list

2005-02-02 Thread Neil Conway
This behavior seems inconsistent: neilc=# create table abc (a int, b int); CREATE TABLE neilc=# create function foo_abc() returns setof abc as 'select * from abc' language sql; CREATE FUNCTION neilc=# insert into abc values (5, 10); INSERT 17234 1 neilc=# insert into abc values (10, 20); INSERT

Re: [HACKERS] subselects in the target list

2005-02-02 Thread Neil Conway
On Wed, 2005-02-02 at 23:22 -0500, Tom Lane wrote: The syntax you are showing is designed to return a scalar. It will (and should) barf on multiple rows as well as multiple columns. I don't understand; the example I posted is of an SRF that returns multiple rows of multiple columns, which is

Re: [HACKERS] [NOVICE] Last ID Problem

2005-02-01 Thread Neil Conway
On Tue, 2005-02-01 at 11:24 -0300, Alvaro Herrera wrote: How about the TID? That wouldn't be sufficiently stable for use by client applications, I believe: a concurrent VACUUM FULL could mean your TID no longer points at what you think it does. -Neil ---(end of

Re: [HACKERS] [NOVICE] Last ID Problem

2005-02-01 Thread Neil Conway
On Tue, 2005-02-01 at 17:50 -0500, Tom Lane wrote: It'd be safe enough within the same transaction, since VACUUM can't kill a tuple inserted by an open transaction; nor could VACUUM FULL touch the table at all, since you'll be holding at least a writer's lock on the table. True, but it still

Re: [HACKERS] FunctionCallN improvement.

2005-01-31 Thread Neil Conway
On Mon, 2005-01-31 at 23:38 +0900, a_ogawa wrote: (b)Define the macro that initialize FunctionCallInfoData, and use it instead of MemSet in all FunctionCallN, DirectFunctionCallN, OidFunctionCallN. This macro is the following. #define InitFunctionCallInfoData(Fcinfo, Flinfo, Nargs)

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Neil Conway
Mike Rylander wrote: For on-disk bitmap indexes, yes. I don't see any reason this couldn't be done with GiST It might be possible to do it with GiST, but GiST is designed for implementing tree-structured indexes; I don't think it's the right tool for the job. -Neil

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Neil Conway
Mike Rylander wrote: For the initial example where the index is implemented as a set of unique keys from the table and a bitmap for each key this would look a unique index, but with an extra datum at at each index node to hold the bitmap for that key. If implemented that way an augmented B-Tree

Re: [HACKERS] bug w/ cursors and savepoints

2005-01-26 Thread Neil Conway
On Wed, 2005-01-26 at 12:02 -0300, Alvaro Herrera wrote: and some regression tests for this behavior, but I'm happy to add that myself if no one beats me to it. Please do. Attached is a patch adding regression tests for this change -- I've already applied it to HEAD. -Neil Index:

Re: [HACKERS] Heads up: upcoming releases in all branches back to

2005-01-26 Thread Neil Conway
On Tue, 2005-01-25 at 13:09 -0500, Tom Lane wrote: Current thought is to wrap these on Thursday for release Friday. If you have any last-minute fixes for the back branches, now's the time to get them in. Sorry for the last minute commit, but I realized that I forgot to backpatch the cursor

Re: [HACKERS] Deferrable Unique Constraints

2005-01-26 Thread Neil Conway
On Wed, 2005-01-26 at 15:48 -0500, Greg Stark wrote: Well presumably you would need a non-unique index created for query execution purposes. The unique index would be purely for enforcing the constraint. Yuck. You could perhaps relax the uniqueness of the index during the transaction itself,

Re: [HACKERS] bug w/ cursors and savepoints

2005-01-25 Thread Neil Conway
On Tue, 2005-01-25 at 02:40 -0500, Tom Lane wrote: Offhand I'd say this should draw a no such cursor as foo error. I'm too tired to look into why foo still exists after the rollback... I'm confused; I wasn't involved in the design discussions about portals and subtransactions this summer, but

Re: [HACKERS] RQ: Prepared statements used by multiple connections

2005-01-25 Thread Neil Conway
Christopher Kings-Lynne wrote: I need a mechanism to keep my queries in optimized state so that multiple processes can use them. You should use stored procedures then. For instance, say you want to keep 'SELECT * FROM table WHERE id=x' prepared. You would go: CREATE OR REPLACE FUNCTION

Re: [HACKERS] Concurrent free-lock

2005-01-25 Thread Neil Conway
Simon Riggs wrote: The one factor which stands out for me from this is that Keir Fraser's and Tim Harris' algorithms are available as *code*, which additionally are covered by a licence which appears to be an MIT/BSD variant licence. If you're referring to their Lock-free library, that is licensed

Re: [HACKERS] bug w/ cursors and savepoints

2005-01-25 Thread Neil Conway
Tom Lane wrote: The routine's comments need a bit of work too. Otherwise it seems OK. Neil or anyone else --- see an issue here? The policy will now be: cursor creation is transaction, but cursor state modifications (FETCH) are non-transactional -- right? I wonder if it wouldn't be more

Re: [HACKERS] Patent issues and 8.1

2005-01-25 Thread Neil Conway
Bruce Momjian wrote: So if we have to address it we call it 8.0.7 or something. My point is that we don't need to address it until we actually find out the patent is being enforced against someone, and that possibility is quite unlikely. IMHO, the patent issue is *not* a potential problem for a

Re: [HACKERS] Status of PostgreSQL for 4+ processors

2005-01-25 Thread Neil Conway
Benjamin Arai wrote: I have been trying to find information on PostgreSQL for running it on greater then 4 processors. Are there any benchmarks out there and have there been any problems or does anybody forsee any issues running PostgreSQL with more then 4 processors? There shouldn't be any

Re: [HACKERS] Patent issues and 8.1

2005-01-25 Thread Neil Conway
Tom Lane wrote: I've already pointed out a couple reasons why I don't have any confidence in its correctness. Well, you've suggested that I should try and reduce the API churn caused by the patch. As I said on -patches, I don't really see this as an issue if we just apply the patch to

Re: [HACKERS] Concurrent free-lock

2005-01-24 Thread Neil Conway
On Mon, 2005-01-24 at 08:35 -0700, Jonah H. Harris wrote: Lock free data structures are cool... but not really applicable to databases. They have a high maintenance overhead, severe complexity, and will fail when there are many concurrent inserts/deletes to the structure. Can you

Re: [HACKERS] Concurrent free-lock

2005-01-24 Thread Neil Conway
On Mon, 2005-01-24 at 16:50 -0700, Jonah H. Harris wrote: Here is some pretty good info on lock-free structures... I'm pretty sure I tested their code in a multithreaded high-concurrency environment and experienced the problems I was discussing. Fair enough, but my hope would be that those

Re: [HACKERS] Concurrent free-lock

2005-01-24 Thread Neil Conway
On Mon, 2005-01-24 at 19:36 -0600, Min Xu (Hsu) wrote: In any case, I think only when contention is high the non-blocking algorithms are worth looking at. So can someone shine some light on where the contention might be? The major point of contention that has been identified in the past is

[HACKERS] bug w/ cursors and savepoints

2005-01-24 Thread Neil Conway
Someone at Fujitsu pointed out the following bug in 8.0: begin; savepoint x; create table abc (a int); insert into abc values (5); declare foo cursor for select * from abc; rollback to x; fetch from foo; -- hits an Assert() commit; The stacktrace is: #2 0x0826367b in ExceptionalCondition

Re: [HACKERS] Extending System Views: proposal for 8.1/8.2

2005-01-22 Thread Neil Conway
Jim C. Nasby wrote: On another naming note; the naming convention for system stuff has always driven me nuts. Some the letter prefix (ie: tab for tables) in front of every field name, with no underscores or anything. Extensive use of abbreviations that you need to remember (ie: indnatts, indexprs,

Re: [HACKERS] ARC patent

2005-01-20 Thread Neil Conway
Simon Riggs wrote: However, I think the ARC replacement should *not* be a fundamental change in behavior: the algorithm should still attempt to balance recency and frequency, to adjust dynamically to changes in workload, to avoid sequential flooding, and to allow constant-time page replacement.

Re: [HACKERS] ARC patent

2005-01-20 Thread Neil Conway
On Fri, 2005-01-21 at 01:26 +, Simon Riggs wrote: Agree with everything apart from the idea that seq scan flooding isn't an issue. I definitely think it is. I agree it's an issue, I just don't think it's an issue of sufficient importance that it needs to be solved in the 8.0.x timeframe.

Re: [HACKERS] Caching of frequently used objects

2005-01-19 Thread Neil Conway
On Wed, 2005-01-19 at 19:14 +0100, Yann Michel wrote: The effect while using a seperate buffer cache for different objects, i.e. using a lru list would stay the same. There would be only two more than one buffer cache for a certain object gourp or class. Multiple buffer caches and pinning

Re: [HACKERS] ARC patent

2005-01-18 Thread Neil Conway
On Mon, 2005-01-17 at 15:11 -0500, Andrew Dunstan wrote: There's a very recent paper at http://carmen.cs.uiuc.edu/~zchen9/paper/TPDS-final.ps on an alternative to ARC which claims superior performance ... From a quick glance, this doesn't look applicable. The authors are discussing buffer

Re: [HACKERS] ARC patent

2005-01-18 Thread Neil Conway
On Mon, 2005-01-17 at 18:43 -0500, Tom Lane wrote: I have already suggested to core that we should insist on 8.1 not requiring an initdb, so as to ensure that people will migrate up to it easily from 8.0. So is it firm policy that changes that require a catversion update cannot be made during

Re: [HACKERS] ARC patent

2005-01-18 Thread Neil Conway
On Tue, 2005-01-18 at 23:26 -0500, Tom Lane wrote: Not yet --- I suggested it but didn't get any yeas or nays. I don't feel this is solely core's decision anyway ... what do the assembled hackers think? I'm not sure it's a great idea. I'm not aware of a recent example of short development

Re: [HACKERS] ARC patent

2005-01-17 Thread Neil Conway
On Mon, 2005-01-17 at 12:15 -0800, Joshua D. Drake wrote: We have just admitted that we knowingly may infringe upon an IBM patent. That really is a different thing than, We have some really smart people that came up with something, like this other technology. The code is clear that it

Re: [HACKERS] ARC patent

2005-01-17 Thread Neil Conway
On Mon, 2005-01-17 at 12:30 -0800, Joshua D. Drake wrote: The biggest problem is going to be that if we release 8 with the patented stuff, then for a minimum of 3 years there will be liability for anyone running 8. We still have people running 7.1 and once you get something into production

Re: [HACKERS] sparse (static analyzer) report

2005-01-16 Thread Neil Conway
Mark Wong wrote: Ah, so you beat me to it Neil. ;) Out of curiosity, how much worse was it before you started fixing things? As I recall, not too different than things are today -- sparse flagged a bunch of stylistic issues that I fixed, like: void some_func() { ... } = void some_func(void) {

Re: [HACKERS] pgdump

2005-01-16 Thread Neil Conway
On Fri, 2005-01-14 at 16:24 +0100, Andreas Joseph Krogh wrote: http://dev.officenet.no/~andreak/pg_dump.c.diff Looks good, except for some minor code cleanups and doc updates. Barring any objections, I'll clean it up and apply it once we branch 8.0. I suppose for consistency we ought to allow

Re: [HACKERS] pgdump

2005-01-16 Thread Neil Conway
On Sun, 2005-01-16 at 23:42 -0500, Bruce Momjian wrote: I don't remember this patch. http://archives.postgresql.org/pgsql-patches/2004-07/msg00331.php How is it related to the other pg_dump patches in the 8.1 pathces queue? I missed the July '04 discussion about the other patches for

Re: [HACKERS] IBM releases 500 patents

2005-01-16 Thread Neil Conway
On Tue, 2005-01-11 at 08:04 -0800, Darcy Buskermolen wrote: IBM has just announced they are waving all rights and providing access to 500 patents. In the list of 500 there are several that relate RDBMS and query optimizations, it may be worth a look. FYI, IBM has applied for a patent on ARC

Re: [HACKERS] pgdump

2005-01-16 Thread Neil Conway
On Mon, 2005-01-17 at 00:54 -0500, Tom Lane wrote: -t s1.t1 [...] without any quoting rules it would then become impossible to deal with names containing dots. Ah, yeah -- sorry, I was focusing on case conversion rather than quoting in general. Are we willing to blow off that case? Or

Re: [HACKERS] pgdump

2005-01-16 Thread Neil Conway
On Mon, 2005-01-17 at 01:19 -0500, Tom Lane wrote: Just to be clear: what I understand the logic to be is OR across multiple switches of the same type, but AND across switches of two types. If I understand you correctly, you're suggesting that we should only report an error if none of the

Re: [HACKERS] sparse (static analyzer) report

2005-01-14 Thread Neil Conway
BTW, perhaps one reason for the relatively small number of legitimate issues picked up by sparse is that I ran sparse on the tree a month or two ago and fixed some of the stylistic issues it reported. Most of the stuff I didn't bother to fix looked like either a sparse bug, or a marginal style

Re: [HACKERS] bgwriter changes

2004-12-16 Thread Neil Conway
Zeugswetter Andreas DAZ SD wrote: This has the disadvantage of converging against 0 dirty pages. A system that has less than maxpages dirty will write every page with every bgwriter run. Yeah, I'm concerned about the bgwriter being overly aggressive if we disable bgwriter_percent. If we leave

Re: [HACKERS] strange regression failure

2004-12-15 Thread Neil Conway
On Wed, 2004-12-15 at 21:33 -0500, Andrew Dunstan wrote: Also, currently buildfarm just runs postgres' own test suites. I'm not averse to supporting a more extensive test suite just for farm members, if people think that's a good idea. I think you'd get more mileage out of expanding the

Re: [HACKERS] bgwriter changes

2004-12-14 Thread Neil Conway
On Tue, 2004-12-14 at 09:23 -0500, Tom Lane wrote: At this point in the release cycle I'm not sure we should be making any significant changes for anything less than a crashing bug. Yes, that's true, and I am definitely hesitant to make changes during RC. That said, adjust bgwriter defaults has

[HACKERS] bgwriter changes

2004-12-13 Thread Neil Conway
In recent discussion[1] with Simon Riggs, there has been some talk of making some changes to the bgwriter. To summarize the problem, the bgwriter currently scans the entire T1+T2 buffer lists and returns a list of all the currently dirty buffers. It then selects a subset of that list (computed

Re: [HACKERS] dropdb/contrib-regression

2004-12-13 Thread Neil Conway
On Mon, 2004-12-13 at 19:29 -0500, Tom Lane wrote: A more radical solution would be to tweak libpq's PQfinish() to be a synchronous close, ie, wait for the backend to exit before returning. I think we've speculated about doing that in the past, but never been fully convinced that it's worth

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Neil Conway
On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Is the plan to make bgwriter_percent = 100 the default setting? Hmm...must confess that my only plan is: i) discover dynamic behaviour of bgwriter ii) fix any bugs or wierdness as quickly

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-11 Thread Neil Conway
I wonder if we even need to retain the bgwriter_percent GUC var. Is there actually a situation in which the combination of bgwriter_maxpages and bgwriter_delay does not give the DBA sufficient flexibility in tuning bgwriter behavior? Simon Riggs wrote: If the bgwriter_percent = 100, then we

Re: [HACKERS] Oid?

2004-12-06 Thread Neil Conway
On Mon, 2004-12-06 at 00:29 -0800, Sibtay Abbas wrote: i see an attribute 'oid_value' with 'Oid' type in the ListCell union. can anyone tell me what does it represents? Given a linked list of type T_OidList, oid_value holds the data in a node of the list. Although int_value and oid_value are

[HACKERS] branch for 8.0?

2004-12-06 Thread Neil Conway
There was some talk of branching REL8_0_STABLE after 8.0.0rc1 was released. Is that still the plan? TIA, Neil ---(end of broadcast)--- TIP 8: explain analyze is your friend

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 10:45 -0500, Tom Lane wrote: (2) I think you lose much of the performance benefit as soon as you have to distinguish cases (b) and (c). Ideally you would use MemoryContextContains for this, but that doesn't work reliably on pointers that point to fields of a tuple. Why

Re: [HACKERS] Code documentation

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 10:58 +0100, Gevik Babakhani wrote: I was wondering if there are any interests or plans for documenting various functions in the code which currently are not documented. I don't know of any systematic effort to do this. I try to document undocumented code as necessary

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 19:07 -0500, Tom Lane wrote: True, but you still have to palloc if it returns the second argument. Is that common? In any case, I don't see how you can _ever_ avoid a palloc if the aggregate returns the second argument. The second argument is in a per-tuple memory context:

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 20:51 -0500, Tom Lane wrote: No. The current code involves two pallocs per cycle, one inside the aggregate function to construct its result value, and then one in datumCopy to copy the result into the proper context. Ah, true -- missed the fact that PG_RETURN_INT64()

Re: [HACKERS] lwlocks and starvation

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 09:59 -0500, Bruce Momjian wrote: OK, either you have to own the issue or I have to add something to the TODO list. Can you add it to the TODO list and assign it to me? -Neil ---(end of broadcast)--- TIP 3: if

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 08:25 +, Simon Riggs wrote: I'd be a little twitchy about new memory contexts at this stage of beta, but if the code is fairly well isolated that could be good. This would be for 8.1 anyway. Would it possible to differentiate between well-known builtin functions and

Re: [HACKERS] New compile warnings for inheritance

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 17:22 -0500, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I am seeing the following regression failures with CVS head and inheritance tests. They look like sort order problems: Does it go away if you initdb? I suspect the size of pg_class is a factor in

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 11:08 -0300, Alvaro Herrera wrote: When would the experimentation end, and 8.0 be finally released? As I said, this is work for 8.1; I don't think it ought to affect the release schedule of 8.0 (perhaps in some marginal way because Tom might spend some time discussing

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 19:34 +0100, [EMAIL PROTECTED] wrote: I regard performance testing as an essential part of the release process of any performance critical software. Most earlier beta releases were fixing functional problems and now the focus has moved to performance related issues. I

Re: [HACKERS] lwlocks and starvation

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 21:51 -0500, Bruce Momjian wrote: Neil, where are we on this? Should we add comments? Add a TODO? A patch? I'm not sure what the right resolution is. As I said, I don't think it's wise to apply a patch that could have a significant impact on performance without (a)

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 15:54 -0500, Tom Lane wrote: This seems like it might work. Instead of copying the result into the aggcontext on every cycle, we could copy it only when we intend to reset the working context. Right. This is problematic since the source tuple will go away on the next

[HACKERS] nodeAgg perf tweak

2004-11-30 Thread Neil Conway
I noticed that we have a bottleneck in aggregate performance in advance_transition_function(): according to callgrind, doing datumCopy() and pfree() for every tuple produced by the transition function is pretty expensive. Some queries bare this out: dvl=# SELECT W.element_id, count(W.element_ID)

Re: [HACKERS] nodeAgg perf tweak

2004-11-30 Thread Neil Conway
On Tue, 2004-11-30 at 23:15 -0500, Tom Lane wrote: And how badly does it leak memory? I do not believe this patch is tenable. Did you read the rest of my mail? Something that occurred to me the other morning in the shower is that we could trivially inline MemoryContextSwitchTo() when using

Re: [HACKERS] Opinions on Usenet ...

2004-11-29 Thread Neil Conway
On Mon, 2004-11-29 at 15:03 -0400, Marc G. Fournier wrote: If there were a comp.databases.postgresql.hackers newsgroup created and carried by all the news servers ... would you move to using it vs using the mailing lists? No. -Neil ---(end of

Re: [HACKERS] [GENERAL] Column n.nsptablespace does not exist error

2004-11-29 Thread Neil Conway
On Tue, 2004-11-30 at 17:54 +1100, Johan Wehtje wrote: I am getting the error Column n.nsptablespace does not exist in my application when I connect using my Administrative tool. This only happens with Version 8, but it does crash my application, does anyone have any ideas ? You need to

Re: [HACKERS] psql and schemas

2004-11-28 Thread Neil Conway
On Sat, 2004-11-27 at 23:11 -0500, Bruce Momjian wrote: Is there a TODO here? Or a few? Sure: you could add a TODO item like Improve psql schema behavior, and assign it to me. I'll send in a patch that implements the behavior I proposed for 8.1 -Neil ---(end of

Re: [HACKERS] -V, --version -- deprecated?

2004-11-28 Thread Neil Conway
On Wed, 2004-11-24 at 20:25 -0500, Bruce Momjian wrote: FreeBSD had a problem with double-dash args but I thought that related to getopt, and I can't remember how that fits in. Maybe we defined '-' in getopt and said it took an argument and tested for '-help' and '-verbose', but now we just

Re: [HACKERS] lwlocks and starvation

2004-11-25 Thread Neil Conway
Simon Riggs wrote: I'd been thinking about lock release order also, thinking that this could be related to the CS storms observed earlier and the apparent lock-step behaviour commented upon previously. As much as I would love to believe this (because it would mean we could probably solve the

[HACKERS] lwlocks and starvation

2004-11-24 Thread Neil Conway
LWLockRelease() currently does something like (simplifying a lot): acquire lwlock spinlock decrement lock count if lock is free if first waiter in queue is waiting for exclusive lock, awaken him; else, walk through the queue and awaken all the shared waiters until we

Re: [HACKERS] Bitmap index

2004-11-24 Thread Neil Conway
On Mon, 2004-11-22 at 07:57 +, PaweX Niewiadomski wrote: I saw discussion about bitmap indexes few weeks ago. I wonder if any of you is working on it (in secret)? For what it's worth, I don't know of anyone working on them. I will be chosing subject of my master thesis and thougth about

<    1   2   3   4   5   6   7   8   9   10   >