Re: [HACKERS] invalidating cached plans

2005-04-02 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > "Neil Conway" <[EMAIL PROTECTED]> writes >> PostgreSQL should invalidate a cached query plan when one of the objects >> the plan depends upon is modified. > It just comes into my mind that current cache invalidation implementation > may need to conside

Re: [HACKERS] invalidating cached plans

2005-04-01 Thread Qingqing Zhou
"Neil Conway" <[EMAIL PROTECTED]> writes > PostgreSQL should invalidate a cached query plan when one of the objects > the plan depends upon is modified. It just comes into my mind that current cache invalidation implementation may need to consider the future query result cache. The question comes

Re: [HACKERS] invalidating cached plans

2005-03-20 Thread Tom Lane
Harald Fuchs <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> One possible approach is to do the invalidation on a sufficiently coarse >> grain that we don't care. For example, I would be inclined to make any >> change in a table's schema invalidate all plans that use that tab

Re: [HACKERS] invalidating cached plans

2005-03-20 Thread Harald Fuchs
In article <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> writes: > One possible approach is to do the invalidation on a sufficiently coarse > grain that we don't care. For example, I would be inclined to make any > change in a table's schema invalidate all plans that use that table at > all;

Re: [HACKERS] invalidating cached plans

2005-03-17 Thread Karel Zak
On Thu, 2005-03-17 at 16:11 +1100, Neil Conway wrote: > Neil Conway wrote: > > Do we want to share plans between call sites? > > After thinking about this a little more, I think the answer is "no" -- > it doesn't really buy us much, and introduces some extra complications > (e.g. resource manage

Re: [HACKERS] invalidating cached plans

2005-03-16 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > BTW, it's quite annoying that the planner scribbles on its input. Yeah ... it would be good to fix that ... regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading throu

Re: [HACKERS] invalidating cached plans

2005-03-16 Thread Neil Conway
Neil Conway wrote: Do we want to share plans between call sites? After thinking about this a little more, I think the answer is "no" -- it doesn't really buy us much, and introduces some extra complications (e.g. resource management). BTW, it's quite annoying that the planner scribbles on its in

Re: [HACKERS] invalidating cached plans

2005-03-15 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > That makes me wonder. What happens if I prepare a query, then use SET SESSION > AUTHORIZATION to change my user. Then try to execute the query? Permissions checks are applied at executor startup, not by the planner, so it should Work Correctly in my view of

Re: [HACKERS] invalidating cached plans

2005-03-15 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Neil Conway <[EMAIL PROTECTED]> writes: > > (BTW, another thing to consider is how the rewriter will effect a plan's > > dependencies: I think we should probably invalidate a plan when a > > modification is made to a view or rule that affected the plan. >

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > This is the key point (say this is point_1) - we must make sure how deep we > have to go to check validity. The recorded plan has to somehow mention all the inlined functions that were expanded out of existence. There might be several levels of such t

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Qingqing Zhou wrote: I don't quite understand the difference between a SQL function and a PL/PgSQL function here - since there is a overlapped functionality that we could implement by SQL function or by PL/PgSQL function. The difference is between an inlined function (which is integrated directly

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Oliver Jowett wrote: >> Does this mean that clients that use PREPARE/Parse need to handle "plan >> invalidated" as a possible response to EXECUTE/Bind, or will the backend >> keep the query string / parse tree around and replan on next execution? > The l

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > (BTW, another thing to consider is how the rewriter will effect a plan's > dependencies: I think we should probably invalidate a plan when a > modification is made to a view or rule that affected the plan. This issue goes away as long as you follow the r

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Tom Lane wrote: I hadn't really gotten as far as working out a reasonable API for the module. The $64 question seems to be what is the input: a textual query string, a raw parse analysis tree, or what? It should be easy enough to accept either, and then convert from the query string into a raw pa

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Qingqing Zhou
"Neil Conway" <[EMAIL PROTECTED]> writes > I don't see that this is a major problem. If a plan A invokes a function > B, then changes to B will need to invalidate A; that should be pretty > easy to arrange. If B is a PL/PgSQL function that invokes a function C, > it will probably cache a plan invo

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Oliver Jowett wrote: Does this mean that clients that use PREPARE/Parse need to handle "plan invalidated" as a possible response to EXECUTE/Bind, or will the backend keep the query string / parse tree around and replan on next execution? The latter -- the client won't be aware that replanning too

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Oliver Jowett
Neil Conway wrote: - it is the responsibility of the call site managing the prepared plan to check whether a previously prepared plan is invalid or not -- and to take the necessary steps to replan it when needed. Does this mean that clients that use PREPARE/Parse need to handle "plan invalidated"

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Qingqing Zhou wrote: Second (as Tom says), some changes can hardly be traced. For example, we only use function A. But function A cites function B, function B cites function C. when C changes, how do we know that we should worry about our plan? I don't see that this is a major problem. If a plan A

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Rod Taylor
On Mon, 2005-03-14 at 20:06 -0500, Bruce Momjian wrote: > Harald Fuchs wrote: > > In article <[EMAIL PROTECTED]>, > > Tom Lane <[EMAIL PROTECTED]> writes: > > > > > One possible approach is to do the invalidation on a sufficiently coarse > > > grain that we don't care. For example, I would be inc

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Christopher Kings-Lynne
This may be totally irrelevant: Our current load distributors, like pgpool, have no way of knowing the side effects of backend functions. It would be interesting if the client could send each potential query to the master saying, "execute this query if there are side effects, otherwise do no operat

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Qingqing Zhou
"Harald Fuchs" <[EMAIL PROTECTED]> writes > How about using an even coarser grain? Whenever something in the > database in question changes, blindly throw away all cached plans for > this DB. > If we clearly define what is "something in database in question", we have to trace all the objects the

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Neil Conway
Bruce Momjian wrote: One idea would be to record if the function uses non-temp tables, temp tables, or both, and invalidate based on the type of table being invalidated, rather than the table name itself. I can imagine this hurting temp table caching, but at least functions using regular tables wo

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Bruce Momjian
Harald Fuchs wrote: > In article <[EMAIL PROTECTED]>, > Tom Lane <[EMAIL PROTECTED]> writes: > > > One possible approach is to do the invalidation on a sufficiently coarse > > grain that we don't care. For example, I would be inclined to make any > > change in a table's schema invalidate all plan

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Harald Fuchs
In article <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> writes: > One possible approach is to do the invalidation on a sufficiently coarse > grain that we don't care. For example, I would be inclined to make any > change in a table's schema invalidate all plans that use that table at > all;

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Greg Stark
Michael Adler <[EMAIL PROTECTED]> writes: > Our current load distributors, like pgpool, have no way of knowing the > side effects of backend functions. It would be interesting if the > client could send each potential query to the master saying, "execute > this query if there are side effects, oth

Re: [HACKERS] invalidating cached plans

2005-03-14 Thread Michael Adler
On Mon, Mar 14, 2005 at 02:53:36AM -0500, Tom Lane wrote: > Probably the first thing to do is look around at the plausible users of > this thing and see what they'd find most convenient. This may be totally irrelevant: Our current load distributors, like pgpool, have no way of knowing the side ef

Re: [HACKERS] invalidating cached plans

2005-03-13 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I would like to see this folded together with creation of a centralized >> plan caching module. > Interesting. Can you elaborate on how you'd envision call sites making > use of this module? I hadn't really gotten as far as working out

Re: [HACKERS] invalidating cached plans

2005-03-13 Thread Neil Conway
Tom Lane wrote: I would like to see this folded together with creation of a centralized plan caching module. Interesting. Can you elaborate on how you'd envision call sites making use of this module? The difficulty with this after-the-fact approach is that the finished plan tree may contain no re

Re: [HACKERS] invalidating cached plans

2005-03-13 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > PostgreSQL should invalidate a cached query plan when one of the objects > the plan depends upon is modified. Agreed. > Implementation sketch: I would like to see this folded together with creation of a centralized plan caching module. We currently have

[HACKERS] invalidating cached plans

2005-03-13 Thread Neil Conway
PostgreSQL should invalidate a cached query plan when one of the objects the plan depends upon is modified. This is the common case of a more general problem: a query plan depends on various parts of the environment at plan-creation time. That environment includes the definitions of database object