Re: [HACKERS] patch: preload dictionary new version
2010/7/8 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2010/7/8 Robert Haas robertmh...@gmail.com: A precompiler can give you all the same memory management benefits. I use mmap(). And with mmap the precompiler are not necessary. Dictionary is loaded only one time - in original ispell format. I think, it is much more simple for administration - just copy ispell files. There are not some possible problems with binary incompatibility, you don't need to solve serialisation, deserialiasation, ...you don't need to copy TSearch ispell parser code to client application - probably we would to support not compiled ispell dictionaries still. Using a precompiler means a new questions for upgrade! You're inventing a bunch of straw men to attack. There's no reason that a precompiler approach would have to put any new requirements on the user. For example, the dictionary-load code could automatically execute the precompile step if it observed that the precompiled copy of the dictionary was missing or had an older file timestamp than the source. uff - just safe activation of precompiler needs lot of low level code - but maybe I see it wrong, and I doesn't work directly with files inside pg. But I can't to see it as simple solution. I like the idea of a precompiler step mainly because it still gives you most of the benefits of the patch on platforms without mmap. (Instead of mmap'ing, just open and read() the precompiled file.) In particular, you would still have a creditable improvement for Windows users without writing any Windows-specific code. the loading cca 10 MB takes on my comp cca 30 ms - it is better than 90ms, but it isn't a win. I think we can divide this problem to three parts a) simple allocator - it can be used not only for TSearch dictionaries. I think that's a waste of time, frankly. There aren't enough potential use cases. b) sharing a data - it is important for large dictionaries Useful but not really essential. c) preloading - it decrease load time of first TSearch query This is the part that is the make-or-break benefit of the patch. You need a solution that cuts load time even when mmap isn't available. I am not sure if this existing, and if it is necessary. Probably main problem is with Czech language - we have a few specialities. For Czech environment is UNIX and Windows platform the most important. I have not information about using Postgres and Fulltext on other platforms here. So, probably the solution doesn't need be core. I am thinking about some pgfoundry project now - some like ispell dictionary preload. I can send only simplified version without preloading and sharing. Just solving a memory issue - I think so there are not different opinions. best regards Pavel Stehule regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch (for 9.1) string functions
hello
2010/7/9 Takahiro Itagaki itagaki.takah...@gmail.com:
2010/7/8 Pavel Stehule pavel.steh...@gmail.com:
sorry, attached fixed patch
Make installcheck for contrib/stringfunc is broken.
Please run regression test with --enable-cassert build.
test stringfunc ... TRAP:
FailedAssertion(!(!lc_ctype_is_c()), File: mbutils.c, Line: 715)
LOG: server process (PID 15121) was terminated by signal 6: Aborted
This patch contains several functions.
- format(fmt text, VARIADIC args any)
- sprintf(fmt text, VARIADIC args any)
- concat(VARIADIC args any)
- concat_ws(separator text, VARIADIC args any)
- concat_json(VARIADIC args any)
- concat_sql(VARIADIC args any)
- rvrs(str text)
- left(str text, n int)
- right(str text, n int)
The first one is in the core, and others are in contrib/stringfunc.
But I think almost
all of them should be in the core, because users want to write portable SQLs.
Contrib modules are not always available. Note that concat() is
supported by Oracle,
MySQL, and DB2. Also left() and right() are supported by MySQL, DB2,
and SQL Server.
Functions that depend on GUC settings should be marked as VOLATILE
instead of IMMUTABLE. I think format(), sprintf(), and all of
concat()s should be
volatile because at least timestamp depends on datestyle parameter.
ok, I'll fix it
concat_ws() and rvrs() should be renamed to non-abbreviated forms.
How about concat_with_sep() and reverse() ?
I used a well known names - concat_ws (MySQL) and rvrs (Oracle rdbms),
I like concat_ws - concat_with_sep is maybe too long. rvrs is too
short, so I'll rename it to reverse - ok?
I think we should avoid concat_json() at the moment because there is another
development project for JSON support. The result type will be JOIN type rather
than text then.
ok
I'm not sure usefulness of concat_sql(). Why don't you just quote all values
with quotes and separate them with comma?
concat_xxx functions are helpers to serialisation. So when when you
would to generate INSERT statements for some export, and you cannot
use a COPY statement, you can do
FOR r IN
SELECT
LOOP
RETURN NEXT 'INSERT INTO tab(..) VALUES (' || concat_sql(r.a, r.b, r.c, ... )
END LOOP;
RETURN;
you don't need to solve anything and output is well formated SQL. Some
databases dislike quoted numeric values - and quoted nums can be
sonfusing
format() function prints NULL as NULL, but RAISE statement in PL/pgSQL
does as NULL.
I prefer just NULL.
maybe some GUC variable
stringfunc.null_string = 'NULL' in future??
We have some choices for NULL representation. For example, empty string,
NULL, NULL, or (null) . What will be our choice? Each of them looks
equally reasonable for me. GUC idea is also good because we need to
mark format() as VOLATILE anyway. We have nothing to lose.
Can ve to solve it other patch? I know to aversion core hackers to new
GUC. Now I propose just NULL. The GUC for NULL representation has
bigger consequences - probably have to related to RAISE statement, and
to proposed functions to_string, to_array.
---
Takahiro Itagaki
Thank You very much, I'do fix it
Pavel
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Assertion failure in get_attstatsslot()
Consider the following small testcase: BEGIN; CREATE OR REPLACE FUNCTION upper(IN varchar, OUT varchar) LANGUAGE SQL STRICT IMMUTABLE AS $$ SELECT pg_catalog.upper($1)::varchar; $$; CREATE TABLE foo(value varchar); INSERT INTO foo SELECT 'helmle' FROM generate_series(1, 1000); CREATE INDEX foo_upper_index ON foo(public.UPPER(value)); ANALYZE foo; -- assertion failure EXPLAIN SELECT 1 FROM foo WHERE UPPER(value) = 'xyz'; COMMIT; While the sense of the UPPER() function is surely debatable (but it is used in this way in the database where the assertion failure was triggered), the EXPLAIN in the script crashes with TRAP: FailedAssertion(!(((array)-elemtype) == elmtype), File: arrayfuncs.c, Line: 2970) LOG: server process (PID 15451) was terminated by signal 6: Abort trap It took me a while to strip that down to this script, because it only happens when the statistics slots of the index are examined for the constant value in the where clause. The stavalues array holds type oid 1043, whereas type oid 25 is expected. I tried it back from current -HEAD to 8.3.11 and managed to reproduce it everywhere. Non-cassert builds are working correctly, so i'm not sure wether this is an over-aggressive assert() or masks a problem from somewhere else. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bug? Concurrent COMMENT ON and DROP object
2010/7/9 KaiGai Kohei kai...@ak.jp.nec.com: (2010/07/07 11:31), Robert Haas wrote: On Tue, Jul 6, 2010 at 10:18 PM, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haasrobertmh...@gmail.com writes: Obviously not. We don't need to acquire an AccessExclusiveLock to comment on an object - just something that will CONFLICT WITH an AccessExclusiveLock. So, use the same locking rules, perhaps, but take a much weaker lock, like AccessShareLock. Well, it probably needs to be a self-conflicting lock type, so that two COMMENTs on the same object can't run concurrently. But I agree AccessExclusiveLock is too strong: that implies locking out read-only examination of the object, which we don't want. Hmm... so, maybe ShareUpdateExclusiveLock? That looks to be the weakest thing that is self-conflicting. The others are ShareRowExclusiveLock, ExclusiveLock, and AccessExclusiveLock. Is it necessary to confirm existence of the database object being commented on after we got acquired the lock, isn't it? Since the logic of AcquireDeletionLock() requires us to provide argument as object-id form, but we have to translate the object name into object-id outside of the critical section, so the object being commented might be already dropped and committed before we got acquired the lock. Yep. I'm going to work up a patch for this. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ExecutorCheckPerms() hook
On Thu, May 20, 2010 at 1:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: As for committing it, I would definitely like to commit the actual hook. If we want the hook without the contrib module that's OK with me, although I generally feel it's useful to have examples of how hooks can be used, which is why I took the time to produce a working example. +1 on committing the hook. As for the contrib module, it doesn't strike me that there's much of a use-case for it as is. I think it's enough that it's available in the -hackers archives. OK, done that way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Stamp HEAD as 9.1devel.
On Fri, Jul 9, 2010 at 12:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: How long should I wait before I start breaking things? Did you have any particular breakage in mind? Well, you can see for yourself what I've submitted for the next CF. You might want to hold off on the get_whatever_oid patches for a bit, but the other stuff I see there looks pretty localized. No objection to pressing forward with CF work otherwise. I can hold off on those for a bit - I don't think there will be enough drift to matter very much, but if it makes you more comfortable, it's not a big deal. What I really want to get committed is this one, which is infrastructure for a further large patch: include Backend ID in relpath for temp rels https://commitfest.postgresql.org/action/patch_view?id=302 However, I need someone fairly knowledgeable to review it first. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 14:06 +, Robert Haas wrote: Log Message: --- Add a hook in ExecCheckRTPerms(). This hook allows a loadable module to gain control when table permissions are checked. It is expected to be used by an eventual SE-PostgreSQL implementation, but there are other possible applications as well. A sample contrib module can be found in the archives at: http://archives.postgresql.org/pgsql-hackers/2010-05/msg01095.php The loadable module doesn't gain control here it simplify kicks-in after, and in addition to, normal checking. That just means you have the option of failing for additional reasons. We're not passing in any form of context other than the rangetable so what additional reasons could there be? This is of no use to anything that uses object labelling. We're not even at the part of the executor where we would be able to identify objects yet, so I can't see what value this brings. Though I am certainly in favour in general terms of simple changes to enhance security configuration features. Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [v9.1] Add security hook on initialization of instance
* Stephen Frost (sfr...@snowman.net) wrote: Guess my first thought was that you'd have a database-level label that would be used by SELinux to validate a connection. A second thought is labels for roles. KaiGai, can you provide your thoughts on this discussion/approach/problems? I realize it's come a bit far-afield from your original proposal. Something else which has come up but is related is the ability to support a pam_tally-like function in PG. Basically, the ability to lock users out if they've had too many failed login attempts. I wonder if we could add this hook (or maybe have more than one if necessary) in a way to support a contrib module for that. Thanks, Stephen signature.asc Description: Digital signature
[HACKERS] FYI: Ubuntu 10.04 lucid strange segfault
Hello list, Due to dependency requirements my development machine has ubuntu repositories jaunty, lucid and intrepid. Today I went testing a patch on a recent PostgreSQL. I had to install autoconf to configure the patched source. While seeing some other packages getting installed on the screen, I remember thinking 'should've installed with aptitude to see change set instead of apt-get install'. I did not get past make install due to a segfault on ./zic in the src/timezone directory. The strange thing is that another source tree I was working on was intact and could configure and make install/run zic without problems. Circumventing the zic segfault showed that after installation the postgres binary had no problems, but psql had similar problems as zic. (0x00659bd0 in strncpy@@GLIBC_2.2.5 () where zic went wrong in strchr). long story of dead ends in problem tree traversal removed At some point read reading http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43556 Changing versions to gcc 4.4.3-4ubuntu5 and binutils 2.20.1-3ubuntu5 made the strange segfaults go away. I don't know which versions I had installed when having the segfault, sorry. regards, Yeb Havinga -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] reducing NUMERIC size for 9.1
EnterpriseDB asked me to develop the attached patch to reduce the on-disk size of numeric and to submit it for inclusion in PG 9.1. After searching the archives, I found a possible design for this by Tom Lane based on an earlier proposal by Simon Riggs. http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php The attached patch implements more or less the design described there, and will essentially knock 2 bytes of the on-disk size of nearly all numeric values anyone is likely to want to store, but without reducing the overall range of the type; so, for people who are storing a lot of numerics, it should save a great deal of storage space and, more importantly, I/O. However, it does so in a way that should be completely backward-compatible from a binary format standpoint, so that pg_upgrade does not break. I'm not entirely happy with the way I handled the variable-length struct, although I don't think it's horrible, either. I'm willing to rework it if someone has a better idea. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company numeric_2b.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, Jul 9, 2010 at 10:51 AM, Simon Riggs si...@2ndquadrant.com wrote: The loadable module doesn't gain control here it simplify kicks-in after, and in addition to, normal checking. That just means you have the option of failing for additional reasons. True. We could change it so that the normal checking is bypassed if the hook is installed, and leave it up to the hook whether to call the standard checks as well, but I don't think there's much of a use case for that. We're not passing in any form of context other than the rangetable so what additional reasons could there be? This is of no use to anything that uses object labelling. We're not even at the part of the executor where we would be able to identify objects yet, so I can't see what value this brings. Though I am certainly in favour in general terms of simple changes to enhance security configuration features. Well, KaiGai Kohei already posted a proof-of-concept patch showing how this could be used by a simple SE-PostgreSQL implementation. Since we don't have a security labelling facility yet, he used the comment on the relation to store the security label (there are other ways it could be done too, of course). Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. Huh. Obviously, I would have refrained from committing the patch had I known that it was going to conflict with work someone else was doing in this area, at least until we reached consensus on which way to go with it, but since you didn't post about it on -hackers, I had no idea that was the case. Sounds like you should probably post your proposal and we can discuss what to do in general and also with respect to this hook. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [v9.1] Add security hook on initialization of instance
On Fri, Jul 9, 2010 at 10:52 AM, Stephen Frost sfr...@snowman.net wrote: * Stephen Frost (sfr...@snowman.net) wrote: Guess my first thought was that you'd have a database-level label that would be used by SELinux to validate a connection. A second thought is labels for roles. KaiGai, can you provide your thoughts on this discussion/approach/problems? I realize it's come a bit far-afield from your original proposal. Something else which has come up but is related is the ability to support a pam_tally-like function in PG. Basically, the ability to lock users out if they've had too many failed login attempts. I wonder if we could add this hook (or maybe have more than one if necessary) in a way to support a contrib module for that. Seems like the hard part would be figuring out where to store the bookkeeping information. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
* Simon Riggs (si...@2ndquadrant.com) wrote: The loadable module doesn't gain control here it simplify kicks-in after, and in addition to, normal checking. That just means you have the option of failing for additional reasons. Right, additional checks (such as the label) can be done. We're not passing in any form of context other than the rangetable so what additional reasons could there be? This is of no use to anything that uses object labelling. We're not even at the part of the executor where we would be able to identify objects yet, so I can't see what value this brings. I'm a bit confused by this. By this point, we've fully planned out the query, looked up info about all the objects involved, and the module can now go look up any other information about them that it needs. It can also use info like what the current user is and information about the connection. There was actually a proof-of-concept module created by KaiGai to do all of this with SELinux using the existing COMMENT tables, I'm pretty sure we would have heard a bit earlier if it was useless. Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. I don't see how you could remove ExecCheckRTPerms..? It's what handles all permissions checking for DML (like, making sure you have SELECT rights on the table you're trying to query). I could see forcing plan invalidation when permissions are updated, sure, but that doesn't mean you can stop doing them altogether anywhere. Where would you move the permissions checking to? Wherever it is, this hook would just need to follow. I don't know that you could (or that I'd be comfortable with) move the permissions checking to the planner and then rely on plan invalidation on permission changes, but if you did, just make sure the hook is included in the decision about allowing the query. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
Simon Riggs si...@2ndquadrant.com writes: Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. That seems both pointless and wrong. Permissions checks should happen at execution time not plan time. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [v9.1] Add security hook on initialization of instance
* Robert Haas (robertmh...@gmail.com) wrote: On Fri, Jul 9, 2010 at 10:52 AM, Stephen Frost sfr...@snowman.net wrote: Something else which has come up but is related is the ability to support a pam_tally-like function in PG. Basically, the ability to lock users out if they've had too many failed login attempts. I wonder if we could add this hook (or maybe have more than one if necessary) in a way to support a contrib module for that. Seems like the hard part would be figuring out where to store the bookkeeping information. pam_tally does it in a flat file on-disk. It's not perfect, but it works. It'd be good enough for me if there was a hook in the right place that I could write the contrib module. Of course, it'd be even nicer to have support for this in-core, since it's a FISMA requirement and not everyone is going to like having to install a contrib module to handle something like this. Of course, so is stuff like remembering old passwords, password aging, etc... I do really wish there was a way PAM could be used by applications *independently* of the system auth (eg: /etc/passwd) to handle all of this. If anyone's aware of a way, I'm all ears.. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] [v9.1] Add security hook on initialization of instance
On Fri, Jul 9, 2010 at 11:19 AM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Fri, Jul 9, 2010 at 10:52 AM, Stephen Frost sfr...@snowman.net wrote: Something else which has come up but is related is the ability to support a pam_tally-like function in PG. Basically, the ability to lock users out if they've had too many failed login attempts. I wonder if we could add this hook (or maybe have more than one if necessary) in a way to support a contrib module for that. Seems like the hard part would be figuring out where to store the bookkeeping information. pam_tally does it in a flat file on-disk. It's not perfect, but it works. It'd be good enough for me if there was a hook in the right place that I could write the contrib module. Of course, it'd be even nicer to have support for this in-core, since it's a FISMA requirement and not everyone is going to like having to install a contrib module to handle something like this. Feel free to propose a patch...! -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: Purely out of interest, since the old repo is still there, I had a quick look at measuring the overhead, using 8.4's pgbench to run two custom scripts: one consisting of a single 'SELECT 1', the other having 100 'SELECT 1' - the latter being probably the worst case scenario. Running 1,2,4,8 clients and 1000-1 transactions gives an overhead in the 5-8% range [1] (i.e transactions/s decrease by this amount with the scheduler turned on [2]). While a lot better than 30% (!) it is certainly higher than we'd like. Hmmm... In my first benchmarks of the serializable patch I was likewise stressing a RAM-only run to see how much overhead was added to a very small database transaction, and wound up with about 8%. By profiling where the time was going with and without the patch, I narrowed it down to lock contention. Reworking my LW locking strategy brought it down to 1.8%. I'd bet there's room for similar improvement in the active transaction limit you describe. In fact, if you could bring the code inside blocks of code already covered by locks, I would think you could get it down to where it would be hard to find in the noise. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Assertion failure in get_attstatsslot()
Bernd Helmle maili...@oopsware.de writes: -- assertion failure EXPLAIN SELECT 1 FROM foo WHERE UPPER(value) = 'xyz'; I tried it back from current -HEAD to 8.3.11 and managed to reproduce it everywhere. Non-cassert builds are working correctly, so i'm not sure wether this is an over-aggressive assert() or masks a problem from somewhere else. I can reproduce it back to 8.0, and I'm not sure it couldn't be done in older branches with a slightly different test case :-(. I think that at bottom the issue is that the planner plays fast and loose with binary-compatible data types by being willing to strip off RelabelType nodes whenever it's trying to arrive at selectivity estimates. This isn't something we can easily change, though. I'm quite hesitant to remove that Assert from deconstruct_array(), but it strikes me that an easy workaround is to make get_attstatsslot() use ARR_ELEMTYPE(statarray) rather than the passed-in atttype. This at least moves the fast-and-looseness into a function that's basically a creature of the planner, rather than fooling with a fundamental utility like deconstruct_array(). If anybody feels really uncomfortable with that, we could add a compensating Assert(IsBinaryCoercible(ARR_ELEMTYPE(statarray), atttype)) into get_attstatsslot(). Not sure if it's worth the cycles. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 11:07 -0400, Robert Haas wrote: Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. Huh. Obviously, I would have refrained from committing the patch had I known that it was going to conflict with work someone else was doing in this area, at least until we reached consensus on which way to go with it, but since you didn't post about it on -hackers, I had no idea that was the case. Sounds like you should probably post your proposal and we can discuss what to do in general and also with respect to this hook. Sorry, yes, you couldn't possibly know I was looking at that. I just meant it was strange those things overlapped. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 11:09 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. That seems both pointless and wrong. Permissions checks should happen at execution time not plan time. Agreed that permission checks should logically be applied at execution time. I am proposing a performance optimisation, not a change in behaviour. Permissions are set much less frequently than plans and connections, so when the only permission check is at table level it makes more sense to optimistically assume that permission checks will never change for a plan and cache the result of the permission check. That way we need only check permissions once at plan time rather than checking them every single execution. The only extra code to do this would be to invalidate plans when permissions change for a table. That doesn't seem hard or invasive. The proposed performance enhancement would be very useful since we have to check permissions of functions, views, tables and every other aspect. We could spend a while quantifying that overhead, though non-zero is all we need to know. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock
On Thu, Jul 8, 2010 at 5:09 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-07-08 at 06:08 -0400, Robert Haas wrote: On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: Rereading the thread, I'm a bit confused by why we're proposing to use a SHARE lock; it seems to me that a self-conflicting lock type would simplify things. There's a bunch of discussion on the thread about how to handle pg_class updates atomically, but doesn't using a self-conflicting lock type eliminate that problem? The use of the SHARE lock had nothing to do with the pg_class update requirement, so that suggestion won't help there. Forgive me if I press on this just a bit further, but ISTM that an atomic pg_class update functionality isn't intrinsically required, because if it were the current code would need it. So what is changing in this patch that makes it necessary when it isn't now? ISTM it must be that the lock is weaker. What am I missing? Not sure I follow that logic. Discussion on the requirement is in the archives. I don't wish to question that aspect myself. The relevant link from the archives is here: http://archives.postgresql.org/pgsql-hackers/2008-10/msg00242.php Tom asked what happens when two transactions attempt to do concurrent actions on the same table. Your response was that we should handle it like CREATE INDEX, and handle the update of the pg_class row non-transactionally. But of course, if you use a self-conflicting lock at the relation level, then the relation locks conflict and you never have to worry about how to update the pg_class entry in the face of concurrent updates. Which, come to think of it, is probably a good thing, because on further reflection I'm pretty sure the proposed approach will fall down completely for some of these operations. heap_inplace_update() can only be used when (a) the new tuple is identical in size to the old tuple, and (b) no action is required on rollback. That's fine for updating things like relpages and reltuples (which are just hints anyway) but it ain't gonna work for changing, say, reloptions, which is variable-length. It's also not going to work for changing things like attstorage, even though a change there can't affect the tuple size, because modifying the tuple in place won't handle rollbacks correctly. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 11:07 -0400, Robert Haas wrote: On Fri, Jul 9, 2010 at 10:51 AM, Simon Riggs si...@2ndquadrant.com wrote: The loadable module doesn't gain control here it simplify kicks-in after, and in addition to, normal checking. That just means you have the option of failing for additional reasons. True. We could change it so that the normal checking is bypassed if the hook is installed, and leave it up to the hook whether to call the standard checks as well, but I don't think there's much of a use case for that. With respect, there doesn't seem to be much use case anyway. I'm sorry to be expressing that opinion now; been away for a while. I am somewhat amazed that Tom isn't dancing on your head for proposing it though. We're not passing in any form of context other than the rangetable so what additional reasons could there be? This is of no use to anything that uses object labelling. We're not even at the part of the executor where we would be able to identify objects yet, so I can't see what value this brings. Though I am certainly in favour in general terms of simple changes to enhance security configuration features. Well, KaiGai Kohei already posted a proof-of-concept patch showing how this could be used by a simple SE-PostgreSQL implementation. Since we don't have a security labelling facility yet, he used the comment on the relation to store the security label (there are other ways it could be done too, of course). What's the difference between that and a GRANT command? GRANT is designed to allow privileges to be defined at table level. I don't see how a plugin whose only API input is a rangetable and which executes before any tuples have been touched can possibly add value here. KaiGai's had an uphill task here and I don't wish to be part of slowing him down. I'm not seeing how this moves label security forwards in any measurable way. Tom's test of a useful plugin has been one where a useful contrib module gets added at the same time. I don't think a useful plugin has been demonstrated or produced, as yet. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 11:09 -0400, Stephen Frost wrote: Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. I don't see how you could remove ExecCheckRTPerms..? It's what handles all permissions checking for DML (like, making sure you have SELECT rights on the table you're trying to query). I could see forcing plan invalidation when permissions are updated, sure, but that doesn't mean you can stop doing them altogether anywhere. Where would you move the permissions checking to? I apologise, when I said removing the check altogether, I meant removing from the executor path. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
Simon, * Simon Riggs (si...@2ndquadrant.com) wrote: With respect, there doesn't seem to be much use case anyway. I'm sorry to be expressing that opinion now; been away for a while. I am somewhat amazed that Tom isn't dancing on your head for proposing it though. I believe it's because we've been through it with him already and explained how and why it's useful. Well, KaiGai Kohei already posted a proof-of-concept patch showing how this could be used by a simple SE-PostgreSQL implementation. Since we don't have a security labelling facility yet, he used the comment on the relation to store the security label (there are other ways it could be done too, of course). What's the difference between that and a GRANT command? GRANT is designed to allow privileges to be defined at table level. I don't see how a plugin whose only API input is a rangetable and which executes before any tuples have been touched can possibly add value here. The *hook*'s API is just a range-table- the plugin has access to a huge amount of additional information. I don't think it makes sense to try to limit that in some fashion by dictating what other information the hook is allowed to gather. Even if we did, it wouldn't be everything the hook needs since we're not going to collect the SE-Linux label from the kernel in the main backend code. The difference between this and a GRANT command would be the whole DAC vs MAC discussion and overall label-based security (this is *not* the same as *row-level* security). KaiGai's had an uphill task here and I don't wish to be part of slowing him down. I'm not seeing how this moves label security forwards in any measurable way. I'm afraid we're talking about different things here. Tom's test of a useful plugin has been one where a useful contrib module gets added at the same time. I don't think a useful plugin has been demonstrated or produced, as yet. We have a plugin which *could* be used to allow SE-Linux in the backend today- but it uses the COMMENT system, which isn't exactly ideal. Robert's already working on a patch which will add the ability to track actual labels in the catalog (using some new catalog tables which are similar to the comment tables, which is why it's not done yet, it's waiting on the get_xxx_oid() infrastructure which will greatly simplify both), which could then be queried by the module. There will be a hook there as well, which will get called whenever someone tries to modify or add a label to an object. We've also discussed new syntax for supporting those catalogs (ALTER SECURITY LABEL ON TABLE x TO y, or something like that, it's in the archives). So, no, it's not done today, but I'm certainly hopeful this will all get into 9.1 and will allow label-based security for DML with SELinux (and maybe Smack too) and this is a necessary step along the way. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-07-09 at 11:09 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. That seems both pointless and wrong. Permissions checks should happen at execution time not plan time. Agreed that permission checks should logically be applied at execution time. I am proposing a performance optimisation, not a change in behaviour. Except that it *is* a change in behavior: the first check will occur too soon. The fact that we're interested in adding plugin permissions checking pretty much destroys the idea anyway. You cannot assume that a plan cache invalidation will happen for any change in external state that a plugin might be consulting. The proposed performance enhancement would be very useful since we have to check permissions of functions, views, tables and every other aspect. We could spend a while quantifying that overhead, though non-zero is all we need to know. No, it's not all we need to know. If you can't prove the overhead involved here is significant, we should not be expending effort and creating subtle behavioral changes in pursuit of a minor optimization. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock
On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: Tom asked what happens when two transactions attempt to do concurrent actions on the same table. Your response was that we should handle it like CREATE INDEX, and handle the update of the pg_class row non-transactionally. But of course, if you use a self-conflicting lock at the relation level, then the relation locks conflict and you never have to worry about how to update the pg_class entry in the face of concurrent updates. From memory, Tom was also worried about the prospect of people updating pg_class directly using SQL. That seems a rare, yet valid concern. I've already agreed with your point that we should use SHARE UPDATE EXCLUSIVE. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
* Tom Lane (t...@sss.pgh.pa.us) wrote: Simon Riggs si...@2ndquadrant.com writes: Agreed that permission checks should logically be applied at execution time. I am proposing a performance optimisation, not a change in behaviour. Except that it *is* a change in behavior: the first check will occur too soon. Yeah, I have to say that I don't see any way you could avoid the behaviour change from doing this. Specifically, you can prepare plans today that you don't have access to run and then run them later after you've been granted the permission. I'm not saying that's a huge use-case or anything, but moving the checks to planner time would definitely change the behavior. No clue if any of this is covered in the SQL spec. The fact that we're interested in adding plugin permissions checking pretty much destroys the idea anyway. You cannot assume that a plan cache invalidation will happen for any change in external state that a plugin might be consulting. Yeah, this would certainly be a problem too, unless we kept the plugin hook in the executor and only used the optimization for the stock PG checks. The proposed performance enhancement would be very useful since we have to check permissions of functions, views, tables and every other aspect. We could spend a while quantifying that overhead, though non-zero is all we need to know. No, it's not all we need to know. If you can't prove the overhead involved here is significant, we should not be expending effort and creating subtle behavioral changes in pursuit of a minor optimization. Agreed. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, Jul 9, 2010 at 1:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-07-09 at 11:09 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: Strangely, I was looking into removing the ExecCheckRTPerms check altogether by forcing plan invalidation when permissions are updated. That would be a performance tweak that would render this change useless. That seems both pointless and wrong. Permissions checks should happen at execution time not plan time. Agreed that permission checks should logically be applied at execution time. I am proposing a performance optimisation, not a change in behaviour. Except that it *is* a change in behavior: the first check will occur too soon. You might be able to get around this by doing the first check on first use of the plan and then going and marking all the plans as needing a recheck whenever a permissions change happens. Whether the performance savings are sufficient to justify such a thing is another matter. The fact that we're interested in adding plugin permissions checking pretty much destroys the idea anyway. You cannot assume that a plan cache invalidation will happen for any change in external state that a plugin might be consulting. This is certainly true, but I also wonder what SE-PostgreSQL plans to do about this. Taking this to its logical exteme, the system security policy could change in mid-query - and while you'd like to think that the system would stop emitting tuples on a dime, that's probably not too feasible in practice. I am assuming that SE-PostgreSQL will want to do some kind of caching, but I wonder how one decides what to cache and for how long, and whether there's any mechanism for propagating cache invalidations. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 13:21 -0400, Tom Lane wrote: Agreed that permission checks should logically be applied at execution time. I am proposing a performance optimisation, not a change in behaviour. Except that it *is* a change in behavior: the first check will occur too soon. Sooner matters why? We already have a lock on the table at plan time so there cannot be a concurrent GRANT against a plan-then-execute transaction. Later transactions would invalidate and replan. The fact that we're interested in adding plugin permissions checking pretty much destroys the idea anyway. You cannot assume that a plan cache invalidation will happen for any change in external state that a plugin might be consulting. Plugin can still be executed at appropriate time, its mostly absent and so cheap. I guess we can keep plugin whatever else I attempt. The proposed performance enhancement would be very useful since we have to check permissions of functions, views, tables and every other aspect. We could spend a while quantifying that overhead, though non-zero is all we need to know. No, it's not all we need to know. If you can't prove the overhead involved here is significant, we should not be expending effort and creating subtle behavioral changes in pursuit of a minor optimization. OK, will gather evidence. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
Robert, * Robert Haas (robertmh...@gmail.com) wrote: This is certainly true, but I also wonder what SE-PostgreSQL plans to do about this. Taking this to its logical exteme, the system security policy could change in mid-query - and while you'd like to think that the system would stop emitting tuples on a dime, that's probably not too feasible in practice. I am assuming that SE-PostgreSQL will want to do some kind of caching, but I wonder how one decides what to cache and for how long, and whether there's any mechanism for propagating cache invalidations. Yes, SE-PG will be doing cacheing and this exact problem has already been addressed (KaiGai's original SE-PG patches included cacheing, actually). It's also not something that's unique to PG in any way. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-07-09 at 13:21 -0400, Tom Lane wrote: Except that it *is* a change in behavior: the first check will occur too soon. Sooner matters why? Consider PREPARE followed only later by EXECUTE. Your proposal would make the PREPARE fail outright, when it currently does not. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FYI: Ubuntu 10.04 lucid strange segfault
On Fri, 2010-07-09 at 16:57 +0200, Yeb Havinga wrote: Hello list, Due to dependency requirements my development machine has ubuntu repositories jaunty, lucid and intrepid. I would not expect anything to work in an environment that is that misconfigured. I would suggest running VirtualBox for your different requirements. Sincerely, Joshua D. Drake -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
On Fri, 2010-07-09 at 14:01 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-07-09 at 13:21 -0400, Tom Lane wrote: Except that it *is* a change in behavior: the first check will occur too soon. Sooner matters why? Consider PREPARE followed only later by EXECUTE. Your proposal would make the PREPARE fail outright, when it currently does not. Just to avoid wasted investigation: are you saying that is important behaviour that is essential we retain in PostgreSQL, or will you hear evidence that supporting that leads to a performance decrease elsewhere? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock
On Fri, Jul 9, 2010 at 1:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: Tom asked what happens when two transactions attempt to do concurrent actions on the same table. Your response was that we should handle it like CREATE INDEX, and handle the update of the pg_class row non-transactionally. But of course, if you use a self-conflicting lock at the relation level, then the relation locks conflict and you never have to worry about how to update the pg_class entry in the face of concurrent updates. From memory, Tom was also worried about the prospect of people updating pg_class directly using SQL. That seems a rare, yet valid concern. Yes, and it's another another reason why we shouldn't use non-transactional updates. http://archives.postgresql.org/pgsql-hackers/2008-11/msg00744.php I've already agreed with your point that we should use SHARE UPDATE EXCLUSIVE. The point you seem to be missing is that once we make that decision, we can throw all the heap_inplace_update() stuff out the window, and the whole problem becomes much simpler. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] - GSoC - snapshot materialized view (work-in-progress) patch
2010/7/8 Pavel Baroš baro...@seznam.cz: Description of patch: 1) can create MV, and is created uninitialized with data CREATE MATERIALIZED VIEW mvname AS SELECT ... This doesn't seem acceptable. It should populate it on creation. 2) can refresh MV ALTER MATERIALIZED VIEW mvname REFRESH 3) MV cannot be modified by DML commands (INSERT, UPDATE and DELETE are not permitted) 4) index can be created and used with MV 5) pg_dump is repaired, in previous patch dump threw error, now dont, but it is sort of dummy, I want to reach state, where refreshing command will be posed after all COPY statements (when all data are in tables). In this patch REFRESH command is right behind CREATE MV command. Hmm... ISTM that you probably need some kind of dependency stuff in here to make the materialized view get created after the tables it depends on have been populated with data. It needs to work with parallel restore, too. I'm not sure exactly how the dependency stuff in pg_dump works, though. A subtle point here is that if you dump and restore a database containing a materialized view, the new database might not be quite the same as the old one, because the materialized view might have been out of date before, and when you recreate it, it'll get refreshed. I'm not sure there's much we can/should do about that, though. 6) psql works too, new command \dm[S+] was added to the list \d[S+] [PATTERN] - lists all db objects like tables, view, materialized view and sequences \dm[S+] [PATTERN] - lists all materialized views 7) there are some docs too, but I guess it is not enough, at least my english will need to correct If we're going to treat materialized views as a separate object type, you probably need to break out the docs for CREATE MATERIALIZED VIEW, ALTER MATERIALIZED VIEW, and DROP MATERIALIZED VIEW into their own pages, rather than having then mixed up with corresponding pages for regular views. 8) some ALTER TABLE commands works, ie. RENAME TO, OWNER TO, SET SCHEMA, SET TABLESPACE 9) MV and columns can be commented 10) also some functions behave as expected, but if you know about some I did not mention and could fail when used with MV, I appreciate your hints pg_get_viewdef() pg_get_ruledef() pg_relation_filenode() pg_relation_filepath() pg_table_size() In progress: - regression tests - behavior of various ALTER commands, ie SET STATISTIC, CLUSTER ON, ENABLE/DISABLE RULE, etc. This isn't right: rhaas=# create view v as select * from t; CREATE VIEW rhaas=# alter view v refresh; ERROR: unrecognized alter table type: 41 Please add your patch here, so that it will be reviewed during the about-to-begin CommitFest. https://commitfest.postgresql.org/action/commitfest_view/open -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
On ons, 2010-07-07 at 16:37 +0100, Mike Fowler wrote: Here's the patch to add the 'xml_is_well_formed' function. I suppose we should remove the function from contrib/xml2 at the same time. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FYI: Ubuntu 10.04 lucid strange segfault
Joshua D. Drake wrote: On Fri, 2010-07-09 at 16:57 +0200, Yeb Havinga wrote: Hello list, Due to dependency requirements my development machine has ubuntu repositories jaunty, lucid and intrepid. I would not expect anything to work in an environment that is that misconfigured. I would suggest running VirtualBox for your different requirements. Thanks for your advice but my message was a FYI. The link I provided suggests it can happen at systems having only lucid as well. regards, Yeb Havinga -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FYI: Ubuntu 10.04 lucid strange segfault
On Fri, Jul 9, 2010 at 4:10 PM, Yeb Havinga yebhavi...@gmail.com wrote: Joshua D. Drake wrote: On Fri, 2010-07-09 at 16:57 +0200, Yeb Havinga wrote: Hello list, Due to dependency requirements my development machine has ubuntu repositories jaunty, lucid and intrepid. I would not expect anything to work in an environment that is that misconfigured. I would suggest running VirtualBox for your different requirements. Thanks for your advice but my message was a FYI. The link I provided suggests it can happen at systems having only lucid as well. It sounds like it's an Ubuntu linker bug, though - not really anything to do with us. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
On Fri, Jul 9, 2010 at 4:06 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-07-07 at 16:37 +0100, Mike Fowler wrote: Here's the patch to add the 'xml_is_well_formed' function. I suppose we should remove the function from contrib/xml2 at the same time. Yep. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks of DML permission checks
2010/6/14 KaiGai Kohei kai...@ak.jp.nec.com: The attached patch tries to rework DML permission checks. It was mainly checked at the ExecCheckRTEPerms(), but same logic was implemented in COPY TO/FROM statement and RI_Initial_Check(). This patch tries to consolidate these permission checks into a common function to make access control decision on DML permissions. It enables to eliminate the code duplication, and improve consistency of access controls. This patch is listed on the CommitFest page, but I'm not sure if it represents the latest work on this topic. At a minimum, it needs to be rebased. I am not excited about moving ExecCheckRT[E]Perms to some other place in the code. It seems to me that will complicate back-patching with no corresponding advantage. I'd suggest we not do that.The COPY and RI code can call ExecCheckRTPerms() where it is. Maybe at some point we will have a grand master plan for how this should all be laid out, but right now I'd prefer localized changes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Add a hook in ExecCheckRTPerms().
Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-07-09 at 14:01 -0400, Tom Lane wrote: Consider PREPARE followed only later by EXECUTE. Your proposal would make the PREPARE fail outright, when it currently does not. Just to avoid wasted investigation: are you saying that is important behaviour that is essential we retain in PostgreSQL, or will you hear evidence that supporting that leads to a performance decrease elsewhere? Well, I think that that problem makes moving the checks into the planner a nonstarter. But as somebody pointed out upthread, you could still get what you want by keeping a flag saying permission checks have been done so the executor could skip the checks on executions after the first. I'd still want to see some evidence showing that it's worth troubling over though. Premature optimization being the root of all evil, and all that. (In this case, the hazard we expose ourselves to seems to be security holes due to missed resets of the flag.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Assertion failure in get_attstatsslot()
Excerpts from Tom Lane's message of vie jul 09 12:16:42 -0400 2010: If anybody feels really uncomfortable with that, we could add a compensating Assert(IsBinaryCoercible(ARR_ELEMTYPE(statarray), atttype)) into get_attstatsslot(). Not sure if it's worth the cycles. Cycles spent only in assert-enabled builds? Why not? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Assertion failure in get_attstatsslot()
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie jul 09 12:16:42 -0400 2010: If anybody feels really uncomfortable with that, we could add a compensating Assert(IsBinaryCoercible(ARR_ELEMTYPE(statarray), atttype)) into get_attstatsslot(). Not sure if it's worth the cycles. Cycles spent only in assert-enabled builds? Why not? I decided not to do it, not so much because of any performance worries as that I didn't want to make lsyscache.c depend on the parser. Maybe sometime we should rethink where the parse_coerce.c functionality lives. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Assertion failure in get_attstatsslot()
1;2401;0cOn Fri, Jul 09, 2010 at 06:49:27PM -0400, Alvaro Herrera wrote: Excerpts from Tom Lane's message of vie jul 09 12:16:42 -0400 2010: If anybody feels really uncomfortable with that, we could add a compensating Assert(IsBinaryCoercible(ARR_ELEMTYPE(statarray), atttype)) into get_attstatsslot(). Not sure if it's worth the cycles. Cycles spent only in assert-enabled builds? Why not? The slower assert-enabled is, the less likely it is that somebody can run serious testing on it - potentially catching bugs way much easier. Contrarily to your statement I would actually like to remove some older asserts. For example AtEOXact_Buffers makes it significantly expensive to do assert tests on larger shbuf setups. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gSoC - ADD MERGE COMMAND - code patch submission
On Fri, Jul 9, 2010 at 10:25 PM, Boxuan Zhai bxzhai2...@gmail.com wrote: Dear All, This is ZHAI BOXUAN, a student of gSoC 2010. My project is to add merge command in postgres. This is the first submission of our codes, which has finished the parser, analyzer and rewriter parts. If you are interested in this project, please download the source code in attachment and have test. There are 3 files in the attachement. Two .patch file is created on the /src/backend and /src/include folders (between orignial psql 8.4.3 and our modified edition). There is a more detailed instruction in readme. Any comments will be highly appreciated. Is there any chance you can submit this as a single patch file? Or if not, can you at least use a zip or tar file instead of a RAR archive? Ideally the patch would be against CVS HEAD, not 8.4.3. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
