Re: [HACKERS] Going, going, GUCs!
On Tue, 20 Oct 2009, David Fetter wrote: commit_delay (no need for this knob) commit_siblings (no need for this knob) Both these knobs do something that's valuable sometimes: help group commits into bigger chunks when there are lots of active clients. I've watched it help a bit on heavy writes at a client site before, and some of the Sun benchmarks that have been published got a little boost out of it too in a similar situation. The use case for both is pretty narrow, and isn't actually the same one the feature was intended to fix (originally this was supposed to be useful more for the person who can't commit very fast at all; it isn't). But there is some value, and the code involved is pretty small, not much beyond the GUC overhead. I'd like to get both pruned out in favor of a better chunking write implementation that's aimed squarely at the problem these two help on accidentally. Until that point, it's hard to justify wiping them out when they do have some value and little code overhead to keep around. As a comment on one of the larger themes brought up in this thread, I don't think picking off a few parameters is of any value to making the postgresql.conf file easier to work with. That needs a much larger pruning before moving in that direction is actually going to help anyone. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
"Marc G. Fournier" wrote: > > In addition, it might be good idea to hide some parameters from the > > default postgresql.conf in order to simplify it, even though we will > > still have those knobs. > > That might be an idea for anything that is meant to be 'deprecated' in the > first place, maybe? Document it in the docs, but dont including it in the > default postgresql.conf? It was just an idea for step-by-step deprecation. I didn't intended to do so for all parameters, but some of them, like default_with_oids, would fit in the way because users should not use it but pg_dump can still use it. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
David Christensen wrote: > Is that only when the default client encoding is set to UTF8 > (PGCLIENTENCODING, whatever), or will it be coded to work with the > following: > > $ psql -f > Where is: > > SET CLIENT ENCODING 'utf8'; Sure. Client encoding is declared in body of a file, but BOM is in head of the file. So, we should always ignore BOM sequence at the file head no matter what client encoding is used. The attached patch replace BOM with while spaces, but it does not change client encoding automatically. I think we can always ignore client encoding at the replacement because SQL command cannot start with BOM sequence. If we don't ignore the sequence, execution of the script must fail with syntax error. This patch does nothing about COPY and \copy commands. It might be possible to add BOM handling code around AllocateFile() in CopyFrom() to support "COPY FROM 'utf8file-with-bom.tsv'", but we need another approach for "COPY FROM STDIN". For example, $ echo utf8bom-1.tsv utf8bom-2.tsv | psql -c "COPY FROM STDIN" might contain BOM sequence in the middle of input stream. Anyway, those changes would come from another patch in the future. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center psql-utf8bom_20091021.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
On Wed, 21 Oct 2009, Itagaki Takahiro wrote: In addition, it might be good idea to hide some parameters from the default postgresql.conf in order to simplify it, even though we will still have those knobs. That might be an idea for anything that is meant to be 'deprecated' in the first place, maybe? Document it in the docs, but dont including it in the default postgresql.conf? Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
I'll hack the makefile and give it a shot. No need to hack it, set CFLAGS during configure: shell]# CFLAGS="-m64" ./configure (options) -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
Zdenek Kotala wrote: Hmm, I'm afraid that 64bit plperl is a problem. Solaris 10 is not shipped with 64bit perl :(. Zdenek Found that the hard way today :-) I'm downloading perl source and will make a 64 bit version. Anybody know if -m64 will work with compiling 64 bit perl? :-) I'll hack the makefile and give it a shot. Thanks! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ProcessUtility_hook
Here is a patch to add ProcessUtility_hook to handle all DDL commands in user modules. (ProcessUtility_hook_20091021.patch) It adds a global variable 'ProcessUtility_hook' and export the original function as 'standard_ProcessUtility'. Euler Taveira de Oliveira wrote: > > pg_stat_statements module also handle DDL commands as same as normal > > queries. Fortunately, autovacuum calls call vacuum() directly, > > so the statistics will not filled with VACUUM and ANALYZE commands > > by autovacuum. > > > I don't look at the code but maybe there is a way to hook something there too. > But I'd leave it alone for now, unless it's few lines of code. I see it is debatable whether pg_stat_statements should support the hook. So I split the change in another patch. (pgss_utility_hook_20091021.patch) Regards, --- ITAGAKI Takahiro NTT Open Source Software Center ProcessUtility_hook_20091021.patch Description: Binary data pgss_utility_hook_20091021.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
Robert Haas wrote: > We just had a conversation about whether or not to massively break > backward compatibility. The consensus was, as I'm sure you are aware, > was "no". We should not discuss serveral kinds of parameters at once. 1. Options for backward compatibility with old PostgreSQL 2. Options for compatibility with other DBMSs and applications 3. Useless parameters 4. Parameters for DBAs who are paranoid about performance 5. Still used I agree we should not drop 1, 2 and 5, but we can drop 3 and some portion of 4. In addition, it might be good idea to hide some parameters from the default postgresql.conf in order to simplify it, even though we will still have those knobs. 1. Options for backward compatibility with old PostgreSQL: > > add_missing_from (should be off) > > array_nulls (should be on) > > default_with_oids (should be off) > > regex_flavor (should be advanced, regex flavor can be controlled on a > > per-regex basis when they're advanced) > > sql_inheritance (should be on) > > password_encryption (should be on) 2. Options for compatibility with other DBMSs and applications: > > standard_conforming_strings (should be on) > > transform_null_equals (should probably be off) 3. Useless parameters: > > commit_delay (no need for this knob) > > commit_siblings (no need for this knob) 4. Parameters for DBAs who are paranoid about performance: > > track_activities (should be on) > > track_counts (should be on) > > update_process_title (should be on) 5. Still used: > > synchronize_seqscans (should be on) -- used by pg_dump Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
Rod Taylor writes: > I tried making a functional index based on an expression containing > the 2 argument regexp_matches() function. Is there a reason why this > function is not marked immutable instead of normal? So I went to see about making the changes to remove regex_flavor, and was astonished to find that all the regex-related functions are already marked immutable, and AFAICS always have been. This is clearly wrong, and we would have to fix it if we weren't about to remove the GUC. (In principle we should advise people to change the markings in existing databases, but given the lack of complaints it's probably not worth the trouble --- I doubt many applications change regex_flavor on the fly.) So, having dismissed my original off-the-cuff answer to Rod, the next question is what's really going wrong for him. I get this from a quick trial: regression=# create table tt1(f1 text, f2 text); CREATE TABLE regression=# create index tt1i on tt1(regexp_matches(f1,f2)); ERROR: index expression cannot return a set IOW the problem is that regexp_matches returns SETOF, not that it's marked stable (as it should have been). I'm not sure what semantics you were expecting the index to have, but we don't have any useful support for indexes on sets. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
On Tue, Oct 20, 2009 at 1:49 PM, David Fetter wrote: > Folks, > > I'd like to see about removing the following GUCs: > > add_missing_from (should be off) > array_nulls (should be on) > commit_delay (no need for this knob) > commit_siblings (no need for this knob) > default_with_oids (should be off) > password_encryption (should be on) > regex_flavor (should be advanced, regex flavor can be controlled on a > per-regex basis when they're advanced) > sql_inheritance (should be on) > standard_conforming_strings (should be on) > synchronize_seqscans (should be on) > track_activities (should be on) > track_counts (should be on) > transform_null_equals (should probably be off) > update_process_title (should be on) > > What say on each? When? We just had a conversation about whether or not to massively break backward compatibility. The consensus was, as I'm sure you are aware, was "no". So why bring it up again, and with absolutely zero justification for the proposed changes? Each and every knob on this list was added for some reason, and we should remove it when, and only when, that reason either no longer applies, or there is some countervailing reason. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha2 release notes
> Actually, change this to I *need* to edit it. There's several errors. > Working on it now, but can't promise to finish by tommorrow AM. Done, actually. Apologies for tab spacing; I can't seem to get this to work right in emacs-sgml. So I've attached the file instead of a patch. I edited several feature descriptions, re-assigned several to appropriate categories, rearranged new features so that they are listed in more-or-less order of importance, and elimitated two new sections which were redundant with existing sections. Please check for further errors. --Josh Berkus Title: Release 8.5alpha2 Overview PostgreSQL alpha releases are snapshots of development code. They are intended to preview and test upcoming features and to provide the possibility for early feedback. They should not be used in production installations or active development projects. While the PostgreSQL code is continuously subject to a number of automated and manual tests, alpha releases might have serious bugs. Also features may be changed incompatibly or removed at any time during the development cycle. The development cycle of a PostgreSQL major release alternates between periods of development and periods of integration work, called commit fests, normally one month each. Alpha releases are planned to be produced at the end of every commit fest, thus every two months. Since the first commit fest starts within a month from the beginning of development altogether, early alpha releases are not indicative of the likely feature set of the final release. The release notes below highlight user visible changes and new features. There are normally numerous bug fixes and performance improvements in every new snapshot of PostgreSQL, and it would be too bulky to attempt to list them all. Note that many bug fixes are also backported to stable releases of PostgreSQL, and you should be using those if you are looking for bug-fix-only upgrades for your current installations. The release notes are cumulative over all alpha releases. Items that are new in the latest alpha release are emphasized. Migration To upgrade from any release to an alpha release or from an alpha release to any other release will most likely require a dump/restore upgrade procedure. It may happen that this is not necessary in particular cases, but that is not verified beforehand. (The server will warn you in any case when a dump/restore is necessary if you attempt to use it with an old data directory.) Note, however, that the dump/restore upgrade procedure is expected to work for alpha releases, and problems in this area should be reported. Testing The primary reason we release alphas is to get users to test new features as early as possible. If you are interested in helping with organized testing, please see the testing information page. Changes SQL Features Add SQL-compliant triggers on columns, which fire only if certain columns are named in the UPDATE's SET list. Add CREATE LIKE INCLUDING COMMENTS and STORAGE, and INCLUDING ALL shortcut to allow users to make an exact copy of a table including all options and features. Define a new, more extensible syntax for COPY options in order to support additional COPY options in the future. Modify the definition of window-function PARTITION BY and ORDER BY clauses so that their elements are always taken as simple expressions over the query's input columns. This fixes a bug. Fix bug with nested WITH RECURSIVE statements. Add surrogate pair support for U& string and identifier syntax. Add Unicode escapes in E'...' strings. DROP COLUMN and DROP CONSTRAINT now support an IF EXISTS clause so that users can avoid fatal errors when running repeatable scripts. UNIQUE constraints can now be DEFERRABLE. This is primarily useful for incremental updates of numerical keys, e.g. "ID = ID + 1" Allow parentheses around the query _expression_ that follows a WITH clause. INFORMATION_SCHEMA, a catalog of standard views of database objects, has been updated to the SQL:2008 standard.
Re: [HACKERS] Going, going, GUCs!
On 10/20/09 5:06 PM, Tom Lane wrote: > Josh Berkus writes: >>> Meanwhile, last chance for anyone to object to killing these two? > >> Nope. People will have to fix their apps, but it's about time. I'll >> try to remember to advertise the breakage ... > > Hmm, did you want to try to include these changes in alpha2? I wasn't > intending to be that quick off the mark... No, no, alpha3. Hence "try to remember" ... --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
Josh Berkus writes: >> Meanwhile, last chance for anyone to object to killing these two? > Nope. People will have to fix their apps, but it's about time. I'll > try to remember to advertise the breakage ... Hmm, did you want to try to include these changes in alpha2? I wasn't intending to be that quick off the mark... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha1 bundled -- please verify
Peter Eisentraut wrote: On Wed, 2009-08-19 at 19:11 +0200, Stefan Kaltenbrunner wrote: correct - svr1 has bison 1.875 and flex 2.5.35 (for the -HEAD builds) and flex 2.5.4 for the back branches. Where? In which directories? /usr/bin/flex is 2.5.4 and /usr/local/bin/flex is 2.5.35. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
> Meanwhile, last chance for anyone to object to killing these two? Nope. People will have to fix their apps, but it's about time. I'll try to remember to advertise the breakage ... --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha2 release notes
On 10/20/09 3:06 PM, Tom Lane wrote: > Josh Berkus writes: >> I'd like to edit these. Where's source? Actually, change this to I *need* to edit it. There's several errors. Working on it now, but can't promise to finish by tommorrow AM. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
"Kevin Grittner" writes: > Magnus Hagander wrote: >> For java, it doesn't even go through libpq, so it wouldn't be set >> for it. And I'd expect the JDBC driver to set it based on Something >> Reasonable (TM) that it can get the information about. After all, >> this thing was listed in the JDBC spec somebody said... > I can't see a good way to get a meaningful default from within the > bowels of the JDBC driver, unless it's taking a system property or > environment variable. [ scratches head... ] I thought the JDBC spec already said exactly how one would set this. Why would we go to significant effort to make it behave contrary to spec? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
David Fetter writes: > add_missing_from (should be off) > regex_flavor (should be advanced, regex flavor can be controlled on a > per-regex basis when they're advanced) I believe that we do have consensus on removing those two, based on discussions here and here respectively: http://archives.postgresql.org/pgsql-hackers/2009-10/msg00958.php http://archives.postgresql.org/pgsql-hackers/2009-10/msg00807.php Each of those threads established that there was a nontrivial identifiable benefit to removing the particular option, not just that somebody thought it was useless. I think the rest of the list needs to be held to a similar standard before removing anything. Meanwhile, last chance for anyone to object to killing these two? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Magnus Hagander wrote: > For java, it doesn't even go through libpq, so it wouldn't be set > for it. And I'd expect the JDBC driver to set it based on Something > Reasonable (TM) that it can get the information about. After all, > this thing was listed in the JDBC spec somebody said... I can't see a good way to get a meaningful default from within the bowels of the JDBC driver, unless it's taking a system property or environment variable. First off, generating a stack trace to try to pick up a class name would be too expensive to be something you'd want to do by default when opening a connection. For another, many applications are started from a bootstrap class (for good reason), so you can't just take the initial class name for a meaningful default. If you were running a GUI application you might try to find the title of some window frame, but that's pretty ugly. In short, I'd suggest supporting a system property for a default name. (For those unfamiliar with Java, a system property is very much like an environment variable, but just for the one Java Virtual Machine.) That seems more sane than an environment variable for the Java world. I don't see much other reasonable alternative, except of course that you should be able to override it when setting up any individual connection. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha2 release notes
Josh Berkus writes: > I'd like to edit these. Where's source? In CVS ... http://archives.postgresql.org/pgsql-committers/2009-10/msg00090.php regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
On Tue, 20 Oct 2009, Bernd Helmle wrote: --On 20. Oktober 2009 10:49:33 -0700 David Fetter wrote: track_activities (should be on) track_counts (should be on) update_process_title (should be on) Removing all track* GUCs would only make sense if we are going to make autovacuum mandatory, i think. And i thought update_process_title was introduced due to performance complaints on some platforms which can't easily be resolved? Yes, update_process_title on FreeBSD defaults to 'off' as the overhead based on how FreeBSD does the update was enough to make a noticeable difference in performance ... suspect that is the same for all the *BSDs, not sure about any other platform though ... maybe Windows? Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
Why? Are they causing problems leaving them there? What sorts of problems will arise by *removing* them? I know with OACS, add_missing_from is required right now, but that code *should* be fixable, and there is an overwhelming reason from an advancement perspective to having it removed (ie. new code that Tom is developing), but what about the rest ... ? On Tue, 20 Oct 2009, David Fetter wrote: Folks, I'd like to see about removing the following GUCs: add_missing_from (should be off) array_nulls (should be on) commit_delay (no need for this knob) commit_siblings (no need for this knob) default_with_oids (should be off) password_encryption (should be on) regex_flavor (should be advanced, regex flavor can be controlled on a per-regex basis when they're advanced) sql_inheritance (should be on) standard_conforming_strings (should be on) synchronize_seqscans (should be on) track_activities (should be on) track_counts (should be on) transform_null_equals (should probably be off) update_process_title (should be on) What say on each? When? Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha2 release notes
On 10/20/09 2:11 PM, Peter Eisentraut wrote: > http://developer.postgresql.org/pgdocs/postgres/release-8-5.html > > If they're OK, I can bundle it tomorrow. I'd like to edit these. Where's source? --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Postgres server goes in recovery mode repeteadly
daveg writes: > We have had this deployed in our test and production environments for a > couple weeks now. We have not seen any further instance of the problem. > Without the patch, we would have expected to see at least a few by now. > So the patch appears to be effective. Cool, thanks for the follow-up. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
daveg writes: > I'd like a default, especially for psql, to help identify interactive > sessions. psql can certainly provide a default, and maybe even do something actually useful like report the -f file it's running. The question here is whether it is worth the trouble for libpq to try to report an appname *without* the surrounding application's cooperation. I don't think so. You are talking dozens if not hundreds of lines of unportable code, versus one line (per app) of entirely portable code. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Postgres server goes in recovery mode repeteadly
On Fri, Oct 02, 2009 at 07:57:13PM -0700, daveg wrote: > On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: > > daveg escribió: > > > > > I work with Kunal and have been looking into this. It appears to be the > > > same > > > as the bug described in: > > > > > > http://archives.postgresql.org/pgsql-bugs/2009-09/msg00355.php > > > > > > as I have localized it to a NULL pointer deference in > > > RelationCacheInitializePhase2() as well. Tom speculates in: > > > > > > http://archives.postgresql.org/pgsql-bugs/2009-09/msg00372.php > > > > > > that large numbers of table drops might trigger this. The system in > > > question > > > creates and drops temp tables at a high rate which tends to confirm this. > > > > Did you test the patch posted by Tom? > > We are testing it since last night in our test environment. If it does not > break anything (unlikely) we will deploy it next week. However, since the > problem is only occasional, only happens every few days on one of 50+ hosts, > it will take some extended time without further segfaults to say anything > confident about the patches effectiveness. We have had this deployed in our test and production environments for a couple weeks now. We have not seen any further instance of the problem. Without the patch, we would have expected to see at least a few by now. So the patch appears to be effective. -dg -- David Gould da...@sonic.net 510 536 1443510 282 0869 If simplicity worked, the world would be overrun with insects. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
On Tue, Oct 20, 2009 at 12:16:42PM -0400, Tom Lane wrote: > Magnus Hagander writes: > > Also, how many platforms can't we do this on? If we have BSD and > > Windows covered already. on linux, I believe you can easily read it > > out of /proc/self/cmdline, no? > > Writing a pile of platform-specific code for this is simply insane from > a support point of view. The feature is NOT worth it. Especially not > since the typical result will be something quite uninformative like > "psql" or "java". The cases that are actually useful are the ones where > the application sets it. I don't think we should have a default at all > --- you don't set it, you don't get a name. -1 I'd like a default, especially for psql, to help identify interactive sessions. -dg -- David Gould da...@sonic.net 510 536 1443510 282 0869 If simplicity worked, the world would be overrun with insects. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] alpha2 release notes
http://developer.postgresql.org/pgdocs/postgres/release-8-5.html If they're OK, I can bundle it tomorrow. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Using pgcrypt to meet PCI compliance?
I have a a postgres database implementation that needs to be enhanced to meet PCI compliance for encrypting sensitive data inside the database. I'm looking at dm-crypt to encrypt my filesystems to prevent against theft of hardware, but we also have a requirement to encrypt a few important fields in a few tables in order to prevent against sql injection/exposure attacks. I'm wondering if anyone on the list has any *specific* experience with the pgcrypto and the key management required in order to meet pci compliance for encrypting data within a postgres db. Thanks! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Magnus Hagander writes: > 2009/10/20 Tom Lane : >> "psql" or "java". The cases that are actually useful are the ones where >> the application sets it. I don't think we should have a default at all >> --- you don't set it, you don't get a name. > > For psql, yes. What about having psql -f foo.sql using 'psql: foo.sql' or sth as the default value? -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Privileges and inheritance
Peter Eisentraut writes: > If I'm seeing this right, it's literally a one-line fix. At least a two-line fix, please: that needs a comment. But yeah, I think that's basically all that would have to change. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha1 bundled -- please verify
On Wed, 2009-08-19 at 19:11 +0200, Stefan Kaltenbrunner wrote: > correct - svr1 has bison 1.875 and flex 2.5.35 (for the -HEAD builds) > and flex 2.5.4 for the back branches. Where? In which directories? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Privileges and inheritance
On Sat, 2009-10-03 at 09:45 +0300, I wrote:
> So let's get rid of that. Selecting (or in general, operating) on a
> table with children only checks the privileges on that table, not the
> children.
If I'm seeing this right, it's literally a one-line fix. Patch attached
with documentation and test updates.
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index e055518..51d9aef 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -2124,20 +2124,20 @@ VALUES ('New York', NULL, NULL, 'NY');
and rejection that apply during CREATE TABLE.
-
- Caveats
-
- Table access permissions are not automatically inherited. Therefore,
- a user attempting to access a parent table must either have permissions
- to do the same operation on all its child tables as well, or must use the
- ONLY notation. When adding a new child table to
- an existing inheritance hierarchy, be careful to grant all the needed
- permissions on it.
+ Note how table access permissions are handled. Querying a parent
+ table can automatically access data in child tables without further
+ access privilege checking. This preserves the appearance that the
+ data is (also) in the parent table. Accessing the child tables
+ directly is, however, not automatically allowed and would require
+ further privileges to be granted.
+
+ Caveats
+
- More generally, note that not all SQL commands are able to work on
+ Note that not all SQL commands are able to work on
inheritance hierarchies. Commands that are used for data querying,
data modification, or schema modification
(e.g., SELECT, UPDATE, DELETE,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index df9eac6..17ddbe2 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -1244,6 +1244,7 @@ expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte, Index rti)
childrte = copyObject(rte);
childrte->relid = childOID;
childrte->inh = false;
+ childrte->requiredPerms = 0;
parse->rtable = lappend(parse->rtable, childrte);
childRTindex = list_length(parse->rtable);
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 16b5581..4decb2b 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -406,11 +406,22 @@ SELECT fx FROM atestp2; -- ok
(0 rows)
-SELECT fy FROM atestp2; -- fail, no privilege on atestc.fy
-ERROR: permission denied for relation atestc
-SELECT atestp2 FROM atestp2; -- fail, no privilege on atestc.fy
-ERROR: permission denied for relation atestc
-SELECT oid FROM atestp2; -- fail, no privilege on atestc.oid
+SELECT fy FROM atestp2; -- ok
+ fy
+
+(0 rows)
+
+SELECT atestp2 FROM atestp2; -- ok
+ atestp2
+-
+(0 rows)
+
+SELECT oid FROM atestp2; -- ok
+ oid
+-
+(0 rows)
+
+SELECT fy FROM atestc; -- fail
ERROR: permission denied for relation atestc
SET SESSION AUTHORIZATION regressuser1;
GRANT SELECT(fy,oid) ON atestc TO regressuser2;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index 943fe36..d4f7282 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -277,9 +277,10 @@ GRANT SELECT(fx) ON atestc TO regressuser2;
SET SESSION AUTHORIZATION regressuser2;
SELECT fx FROM atestp2; -- ok
-SELECT fy FROM atestp2; -- fail, no privilege on atestc.fy
-SELECT atestp2 FROM atestp2; -- fail, no privilege on atestc.fy
-SELECT oid FROM atestp2; -- fail, no privilege on atestc.oid
+SELECT fy FROM atestp2; -- ok
+SELECT atestp2 FROM atestp2; -- ok
+SELECT oid FROM atestp2; -- ok
+SELECT fy FROM atestc; -- fail
SET SESSION AUTHORIZATION regressuser1;
GRANT SELECT(fy,oid) ON atestc TO regressuser2;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page writes: > I just realised there's a nasty problem with this. In my client > application, I can use PQconninfoParse to determine if > application_name (or fallback_application_name) are valid connection > string options for the version of libpq that I have. > However, there is no way that I can see of doing a comparable test in > libpq itself, to ensure that the server understands the parameter, so > if I connect to an 8.5 server, everything works fine, but if connect > to an older server with my new libpq, the connection fails because of > the unknown parameter in the startup packet. Hmm, yeah, that's a good point. It seems like you will have to send the appname SET command as a separate packet, after you have gotten the initial response and know what version the server is. Kind of annoying from a performance standpoint, but I believe it won't be too hard to shoehorn into libpq --- it already has a code path for that for older servers, IIRC. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Controlling changes in plpgsql variable resolution
>> >> I don't thing, so drop some implicit-casts was huge problem. Somebody >> could to use Peter's patch, that recreate missing casts. > > True, but we should have had those compatibility pathes (Peter's patch) > ready before we released, and advertised them in the release notes. sure Maybe we have to have a compatibility group. I don't support 100% compatibility - mainly with things that should be source of bugs - like these implicit casts. I migrate two applications (not own) and I really found bugs there. But change of every feature should be well supported. Pavel > > -- > Bruce Momjian http://momjian.us > EnterpriseDB http://enterprisedb.com > > + If your life is a hard drive, Christ can be your backup. + > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY enhancements
Emmanuel Cecchet writes: > Tom Lane wrote: >> The key word in my sentence above is "arbitrary". You don't know what >> a datatype input function might try to do, let alone triggers or other >> functions that COPY might have to invoke. They might do things that >> need to be cleaned up after, and subtransaction rollback is the only >> mechanism we have for that. > Is it possible to use the try/catch mechanism to detect errors and log > them and only abort the command at the end of the processing? No, I wouldn't trust that. The point here is that various backend subsystems (eg, locks, buffers) might need to be cleaned up after an error before they can be used further. It might look like it works, until you stumble across the cases where it doesn't. An example of the sort of situation I'm concerned about is somebody throwing an elog while holding a buffer lock. If you try to keep processing and clean up later, it will work fine ... until the day comes that the subsequent processing chances to try to lock that same buffer again, and then the backend will freeze up. We've got twenty years worth of code development that assumes that transaction abort will clean up anything that was left hanging when an error was thrown. It was difficult enough getting that to work for subtransaction abort as well as main transaction abort. It's not happening for anything less than a subtransaction abort. This approach is a good tradeoff most of the time: the code is simpler, much more reliable, and faster in the normal case than it would be if we tried to do post-error cleanup differently. Error-tolerant COPY is going to pay for all that, but we're not revisiting the decision. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
David Fetter wrote: Folks, I'd like to see about removing the following GUCs: add_missing_from (should be off) array_nulls (should be on) commit_delay (no need for this knob) commit_siblings (no need for this knob) default_with_oids (should be off) password_encryption (should be on) regex_flavor (should be advanced, regex flavor can be controlled on a per-regex basis when they're advanced) sql_inheritance (should be on) standard_conforming_strings (should be on) synchronize_seqscans (should be on) track_activities (should be on) track_counts (should be on) transform_null_equals (should probably be off) update_process_title (should be on) What say on each? When? We don't do wholesale changes like this. If you have a case to make then make it for each one individually. I will observe, however, that removing standard_conforming_strings and making it always on will break thousands and thousands of apps. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY enhancements
Tom, Emmanuel Cecchet writes: Tom Lane wrote: There aren't any. You can *not* put a try/catch around arbitrary code without a subtransaction. Don't even think about it Well then why the tests provided with the patch are working? Because they carefully exercise only a tiny fraction of the system. The key word in my sentence above is "arbitrary". You don't know what a datatype input function might try to do, let alone triggers or other functions that COPY might have to invoke. They might do things that need to be cleaned up after, and subtransaction rollback is the only mechanism we have for that. Is it possible to use the try/catch mechanism to detect errors and log them and only abort the command at the end of the processing? This would have the advantage of being to be able to log all errors without the overhead of subtransactions and even add some additional information on where the error happened (parsing, constraints, trigger, ...) for further automated treatment. The result would still be clean since we would abort the COPY command in case of an error as it does currently (but we would not stop on the first error). I guess that it would make more sense to log to a file than to a table in that case. Emmanuel -- Emmanuel Cecchet Aster Data Systems Web: http://www.asterdata.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
In response to Bernd Helmle : > > > --On 20. Oktober 2009 10:49:33 -0700 David Fetter wrote: > > > track_activities (should be on) > > track_counts (should be on) > > update_process_title (should be on) I'm not replying in the correct part of the thread, but I just happened to notice this by accident. update_process_title is a known performance killer on FreeBSD (unless this has been fixed in 8.x). As a result, it would hurt me if that were removed. I always have it turned off. Even if the performance problem is fixed in FreeBSD 8.x, this should be left in for a few more releases until newer versions of FreeBSD are the standard. -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
--On 20. Oktober 2009 10:49:33 -0700 David Fetter wrote: track_activities (should be on) track_counts (should be on) update_process_title (should be on) Removing all track* GUCs would only make sense if we are going to make autovacuum mandatory, i think. And i thought update_process_title was introduced due to performance complaints on some platforms which can't easily be resolved? -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
Jeff Davis writes: > On Tue, 2009-10-20 at 10:49 -0700, David Fetter wrote: >> synchronize_seqscans (should be on) > Right now this is used for pg_dump, because pg_dump could un-cluster a > previously clustered table (I believe Greg Stark made this observation). In general, the setting results in producing indeterminate output where previous versions produced deterministic results; we did get complaints about that when it came out. Furthermore the performance gain may be marginal or nonexistent depending on your application. The code savings from removing the setting would certainly be marginal or nonexistent. I can't see a good argument for taking this out. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
David Fetter writes: > I'd like to see about removing the following GUCs: > add_missing_from (should be off) > array_nulls (should be on) > commit_delay (no need for this knob) > commit_siblings (no need for this knob) > default_with_oids (should be off) > password_encryption (should be on) > regex_flavor (should be advanced, regex flavor can be controlled on a > per-regex basis when they're advanced) > sql_inheritance (should be on) > standard_conforming_strings (should be on) > synchronize_seqscans (should be on) > track_activities (should be on) > track_counts (should be on) > transform_null_equals (should probably be off) > update_process_title (should be on) There was consensus on removing about two of these. Back off. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
u235sentinel píše v út 20. 10. 2009 v 12:22 -0600: > Now I'm running and will add a few more things in like readline. The > goal is to build plr and plperl and load them into postgres. Hmm, I'm afraid that 64bit plperl is a problem. Solaris 10 is not shipped with 64bit perl :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
On Tue, 2009-10-20 at 10:49 -0700, David Fetter wrote: > synchronize_seqscans (should be on) Right now this is used for pg_dump, because pg_dump could un-cluster a previously clustered table (I believe Greg Stark made this observation). This is actually a stats/planner issue more than anything else, because the table isn't _really_ unclustered, but it is still an issue (seems minor to me, but not insignificant). Also, it seems reasonable that testers might want to use something like this, if they don't want to ORDER BY. For instance, if testing postgresql itself, ORDER BY would change what you're testing. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Going, going, GUCs!
David Fetter wrote: > I'd like to see about removing the following GUCs: > sql_inheritance (should be on) I'd rather see that stay, so that I can make sure it's off. That said, we have other ways to enforce shop policy on this, if need be. > track_counts (should be on) I believe we found a slight performance benefit to turning this off for bulk loads (during which we also turn off autovacuum). We finish the process by running VACUUM FREEZE ANALYZE on the database and restoring the normal postgresql.conf file (with these on). A minimal hit on bulk load operations wouldn't be the end of the world, but I'd rather avoid it. > transform_null_equals (should probably be off) If we're eliminating the GUC, it *better* be treated as off; otherwise a lot of code written to the SQL specification will be broken. The others don't matter much to me, personally; but I wonder why you are pushing this in the face of the concurrent threads, where the consensus seems to be that there is not much to be gained by eliminating these, and risk of problems. What makes the GUCs on you list worth expending the effort? (I'd say that each one merits separate justification.) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
Zdenek Kotala wrote:
Andrew Chernow píše v po 19. 10. 2009 v 14:14 -0400:
# ./pg_ctl
ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file
/usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value
0xfd7fff1cf210 does not fit
Killed
{snip}
/usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib AMD64
Version 1 [SSE CMOV], dynamically linked, not stripped
Ok. So looking good. Maybe there is a library or header libpq needs
that I'm missing in 64 bit?
# ldd /usr/local/postgres64/lib/libpq.so.5
Are you sure that all pg_ctl referenced libraries and all libpq.so
referenced libraries were built as 64-bit using PIC? Are you linking
with any static library that may contain 32-bit objects? That error is
most commonly PIC or arch-mismatch.
Agree, I went through linker bugs and missing PIC is often root cause of
this problem. See
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6261066
Problem was that ./configure badly setup PIC switch on amd64 platform.
Please, could you compile pure postgreSQL without other own libraries
like readline and openssl? It should help to find which library is
culprit.
Zdenek
I really appreciate all the comments about this problem. I'm not a
developer but I've been around the block a few times here.
I "think" I have it working now. At least it compiled and I was able to
run initdb, pg_ctl and psql to login to my new test database. All in 64
bit with openssl compiled in.
How I did it.
I've been SO focued on postgres thinking it was the entire problem that
I forgot about ssl. Even though openssl compiled just fine, it
contributed to the problem. After recompiling it with gcc and using
-fpic and -m64 I recompiled postgres also with -fpic and -m64. Seems
that was the magic sauce here.
Now I'm running and will add a few more things in like readline. The
goal is to build plr and plperl and load them into postgres.
Crossing my fingers those will go smoother.
Thanks a bunch!!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Going, going, GUCs!
Folks, I'd like to see about removing the following GUCs: add_missing_from (should be off) array_nulls (should be on) commit_delay (no need for this knob) commit_siblings (no need for this knob) default_with_oids (should be off) password_encryption (should be on) regex_flavor (should be advanced, regex flavor can be controlled on a per-regex basis when they're advanced) sql_inheritance (should be on) standard_conforming_strings (should be on) synchronize_seqscans (should be on) track_activities (should be on) track_counts (should be on) transform_null_equals (should probably be off) update_process_title (should be on) What say on each? When? Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Controlling changes in plpgsql variable resolution
Pavel Stehule wrote: > 2009/10/20 Tom Lane : > > Merlin Moncure writes: > >> How about warning for release before making the big switch? ?The text > >> cast change, while ultimately good, maybe could have been stretched > >> out for a release or two...it was painful. ?I do though absolutely > >> think that it was good in the end to not support a compatibility > >> option in core. > > > > As long as we provide a backwards-compatible setting, I don't believe > > this change will be a huge deal. ?The problem with the implicit-casts > > business was that there was no reasonable way to provide a control > > switch --- the catalog entries were either there or not :-(. ?So far > > as I can tell at the moment, there won't be any large technical cost > > to providing a backwards-compatible option for this plpgsql change. > > I don't thing, so drop some implicit-casts was huge problem. Somebody > could to use Peter's patch, that recreate missing casts. True, but we should have had those compatibility pathes (Peter's patch) ready before we released, and advertised them in the release notes. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
David Fetter writes: > This isn't about a "passion for neatness." It's about recognizing > that some experiments have failed and weeding out the failures. The > RULE system, for example, was a ground-breaking innovation in the > sense of being a truly new idea. Evidence over the decades since has > shown that it was a *bad* idea, and I like to think we're going with > an evidence-based approach. Things like add_missing_from and > regex_flavor, to name two examples, are just bletcherous hacks > invented to solve no-longer-extant problems. The above examples seem to me to show that your argument is nonsense. regex_flavor, in particular, is not about "failed experiments", it's about backwards compatibility with a previous version that simply worked differently. It might be that we can have a sunset provision for backwards compatibility, but to argue that it's not important pretty much flies in the face of most of the discussions we've had on the topic. I agree that the RULE system is a failed experiment and we need to find something better, but we're unlikely to rip that out either until the replacement has been in use for a few releases. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] OpenACS vs Postgres
OpenACS uses three compatibility settings in its recommended config, two
of which it has been proposed to remove.
I have been investigating all three.
1. default_with_oids= true
While it's not proposed to remove this option, its use seems
quite unnecessary in OpenACS. AFAICS, the code uses oids on
exactly two tables, so these can be created with OIDS, and the
flag turned off. The use it makes of OIDS is kludgey anyway,
but that need not concern us.
2. regex_flavor = 'extended'
I have not found anywhere in the current release that actually
uses POSIX regex matching, so I suspect this was recommended
from an over abundance of caution, or else it was used in code
that is no longer present.
3. add_missing_from = on
In a particularly hacky gadget, OpenACS creates views that
look like this:
create view foo_seq as select nextval('real_foo_seq') as
nextval;
which then lets them use Oracle-like syntax such as:
select foo_seq.nextval from dual;
relying on add_missing_from to do the right thing. I have
discussed this with my client that uses OpenACS, and they will
patch OpenACS as necessary when they upgrade to Postgres 8.5
to remove this kludge.
So, bottom line, I withdraw my objection to the proposed changes.
Personally, I'll be quite glad to see the back of add_missing_from, and
so will my client.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
2009/10/20 Tom Lane : > Magnus Hagander writes: >> Also, how many platforms can't we do this on? If we have BSD and >> Windows covered already. on linux, I believe you can easily read it >> out of /proc/self/cmdline, no? > > Writing a pile of platform-specific code for this is simply insane from > a support point of view. The feature is NOT worth it. Especially not > since the typical result will be something quite uninformative like > "psql" or "java". The cases that are actually useful are the ones where > the application sets it. I don't think we should have a default at all > --- you don't set it, you don't get a name. For psql, yes. For java, it doesn't even go through libpq, so it wouldn't be set for it. And I'd expect the JDBC driver to set it based on Something Reasonable (TM) that it can get the information about. After all, this thing was listed in the JDBC spec somebody said... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
On Oct 20, 2009, at 10:51 AM, Tom Lane wrote: Andrew Dunstan writes: What I think we might sensibly do is to eat the leading BOM of an SQL file iff the client encoding is UTF8, and otherwise treat it as just bytes in whatever the encoding is. That seems relatively non-risky. Is that only when the default client encoding is set to UTF8 (PGCLIENTENCODING, whatever), or will it be coded to work with the following: $ PGCLIENTENCODING=...nonutf8... $ psql -f Where is: ... SET CLIENT ENCODING 'utf8'; ... EOF Regards, David -- David Christensen End Point Corporation da...@endpoint.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Magnus Hagander writes: > Also, how many platforms can't we do this on? If we have BSD and > Windows covered already. on linux, I believe you can easily read it > out of /proc/self/cmdline, no? Writing a pile of platform-specific code for this is simply insane from a support point of view. The feature is NOT worth it. Especially not since the typical result will be something quite uninformative like "psql" or "java". The cases that are actually useful are the ones where the application sets it. I don't think we should have a default at all --- you don't set it, you don't get a name. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Controlling changes in plpgsql variable resolution
2009/10/20 Tom Lane : > Merlin Moncure writes: >> How about warning for release before making the big switch? The text >> cast change, while ultimately good, maybe could have been stretched >> out for a release or two...it was painful. I do though absolutely >> think that it was good in the end to not support a compatibility >> option in core. > > As long as we provide a backwards-compatible setting, I don't believe > this change will be a huge deal. The problem with the implicit-casts > business was that there was no reasonable way to provide a control > switch --- the catalog entries were either there or not :-(. So far > as I can tell at the moment, there won't be any large technical cost > to providing a backwards-compatible option for this plpgsql change. I don't thing, so drop some implicit-casts was huge problem. Somebody could to use Peter's patch, that recreate missing casts. regards Pavel > > regards, tom lane > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Tom Lane wrote: > if your software is written to depend on the appname being set a > particular way then you're not using for its intended purpose, I should think. Since any client can set this to whatever they want, having the application name as a default, rather than NULL (at least for clients on Windows, BSD, and Linux) would be a convenience for those using it for its intended purpose. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
On Tue, Oct 20, 2009 at 11:48 AM, David Fetter wrote: > On Tue, Oct 20, 2009 at 10:46:10AM -0400, Andrew Dunstan wrote: >> Robert Haas wrote: >>> I think the real issue, though, is that answer to Ron's original >>> question is "No". When backward compatibility gets in the way of >>> cool new features, that's worth considering. But removing backward >>> compatibility just for the sake of removing backward compatibility >>> doesn't really buy us anything. It's basically doing extra work >>> for no benefit and some possible harm. >> >> Well said. >> >> I am singularly unimpressed by arguments for removing backwards >> compatibility features to satisfy someone's passion for neatness, or >> to force people to conform to how they think their software should >> be managed. I occasionally shake my head in amazement at the >> willingness of some people to throw other users under the bus. > > This isn't about a "passion for neatness." It's about recognizing > that some experiments have failed and weeding out the failures. The > RULE system, for example, was a ground-breaking innovation in the > sense of being a truly new idea. Evidence over the decades since has > shown that it was a *bad* idea, and I like to think we're going with > an evidence-based approach. Things like add_missing_from and > regex_flavor, to name two examples, are just bletcherous hacks > invented to solve no-longer-extant problems. > > The burden of keeping things like this in the code base is large and > increasing. I don't think that's true. That's just an assertion on your part, and I don't think there's any evidence to back it up. > There's even a name for it: technical debt. > > http://en.wikipedia.org/wiki/Technical_debt > > If we're to remain a successful project, the vast majority of our > community is that of future users, not present or past, so worshipping > backward compatibility is rejecting the needs of the many in favor of > the few. Let's at least think a bit before we make a decision to do > such a thing. I think we usually think pretty carefully about these decisions on a case-by-case basis. Making a blanket policy, as proposed here, would NOT be thinking carefully. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Controlling changes in plpgsql variable resolution
Merlin Moncure writes: > How about warning for release before making the big switch? The text > cast change, while ultimately good, maybe could have been stretched > out for a release or two...it was painful. I do though absolutely > think that it was good in the end to not support a compatibility > option in core. As long as we provide a backwards-compatible setting, I don't believe this change will be a huge deal. The problem with the implicit-casts business was that there was no reasonable way to provide a control switch --- the catalog entries were either there or not :-(. So far as I can tell at the moment, there won't be any large technical cost to providing a backwards-compatible option for this plpgsql change. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] per table random-page-cost?
On Tue, 20 Oct 2009, Kevin Grittner wrote: How often do you have to print a list of past due accounts? I've generally seen that done weekly or monthly, in the same places that there are many people standing full time in payment windows just to collect money from those lining up to pay. This is only because your A/R collections staff includes people with guns. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
2009/10/20 Tom Lane : > Andrew Dunstan writes: >> What I think we might sensibly do is to eat the leading BOM of an SQL >> file iff the client encoding is UTF8, and otherwise treat it as just >> bytes in whatever the encoding is. > > That seems relatively non-risky. +1. >> Should we also do the same for files passed via \copy? What about >> streams on stdin? What about files read from the backend via COPY? > > Not thrilled about doing this on stdin --- you have no good > justification for assuming that start of stdin corresponds to a file > boundary somewhere. COPY files, maybe. Yeah, that seems a lot more error-prone. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
2009/10/20 Tom Lane : > Heikki Linnakangas writes: >> Tom Lane wrote: >>> It would be a seriously bad idea for this to behave one way on some >>> platforms and differently on others. > >> Why would that be so bad? On platforms that support getting argv[0], >> you'd get "mycoolapp" in the application name by default. On others, >> you'd get something like "unknown libpq app". > > Right, and if your software is written to depend on the appname being > set a particular way, you suddenly find you have a portability problem. > I don't think we should be in the business of creating hidden vendor > lock-ins. If your application depends on it being set properly, then have the application set it. This would just be a *default*, you could still override it - and should if it's really that important to your application. Also, how many platforms can't we do this on? If we have BSD and Windows covered already. on linux, I believe you can easily read it out of /proc/self/cmdline, no? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
Andrew Dunstan wrote: > What I think we might sensibly do is to eat the leading BOM of an > SQL file iff the client encoding is UTF8, and otherwise treat it as > just bytes in whatever the encoding is. Only at the beginning of the file or stream? What happens when people concatenate files? Would it make sense to treat BOM as whitespace in UTF-8, or maybe ignore it entirely? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Controlling changes in plpgsql variable resolution
On Tue, Oct 20, 2009 at 10:32 AM, Tom Lane wrote: > Bruce Momjian writes: >> Tom Lane wrote: >>> 1. Invent a GUC that has the settings backwards-compatible, >>> oracle-compatible, throw-error (exact spellings TBD). Factory default, >>> at least for a few releases, will be throw-error. Make it SUSET so that >>> unprivileged users can't break things by twiddling it; but it's still >>> possible for the DBA to set it per-database or per-user. > >> I don't see the logic to making the setting SUSET. The user wrote the >> function; what logic is there to say the resolution rules are not under >> their control? > > That's only sane if you are 100% certain that there could not be a > security issue arising from the change of behavior. Otherwise someone > could for instance subvert a security-definer function by running it > under the setting it wasn't written for. Personally I am not 100% > certain of that. > >> Also, I think to GUC that throws an error or not is a lot safer than one >> that changes resolution semantics. Changing resolution semantics sounds >> like the autocommit GUC to me. :-O > > Yeah, that's another reason to not allow it to be changed too easily. > >> Also, I am not really keen on the "keep it for a few releases" > > Well, I'm not necessarily saying we would ever change it. Maybe the > default could always stay at "error". > >> ... maybe just error/no error >> and using Oracle semantics is the way to go, with 'error' as the >> default. > > I'd personally be entirely happy with that, but people with large > plpgsql code bases will not be. They're going to want a > backward-compatible setting so that this doesn't become a show stopper > for migration to 8.5. Any time you can allow someone to deal with a > migration issue later instead of right away, it becomes easier for them > to migrate. How about warning for release before making the big switch? The text cast change, while ultimately good, maybe could have been stretched out for a release or two...it was painful. I do though absolutely think that it was good in the end to not support a compatibility option in core. Didn't we have a long discussion on big compatibility changes with the consensus that we were to going give a transition release before we dropped a backwards compatibility bomb? I can't help feeling that we are about to jump off into the abyss... merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
Andrew Dunstan writes: > What I think we might sensibly do is to eat the leading BOM of an SQL > file iff the client encoding is UTF8, and otherwise treat it as just > bytes in whatever the encoding is. That seems relatively non-risky. > Should we also do the same for files passed via \copy? What about > streams on stdin? What about files read from the backend via COPY? Not thrilled about doing this on stdin --- you have no good justification for assuming that start of stdin corresponds to a file boundary somewhere. COPY files, maybe. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
On Tue, Oct 20, 2009 at 10:46:10AM -0400, Andrew Dunstan wrote: > Robert Haas wrote: >> I think the real issue, though, is that answer to Ron's original >> question is "No". When backward compatibility gets in the way of >> cool new features, that's worth considering. But removing backward >> compatibility just for the sake of removing backward compatibility >> doesn't really buy us anything. It's basically doing extra work >> for no benefit and some possible harm. > > Well said. > > I am singularly unimpressed by arguments for removing backwards > compatibility features to satisfy someone's passion for neatness, or > to force people to conform to how they think their software should > be managed. I occasionally shake my head in amazement at the > willingness of some people to throw other users under the bus. This isn't about a "passion for neatness." It's about recognizing that some experiments have failed and weeding out the failures. The RULE system, for example, was a ground-breaking innovation in the sense of being a truly new idea. Evidence over the decades since has shown that it was a *bad* idea, and I like to think we're going with an evidence-based approach. Things like add_missing_from and regex_flavor, to name two examples, are just bletcherous hacks invented to solve no-longer-extant problems. The burden of keeping things like this in the code base is large and increasing. There's even a name for it: technical debt. http://en.wikipedia.org/wiki/Technical_debt If we're to remain a successful project, the vast majority of our community is that of future users, not present or past, so worshipping backward compatibility is rejecting the needs of the many in favor of the few. Let's at least think a bit before we make a decision to do such a thing. > Upgrading a database installation is hard enough without us > gratuitously making it harder, and we positively don't want to make > people stay on older releases if they don't have to, I should have > thought. Upgrading a database installation should be made easier in a lot of different ways, not least that a postmaster should be able to point to an older database instance, possibly with some kind of prep procedure, and do the right thing. What we have now (Slony, pg_upgrade) is a bit better than, "fire up pg_dump and take the down time," but it needs a lot more love. This is for the benefit of our entire community, not just the present one. :) Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Heikki Linnakangas writes: > Tom Lane wrote: >> It would be a seriously bad idea for this to behave one way on some >> platforms and differently on others. > Why would that be so bad? On platforms that support getting argv[0], > you'd get "mycoolapp" in the application name by default. On others, > you'd get something like "unknown libpq app". Right, and if your software is written to depend on the appname being set a particular way, you suddenly find you have a portability problem. I don't think we should be in the business of creating hidden vendor lock-ins. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
Tom Lane wrote: Bruce Momjian writes: Seems there is community support for accepting BOM: http://archives.postgresql.org/pgsql-hackers/2009-09/msg01625.php That discussion has approximately nothing to do with the much-more-invasive change that Itagaki-san is suggesting. In particular I think an automatic change of client_encoding isn't particularly a good idea --- wouldn't you have to change it back later, and is there any possibility of creating a security issue from such behavior? Remember that client_encoding *IS* tied to security issues such as backslash escape handling. Yeah, I don't think we should be second-guessing the user on the encoding. What I think we might sensibly do is to eat the leading BOM of an SQL file iff the client encoding is UTF8, and otherwise treat it as just bytes in whatever the encoding is. Should we also do the same for files passed via \copy? What about streams on stdin? What about files read from the backend via COPY? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
"Greg Sabino Mullane" writes: > That particular example is a very poor one for illustrating your > point. You severely underestimate "get away with" for the implicit > cast changes in 8.3. This was a really big deal for many, many users > of Postgres, and continues to cause many problems to this day. Actually, that was *exactly* my point. What's being blithely discussed here is to make tens of changes that would be equally as painful as that one, all at once. You really think that would fly? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Tom Lane wrote: > Magnus Hagander writes: >> 2009/10/20 Dave Page : >>> Yeah, and there's a similar API on *BSD I believe, but nothing standard. > >> Right, but it might be worth investigating using the API that's >> available on the platform, if one is. It's a fairly simple operation >> after all, so it won't take huge amounts of platform-specific code. > > It would be a seriously bad idea for this to behave one way on some > platforms and differently on others. Why would that be so bad? On platforms that support getting argv[0], you'd get "mycoolapp" in the application name by default. On others, you'd get something like "unknown libpq app". And if you care enough, you can override it in the application or with the environment variable. Seems quite sane and harmless to me. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
Robert Haas wrote: I think the real issue, though, is that answer to Ron's original question is "No". When backward compatibility gets in the way of cool new features, that's worth considering. But removing backward compatibility just for the sake of removing backward compatibility doesn't really buy us anything. It's basically doing extra work for no benefit and some possible harm. Well said. I am singularly unimpressed by arguments for removing backwards compatibility features to satisfy someone's passion for neatness, or to force people to conform to how they think their software should be managed. I occasionally shake my head in amazement at the willingness of some people to throw other users under the bus. Upgrading a database installation is hard enough without us gratuitously making it harder, and we positively don't want to make people stay on older releases if they don't have to, I should have thought. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
Magnus Hagander writes: > 2009/10/20 Dave Page : >> Yeah, and there's a similar API on *BSD I believe, but nothing standard. > Right, but it might be worth investigating using the API that's > available on the platform, if one is. It's a fairly simple operation > after all, so it won't take huge amounts of platform-specific code. It would be a seriously bad idea for this to behave one way on some platforms and differently on others. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
Bruce Momjian writes: > Seems there is community support for accepting BOM: > http://archives.postgresql.org/pgsql-hackers/2009-09/msg01625.php That discussion has approximately nothing to do with the much-more-invasive change that Itagaki-san is suggesting. In particular I think an automatic change of client_encoding isn't particularly a good idea --- wouldn't you have to change it back later, and is there any possibility of creating a security issue from such behavior? Remember that client_encoding *IS* tied to security issues such as backslash escape handling. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Controlling changes in plpgsql variable resolution
Bruce Momjian writes: > Tom Lane wrote: >> 1. Invent a GUC that has the settings backwards-compatible, >> oracle-compatible, throw-error (exact spellings TBD). Factory default, >> at least for a few releases, will be throw-error. Make it SUSET so that >> unprivileged users can't break things by twiddling it; but it's still >> possible for the DBA to set it per-database or per-user. > I don't see the logic to making the setting SUSET. The user wrote the > function; what logic is there to say the resolution rules are not under > their control? That's only sane if you are 100% certain that there could not be a security issue arising from the change of behavior. Otherwise someone could for instance subvert a security-definer function by running it under the setting it wasn't written for. Personally I am not 100% certain of that. > Also, I think to GUC that throws an error or not is a lot safer than one > that changes resolution semantics. Changing resolution semantics sounds > like the autocommit GUC to me. :-O Yeah, that's another reason to not allow it to be changed too easily. > Also, I am not really keen on the "keep it for a few releases" Well, I'm not necessarily saying we would ever change it. Maybe the default could always stay at "error". > ... maybe just error/no error > and using Oracle semantics is the way to go, with 'error' as the > default. I'd personally be entirely happy with that, but people with large plpgsql code bases will not be. They're going to want a backward-compatible setting so that this doesn't become a show stopper for migration to 8.5. Any time you can allow someone to deal with a migration issue later instead of right away, it becomes easier for them to migrate. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
"Greg Sabino Mullane" writes: > I'm sure the casting changes broke more applications and prevented more > people from upgrading than every thing on Ron's list for a clean 9.0 would. > Not that I'm necessarily promoting his idea, but 8.3 was already a > "all-at-once breakage" release. I'm having to support 8.2 because that's the most recent upgrade we can afford on some project here, for this very problem. I don't think removing support for add_missing_from would stop us from upgrading to 8.5 (or call it 9.0) from 8.3+. Or more to the point, that the cost to migrate from pre-8.3 to any 8.[345] releases would change much. As far as breaking historical compatibility where it matters for a next release, I think it boils down to older releases mainteance. The way I see it, if 8.5 or 9.0 breaks a lot of warts but 8.0 (or 8.1) to 8.4 are still maintained the way they are now, this will be just an usual PostgreSQL major upgrade headache (and revenue source for contractants). It seems such a little game changer that I won't care voting for or against it. Please continue considering code maintenance as a higher priority target than upgrade easyness, that seems to open the road for more stability and features in the long run. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
On Tue, Oct 20, 2009 at 10:07 AM, Greg Sabino Mullane wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: RIPEMD160 > > Tom Lane replied to Ron Mayer: >>> Would postgres get considerably cleaner if a hypothetical 9.0 release >>> skipped backward compatibility and removed anything that's only >>> maintained for historical reasons? >> >> Yeah, and our user community would get a lot smaller too :-( >> >> Actually, I think any attempt to do that would result in a fork, >> and a consequent splintering of the community. We can get away >> with occasionally cleaning up individual problematic behaviors >> (example: implicit casts to text), but any sort of all-at-once >> breakage would result in a lot of people Just Saying No. > > That particular example is a very poor one for illustrating your > point. You severely underestimate "get away with" for the implicit > cast changes in 8.3. This was a really big deal for many, many users > of Postgres, and continues to cause many problems to this day. > > I'm sure the casting changes broke more applications and prevented more > people from upgrading than every thing on Ron's list for a clean 9.0 would. > Not that I'm necessarily promoting his idea, but 8.3 was already a > "all-at-once breakage" release. This is a fair point. I think the real issue, though, is that answer to Ron's original question is "No". When backward compatibility gets in the way of cool new features, that's worth considering. But removing backward compatibility just for the sake of removing backward compatibility doesn't really buy us anything. It's basically doing extra work for no benefit and some possible harm. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Tue, Oct 20, 2009 at 9:42 AM, Tom Lane wrote: > Magnus Hagander writes: >> 2009/10/19 Tom Lane : >>> Now we have a user with name equal to password, which no sane security >>> policy will think is a good thing, but the plugin had no chance to >>> prevent it. > >> The big difference is that you need to be superuser to change the name >> of a user, but not to change your own password. > > True, but the superuser doesn't necessarily know what the user has > set his password to. Yeah, but I'm not sure this case is worth worrying about. People who actually care password security are likely to have checks that are substantially stronger than "!= username". ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Tom Lane replied to Ron Mayer: >> Would postgres get considerably cleaner if a hypothetical 9.0 release >> skipped backward compatibility and removed anything that's only >> maintained for historical reasons? > > Yeah, and our user community would get a lot smaller too :-( > > Actually, I think any attempt to do that would result in a fork, > and a consequent splintering of the community. We can get away > with occasionally cleaning up individual problematic behaviors > (example: implicit casts to text), but any sort of all-at-once > breakage would result in a lot of people Just Saying No. That particular example is a very poor one for illustrating your point. You severely underestimate "get away with" for the implicit cast changes in 8.3. This was a really big deal for many, many users of Postgres, and continues to cause many problems to this day. I'm sure the casting changes broke more applications and prevented more people from upgrading than every thing on Ron's list for a clean 9.0 would. Not that I'm necessarily promoting his idea, but 8.3 was already a "all-at-once breakage" release. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation PGP Key: 0x14964AC8 200910201005 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkrdxBEACgkQvJuQZxSWSsgDigCfcEXFz/+4GvcNstAEYh05rkYR RJcAoICN46WCy1jLI9umMfGn5j9taqEt =9Iq7 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] per table random-page-cost?
Greg Smith wrote: > The unfortunate reality of accounts receivable is that reports run > to list people who owe one money happen much more often than posting > payments into the system does. How often do you have to print a list of past due accounts? I've generally seen that done weekly or monthly, in the same places that there are many people standing full time in payment windows just to collect money from those lining up to pay. When they bring in a document which identifies the receivable (like, in our case, a traffic or parking ticket), there's no need to look at any older data in the database. Heck, even our case management applications likely follow the 90% to 95% cache hit pattern in counties which aren't fully cached, since there's a lot more activity on court cases filed this year and last year than on cases filed 30 years ago. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Magnus Hagander writes: > 2009/10/19 Tom Lane : >> Now we have a user with name equal to password, which no sane security >> policy will think is a good thing, but the plugin had no chance to >> prevent it. > The big difference is that you need to be superuser to change the name > of a user, but not to change your own password. True, but the superuser doesn't necessarily know what the user has set his password to. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Tue, Oct 13, 2009 at 4:11 PM, Andrew Dunstan wrote: > >> On the second question, obviously the user can call SET to set a >> value, as I've done for now in psql, however in other DBMSs, it may be >> set in the connection string. My feeling would be to do that, and >> possibly add PQsetApplicationName(PGconn *conn, char *name) and char >> *PQapplicationName(PGconn *conn);. What do others think? >> >> > > > Doing it with a GUC will not be nearly so useful as having it in the wire > protocol, IMNSHO. Just one example: it wouldn't be present in connection > records, because it wouldn't be set yet. I just realised there's a nasty problem with this. In my client application, I can use PQconninfoParse to determine if application_name (or fallback_application_name) are valid connection string options for the version of libpq that I have. However, there is no way that I can see of doing a comparable test in libpq itself, to ensure that the server understands the parameter, so if I connect to an 8.5 server, everything works fine, but if connect to an older server with my new libpq, the connection fails because of the unknown parameter in the startup packet. Am I missing a way around this? -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com PGDay.EU 2009 Conference: http://2009.pgday.eu/start -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ProcessUtility_hook
Itagaki Takahiro escreveu: > We can hook SELECT, INSERT, UPDATE and DELETE with ExecutorXxx_hooks, > but there is no way to hook DDL commands. Can I submit a patch to add > ProcessUtility_hook? > +1. Hey, I was thinking about that yesterday. ;) > pg_stat_statements module also handle DDL commands as same as normal > queries. Fortunately, autovacuum calls call vacuum() directly, > so the statistics will not filled with VACUUM and ANALYZE commands > by autovacuum. > I don't look at the code but maybe there is a way to hook something there too. But I'd leave it alone for now, unless it's few lines of code. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UTF8 with BOM support in psql
On Tue, 2009-10-20 at 14:41 +0900, Itagaki Takahiro wrote: > UTF8 encoding text files with BOM (Byte Order Mark) are commonly > used in Windows, though BOM was designed for UTF16 text originally. > However, psql cannot read such format even if we set client encoding > to UTF8. Is it worth supporting those format in psql? psql doesn't have a problem, but the backend's lexer doesn't parse the BOM as whitespace. Since the lexer is byte-based, it will presumably have problems with anything outside of ASCII that Unicode considers whitespace. > When psql opens a file with -f or \i, it checks first 3 bytes of the > file. If they are BOM, discard the 3 bytes and change client encoding > to UTF8 automatically. While I see that the Unicode standard supports using a UTF-8 encoded BOM as UTF-8 signature, I wonder if those bytes can usefully appear in a leading position in other encodings. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] ProcessUtility_hook
We can hook SELECT, INSERT, UPDATE and DELETE with ExecutorXxx_hooks, but there is no way to hook DDL commands. Can I submit a patch to add ProcessUtility_hook? pg_stat_statements module also handle DDL commands as same as normal queries. Fortunately, autovacuum calls call vacuum() directly, so the statistics will not filled with VACUUM and ANALYZE commands by autovacuum. There might be another usage for the hook. Since we can filter all SQL commands using ExecutorRun_hook and ProcessUtility_hook, so we could write query text for database audit with programmable filter in the hooks. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, pausing recovery
On Tue, 2009-10-20 at 09:43 +0100, Simon Riggs wrote: > On Tue, 2009-10-20 at 10:33 +0300, Heikki Linnakangas wrote: > > At this point, I'd like to cut out all those control functions to > > pause/stop at various points from the patch. > > OK Maybe OK, I should say. Some parts are important for testing HS itself and for debugging situations that occur. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, pausing recovery
On Tue, 2009-10-20 at 10:33 +0300, Heikki Linnakangas wrote: > At this point, I'd like to cut out all those control functions to > pause/stop at various points from the patch. OK -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] per table random-page-cost?
On Tue, Oct 20, 2009 at 1:21 AM, Tom Lane wrote: > If the parameter is defined as "the chance that a page is in cache" >> there is very real physical meaning to it. > > We have no such parameter... What a simple person like me would think would work is: - call the parameter "cached_probability". - Invent a way to store it (I`d actually try to do it the exact same way recent "alter table set statistics distinct" does it) a) less radical idea: replace all usage of random_page_cost with seq_page_cost * cached_probability + random_page_cost * (1-cached_probability) b) more radical idea: b1) invent a new GUC: cached_page_cost b2) replace all usage of seq_page_cost with cached_page_cost * cached_probability + seq_page_cost * (1-cached_probability) b3) replace all usage of random_page_cost with cached_page_cost * cached_probability + random_page_cost * (1-cached_probability) > How would you come up with sensible figures for this hypothetical parameter? select schemaname,relname,heap_blks_hit / cast(heap_blks_read+heap_blks_hit+1 as float) from pg_statio_all_tables Would be a nice starting point. Greetings Marcin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
2009/10/20 Dave Page : > On Mon, Oct 19, 2009 at 9:34 PM, Magnus Hagander wrote: >> 2009/10/19 Dave Page : >>> On Mon, Oct 19, 2009 at 3:42 PM, Massa, Harald Armin wrote: Would'nt this also make sense for PostgreSQL? That is, when no environment is set, and no SET-command is issued, that the application name becomes the default? >>> >>> That needs to be set by the application. As discussed previously, >>> there's no way for libpq to get at argv[0]. >> >> Um, that has to be platform dependent, no? On windows for example, you >> can use GetCommandLine(). > > Yeah, and there's a similar API on *BSD I believe, but nothing standard. Right, but it might be worth investigating using the API that's available on the platform, if one is. It's a fairly simple operation after all, so it won't take huge amounts of platform-specific code. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v2
On Mon, Oct 19, 2009 at 9:34 PM, Magnus Hagander wrote: > 2009/10/19 Dave Page : >> On Mon, Oct 19, 2009 at 3:42 PM, Massa, Harald Armin wrote: >>> >>> Would'nt this also make sense for PostgreSQL? That is, when no environment >>> is set, and no SET-command is issued, that the application name becomes the >>> default? >> >> That needs to be set by the application. As discussed previously, >> there's no way for libpq to get at argv[0]. > > Um, that has to be platform dependent, no? On windows for example, you > can use GetCommandLine(). Yeah, and there's a similar API on *BSD I believe, but nothing standard. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com PGDay.EU 2009 Conference: http://2009.pgday.eu/start -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, pausing recovery
Simon Riggs wrote: > On Fri, 2009-10-16 at 09:33 +0300, Heikki Linnakangas wrote: >> If you pause recovery, and then continue, we reset to "target none" >> mode, even if a stopping point was set previously. > > Yes, currently. Resetting the target mode will do what you want, rather > than continue. > >> That doesn't seem right to me. > > We can define these things various ways and I'm happy to change it to be > the way people find most intuitive and useful. You're the first person > to give me feedback on how these things should work, so I'm happy to > hear how you would like it to work. Thinking about the interface a bit more: pg_recovery_advance() isn't very useful except for debugging hot standby itself. Normal users don't care about individual WAL records, since the effect of record X isn't visible until the transaction commit record anyway. A function to advance to next *commit* record would make a lot more sense, allowing you to step through transactions one at a time. A function to step forward X minutes would be useful too. At this point, I'd like to cut out all those control functions to pause/stop at various points from the patch. They're not required for Hot Standby, and the less stuff I have to review right now the better. Don't get me wrong, I absolutely love those functions, it's a great feature that I do want to get in before release, but we need more discussion on the exact set of functions and how they should work. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
2009/10/19 Tom Lane : > I wrote: >> A server-side plugin can provide a guarantee that there are no bad >> passwords (for some value of bad, and with some possible adverse >> consequences). We don't have that today. > > BTW, it strikes me that ALTER USER RENAME introduces an interesting > hazard for such a plugin. Consider > > CREATE USER joe; > ALTER USER joe PASSWORD joe; -- presumably, plugin will reject this > ALTER USER joe PASSWORD mumblefrotz; -- assume this is considered OK > ALTER USER joe RENAME TO mumblefrotz; > > Now we have a user with name equal to password, which no sane security > policy will think is a good thing, but the plugin had no chance to > prevent it. The big difference is that you need to be superuser to change the name of a user, but not to change your own password. I know for example the Windows password policy thing has the same issue - if you rename the user, it doesn't have the password around to check, but you are an administrator so that's considered ok. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?
Aidan Van Dyk wrote: > * Tom Lane [091019 18:45]: >> Actually, I think any attempt to do that would result in a fork, >> and a consequent splintering of the community. We can get away >> with occasionally cleaning up individual problematic behaviors >> (example: implicit casts to text), but any sort of all-at-once >> breakage would result in a lot of people Just Saying No. > > I don't know... What if this hypothetical "baggage-free" version came > with configurable syncrhonous master-slave replication, writable CTEs, > and everything ;-) I know your joking, but what would actually happen is that the people maintaining the backwards-compatible fork would simply port over those features as well, or do the fork after those features have went in. None of the historical features or warts we maintain for backwards-compatibility are in the way of those cool new features. It makes sense to remove old warts whenever they do get in the way. Like add_missing_from that's been discussed at the moment. Other than that, there's very little harm in keeping them. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
