Re: [HACKERS] New VACUUM FULL
Happy New Year, On Mon, 2010-01-04 at 11:50 +0900, Takahiro Itagaki wrote: Robert Haas robertmh...@gmail.com wrote: So, what is the roadmap for getting this done? It seems like to get rid of VFI completely, we would need to implement something like what Tom described here: http://archives.postgresql.org/pgsql-hackers/2009-09/msg00249.php I'm not sure whether the current patch is a good intermediate step towards that ultimate goal, or whether events have overtaken it. I think the most desirable roadmap is: 1. Enable CLUSTER to non-critical system catalogs. 2. Also enable CLUSTER and REINDEX to critical system catalogs. 3. Remove VFI and re-implement VACUUM FULL with CLUSTER-based approach. It should be also optimized as Simon's suggestion. My patch was intended to do 3, but we should not skip 1 and 2. In the roadmap, we don't have two versions of VACUUM FULL (INPLACE and REWRITE) at a time. I think we can do 1 immediately. The comment in cluster says might work, and I also think so. CLUSTERable toast tables are obviously useful. You make some good points. I would prefer this slightly modified version 1. Commit your patch, as-is (you/me) 2. Work on infrastructure for VFC (VACUUM FULL using CLUSTER) for system relations (Simon) 3. Enable CLUSTER and REINDEX on critical system catalogs (Itagaki) 4. Optimise VFC, as discussed earlier (Itagaki) I have put names in brackets, but this is just a suggestion. This differs from your sequence in only a few ways * We implement the basic VFC now, so everybody knows what we have * We separate the infrastructure for (2) from the enabling of this infrastructure for CLUSTER and REINDEX. There may be additional issues to consider for those cases and we should think through and test them as a different task * We do not remove VFI in this release This is a more cautious approach. Completely removing VFI in this release is a big risk that we need not take; we have little to gain from doing so and putting it back again will be harder. I am always keen to push forwards when a new feature is worthwhile, but cleaning up code is not an important thing this late in release cycle. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: Add hstore_to_json()
2010/1/4 David E. Wheeler da...@kineticode.com:
On Jan 3, 2010, at 4:18 PM, Hitoshi Harada wrote:
That sounds good and seems possible, as far as operator returns JSON
always. Perhaps every JSON fetching returns JSON even if the result
would be a number. You can cast it.
% SELECT ('{foo:{bar:[a,b,c]}}' - '[foo][1]')::text;
1
-
b
No, because 'b' isn't valid JSON. So if we want an interface that returns
scalars, they can't be JSON.
AFAIK string value can be parsed as JSON. At least my local v8 shell answers:
JSON.stringify({foo: {bar: [a, b, c]}})
{foo:{bar:[a,b,c]}}
JSON.stringify(b)
b
Regards,
--
Hitoshi Harada
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] TODO: Allow substring/replace() to get/set bit values
Hi all, I would like to work on Allow substring/replace() to get/set bit values, since it looks like a simple task. The item is not marked as easy on the TODO though. Before proceding to a discussion on how this functions should be implemented (I got from the messages on the mailing list that bit substring/replace functions should do it) I would like to know if it's a complicated task. Leonardo -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUG?] strange behavior in ALTER TABLE ... RENAME TO on inherited columns
The method I suggested would allow the necessary information to be extracted during the initial search for child tables, which we have to do anyway. find_all_inheritors() returns a clean list which does not contain duplicated OID of the inherited relation, so it seems to me we need to change the function prototype but it affects other parts, or to add a new function which also returns number of duplications, not only OIDs. Or, we can call find_inheritance_children() in renameatt() as if find_all_inheritors() doing except for list_concat_unique_oid(). The attached patch modified the condition to prevent renaming. It computes an expected inhcount for each child relations on the initial call of the renameatt() for the parent relation. The find_all_inheritors_with_inhcount() returns OID of the inherited relations and the expected inhcoundt. If a child relation has diamond inheritance tree, it has its expected inhcount larger than 1. This patch raises an error, if pg_attribute.inhcount is larger than the expected inhcount. It can be happen when the attribute to be renamed is merged from any other unrelated relations in the child relations. See the example: postgres=# CREATE TABLE t1 (a int); CREATE TABLE postgres=# CREATE TABLE t2 (b int) inherits (t1); CREATE TABLE postgres=# CREATE TABLE t3 (c int) inherits (t1); CREATE TABLE postgres=# CREATE TABLE t4 (d int) inherits (t2, t3); NOTICE: merging multiple inherited definitions of column a CREATE TABLE postgres=# ALTER TABLE t1 RENAME a TO x; ALTER TABLE postgres=# \d t4 Table public.t4 Column | Type | Modifiers +-+--- x | integer | b | integer | c | integer | d | integer | Inherits: t2, t3 We can rename a of t1, t2, t3 and t4 correctly, although t4.a has inherited from multiple relations. postgres=# CREATE TABLE s1 (x int); CREATE TABLE postgres=# CREATE TABLE t5 (e int) inherits (t2, t3, s1); NOTICE: merging multiple inherited definitions of column x NOTICE: merging multiple inherited definitions of column x CREATE TABLE postgres=# ALTER TABLE t1 RENAME x TO y; ERROR: cannot rename multiple inherited column x But, the new t5 prevent to rename x to y, because t5.x is also inherited from the s1 and merged. So, its inhcount is 3 larger than expected inhcount (=2). postgres=# SELECT attname, attinhcount FROM pg_attribute where attname='x' and attrelid='t5'::regclass; attname | attinhcount -+- x | 3 (1 row) Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com pgsql-fix-inherit-rename.3.patch Description: application/octect-stream -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
I: [HACKERS] TODO: Allow substring/replace() to get/set bit values
Re-reading the docs it looks like the only thing missing is get/set_bit for bit string. Substring is already implemented for bit string, and I don't really know if replace is useful at all. (sorry if the other mail came with a different sender name) Leonardo I would like to work on Allow substring/replace() to get/set bit values, since it looks like a simple task. The item is not marked as easy on the TODO though. Before proceding to a discussion on how this functions should be implemented (I got from the messages on the mailing list that bit substring/replace functions should do it) I would like to know if it's a complicated task. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting psql varible as identifier
hello 2010/1/2 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: here is patch I looked at this patch a bit, and I think the real problem with it is that it's not multibyte safe. You've copied backend code that is allowed to assume it's in a safe encoding (ie, one where multibyte characters can't contain non-high-bit-set bytes). This is not okay on the client side, see SJIS and similar encodings. Where you need to start out is by cloning PQescapeStringConn, which does a similar type of transformation correctly even in unsafe encodings. I think we'd agreed upthread that libpq should provide PQescapeIdentifier functionality anyhow. I am looking on psql directory. Now I found, so in this directory is linked dumputil.c - It could little bit to help us. I have one question. If I understand well, the function fmtId isn't multibyte safe? So why is possible to use it in pg_dump? Pavel Once you've actually read that code, you'll realize that it's okay to treat the error result as a warning, which resolves the other point of concern. Just print the message and use the result anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] invalid UTF-8 via pl/perl
On sön, 2010-01-03 at 18:40 -0500, Andrew Dunstan wrote: Incidentally, I guess we need to look at plpython and pltcl for similar issues. I confirm that the same issue exists in plpython. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] why fmtId isn't multibyte safe?
Hello I am looking for some quoting implementation in PostgreSQL. I cannot to understand, why we can to use multibyte unsafe functions quote_identifier or fmtId, and we have to use multibyte safe functions quote_literal (and similar). Can somebody explain it? Thank you Pavel Stehule -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Backup history file should be replicated in Streaming Replication?
Heikki Linnakangas wrote: Here's is modified patch that adds a new backupStartPoint field to pg_control for that + some other minor editorialization. I've committed this now. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting psql varible as identifier
Hello I talked with Hitoshi Harada, and fmtId function is safe (minimally for Japanese case). This function working without any errors, so we must not duplicate a code. Pavel 2010/1/4 Pavel Stehule pavel.steh...@gmail.com: hello 2010/1/2 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: here is patch I looked at this patch a bit, and I think the real problem with it is that it's not multibyte safe. You've copied backend code that is allowed to assume it's in a safe encoding (ie, one where multibyte characters can't contain non-high-bit-set bytes). This is not okay on the client side, see SJIS and similar encodings. Where you need to start out is by cloning PQescapeStringConn, which does a similar type of transformation correctly even in unsafe encodings. I think we'd agreed upthread that libpq should provide PQescapeIdentifier functionality anyhow. I am looking on psql directory. Now I found, so in this directory is linked dumputil.c - It could little bit to help us. I have one question. If I understand well, the function fmtId isn't multibyte safe? So why is possible to use it in pg_dump? Pavel Once you've actually read that code, you'll realize that it's okay to treat the error result as a warning, which resolves the other point of concern. Just print the message and use the result anyway. regards, tom lane variable_escaping.diff Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql: When estimating the selectivity of an inequality column
On Mon, Jan 4, 2010 at 2:44 AM, Tom Lane t...@postgresql.org wrote: Log Message: --- When estimating the selectivity of an inequality column constant or column constant, and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. So This is pretty cool but it's worth discussing the downsides. This is the second bit of live information the planning process is peeking at to supplement the statistics (the first is the physical size of the table). There are two use cases that I see being negatively impacted by these features, neither of which we support currently but I expect we'll eventually support and when we do we'll have to work around these problems. The two problematic use cases I see are: plan stability and exporting statistics to a another machine to reproduce behaviour on a test machine. I'm not sure what approach we'll have to take to work around these problems. Perhaps we just need a way to disable checking these bits of information and have a fallback strategy? perhaps we need a hook for each of these fetches so you can store the size and upper bound of the table on the production system you're trying to replicate the behaviour of and a hook which substitutes these values for the dynamic value? Perhaps the hook could even use a dbilink connection to get the live production values. I also wonder if we want to run these lookups on every single planner invocation. If the table is being rarely updated analyze will never fire and the stats value will never be updated. I don't think we want the planner doing write operations to the stats either though. It would be nice if there was a low-cost mode of operation for ANALYZE which simulates precisely this update, the planner could send a stats message saying that autovacuum should schedule one of these operations sometime even if it doesn't see a need for a full analyze run. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add subdirectory support for DATA/DOCS with PGXS
Tom Lane wrote: Why do DOCS still go into doc/contrib? Shouldn't that become doc/$MODULEDIR for consistency? Hmmm it looks as if the code was correct but I missed the comment at the top of the file. Sorry for the confusion - revised version attached. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs diff --git a/src/makefiles/pgxs.mk b/src/makefiles/pgxs.mk index a83dad3..76b585f 100644 --- a/src/makefiles/pgxs.mk +++ b/src/makefiles/pgxs.mk @@ -19,11 +19,14 @@ # # MODULES -- list of shared objects to be build from source file with # same stem (do not include suffix in this list) -# DATA -- random files to install into $PREFIX/share/contrib -# DATA_built -- random files to install into $PREFIX/share/contrib, +# MODULEDIR -- subdirectory under contrib into which DATA and DOCS are +# installed (if not set, DATA and DOCS files are installed directly +# into the contrib/ directory) +# DATA -- random files to install into $PREFIX/share/$MODULEDIR +# DATA_built -- random files to install into $PREFIX/share/$MODULEDIR, # which need to be built first # DATA_TSEARCH -- random files to install into $PREFIX/share/tsearch_data -# DOCS -- random files to install under $PREFIX/doc/contrib +# DOCS -- random files to install under $PREFIX/doc/$MODULEDIR # SCRIPTS -- script files (not binaries) to install into $PREFIX/bin # SCRIPTS_built -- script files (not binaries) to install into $PREFIX/bin, # which need to be built first @@ -86,12 +89,19 @@ include $(top_srcdir)/src/Makefile.shlib all: all-lib endif # MODULE_big +ifndef MODULEDIR +datamoduledir = contrib +docmoduledir = contrib +else +datamoduledir = $(MODULEDIR) +docmoduledir = $(MODULEDIR) +endif install: all installdirs ifneq (,$(DATA)$(DATA_built)) @for file in $(addprefix $(srcdir)/, $(DATA)) $(DATA_built); do \ - echo $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/contrib'; \ - $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/contrib'; \ + echo $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/$(datamoduledir)'; \ + $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/$(datamoduledir)'; \ done endif # DATA ifneq (,$(DATA_TSEARCH)) @@ -109,8 +119,8 @@ endif # MODULES ifdef DOCS ifdef docdir @for file in $(addprefix $(srcdir)/, $(DOCS)); do \ - echo $(INSTALL_DATA) $$file '$(DESTDIR)$(docdir)/contrib'; \ - $(INSTALL_DATA) $$file '$(DESTDIR)$(docdir)/contrib'; \ + echo $(INSTALL_DATA) $$file '$(DESTDIR)$(docdir)/$(docmoduledir)'; \ + $(INSTALL_DATA) $$file '$(DESTDIR)$(docdir)/$(docmoduledir)'; \ done endif # docdir endif # DOCS @@ -137,7 +147,7 @@ endif # MODULE_big installdirs: ifneq (,$(DATA)$(DATA_built)) - $(MKDIR_P) '$(DESTDIR)$(datadir)/contrib' + $(MKDIR_P) '$(DESTDIR)$(datadir)/$(datamoduledir)' endif ifneq (,$(DATA_TSEARCH)) $(MKDIR_P) '$(DESTDIR)$(datadir)/tsearch_data' @@ -147,7 +157,7 @@ ifneq (,$(MODULES)) endif ifdef DOCS ifdef docdir - $(MKDIR_P) '$(DESTDIR)$(docdir)/contrib' + $(MKDIR_P) '$(DESTDIR)$(docdir)/$(docmoduledir)' endif # docdir endif # DOCS ifneq (,$(PROGRAM)$(SCRIPTS)$(SCRIPTS_built)) @@ -161,7 +171,7 @@ endif # MODULE_big uninstall: ifneq (,$(DATA)$(DATA_built)) - rm -f $(addprefix '$(DESTDIR)$(datadir)'/contrib/, $(notdir $(DATA) $(DATA_built))) + rm -f $(addprefix '$(DESTDIR)$(datadir)'/$(datamoduledir)/, $(notdir $(DATA) $(DATA_built))) endif ifneq (,$(DATA_TSEARCH)) rm -f $(addprefix '$(DESTDIR)$(datadir)'/tsearch_data/, $(notdir $(DATA_TSEARCH))) @@ -170,7 +180,7 @@ ifdef MODULES rm -f $(addprefix '$(DESTDIR)$(pkglibdir)'/, $(addsuffix $(DLSUFFIX), $(MODULES))) endif ifdef DOCS - rm -f $(addprefix '$(DESTDIR)$(docdir)'/contrib/, $(DOCS)) + rm -f $(addprefix '$(DESTDIR)$(docdir)'/$(docmoduledir)/, $(DOCS)) endif ifdef PROGRAM rm -f '$(DESTDIR)$(bindir)/$(PROGRAM)$(X)' -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] invalid UTF-8 via pl/perl
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: andrew=# select 'a' || invalid_utf_seq() || 'b'; ERROR: invalid byte sequence for encoding UTF8: 0xd0 HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by client_encoding. CONTEXT: PL/Perl function invalid_utf_seq That hint seems rather misleading. I'm not sure what we can do about it though. If we set the noError param on pg_verifymbstr() we would miss the error message that actually identified the bad data, so that doesn't seem like a good plan. Yeah, we want the detailed error info. The problem is that the hint is targeted to the case where we are checking data coming from the client. We could add another parameter to pg_verifymbstr to indicate the context, perhaps. I'm not sure how to do it exactly --- just a bool that suppresses the hint, or do we want to make a provision for some other hint or detail message? This is a mess. It affects four or five levels of visible functions that are called in about 18 files. How about we just change the hint so it also refers to the possibility that the data comes from a PL? That would save lots of trouble. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql: When estimating the selectivity of an inequality column
Greg Stark wrote: I also wonder if we want to run these lookups on every single planner invocation. If the table is being rarely updated analyze will never fire and the stats value will never be updated. I don't think we want the planner doing write operations to the stats either though. It would be nice if there was a low-cost mode of operation for ANALYZE which simulates precisely this update, the planner could send a stats message saying that autovacuum should schedule one of these operations sometime even if it doesn't see a need for a full analyze run. Maybe autovac could run such a cheap ANALYZE frequently on tables with large number of inserts (but not large enough to trigger a regular ANALYZE) ... say a fixed number of tuples (not depending on pg_class.reltuples) -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting psql varible as identifier
Pavel Stehule pavel.steh...@gmail.com writes: I have one question. If I understand well, the function fmtId isn't multibyte safe? So why is possible to use it in pg_dump? pg_dump is only guaranteed to work correctly in the server encoding. If you force it to use a client_encoding different from the server's, it might or might not work, for reasons far beyond that one --- the big problem usually is data containing characters that have no equivalent in the client encoding. So I'm not particularly excited about whether fmtId is multibyte safe within pg_dump. If we were to try to use it in more general contexts, it would probably need more work. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] invalid UTF-8 via pl/perl
Andrew Dunstan and...@dunslane.net writes: How about we just change the hint so it also refers to the possibility that the data comes from a PL? That would save lots of trouble. Maybe just lose the hint altogether. It's not adding that much, and I seem to recall that there have already been complaints about other cases where it's misleading. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Bruce Momjian wrote: pg_migrator has become more popular recently, so it seems time to look at some enhancements that would improve pg_migrator. None of these are required, but rather changes that would be nice to have: 1) Right now pg_migrator preserves relfilenodes for TOAST files because this is required for proper migration. Now that we have shown that strategically-placed global variables with a server-side function to set them is a viable solution, it would be nice to preserve all relfilenodes from the old server. This would simplify pg_migrator by no long requiring place-holder relfilenodes or the renaming of TOAST files. A simpler solution would just be to allow TOAST table creation to automatically remove placeholder files and create specified relfilenodes via global variables. Getting rid of the need for placeholders is a good idea. +1 on getting TOAST tables created with the correct relfilenode from the start. I don't know that preserving any other relfilenode is useful; however if it means you no longer have to rename the files underlying each table, it would probably also be a good idea. (I don't know how does pg_migrator deal with such things currently -- does it keep a map of table name to relfilenode?) 2) Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? -1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. 3) There is no easy way to analyze all databases. vacuumdb --analyze does analyze _and_ vacuum, which for an 8.4 to 8.5 migration does an unnecessary vacuum. Right now I recommend ANALYZE in every database, but it would be nice if there were a single command which did this. +1 for vacuumdb --analyze-only 4) I have implemented the ability to run pg_migrator --check on a live old server. However, pg_migrator uses information from controldata to check things, and it also needs xid information that is only available via pg_resetxlog -n(no update) to perform the migration. Unfortunately, pg_resetxlog -n cannot be run on a live server, so pg_migrator runs pg_controldata for --check and pg_resetxlog -n for real upgrades. It would simplify pg_migrator if I would run pg_resetxlog -n on a live server, but I can understand if people don't want to do that because the xid information reported on a live server is inaccurate. What xid info does it need? Would it be good enough to use the next XID from most recent checkpoint from pg_controldata? It is a bit outdated, but can't you simply add some value to it to have a safety margin? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Alvaro Herrera alvhe...@commandprompt.com writes: Getting rid of the need for placeholders is a good idea. +1 on getting TOAST tables created with the correct relfilenode from the start. I don't know that preserving any other relfilenode is useful; however if it means you no longer have to rename the files underlying each table, it would probably also be a good idea. I think this is an all-or-nothing proposition: if you try to preserve only some relfilenodes, you risk collisions with automatically assigned ones. It's just like the situation with pg_type OIDs. I concur that trying to preserve them looks like it would be less work than the current method. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and non-blocking I/O
I've merged the replication branch with PostgreSQL CVS HEAD now, including the patch for end-of-backup WAL records I committed earlier today. See 'replication' branch in my git repository. There's also a couple of other small changes: I believe the SSL stuff isn't really necessary, so I removed it. I also moved the START_REPLICATION phase from the walreceiver main loop to WalRcvConnect, as it's simpler that way. I will continue reviewing.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Change to config.pl processing in the msvc build environment
Andrew Dunstan wrote: I don't think the parens on trailing conditions issue is anything other than just a matter of taste. Agreed -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
On Mon, Jan 4, 2010 at 3:04 AM, Simon Riggs si...@2ndquadrant.com wrote: This is a more cautious approach. Completely removing VFI in this release is a big risk that we need not take; we have little to gain from doing so and putting it back again will be harder. I am always keen to push forwards when a new feature is worthwhile, but cleaning up code is not an important thing this late in release cycle. I don't have a strong opinion one way or the other on whether we should remove VFI this release cycle, but I thought the reason why there was pressure to do that was because we will otherwise need to make changes to Hot Standby to cope with VFI. Or in other words, I thought that in order to wrap a release we would need to do one of (1) remove VFI and (2) fix HS to cope with VFI, and maybe there was a theory that the former was easier than the latter. But it's possible I may have totally misunderstood the situation. What is your thought on how to handle this? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Setting oom_adj on linux?
I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/proc.txt;h=220cc6376ef80e0c9bcfec162d45552e729cdf5a;hb=45d28b097280a78893ce25a5d0db41e6a2717853 section 3.1. To get the best benefit, I think it needs to be done in cooperation between the startup scripts and PostgreSQL. We'd want -17 (never oom kill) on the postmaster, but some different value on regular backends (since if it has to kill someone, it's better to pick a regular backend so we can do a controlled restart). Only root can drop the value, so the startup script needs to be part of it. But if we then want to increase it for sub-processes, that'd require something in the backend itself... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
Robert Haas escribió: Hmm, I see this needs to be rebased over Tom's latest changes, but the conflict I got was in syscache.h, rather than syscache.c. Not sure if that's what you were going for or if there's another issue. Updated patch attached. FWIW I think the reloptions code in this patch is sane enough. The fact that it was this easily written means that the API for reloptions was reasonably chosen, thanks :-) Hmm, it seems we're missing a need_initialization = false at the bottom of initialize_reloptions ... I'm wondering what happened to that?? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql: When estimating the selectivity of an inequality column
On Mon, Jan 4, 2010 at 2:50 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Maybe autovac could run such a cheap ANALYZE frequently on tables with large number of inserts (but not large enough to trigger a regular ANALYZE) ... say a fixed number of tuples (not depending on pg_class.reltuples) Well that might cut down on the number of plans that need to do it themselves. But I'm more concerned about a database that *doesn't* have a frequent number of inserts. Such a database should presumably trigger a real analyze reasonably quickly. But consider a database that has one new record inserted per day but thousands of queries per minute looking up the maximum value in the table. This change has basically doubled the work that query needs to do since the planner now needs to do the same lookup that the query itself was going to do. And autovacuum won't fire for a long long time against this table. Admittedly the fact that there is 100% overhead isn't terribly interesting since it's really a fixed overhead and only 100% if that query happens to be correspondingly cheap. But it's still annoying to me that we'll potentially never figure out what the new stats should be and stop doing the lookup no matter how long the new row sits there unless some unrelated activity triggers a real analyze. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Grr. I had zero recollectoin of that :S Can't find a useful consensus though? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Mon, 4 Jan 2010 16:57:28 +0100, Magnus Hagander mag...@hagander.net wrote: On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Grr. I had zero recollectoin of that :S Can't find a useful consensus though? I don't think we should set a setting like that automatically. Perhaps a warning on startup? Joshua D. Drake -- PostgreSQL - XMPP: jdrake(at)jabber(dot)postgresql(dot)org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander wrote: On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Grr. I had zero recollectoin of that :S Can't find a useful consensus though? It is probably worth trying to protect the postmaster in the init script. Beyond that things probably start to get fairly difficult. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Thoughts on statistics for continuously advancing columns
On Wed, 2009-12-30 at 17:16 +0100, Tom Lane wrote: I think the cleanest solution to this would be to make ANALYZE cheaper, perhaps by finding some way for it to work incrementally. What if when inserting/deleting a tuple, some random sample of them would be passed into an auto-analyze buffer ? Then a special process (the auto-analyze daemon) would process them and update the statistics incrementally based on the new values found (which might or might not be mathematically feasible). The overhead for each backend process would be kept in limits by the rate at which you randomly send or not send the change to the analyze buffer. The processing overhead would be kept in limits by the processing rate of the auto-analyze process, which can be made to periodically sleep or it could be made to span multiple processes (on multiprocessor systems). If the buffer is full, then you skip putting in it... so it also could autotune itself to a sustainable rate. Of course as with all my other posts on hackers, this is all mostly hand-waving, I have no clue about the feasibility of all this with regard to the current state of the code (which I didn't read, I unfortunately found myself hating reading C code beyond reason, and writing any of it till now resumed to copy-paste-modify). Cheers, Csaba. Csaba Nagy Software Engineer eCircle P: +49 (0)89 / 120 09-783 | F: +49 (0)89 / 120 09-750 E: c.n...@ecircle.com Nymphenburger Str. 86, 80636 München Stay in touch Web: www.ecircle.com/de | Newsletter: www.ecircle.com/index.php?id=63L=0 Für Hilfe mit dem eC-messenger wenden Sie sich bitte an unseren Support: support...@ecircle.com. Neuste Untersuchungen Ein unschlagbares Doppel: E-mail-Marketing Webanalyse Download Whitepaper: www.ecircle.com/index.php?id=61L=0 eCircle AG, HRB 136 334, Handelsregister München Vorstand: Volker Wiewer (Vorsitzender), Thomas Wilke, Lars Wössner, Alexander Meyer Vorsitzender des Aufsichtsrates: Dr. Mark Wössner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Mon, Jan 4, 2010 at 17:07, Andrew Dunstan and...@dunslane.net wrote: Magnus Hagander wrote: On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Grr. I had zero recollectoin of that :S Can't find a useful consensus though? It is probably worth trying to protect the postmaster in the init script. Beyond that things probably start to get fairly difficult. Right. But AFAICS (though I haven't tested with -17), it will become inherited to children, which is something we'd want to *undo*, no? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander wrote: On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Can't find a useful consensus though? In http://archives.postgresql.org/pgsql-hackers/2008-02/msg00049.php Tom points out that while you could make this adjustment in the init scripts for PostgreSQL, actually doing so is quite questionable as a packaging decision. That's where that thread ended as far as I was concerned. The best I think anyone could do here is to add such a capability into some of the init scripts, but it would probably need to be disabled by default. Since that sort of defeats the purpose of the change, I'm not sure what the benefit there is--if you have to turn it on, you might as well do something at a higher level instead. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com
Re: [HACKERS] Add subdirectory support for DATA/DOCS with PGXS
Mark Cave-Ayland mark.cave-ayl...@siriusit.co.uk writes: Hmmm it looks as if the code was correct but I missed the comment at the top of the file. Sorry for the confusion - revised version attached. Applied with minor fixups (mostly improving the documentation, which was not in very good shape beforehand...) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander wrote: On Mon, Jan 4, 2010 at 17:07, Andrew Dunstan and...@dunslane.net wrote: Magnus Hagander wrote: On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Grr. I had zero recollectoin of that :S Can't find a useful consensus though? It is probably worth trying to protect the postmaster in the init script. Beyond that things probably start to get fairly difficult. Right. But AFAICS (though I haven't tested with -17), it will become inherited to children, which is something we'd want to *undo*, no? [experiments] Yes, darnit, you're right. But it looks like the oom_adj file can be set to the default by the process owner: [and...@sophia ~]$ ls -l /proc/6520/oom_adj -rw-r--r-- 1 andrew andrew 0 2010-01-04 12:37 /proc/6520/oom_adj [and...@sophia ~]$ cat /proc/6520/oom_adj 0 [and...@sophia ~]$ id uid=500(andrew) gid=500(andrew) groups=10(wheel),500(andrew) [and...@sophia ~]$ echo -17 /proc/6520/oom_adj -bash: echo: write error: Permission denied [and...@sophia ~]$ echo 0 /proc/6520/oom_adj [and...@sophia ~]$ echo -17 /proc/6520/oom_adj -bash: echo: write error: Permission denied [and...@sophia ~]$ But that would be a pain to have to do. OTOH, disabling the OOM killer is not always an option. I recently tried it on one system and had to revert it rapidly because the system stopped working in minutes. Some software just doesn't live well in such environments, sadly. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Mon, Jan 4, 2010 at 17:40, Andrew Dunstan and...@dunslane.net wrote: Magnus Hagander wrote: On Mon, Jan 4, 2010 at 17:07, Andrew Dunstan and...@dunslane.net wrote: Magnus Hagander wrote: On Mon, Jan 4, 2010 at 16:45, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://archives.postgresql.org/message-id/20080201223336.GC24780%40alvh.no-ip.org Grr. I had zero recollectoin of that :S Can't find a useful consensus though? It is probably worth trying to protect the postmaster in the init script. Beyond that things probably start to get fairly difficult. Right. But AFAICS (though I haven't tested with -17), it will become inherited to children, which is something we'd want to *undo*, no? [experiments] Yes, darnit, you're right. But it looks like the oom_adj file can be set to the default by the process owner: [and...@sophia ~]$ ls -l /proc/6520/oom_adj -rw-r--r-- 1 andrew andrew 0 2010-01-04 12:37 /proc/6520/oom_adj [and...@sophia ~]$ cat /proc/6520/oom_adj 0 [and...@sophia ~]$ id uid=500(andrew) gid=500(andrew) groups=10(wheel),500(andrew) [and...@sophia ~]$ echo -17 /proc/6520/oom_adj -bash: echo: write error: Permission denied [and...@sophia ~]$ echo 0 /proc/6520/oom_adj [and...@sophia ~]$ echo -17 /proc/6520/oom_adj -bash: echo: write error: Permission denied [and...@sophia ~]$ But that would be a pain to have to do. OTOH, disabling the OOM killer is not always an option. I recently tried it on one system and had to revert it rapidly because the system stopped working in minutes. Some software just doesn't live well in such environments, sadly. Right. Which is why I like the idea of disabling the OOM killer for the *postmaster*, but not the regular backends. Gives it a chance to recover. It's not nice, but it's better than nothing. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Greg Smith g...@2ndquadrant.com writes: In http://archives.postgresql.org/pgsql-hackers/2008-02/msg00049.php Tom points out that while you could make this adjustment in the init scripts for PostgreSQL, actually doing so is quite questionable as a packaging decision. I just wondered if it would be questioned, I didn't say there was a problem. However, the long and the short of this is that we can't do anything without the close cooperation of an init script. I think that moves it out of the realm of what Postgres as a project should be doing. It seems more like a patch that the Linux-based packagers should be carrying. Memo to self: get off duff and prepare such a patch for the Red Hat/Fedora packages. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander wrote: Right. Which is why I like the idea of disabling the OOM killer for the *postmaster*, but not the regular backends. Gives it a chance to recover. It's not nice, but it's better than nothing. It doesn't sound like the init script can reenable the killer for the child processes though. So, if there's anything that the core code ought to do, is re-enable OOM-killer for postmaster children, after being disabled by the initscript. BTW, is it possible for pg_ctl to disable OOM-killer? I guess not, since it's not run by root ... Is there a way to disable memory overcommit for particular processes? That would be very useful -- just disable overcommit for all Postgres processes, and there shouldn't be much need to enable the killer for backends. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
On Mon, 2010-01-04 at 10:31 -0500, Robert Haas wrote: On Mon, Jan 4, 2010 at 3:04 AM, Simon Riggs si...@2ndquadrant.com wrote: This is a more cautious approach. Completely removing VFI in this release is a big risk that we need not take; we have little to gain from doing so and putting it back again will be harder. I am always keen to push forwards when a new feature is worthwhile, but cleaning up code is not an important thing this late in release cycle. I don't have a strong opinion one way or the other on whether we should remove VFI this release cycle, but I thought the reason why there was pressure to do that was because we will otherwise need to make changes to Hot Standby to cope with VFI. What I should have said, in addition: VFI will be kept as a non-default option, in case it is required. We will document that use of VFI will not work correctly with HS and that its use is deprecated and should be in emergencies only in any case. I will enjoy removing VFI when that eventually occurs, but its not a priority. (And if you think, why keep it? I'll say - how else can we run a VFI - not by a stored proc, certainly). -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Mon, Jan 4, 2010 at 17:55, Alvaro Herrera alvhe...@commandprompt.com wrote: Magnus Hagander wrote: Right. Which is why I like the idea of disabling the OOM killer for the *postmaster*, but not the regular backends. Gives it a chance to recover. It's not nice, but it's better than nothing. It doesn't sound like the init script can reenable the killer for the child processes though. So, if there's anything that the core code ought to do, is re-enable OOM-killer for postmaster children, after being disabled by the initscript. Yeah, that's why the backend code would need to be involved. BTW, is it possible for pg_ctl to disable OOM-killer? I guess not, since it's not run by root ... No, it has to run as root. Is there a way to disable memory overcommit for particular processes? That would be very useful -- just disable overcommit for all Postgres processes, and there shouldn't be much need to enable the killer for backends. Not that I've been able to find. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Alvaro Herrera alvhe...@commandprompt.com writes: Is there a way to disable memory overcommit for particular processes? I would think not --- the very essence of overcommit is that you're promising more total memory than the system has got, and that's inherently a global proposition. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander wrote: On Mon, Jan 4, 2010 at 17:55, Alvaro Herrera alvhe...@commandprompt.com wrote: BTW, is it possible for pg_ctl to disable OOM-killer? I guess not, since it's not run by root ... No, it has to run as root. We could at least make it work on Windows, since it is often run as Administrator and drops privileges afterwards ... ... oh, wait ... -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] More frame options in window functions
2009/12/31 Hitoshi Harada umi.tan...@gmail.com: Attached is the fix pointed out in the previous CommitFest plus RANGE offset support. Improved version attached. In this revision I fixed type mismatch case like ORDER BY int4_data RANGE BETWEEN int8_data PRECEDING Update of comments and fix typos in documents are also included. Regards, -- Hitoshi Harada more_frame_options.20100105.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Windows x64 [repost]
On Fri, Jan 1, 2010 at 20:45, Magnus Hagander mag...@hagander.net wrote: On Fri, Dec 4, 2009 at 11:42, Tsutomu Yamada tsut...@sraoss.co.jp wrote: 2) use appropriate macro and datatypes for Windows API. enables more than 32bits shared memory. Are you sure this one should use __noop, and not __nop? __noop: http://msdn.microsoft.com/en-us/library/s6btaxcs.aspx __nop: http://msdn.microsoft.com/en-us/library/aa983381.aspx I think __nop is what we want? Also, that turns it into nop and not rep nop, no? I did some more research, and __nop() is at least closer than __noop(), but it's still not the same. Should we perhaps instead use __yield, per: http://msdn.microsoft.com/en-us/library/2b2h26kx.aspx On further reading, __yield() is only available on Itanium. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Alvaro Herrera wrote: 3) There is no easy way to analyze all databases. vacuumdb --analyze does analyze _and_ vacuum, which for an 8.4 to 8.5 migration does an unnecessary vacuum. Right now I recommend ANALYZE in every database, but it would be nice if there were a single command which did this. +1 for vacuumdb --analyze-only OK, I have implemented this using --only-analyze to avoid having the '--anal' option spelling be ambiguous, which might confuse/frustrate users. I also moved the --freeze option documention mention into a more logical place. Patch attached. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/ref/vacuumdb.sgml === RCS file: /cvsroot/pgsql/doc/src/sgml/ref/vacuumdb.sgml,v retrieving revision 1.45 diff -c -c -r1.45 vacuumdb.sgml *** doc/src/sgml/ref/vacuumdb.sgml 27 Nov 2009 17:41:26 - 1.45 --- doc/src/sgml/ref/vacuumdb.sgml 4 Jan 2010 17:21:34 - *** *** 24,32 commandvacuumdb/command arg rep=repeatreplaceableconnection-option/replaceable/arg grouparg--full/argarg-f/arg/group grouparg--verbose/argarg-v/arg/group grouparg--analyze/argarg-z/arg/group !grouparg--freeze/argarg-F/arg/group arg--table | -t replaceabletable/replaceable arg( replaceable class=parametercolumn/replaceable [,...] )/arg /arg --- 24,33 commandvacuumdb/command arg rep=repeatreplaceableconnection-option/replaceable/arg grouparg--full/argarg-f/arg/group +grouparg--freeze/argarg-F/arg/group grouparg--verbose/argarg-v/arg/group grouparg--analyze/argarg-z/arg/group !grouparg--only-analyze/argarg-o/arg/group arg--table | -t replaceabletable/replaceable arg( replaceable class=parametercolumn/replaceable [,...] )/arg /arg *** *** 36,44 arg rep=repeatreplaceableconnection-options/replaceable/arg grouparg--all/argarg-a/arg/group grouparg--full/argarg-f/arg/group grouparg--verbose/argarg-v/arg/group grouparg--analyze/argarg-z/arg/group !grouparg--freeze/argarg-F/arg/group /cmdsynopsis /refsynopsisdiv --- 37,46 arg rep=repeatreplaceableconnection-options/replaceable/arg grouparg--all/argarg-a/arg/group grouparg--full/argarg-f/arg/group +grouparg--freeze/argarg-F/arg/group grouparg--verbose/argarg-v/arg/group grouparg--analyze/argarg-z/arg/group !grouparg--only-analyze/argarg-o/arg/group /cmdsynopsis /refsynopsisdiv *** *** 56,63 para applicationvacuumdb/application is a wrapper around the SQL command xref linkend=SQL-VACUUM endterm=SQL-VACUUM-title. !There is no effective difference between vacuuming databases via !this utility and via other methods for accessing the server. /para /refsect1 --- 58,66 para applicationvacuumdb/application is a wrapper around the SQL command xref linkend=SQL-VACUUM endterm=SQL-VACUUM-title. !There is no effective difference between vacuuming and analyzing !databases via this utility and via other methods for accessing the !server. /para /refsect1 *** *** 117,122 --- 120,145 /varlistentry varlistentry + termoption-F/option/term + termoption--freeze/option/term + listitem +para + Aggressively quotefreeze/quote tuples. +/para + /listitem + /varlistentry + + varlistentry + termoption-o/option/term + termoption--only-analyze/option/term + listitem +para + Only calculate statistics for use by the optimizer (no vacuum). +/para + /listitem + /varlistentry + + varlistentry termoption-q//term termoption--quiet//term listitem *** *** 133,139 para Clean or analyze replaceable class=parametertable/replaceable only. Column names can be specified only in conjunction with ! the option--analyze/option option. /para tip para --- 156,162 para Clean or analyze replaceable class=parametertable/replaceable only. Column names can be specified only in conjunction with ! the option--analyze/option or option--only-analyze/option options. /para tip para *** *** 164,178 /listitem /varlistentry - varlistentry - termoption-F/option/term - termoption--freeze/option/term - listitem -para - Aggressively quotefreeze/quote tuples. -/para - /listitem - /varlistentry /variablelist /para --- 187,192 Index: src/bin/scripts/vacuumdb.c
Re: [HACKERS] pg_migrator issues
Robert Haas wrote: 3) ?There is no easy way to analyze all databases. ?vacuumdb --analyze does analyze _and_ vacuum, which for an 8.4 to 8.5 migration does an unnecessary vacuum. ?Right now I recommend ANALYZE in every database, but it would be nice if there were a single command which did this. Something like vacuumdb --analyze-only? It seems like overkill to create a whole new command for this, even though vacuumdb doesn't quite make sense. Yea, I am not excited about having vacuumdb do only analyze, but it seems the most minimal solution. I spelled it --only-analyze and just posted the reason and patch. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander mag...@hagander.net writes: I realize this is a very platform-specific thing, but should we consider setting the value of /proc/pid/oom_adj when running on linux? See: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/proc.txt;h=220cc6376ef80e0c9bcfec162d45552e729cdf5a;hb=45d28b097280a78893ce25a5d0db41e6a2717853 One interesting thing I read there is: Swapped out tasks are killed first. Half of each child's memory size is added to the parent's score if they do not share the same memory. This suggests that PG's shared memory ought not be counted in the postmaster's OOM score, which would mean that the problem shouldn't be quite as bad as we've believed. I wonder if that is a recent change? Or maybe it's supposed to be that way and is not implemented correctly? BTW, the given link shows only chapter 1, see http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/filesystems/proc.txt;hb=45d28b097280a78893ce25a5d0db41e6a2717853 for the whole file. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Robert Haas wrote: 4) ?I have implemented the ability to run pg_migrator --check on a live old server. ?However, pg_migrator uses information from controldata to check things, and it also needs xid information that is only available via pg_resetxlog -n(no update) to perform the migration. ?Unfortunately, pg_resetxlog -n cannot be run on a live server, so pg_migrator runs pg_controldata for --check and pg_resetxlog -n for real upgrades. ?It would simplify pg_migrator if I would run pg_resetxlog -n on a live server, but I can understand if people don't want to do that because the xid information reported on a live server is inaccurate. I don't really have a specific thought on this issue, except that it sounds like you're launching a lot of shell commands, and I wonder whether it would be better to try to do this through either C code or by exposing the appropriate stuff at the SQL level. I considered that but realize that pg_migrator has to read pg_controldata in both the old and new servers, meaning it would need access to both C structures, and considering they both have the same structure names, that would require some odd C tricks. Add to that you don't know which version of Postgres you are migrating from/to during compile and the idea of using C becomes even less attractive. Doing this in C would require pg_migrator to track all changes in the pg_controldata structure layout, which seems excessively complex/error-prone. Right now I only have to track changes to the naming of the output fields. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUG?] strange behavior in ALTER TABLE ... RENAME TO on inherited columns
On Sun, Jan 3, 2010 at 11:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: 2010/1/3 KaiGai Kohei kai...@ak.jp.nec.com: if (number_of_attribute_origin(myrelid, oldattname) 1) ereport(ERROR, ...); Am I missing something? That sounds about right to me, It looks remarkably inefficient to me. Do you propose to search the entire database's inheritance tree to derive that number? And do it over again at each child table? The method I suggested would allow the necessary information to be extracted during the initial search for child tables, which we have to do anyway. I haven't read the code in enough detail to have an educated opinion about whether that would induce enough overhead to be worth worrying about it, so I will refrain from comment on this until I have done my homework. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Bruce Momjian escribió: I considered that but realize that pg_migrator has to read pg_controldata in both the old and new servers, meaning it would need access to both C structures, and considering they both have the same structure names, that would require some odd C tricks. Add to that you don't know which version of Postgres you are migrating from/to during compile and the idea of using C becomes even less attractive. However, keep in mind that this might not be the last time on which we will want to read something from a C struct, so perhaps it would be good to bite the bullet and write the odd tricks. Does it already have access (at compile time) to the old and new source trees? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Tom Lane t...@sss.pgh.pa.us wrote: This suggests that PG's shared memory ought not be counted in the postmaster's OOM score, which would mean that the problem shouldn't be quite as bad as we've believed. I wonder if that is a recent change? Or maybe it's supposed to be that way and is not implemented correctly? I've wondered about that based on my experience. When I found that memory leak back in 8.2devel, running on a SLES 9 SP 3 system, the OOM killer killed the offending backend rather than the postmaster, although it took out a couple Java middle tier processes before starting in on PostgreSQL. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
Robert Haas robertmh...@gmail.com writes: Hmm, I see this needs to be rebased over Tom's latest changes, but the conflict I got was in syscache.h, rather than syscache.c. Not sure if that's what you were going for or if there's another issue. Updated patch attached. I'm planning to go look at Naylor's bki refactoring patch now. Assuming there isn't any showstopper problem with that, do you object to it getting committed first? Either order is going to create a merge problem, but it seems like we'd be best off to get Naylor's patch in so people can resync affected patches before the January commitfest starts. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
On Mon, Jan 4, 2010 at 1:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Hmm, I see this needs to be rebased over Tom's latest changes, but the conflict I got was in syscache.h, rather than syscache.c. Not sure if that's what you were going for or if there's another issue. Updated patch attached. I'm planning to go look at Naylor's bki refactoring patch now. Assuming there isn't any showstopper problem with that, do you object to it getting committed first? Either order is going to create a merge problem, but it seems like we'd be best off to get Naylor's patch in so people can resync affected patches before the January commitfest starts. My only objection to that is that if we're going to add attoptions also, I'd like to get this committed first before I start working on that, and we're running short on time. If you can commit his patch in the next day or two, then I am fine with rebasing mine afterwards, but if it needs more work than that then I would prefer to commit mine so I can move on. Is that reasonable? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Thoughts on statistics for continuously advancing columns
Josh Berkus j...@agliodbs.com writes: I've applied a patch to HEAD that does the above. Can you test it to see how well it fixes your problem? Sure. It'll take us a while to set up a test environment; the database is 1TB in size so converting it to 8.5 isn't quick. Great. When you have it set up, you might want to play with enabling the mergejoinscansel change as well, and see if that is a net plus or minus for you. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
Robert Haas robertmh...@gmail.com writes: My only objection to that is that if we're going to add attoptions also, I'd like to get this committed first before I start working on that, and we're running short on time. If you can commit his patch in the next day or two, then I am fine with rebasing mine afterwards, but if it needs more work than that then I would prefer to commit mine so I can move on. Is that reasonable? Fair enough --- if I can't get it done today I will let you know and hold off. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Thoughts on statistics for continuously advancing columns
I've applied a patch to HEAD that does the above. Can you test it to see how well it fixes your problem? Sure. It'll take us a while to set up a test environment; the database is 1TB in size so converting it to 8.5 isn't quick. Will report back. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Alvaro Herrera wrote: Bruce Momjian wrote: pg_migrator has become more popular recently, so it seems time to look at some enhancements that would improve pg_migrator. None of these are required, but rather changes that would be nice to have: 1) Right now pg_migrator preserves relfilenodes for TOAST files because this is required for proper migration. Now that we have shown that strategically-placed global variables with a server-side function to set them is a viable solution, it would be nice to preserve all relfilenodes from the old server. This would simplify pg_migrator by no long requiring place-holder relfilenodes or the renaming of TOAST files. A simpler solution would just be to allow TOAST table creation to automatically remove placeholder files and create specified relfilenodes via global variables. Getting rid of the need for placeholders is a good idea. +1 on getting TOAST tables created with the correct relfilenode from the start. I don't know that preserving any other relfilenode is useful; however if it means you no longer have to rename the files underlying each table, it would probably also be a good idea. (I don't know how does pg_migrator deal with such things currently -- does it keep a map of table name to relfilenode?) Yea, as Tom said later, there are two options. Either we create placeholder files and then remove the place-holders when we create the toast tables or we just preserve all relfilenodes --- I think the later is easier. 4) I have implemented the ability to run pg_migrator --check on a live old server. However, pg_migrator uses information from controldata to check things, and it also needs xid information that is only available via pg_resetxlog -n(no update) to perform the migration. Unfortunately, pg_resetxlog -n cannot be run on a live server, so pg_migrator runs pg_controldata for --check and pg_resetxlog -n for real upgrades. It would simplify pg_migrator if I would run pg_resetxlog -n on a live server, but I can understand if people don't want to do that because the xid information reported on a live server is inaccurate. What xid info does it need? Would it be good enough to use the next XID from most recent checkpoint from pg_controldata? It is a bit outdated, but can't you simply add some value to it to have a safety margin? Well, I am not much into 'safety margins' with pg_migrator, meaning I want to get the most reliable value I can --- I have no idea what that safety margin would be. Right now pg_migrator works fine by calling pg_controldata or pg_resetxlog as appropriate. I was hoping to allow pg_resetxlog -n on a live server. Is that something we should avoid? I really don't need the change --- it would just simplify pg_migrator. I was just really asking if disallowing pg_resetxlog -n on a live server is planned behavior or an oversight. I can see the logic that it should be disallowed but I am just looking for confirmation from someone and I can then drop the issue. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Alvaro Herrera wrote: Bruce Momjian escribi?: I considered that but realize that pg_migrator has to read pg_controldata in both the old and new servers, meaning it would need access to both C structures, and considering they both have the same structure names, that would require some odd C tricks. Add to that you don't know which version of Postgres you are migrating from/to during compile and the idea of using C becomes even less attractive. However, keep in mind that this might not be the last time on which we will want to read something from a C struct, so perhaps it would be good to bite the bullet and write the odd tricks. Does it already have access (at compile time) to the old and new source trees? No, only the new soure tree, or actually any source tree, but ideally the new one. Remember we have Win32 binaries being built, and right now there is limited linkage between pg_migrator and the backend code. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
On Mon, Jan 4, 2010 at 10:42 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Robert Haas escribió: Hmm, I see this needs to be rebased over Tom's latest changes, but the conflict I got was in syscache.h, rather than syscache.c. Not sure if that's what you were going for or if there's another issue. Updated patch attached. FWIW I think the reloptions code in this patch is sane enough. The fact that it was this easily written means that the API for reloptions was reasonably chosen, thanks :-) :-) Actually, there are some things about it that I'm not entirely happy with, but I haven't brought them up because I don't have a clear idea what I think we should do about them. The special-case hack to handle the oids option is one of them another, possibly related, is that I wish we could decouple the options-validation logic from the backend storage representation. But those are issues for a future thread. I do think it's pretty well-done overall. Hmm, it seems we're missing a need_initialization = false at the bottom of initialize_reloptions ... I'm wondering what happened to that?? It appears that it has never been there. $ git log -Sneed_initialization master src/backend/access/common/reloptions.c commit f35e4442a6c9893e72fe870d9e1756262d542027 Author: Alvaro Herrera alvhe...@alvh.no-ip.org Date: Mon Jan 5 17:14:28 2009 + Change the reloptions machinery to use a table-based parser, and provide a more complete framework for writing custom option processing routines by user-defined access methods. Catalog version bumped due to the general API changes, which are going to affect user-defined amoptions routines. That was the original patch that added need_initialization, and it didn't add that line. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG SQLDA support
Boszormenyi Zoltan wrote: I happened to notice this by chance: + #if (LONG_BIT == 64) + #define SQLINT8 ECPGt_long + #define SQLSERIAL8 ECPGt_long + #else + #define SQLINT8 ECPGt_long_long + #define SQLSERIAL8 ECPGt_long_long + #endif I'm not sure how portable is the LONG_BIT business. We don't seem to use it anywhere else (hmm, but then we do use CHAR_BIT elsewhere) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Alvaro Herrera wrote: 2) Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? -1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. Is that not a supported setup? There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. I have updated the pg_migrator INSTALL file to mention this issue. As far as .old, we could create the tablespaces as *.new, but that kind of defeats the existing recommended pg_migrator usage where we tell the user to rename PGDATA to .old before running pg_migrator. It was actually Tom's idea months ago to put a version-specific directory in the tablespace. I don't think it is necessary, and we can live with the mount point limitation. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG SQLDA support
Alvaro Herrera írta: Boszormenyi Zoltan wrote: I happened to notice this by chance: + #if (LONG_BIT == 64) + #defineSQLINT8 ECPGt_long + #defineSQLSERIAL8 ECPGt_long + #else + #defineSQLINT8 ECPGt_long_long + #defineSQLSERIAL8 ECPGt_long_long + #endif I'm not sure how portable is the LONG_BIT business. We don't seem to use it anywhere else (hmm, but then we do use CHAR_BIT elsewhere) I specifically looked for a portable solution, as #if sizeof(...) == N is not evaluated at compile time. Best regards, Zoltán Böszörményi -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
On Mon, Jan 4, 2010 at 1:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: My only objection to that is that if we're going to add attoptions also, I'd like to get this committed first before I start working on that, and we're running short on time. If you can commit his patch in the next day or two, then I am fine with rebasing mine afterwards, but if it needs more work than that then I would prefer to commit mine so I can move on. Is that reasonable? Fair enough --- if I can't get it done today I will let you know and hold off. OK. I just took a really fast look at that the bki patch and it looks pretty nice, so I hope you're able to get it in. Of course, I'm biased because it's based on earlier work of my own, but biased != wrong. :-) A lot more work will need to be done to escape the insanity that is our current method of handling system catalogs, but this seems like a good step in the right direction. I also observe that it applies cleanly over my current spcoptions branch, so the merge conflicts might be a non-issue. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Bruce Momjian br...@momjian.us writes: I was just really asking if disallowing pg_resetxlog -n on a live server is planned behavior or an oversight. I can see the logic that it should be disallowed but I am just looking for confirmation from someone and I can then drop the issue. Well, it's not only a matter of are we going to clobber live state, it's also is the state that we are looking at changing under us?. The -n switch only covers the first point. I think it would require some careful analysis, and testing that's never been done, before having any confidence in the results of pg_resetxlog on a live server. Why should you need this anyway? pg_migrator should not be having to run pg_resetxlog on the old installation, I would think. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG SQLDA support
Alvaro Herrera alvhe...@commandprompt.com writes: I'm not sure how portable is the LONG_BIT business. I think checking SIZEOF_LONG would be preferred, since that's what we use elsewhere. Although actually I wonder why this code exists at all --- wouldn't it be easier to make these depend on int64? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
On Mon, Jan 4, 2010 at 2:11 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: 2) Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? -1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. Is that not a supported setup? There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. I have updated the pg_migrator INSTALL file to mention this issue. As far as .old, we could create the tablespaces as *.new, but that kind of defeats the existing recommended pg_migrator usage where we tell the user to rename PGDATA to .old before running pg_migrator. It was actually Tom's idea months ago to put a version-specific directory in the tablespace. I don't think it is necessary, and we can live with the mount point limitation. What doesn't work if we just don't rename the tablespace at all? And can't we put some smarts into the backend to handle that thing? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG SQLDA support
Tom Lane írta: Alvaro Herrera alvhe...@commandprompt.com writes: I'm not sure how portable is the LONG_BIT business. I think checking SIZEOF_LONG would be preferred, since that's what we use elsewhere. Although actually I wonder why this code exists at all --- wouldn't it be easier to make these depend on int64? regards, tom lane Don't ask me why ECPGt_long_long and ECPGt_unsigned_long_long exist. But they do, and the libecpg code has some #ifdef HAVE_LONG_LONG_INT_64 surrounding code handling them. Maybe it would've been better to be consistent with that coding. -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: I: [HACKERS] TODO: Allow substring/replace() to get/set bit values
On Mon, Jan 4, 2010 at 5:17 AM, Leonardo F m_li...@yahoo.it wrote: Re-reading the docs it looks like the only thing missing is get/set_bit for bit string. Substring is already implemented for bit string, and I don't really know if replace is useful at all. (sorry if the other mail came with a different sender name) You might want to search the archives (or the wiki history, or the CVS history if it's been there since before we moved the TODO list to the wiki) for discussion of why that item was added to the TODO in the first place. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting psql varible as identifier
2010/1/4 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I have one question. If I understand well, the function fmtId isn't multibyte safe? So why is possible to use it in pg_dump? pg_dump is only guaranteed to work correctly in the server encoding. If you force it to use a client_encoding different from the server's, it might or might not work, for reasons far beyond that one --- the big problem usually is data containing characters that have no equivalent in the client encoding. So I'm not particularly excited about whether fmtId is multibyte safe within pg_dump. If we were to try to use it in more general contexts, it would probably need more work. I could agree with this explanation for quote_identifier function, but not in 100% for fmtId function. We can change encoding for pg_dump (option -E). I don't have a problem to write second and safe fmtId function (with technique used in dumputils don't need to modify libpq), although fmtId do exactly what I need. I would to understand to behave. Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I was just really asking if disallowing pg_resetxlog -n on a live server is planned behavior or an oversight. I can see the logic that it should be disallowed but I am just looking for confirmation from someone and I can then drop the issue. Well, it's not only a matter of are we going to clobber live state, it's also is the state that we are looking at changing under us?. The -n switch only covers the first point. I think it would require some careful analysis, and testing that's never been done, before having any confidence in the results of pg_resetxlog on a live server. Yea, that was my analysis too. I will discard the idea and just keep the pg_migrator code that does either. Why should you need this anyway? pg_migrator should not be having to run pg_resetxlog on the old installation, I would think. Well, the same code is run on the new and old server. The complex issue is that the same code that checks for matching controldata settings (check mode) is the same that pulls the xid from the old server to set it on the new one. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Robert Haas wrote: On Mon, Jan 4, 2010 at 2:11 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: 2) ?Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. ?I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? ?-1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. ?Is that not a supported setup? ?There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. ?I have updated the pg_migrator INSTALL file to mention this issue. As far as .old, we could create the tablespaces as *.new, but that kind of defeats the existing recommended pg_migrator usage where we tell the user to rename PGDATA to .old before running pg_migrator. It was actually Tom's idea months ago to put a version-specific directory in the tablespace. ?I don't think it is necessary, and we can live with the mount point limitation. What doesn't work if we just don't rename the tablespace at all? And can't we put some smarts into the backend to handle that thing? Well, when you restore the old dump's schema into the new server, the tablespace directory path will be the same, so we had better not have any directory there. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
On Mon, Jan 4, 2010 at 11:57 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2010-01-04 at 10:31 -0500, Robert Haas wrote: On Mon, Jan 4, 2010 at 3:04 AM, Simon Riggs si...@2ndquadrant.com wrote: This is a more cautious approach. Completely removing VFI in this release is a big risk that we need not take; we have little to gain from doing so and putting it back again will be harder. I am always keen to push forwards when a new feature is worthwhile, but cleaning up code is not an important thing this late in release cycle. I don't have a strong opinion one way or the other on whether we should remove VFI this release cycle, but I thought the reason why there was pressure to do that was because we will otherwise need to make changes to Hot Standby to cope with VFI. What I should have said, in addition: VFI will be kept as a non-default option, in case it is required. We will document that use of VFI will not work correctly with HS and that its use is deprecated and should be in emergencies only in any case. I will enjoy removing VFI when that eventually occurs, but its not a priority. (And if you think, why keep it? I'll say - how else can we run a VFI - not by a stored proc, certainly). If we go this route, can we make it fail in a relatively detectable way with Hot Standby? Like an error message that says oh, crap, you did a VFI, you need a new base backup? Or will it do something goofier than that? I don't have an informed opinion on whether or not we should try to remove VFI in this release, and I leave that discussion to yourself and other people who are more qualified to speak to that issue than I am. I am somewhat dismayed that there are not more people weighing in on this, because it seems to me that this is a critical issue for the forthcoming release, so we really need to make sure we have consensus on the way forward NOW, not a month from now. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Thoughts on statistics for continuously advancing columns
Great. When you have it set up, you might want to play with enabling the mergejoinscansel change as well, and see if that is a net plus or minus for you. How would I *disable* it? --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Thoughts on statistics for continuously advancing columns
Josh Berkus j...@agliodbs.com writes: Great. When you have it set up, you might want to play with enabling the mergejoinscansel change as well, and see if that is a net plus or minus for you. How would I *disable* it? It's #ifdef NOT_USED in CVS at the moment. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
What I should have said, in addition: VFI will be kept as a non-default option, in case it is required. We will document that use of VFI will not work correctly with HS and that its use is deprecated and should be in emergencies only in any case. I will enjoy removing VFI when that eventually occurs, but its not a priority. (And if you think, why keep it? I'll say - how else can we run a VFI - not by a stored proc, certainly). Isn't there some way we can tell if a server is an HS master, and prevent VFI from being run? --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
On Mon, Jan 4, 2010 at 2:52 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Mon, Jan 4, 2010 at 2:11 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: 2) ?Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. ?I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? ?-1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. ?Is that not a supported setup? ?There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. ?I have updated the pg_migrator INSTALL file to mention this issue. As far as .old, we could create the tablespaces as *.new, but that kind of defeats the existing recommended pg_migrator usage where we tell the user to rename PGDATA to .old before running pg_migrator. It was actually Tom's idea months ago to put a version-specific directory in the tablespace. ?I don't think it is necessary, and we can live with the mount point limitation. What doesn't work if we just don't rename the tablespace at all? And can't we put some smarts into the backend to handle that thing? Well, when you restore the old dump's schema into the new server, the tablespace directory path will be the same, so we had better not have any directory there. Well that seems like something you could work around by hacking the contents of the dump... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Robert Haas wrote: On Mon, Jan 4, 2010 at 2:52 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Mon, Jan 4, 2010 at 2:11 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: 2) ?Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. ?I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? ?-1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. ?Is that not a supported setup? ?There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. ?I have updated the pg_migrator INSTALL file to mention this issue. As far as .old, we could create the tablespaces as *.new, but that kind of defeats the existing recommended pg_migrator usage where we tell the user to rename PGDATA to .old before running pg_migrator. It was actually Tom's idea months ago to put a version-specific directory in the tablespace. ?I don't think it is necessary, and we can live with the mount point limitation. What doesn't work if we just don't rename the tablespace at all? ?And can't we put some smarts into the backend to handle that thing? Well, when you restore the old dump's schema into the new server, the tablespace directory path will be the same, so we had better not have any directory there. Well that seems like something you could work around by hacking the contents of the dump... True, in --binary-upgrade mode, but what do we make it? *.new? What if they want to have the same tablespace names after the upgrade? It really gets ugly if we are on a mount point because the tablespaces will be in different file systems, which makes --link mode impossible, and might create files in a filesystem that doesn't have enough space. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Bruce Momjian wrote: Alvaro Herrera wrote: 2) Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? -1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. Is that not a supported setup? There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. I have updated the pg_migrator INSTALL file to mention this issue. Oh, the actual INSTALL warning is: If you are using tablespaces, there must be sufficient directory permissions to allow each tablespace directory to be renamed with a .old suffix. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
On Mon, 2010-01-04 at 12:05 -0800, Josh Berkus wrote: What I should have said, in addition: VFI will be kept as a non-default option, in case it is required. We will document that use of VFI will not work correctly with HS and that its use is deprecated and should be in emergencies only in any case. I will enjoy removing VFI when that eventually occurs, but its not a priority. (And if you think, why keep it? I'll say - how else can we run a VFI - not by a stored proc, certainly). Isn't there some way we can tell if a server is an HS master, and prevent VFI from being run? I'm proposing that VFI is only accessible by explicit request using new syntax; no existing code would call VFI. The VFI problems would only apply to system relations anyway, not to all tables. I propose we have a WARNING if VFI being run when recovery_connections = on, since I probably know what I'm doing if I go out of my way to use new syntax after presumably having read the manual. Just as a point of note, I'm worried that the act of removing VFI would introduce more bugs than leaving it alone; if its there we may as well keep it runnable. Changes required to remove it are at least these places * most of vacuum.c * visibility checks * heap tuple flags and xvac * nontransactional validation * minor points and follow up in 7 files, 12 places -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Tom Lane wrote:
Magnus Hagander mag...@hagander.net writes:
...oom_adj...
One interesting thing I read there is:
Swapped out tasks are killed first. Half of each child's memory size is
added to the parent's score if they do not share the same memory.
This suggests that PG's shared memory ought not be counted in the
postmaster's OOM score, which would mean that the problem shouldn't be
quite as bad as we've believed. I wonder if that is a recent change?
Or maybe it's supposed to be that way and is not implemented correctly?
The code for oom_kill.c looks fairly readable (link below [1]):
96 points = mm-total_vm;
117 list_for_each_entry(child, p-children, sibling) {
118 task_lock(child);
119 if (child-mm != mm child-mm)
120 points += child-mm-total_vm/2 + 1;
121 task_unlock(child);
122 }
Which seems to add points for each child who doesn't share the
same mm structure as the parent. Which I think is a quite a bit
stricter interpretation of if they do not share the same memory.
[1]
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=mm/oom_kill.c;h=f52481b1c1e5442c9a5b16b06b1b75b9bb7c;hb=HEAD
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v3
Le 29/12/2009 14:12, Guillaume Lelarge a écrit :
Le 29/12/2009 00:03, Guillaume Lelarge a écrit :
Le 28/12/2009 22:59, Tom Lane a écrit :
Guillaume Lelarge guilla...@lelarge.info writes:
Le 28/12/2009 17:06, Tom Lane a écrit :
I think we were stalled on the question of whether to use one array
or two parallel arrays. Do you want to try coding up a sample usage
of each possibility so we can see which one seems more useful?
I'm interested in working on this. But I don't find the thread that talk
about this.
Try here
http://archives.postgresql.org/message-id/4aae8ccf.9070...@esilo.com
Thanks. I've read all the new version of PQconnectdb and Determining
client_encoding from client locale threads. I think I understand the
goal. Still need to re-read this one
(http://archives.postgresql.org/message-id/6222.1253734...@sss.pgh.pa.us) and
completely understand it (will probably need to look at the code, at
least the PQconnectdb one). But I'm definitely working on this.
If I try to sum up my readings so far, this is what we still have to do:
1. try the one-array approach
PGconn *PQconnectParams(const char **params)
2. try the two-arrays approach
PGconn *PQconnectParams(const char **keywords, const char **values)
Instead of doing a wrapper around PQconnectdb, we need to refactor the
whole function, so that we can get rid of the parsing of the conninfo
string (which is quite complicated).
Using psql as an example would be a good idea, AFAICT.
Am I right? did I misunderstand or forget something?
I supposed I was right since noone yell at me :)
I worked on this tonight. You'll find two patches attached, one for the
one-array approach, one for the two-arrays approach. I know some more
factoring can be done (at least, the get the fallback resources...
part). I'm OK to do them. I just need to know if I'm on the right track.
--
Guillaume.
http://www.postgresqlfr.org
http://dalibo.com
Index: src/bin/psql/startup.c
===
RCS file: /opt/cvsroot_postgresql/pgsql/src/bin/psql/startup.c,v
retrieving revision 1.158
diff -c -p -c -r1.158 startup.c
*** src/bin/psql/startup.c 2 Jan 2010 16:57:59 - 1.158
--- src/bin/psql/startup.c 4 Jan 2010 21:04:13 -
*** main(int argc, char *argv[])
*** 171,181
/* loop until we have a password if requested by backend */
do
{
! new_pass = false;
! pset.db = PQsetdbLogin(options.host, options.port, NULL, NULL,
! options.action == ACT_LIST_DB options.dbname == NULL ?
! postgres : options.dbname,
! options.username, password);
if (PQstatus(pset.db) == CONNECTION_BAD
PQconnectionNeedsPassword(pset.db)
--- 171,190
/* loop until we have a password if requested by backend */
do
{
! const char *params[] = {
! host, options.host,
! port, options.port,
! dbname, (options.action == ACT_LIST_DB
!options.dbname == NULL) ? postgres : options.dbname,
! user, options.username,
! password, password,
! application_name, pset.progname,
! NULL, NULL
! };
!
! new_pass = false;
!
! pset.db = PQconnectdbParams(params);
if (PQstatus(pset.db) == CONNECTION_BAD
PQconnectionNeedsPassword(pset.db)
Index: src/interfaces/libpq/exports.txt
===
RCS file: /opt/cvsroot_postgresql/pgsql/src/interfaces/libpq/exports.txt,v
retrieving revision 1.23
diff -c -p -c -r1.23 exports.txt
*** src/interfaces/libpq/exports.txt 31 Mar 2009 01:41:27 - 1.23
--- src/interfaces/libpq/exports.txt 4 Jan 2010 20:51:13 -
*** PQresultSetInstanceData 150
*** 153,155
--- 153,157
PQfireResultCreateEvents 151
PQconninfoParse 152
PQinitOpenSSL 153
+ PQconnectdbParams 154
+ PQconnectStartParams 155
Index: src/interfaces/libpq/fe-connect.c
===
RCS file: /opt/cvsroot_postgresql/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.382
diff -c -p -c -r1.382 fe-connect.c
*** src/interfaces/libpq/fe-connect.c 2 Jan 2010 16:58:11 - 1.382
--- src/interfaces/libpq/fe-connect.c 4 Jan 2010 20:54:12 -
*** static bool connectOptions2(PGconn *conn
*** 259,264
--- 259,265
static int connectDBStart(PGconn *conn);
static int connectDBComplete(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
+ static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
static PQconninfoOption *conninfo_parse(const char *conninfo,
*** pgthreadlock_t pg_g_threadlock = default
*** 299,304
--- 300,337
*/
Re: [HACKERS] New VACUUM FULL
On Mon, Jan 4, 2010 at 3:51 PM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2010-01-04 at 12:05 -0800, Josh Berkus wrote: What I should have said, in addition: VFI will be kept as a non-default option, in case it is required. We will document that use of VFI will not work correctly with HS and that its use is deprecated and should be in emergencies only in any case. I will enjoy removing VFI when that eventually occurs, but its not a priority. (And if you think, why keep it? I'll say - how else can we run a VFI - not by a stored proc, certainly). Isn't there some way we can tell if a server is an HS master, and prevent VFI from being run? I'm proposing that VFI is only accessible by explicit request using new syntax; no existing code would call VFI. The VFI problems would only apply to system relations anyway, not to all tables. I propose we have a WARNING if VFI being run when recovery_connections = on, since I probably know what I'm doing if I go out of my way to use new syntax after presumably having read the manual. I think I'd vote for throwing an ERROR. By the time you see the WARNING it may be too late. Since this is only for emergencies, the user can shut off recovery_connections first if they really need it. Just as a point of note, I'm worried that the act of removing VFI would introduce more bugs than leaving it alone; if its there we may as well keep it runnable. Changes required to remove it are at least these places * most of vacuum.c * visibility checks * heap tuple flags and xvac * nontransactional validation * minor points and follow up in 7 files, 12 places Doesn't sound trivial. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
On Mon, Jan 4, 2010 at 3:33 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Mon, Jan 4, 2010 at 2:52 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Mon, Jan 4, 2010 at 2:11 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: 2) ?Right now pg_migrator renames old tablespaces to .old, which fails if the tablespaces are on mount points. ?I have already received a report of such a failure. I thought it was impossible to use bare mountpoints as tablespaces due to ownership problems ... Is that not the case? ?-1 for special hacks that work around bogus setups, if that means intrusive changes to the core code. I talked to the person who reported the problem and he and I confirmed that it is quite easy to make the mount point be owned by the postgres user and have that function as a tablespace. ?Is that not a supported setup? ?There is probably a larger problem that the tablespace must be located in a directory that has directory rename permission for postgres. ?I have updated the pg_migrator INSTALL file to mention this issue. As far as .old, we could create the tablespaces as *.new, but that kind of defeats the existing recommended pg_migrator usage where we tell the user to rename PGDATA to .old before running pg_migrator. It was actually Tom's idea months ago to put a version-specific directory in the tablespace. ?I don't think it is necessary, and we can live with the mount point limitation. What doesn't work if we just don't rename the tablespace at all? ?And can't we put some smarts into the backend to handle that thing? Well, when you restore the old dump's schema into the new server, the tablespace directory path will be the same, so we had better not have any directory there. Well that seems like something you could work around by hacking the contents of the dump... True, in --binary-upgrade mode, but what do we make it? *.new? What if they want to have the same tablespace names after the upgrade? It really gets ugly if we are on a mount point because the tablespaces will be in different file systems, which makes --link mode impossible, and might create files in a filesystem that doesn't have enough space. But can't we just call a special function first before running the CREATE TABLESPACE, like: pg_tablespace_dont_really_create_this_tablespace_because_we_are_in_pg_migrator()? Sorta like what you did to preserve ENUM OIDs, etc.? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Robert Haas wrote: What doesn't work if we just don't rename the tablespace at all? ?And can't we put some smarts into the backend to handle that thing? Well, when you restore the old dump's schema into the new server, the tablespace directory path will be the same, so we had better not have any directory there. Well that seems like something you could work around by hacking the contents of the dump... True, in --binary-upgrade mode, but what do we make it? ?*.new? ?What if they want to have the same tablespace names after the upgrade? ?It really gets ugly if we are on a mount point because the tablespaces will be in different file systems, which makes --link mode impossible, and might create files in a filesystem that doesn't have enough space. But can't we just call a special function first before running the CREATE TABLESPACE, like: pg_tablespace_dont_really_create_this_tablespace_because_we_are_in_pg_migrator()? Sorta like what you did to preserve ENUM OIDs, etc.? Well, the problem is that we are creating something in a file system, and the old and new contents of the tablespace directories must exist after the migration (in case the migration is reverted). We were able to get away with this for enum because we were only creating this in the _new_ database. With the file system, we have a resource/namespace shared between the old and new server. What were you thinking this function call would do? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
On Mon, Jan 4, 2010 at 4:53 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: What doesn't work if we just don't rename the tablespace at all? ?And can't we put some smarts into the backend to handle that thing? Well, when you restore the old dump's schema into the new server, the tablespace directory path will be the same, so we had better not have any directory there. Well that seems like something you could work around by hacking the contents of the dump... True, in --binary-upgrade mode, but what do we make it? ?*.new? ?What if they want to have the same tablespace names after the upgrade? ?It really gets ugly if we are on a mount point because the tablespaces will be in different file systems, which makes --link mode impossible, and might create files in a filesystem that doesn't have enough space. But can't we just call a special function first before running the CREATE TABLESPACE, like: pg_tablespace_dont_really_create_this_tablespace_because_we_are_in_pg_migrator()? Sorta like what you did to preserve ENUM OIDs, etc.? Well, the problem is that we are creating something in a file system, and the old and new contents of the tablespace directories must exist after the migration (in case the migration is reverted). We were able to get away with this for enum because we were only creating this in the _new_ database. With the file system, we have a resource/namespace shared between the old and new server. Oh, I thought you were hard-linking the files, not copying them, so the old directory would be destroyed anyway. What were you thinking this function call would do? Basically, make PostgreSQL not reinitialize the directory as it normally would when a new tablespace is created. Or make it ignore the existence of the directory until told to stop ignoring it. Or whatever is needed to avoid having to move the thing around. Sorry, I'm hand-waving wildly here; I haven't read the code. Maybe I should shut up. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Robert Haas wrote: But can't we just call a special function first before running the CREATE TABLESPACE, like: pg_tablespace_dont_really_create_this_tablespace_because_we_are_in_pg_migrator()? Sorta like what you did to preserve ENUM OIDs, etc.? Well, the problem is that we are creating something in a file system, and the old and new contents of the tablespace directories must exist after the migration (in case the migration is reverted). ?We were able to get away with this for enum because we were only creating this in the _new_ database. ?With the file system, we have a resource/namespace shared between the old and new server. Oh, I thought you were hard-linking the files, not copying them, so the old directory would be destroyed anyway. The default mode is COPY but there is a --link option. You are right that if we only did linking things would be much simpler. What were you thinking this function call would do? Basically, make PostgreSQL not reinitialize the directory as it normally would when a new tablespace is created. Or make it ignore the existence of the directory until told to stop ignoring it. Or whatever is needed to avoid having to move the thing around. Sorry, I'm hand-waving wildly here; I haven't read the code. Maybe I should shut up. Sorry, I can't figure out how that would work. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] KNNGiST for knn-search (WIP)
I'm sure whatever conclusion -hackers comes to in the end will be the best for pgsql, and I'll be supportive. But until then, let me note from the PostGIS point-of-view: sure would be great to get this in for 8.5 :) P. On Thu, Dec 31, 2009 at 4:26 AM, Greg Stark gsst...@mit.edu wrote: On Wed, Dec 30, 2009 at 4:56 PM, Robert Haas robertmh...@gmail.com wrote: From my point of view, what makes a patch invasive is the likelihood that it might break something other than itself. For example, your patch touches the core planner code and the core GIST code, so it seems possible that adding support for this feature might break something else in one of those areas. It doesn't seem obvious to me that this is a high-risk patch. It's touching the planner which is tricky but it's not the kind of massive overhaul that touches every module that HOT or HS were. I'm really glad HS got in before the end because lots of people with different areas of expertise and different use cases are going to get to exercise it in the time remaining. This patch I would expect relatively few people to need to try it out before any oversights are caught. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] KNNGiST for knn-search (WIP)
On Mon, Jan 4, 2010 at 5:33 PM, Paul Ramsey pram...@cleverelephant.ca wrote: I'm sure whatever conclusion -hackers comes to in the end will be the best for pgsql, and I'll be supportive. But until then, let me note from the PostGIS point-of-view: sure would be great to get this in for 8.5 :) That's good to know. The current status is that I've been waiting for a patch that applies cleanly for 6 days, and we have 41 days left until the end of the last CommitFest. There's not much I can do to move this along until I have a clean patch to work with. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Testing with concurrent sessions
David E. Wheeler da...@kineticode.com wrote: I think the consensus was, failing support for concurrent sessions in psql, to use a Perl script to control multiple psql sessions and perhaps use Test::More to do the testing. Are there any examples of that? While I can hack my way through regular expressions when I need them, perl as a language is something I don't know at all; with an example I might be able to come up to speed quickly, though. Although pgTAP might make sense, too, if the tests ought to run in the database. I need to run statements against a database; I don't particularly need any special features of psql for this. Can anyone confirm that pgTAP can let you interleave specific statements against specific connections in a specific sequence? (The answer to that didn't leap out at me in a quick scan of the docs.) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Testing with concurrent sessions
On mån, 2010-01-04 at 17:10 -0600, Kevin Grittner wrote: David E. Wheeler da...@kineticode.com wrote: I think the consensus was, failing support for concurrent sessions in psql, to use a Perl script to control multiple psql sessions and perhaps use Test::More to do the testing. Are there any examples of that? While I can hack my way through regular expressions when I need them, perl as a language is something I don't know at all; with an example I might be able to come up to speed quickly, though. If you're not deep into Perl, perhaps ignore the Test::More comment for the moment and just use DBI to connect to several database sessions, execute your queries and check if the results are what you want. Once you have gotten somewhere with that, wrapping a test harness around it is something others will be able to help with. Although pgTAP might make sense, too, if the tests ought to run in the database. I need to run statements against a database; I don't particularly need any special features of psql for this. Can anyone confirm that pgTAP can let you interleave specific statements against specific connections in a specific sequence? (The answer to that didn't leap out at me in a quick scan of the docs.) pgTAP isn't really going to help you here, as it runs with *one* database session, and its main functionality is to format the result of SQL functions into TAP output, which is not very much like what you ought to be doing. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Initial refactoring of plperl.c - rebased [PATCH]
Andrew Dunstan wrote: Yes. I believe the test is highlighting an existing problem: that plperl function in non-PG_UTF8 databases can't use regular expressions that require unicode character meta-data. Either the (GetDatabaseEncoding() == PG_UTF8) test in plperl_safe_init() should be removed, so the utf8fix function is always called, or the test should be removed (or hacked to only apply to PG_UTF8 databases). I tried forcing the test, but it doesn't seem to work, possibly because in the case that the db is not utf8 we aren't forcing argument strings to UTF8 :-( I think we might need to remove the test from the patch. I have not been able to come up with a fix for this - the whole thing seems very fragile. I'm going to commit what remains of this patch, but not add the extra regression test. I'll add a TODO to allow plperl to do utf8 operations in non-utf8 databases. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Testing with concurrent sessions
On Jan 4, 2010, at 3:29 PM, Peter Eisentraut wrote: If you're not deep into Perl, perhaps ignore the Test::More comment for the moment and just use DBI to connect to several database sessions, execute your queries and check if the results are what you want. Once you have gotten somewhere with that, wrapping a test harness around it is something others will be able to help with. Last I heard, Andrew was willing to require Test::More for testing, so that a Perl script could handle multiple psql connections (perhaps forked) and output test results based on them. But he wasn't as interested in requiring DBI and DBD::Pg, neither of which are in the Perl core and are more of a PITA to install (not huge, but the barrier might as well stay low). pgTAP isn't really going to help you here, as it runs with *one* database session, and its main functionality is to format the result of SQL functions into TAP output, which is not very much like what you ought to be doing. Right, exactly. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] What's the current theory about derived files in VPATH builds?
It seems like VPATH building is broken at the moment, at least if you are working from a maintainer-clean source tree. make -C bootstrap all make[3]: Entering directory `/home/postgres/build/src/backend/bootstrap' /usr/local/bin/bison -o bootparse.c /home/postgres/pgsql/src/backend/bootstrap/bootparse.y /usr/local/bin/flex -o'bootscanner.c' /home/postgres/pgsql/src/backend/bootstrap/bootscanner.l gcc -O1 -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -g -I/home/postgres/pgsql/src/backend/bootstrap -I../../../src/include -I/home/postgres/pgsql/src/include -D_XOPEN_SOURCE_EXTENDED -c -o bootparse.o bootparse.c /home/postgres/pgsql/src/backend/bootstrap/bootparse.y:401: bootscanner.c: No such file or directory make[3]: *** [bootparse.o] Error 1 I thought the current plan was that derived files should be generated in the build tree (as indeed it seems to be doing) ... but there are apparently parts of the Makefiles that are not in sync with this plan. Has this just not been tested recently, or am I confused? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What's the current theory about derived files in VPATH builds?
On mån, 2010-01-04 at 18:49 -0500, Tom Lane wrote: It seems like VPATH building is broken at the moment, at least if you are working from a maintainer-clean source tree. make -C bootstrap all make[3]: Entering directory `/home/postgres/build/src/backend/bootstrap' /usr/local/bin/bison -o bootparse.c /home/postgres/pgsql/src/backend/bootstrap/bootparse.y /usr/local/bin/flex -o'bootscanner.c' /home/postgres/pgsql/src/backend/bootstrap/bootscanner.l gcc -O1 -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -g -I/home/postgres/pgsql/src/backend/bootstrap -I../../../src/include -I/home/postgres/pgsql/src/include -D_XOPEN_SOURCE_EXTENDED -c -o bootparse.o bootparse.c /home/postgres/pgsql/src/backend/bootstrap/bootparse.y:401: bootscanner.c: No such file or directory make[3]: *** [bootparse.o] Error 1 I thought the current plan was that derived files should be generated in the build tree (as indeed it seems to be doing) ... but there are apparently parts of the Makefiles that are not in sync with this plan. Has this just not been tested recently, or am I confused? This should in principle work. My guess is an old make version being confused. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What's the current theory about derived files in VPATH builds?
Peter Eisentraut pete...@gmx.net writes: On mån, 2010-01-04 at 18:49 -0500, Tom Lane wrote: I thought the current plan was that derived files should be generated in the build tree (as indeed it seems to be doing) ... but there are apparently parts of the Makefiles that are not in sync with this plan. Has this just not been tested recently, or am I confused? This should in principle work. My guess is an old make version being confused. Well, I am testing with relatively old make and gcc too, but what it looks like to me is that we need to add a -I. switch in places where we might need to #include a file out of the current build directory. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
I think I'd vote for throwing an ERROR. By the time you see the WARNING it may be too late. Since this is only for emergencies, the user can shut off recovery_connections first if they really need it. I'm with Robert on this one. If running VFI will cause unrecoverable failure on the slave, it should be prevented. If this is only an issue with system tables, then we only need to error on system tables. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What's the current theory about derived files in VPATH builds?
I wrote: Peter Eisentraut pete...@gmx.net writes: This should in principle work. My guess is an old make version being confused. Well, I am testing with relatively old make and gcc too, but what it looks like to me is that we need to add a -I. switch in places where we might need to #include a file out of the current build directory. Hm, it does seem to work as expected on a Fedora 11 box. I suspect what is happening is that gcc's rule for searching for #include's has changed slightly since the old gcc version I have on my HPUX box. The situation is that we are trying to #include bootscanner.c from bootparse.c, where both of those files are in the current directory, but . is *not* named anywhere in the -I options. In principle the #include ought to fail, but gcc has a special exception that causes it to look in the directory of the current input file for #include files. As can be seen from the error message, my older gcc seems to think that the current input file is /home/postgres/pgsql/src/backend/bootstrap/bootparse.y --- that is, it's probably believing the # directives rather than the originally opened file name. The same thing is happening in parser/gram.c, and probably in the other places where we #include flex output from bison output. This might be considered a gcc bug, but since we don't know when the behavior changed, or whether other compilers have any such exception at all, I think we ought to accommodate both ways --- ie add -I. in the Makefiles that require this case to work. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
On Mon, 2010-01-04 at 16:41 -0500, Robert Haas wrote: I propose we have a WARNING if VFI being run when recovery_connections = on, since I probably know what I'm doing if I go out of my way to use new syntax after presumably having read the manual. I think I'd vote for throwing an ERROR. By the time you see the WARNING it may be too late. Since this is only for emergencies, the user can shut off recovery_connections first if they really need it. OK -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch - per-tablespace random_page_cost/seq_page_cost
Tom, It seems I introduced a couple errors in src/tools/msvc/clean.bat in the bki patch. I'm attaching a cumulative fix. I can resend the complete updated patch, if you like... Sorry! :-) John I'm planning to go look at Naylor's bki refactoring patch now. Assuming there isn't any showstopper problem with that, do you object to it getting committed first? Either order is going to create a merge problem, but it seems like we'd be best off to get Naylor's patch in so people can resync affected patches before the January commitfest starts. diff --git a/src/tools/msvc/clean.bat b/src/tools/msvc/clean.bat index 8cce31e..1d3ea65 100755 *** a/src/tools/msvc/clean.bat --- b/src/tools/msvc/clean.bat *** REM Delete files created with GenerateFi *** 20,35 if exist src\include\pg_config.h del /q src\include\pg_config.h if exist src\include\pg_config_os.h del /q src\include\pg_config_os.h if %DIST%==1 if exist src\backend\parser\gram.h del /q src\backend\parser\gram.h ! if exist src\include\utils\fmgroids.h del /q src\include\utils\fmgroids.h ! if exist src\include\catalog\schemapg.h del /q src\include\catalog\schemapg.h if exist src\include\utils\probes.h del /q src\include\utils\probes.h - if %DIST%==1 if exist src\backend\utils\fmgroids.h del /q src\backend\utils\fmgroids.h if %DIST%==1 if exist src\backend\utils\fmgrtab.c del /q src\backend\utils\fmgrtab.c if %DIST%==1 if exist src\backend\catalog\postgres.bki del /q src\backend\catalog\postgres.bki if %DIST%==1 if exist src\backend\catalog\postgres.description del /q src\backend\catalog\postgres.description if %DIST%==1 if exist src\backend\catalog\postgres.shdescription del /q src\backend\catalog\postgres.shdescription - if %DIST%==1 if exist src\backend\catalog\schemapg.h del /q src\backend\catalog\schemapg.h if %DIST%==1 if exist src\backend\parser\scan.c del /q src\backend\parser\scan.c if %DIST%==1 if exist src\backend\parser\gram.c del /q src\backend\parser\gram.c if %DIST%==1 if exist src\backend\bootstrap\bootscanner.c del /q src\backend\bootstrap\bootscanner.c --- 20,33 if exist src\include\pg_config.h del /q src\include\pg_config.h if exist src\include\pg_config_os.h del /q src\include\pg_config_os.h if %DIST%==1 if exist src\backend\parser\gram.h del /q src\backend\parser\gram.h ! if %DIST%==1 if exist src\backend\utils\fmgroids.h del /q src\backend\utils\fmgroids.h ! if %DIST%==1 if exist src\backend\catalog\schemapg.h del /q src\backend\catalog\schemapg.h if exist src\include\utils\probes.h del /q src\include\utils\probes.h if %DIST%==1 if exist src\backend\utils\fmgrtab.c del /q src\backend\utils\fmgrtab.c if %DIST%==1 if exist src\backend\catalog\postgres.bki del /q src\backend\catalog\postgres.bki if %DIST%==1 if exist src\backend\catalog\postgres.description del /q src\backend\catalog\postgres.description if %DIST%==1 if exist src\backend\catalog\postgres.shdescription del /q src\backend\catalog\postgres.shdescription if %DIST%==1 if exist src\backend\parser\scan.c del /q src\backend\parser\scan.c if %DIST%==1 if exist src\backend\parser\gram.c del /q src\backend\parser\gram.c if %DIST%==1 if exist src\backend\bootstrap\bootscanner.c del /q src\backend\bootstrap\bootscanner.c -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Red-black tree for GIN
Robert Haas escribió: I did a quick read-through of this, and one question that immediately occurred to me is that rbtree.c says that it is adopted from http://algolist.manual.ru/ds/rbtree.php. But I'm not sure what license that code is under, so I'm not sure whether it's OK for us to use it. This code comes from Thomas Niemann's Sorting and Searching Algorithms: A Cookbook, http://www.cs.auckland.ac.nz/software/AlgAnim/niemann/s_man.htm specifically http://www.cs.auckland.ac.nz/software/AlgAnim/niemann/s_rbt.htm The code is in the public domain; that web page says Source code, when part of a software project, may be used freely without reference to the author. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Red-black tree for GIN
On Mon, Jan 4, 2010 at 8:12 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Robert Haas escribió: I did a quick read-through of this, and one question that immediately occurred to me is that rbtree.c says that it is adopted from http://algolist.manual.ru/ds/rbtree.php. But I'm not sure what license that code is under, so I'm not sure whether it's OK for us to use it. This code comes from Thomas Niemann's Sorting and Searching Algorithms: A Cookbook, http://www.cs.auckland.ac.nz/software/AlgAnim/niemann/s_man.htm specifically http://www.cs.auckland.ac.nz/software/AlgAnim/niemann/s_rbt.htm The code is in the public domain; that web page says Source code, when part of a software project, may be used freely without reference to the author. That is excellent. Perhaps we should document that information in the code comments where the URL is currently mentioned. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New VACUUM FULL
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 4, 2010 at 3:51 PM, Simon Riggs si...@2ndquadrant.com wrote: Changes required to remove it are at least these places * most of vacuum.c * visibility checks * heap tuple flags and xvac * nontransactional validation * minor points and follow up in 7 files, 12 places Doesn't sound trivial. The above is a vast overstatement of the problem. Simon is not only talking about removing VACUUM FULL, he's talking about removing every trace that it ever existed, eg deleting support for MOVED_OFF/MOVED_IN tuple status flags. We are *not* doing that, not now nor in the foreseeable future. As long as we have any ambition of having in-place upgrade from pre-8.5 we have to handle the MOVED status bits the same as we do now. AFAICS, ripping out most of the guts of vacuum.c is about all that's likely to happen for 8.5. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
