Re: [HACKERS] Testing extension upgrade scripts
da...@kineticode.com (David E. Wheeler) writes: You should blog this. He just did, using the SMTP protocol... -- select 'cbbrowne' || '@' || 'acm.org'; http://linuxdatabases.info/info/postgresql.html Where do you want to Tell Microsoft To Go Today? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Why our counters need to be time-based WAS: WIP: cross column correlation ...
j...@agliodbs.com (Josh Berkus) writes: I don't understand what you're talking about at all here. I think there are a lot of unsolved problems in monitoring but the one thing I think everyone is pretty clear on is that the right way to export metrics like these is to export a counter and then have some external component periodically copy the counter into some history table and calculate the derivative, second derivative, running average of the first derivative, etc. You missed the original point of the discussion, which was to have stats we could use for auto-tuning internally. Not to export them. For example, there are optimizations we could make with the query planner if we knew which tables and indexes were hot in general. That's how we started this discussion, and it's not solved by storing the stats history on another server. There's value to both, and there's no dearth of monitoring frameworks that people keep on replacing with successors, so there's certainly room for both ;-). Recent stuff about such... https://lopsa.org/content/philosophy-monitoring https://labs.omniti.com/labs/reconnoiter I'm not quite sure what ought to be in PostgreSQL as a built-in; I suspect that what's eventually needed is to be able to correlate things across database instances, so that when Tom says, I need to know what data the planner's working on, the answer can be OK, got that... This data is surely useful to get out of the system, so I'd bias towards something sorta like what Greg suggests. And the closed-ended answer may prevent us from asking more sophisticated questions, also not a notably good thing... -- (reverse (concatenate 'string moc.liamg @ enworbbc)) If tautologies do not convey information, mathematicians would not be surprised by them. -- Mark Miller -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2011-01 as of 2011-02-04
robertmh...@gmail.com (Robert Haas) writes:
It does, but frankly I don't see much reason to change it, since it's
been working pretty well on the whole. Andrew was on point when he
mentioned that it's not obvious what committers get out of working on
other people's patches. Obviously, the answer is, well, they get a
better PostgreSQL, and that's ultimately good for all of us. But the
trickiest part of this whole process is that, on the one hand, it's
not fair for committers to ignore other people's patches, but on the
other hand, it's not fair to expect committers to sacrifice getting
their own projects done to get other people's projects done.
I had two interesting germane comments in my RSS feed this morning, both
entitled Please send a patch
http://www.lucas-nussbaum.net/blog/?p=630
Where Lucas suggests that, when someone requests an enhancement, the
retort Please send a patch mayn't be the best idea, because the
one receiving the requests may be many times better at contributing
such changes than the one making the request.
http://hezmatt.org/~mpalmer/blog/general/please_send_a_patch.html
On the other hand, Lucas, remember that each time you ask someone
to take some time to implement your pet feature request, you take
some time away from her that could be used to contribute something
in an area where she gives a damn.
These are *both* true statements, and, in order to grow the community
that is capable of enhancing the system, there is merit to the careful
application of both positions.
There's stuff that Tom should do :-). And absent the general
availability of cloning machines, we need to have people improving their
skills so that there are more that are capable of submitting (and
evaluating and committing) usable patches.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://linuxdatabases.info/info/languages.html
Signs of a Klingon Programmer - 14. Our competitors are without
honor!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why two dashes in extension load files
t...@sss.pgh.pa.us (Tom Lane) writes:
Peter Eisentraut pete...@gmx.net writes:
On mån, 2011-02-14 at 10:13 -0500, Tom Lane wrote:
Peter Eisentraut pete...@gmx.net writes:
Why do the extension load files need two dashes, like xml2--1.0.sql?
Why isn't one enough?
Because we'd have to forbid dashes in extension name and version
strings. This was judged to be a less annoying solution. See
yesterday's discussion.
I'm not convinced. There was nothing in that discussion why any
particular character would have to be allowed in a version number.
Well, there's already a counterexample in the current contrib stuff:
uuid-ossp. We could rename that to uuid_ossp of course, but it's
not clear to me that there's consensus for forbidding dashes here.
I suspect that _ might be troublesome.
Let me observe on Debian policy... It requires that package names
consist as follows:
Package names (both source and binary, see Package, Section 5.6.7)
must consist only of lower case letters (a-z), digits (0-9), plus (+)
and minus (-) signs, and periods (.). They must be at least two
characters long and must start with an alphanumeric character.
http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Source
I suspect that we'll need to have a policy analagous to that.
Also worth observing: Debian package files are of the form:
${package}_${version}-${dversion}_${arch}.deb
where package and version have fairly obvious interpretation, and...
- dversion indicates a sequence handled by Debian
- arch indicates CPU architecture (i386, amd64, ...)
Probably the dversion/arch bits aren't of interest to us, but the
remainder of the notation used by Debian seems not inapplicable for us.
--
let name=cbbrowne and tld=gmail.com in String.concat @ [name;tld];;
http://linuxdatabases.info/info/languages.html
Signs of a Klingon Programmer - 4. You cannot really appreciate
Dilbert unless you've read it in the original Klingon.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postponing some large patches to 9.2
pg...@j-davis.com (Jeff Davis) writes: On Tue, 2011-02-08 at 15:10 -0500, Chris Browne wrote: It's more than a bit sad... The RangeType change has the massive merit of enabling some substantial development changes, where we can get rid of whole classes of comparison clauses, and hopefully whole classes of range errors. That was my favorite would-be feature for 9.1. I appreciate the support. If you take the feature for a quick spin before the next commitfest, that would be a big help. If I get it in the first commitfest of 9.2 that may mean some follow-up features, like RANGE KEYs/FKs, and maybe even RANGE JOIN might have a chance for 9.2 as well. Or, maybe some other features might find it useful, like partitioning or audit logs. I've found my wish item... I wish that queries could expand ranges in much the same fashion that BETWEEN expands into two query nodes. That way, you can use a range to pick data from a large table, and not revert to a Seq Scan+Filter, which is what I'm seeing for the following sort of query: select * from some_data where '[2010-01-01,2010-02-01)'::daterange @ whensit; -- let name=cbbrowne and tld=gmail.com in String.concat @ [name;tld];; http://linuxfinances.info/info/lsf.html Rules of the Evil Overlord #162. If I steal something very important to the hero, I will not put it on public display. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Range Types - efficiency
One of the things I'd particularly like to use range types for is to
make it easier to construct range-related queries. Classic example is
that of reports that work on date ranges.
I create a table that will have transaction data:
CREATE TABLE some_data (
id serial,
whensit date
-- And it'll have other attributes, but those don't matter here...
);
CREATE INDEX some_when ON some_data USING btree (whensit);
I then populate it with a bunch of date-based data...
rangetest@localhost- select count(*), min(whensit), max(whensit) from
some_data;
count |min |max
---++
37440 | 2007-01-01 | 2014-12-27
(1 row)
Here's the traditional way of doing a range-based query on this data:
rangetest@localhost- explain analyze select * from some_data where whensit
= '2010-01-01' and whensit '2010-02-01';
QUERY PLAN
---
Bitmap Heap Scan on some_data (cost=12.30..184.23 rows=395 width=8) (actual
time=0.064..0.150 rows=390 loops=1)
Recheck Cond: ((whensit = '2010-01-01'::date) AND (whensit
'2010-02-01'::date))
- Bitmap Index Scan on some_when (cost=0.00..12.21 rows=395 width=0)
(actual time=0.054..0.054 rows=390 loops=1)
Index Cond: ((whensit = '2010-01-01'::date) AND (whensit
'2010-02-01'::date))
Total runtime: 0.197 ms
(5 rows)
The RangeType-based equivalent is the following:
rangetest@localhost- explain analyze select * from some_data where
'[2010-01-01,2010-02-01)'::daterange @ whensit;
QUERY PLAN
-
Seq Scan on some_data (cost=0.00..634.00 rows=1 width=8) (actual
time=1.045..111.739 rows=390 loops=1)
Filter: ('[ 2010-01-01, 2010-02-01 )'::daterange @ whensit)
Total runtime: 111.780 ms
(3 rows)
This, alas, reverts to a seq scan on the table, rather than restricting
itself to the tuples of interest.
I realize that, after a fashion, I'm using this backwards. But when I'm
doing temporal stuff, that tends to be the pattern:
- There is a set of temporal configuration, indicating criteria that
are true for particular date ranges
- There is then event data, which has but a single date, but which
needs to be matched against the temporal configuration.
It sure would be nice to expand that filter into subqueries involving
the two criteria, in much the same fashion that is true today for
BETWEEN. I imagine that would allow many queries with this kind of
pattern to make use of indexes, making them visibly thousands of times
faster.
--
I have traveled the length and breadth of this country and talked
with the best people, and can assure you that data processing is a fad
that won't last out the year. -- Business books editor, Prentice
Hall 1957
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Range Types - efficiency
pg...@j-davis.com (Jeff Davis) writes:
On Wed, 2011-02-09 at 16:20 -0500, Chris Browne wrote:
rangetest@localhost- explain analyze select * from some_data where
'[2010-01-01,2010-02-01)'::daterange @ whensit;
QUERY PLAN
-
Seq Scan on some_data (cost=0.00..634.00 rows=1 width=8) (actual
time=1.045..111.739 rows=390 loops=1)
Filter: ('[ 2010-01-01, 2010-02-01 )'::daterange @ whensit)
Total runtime: 111.780 ms
(3 rows)
This, alas, reverts to a seq scan on the table, rather than restricting
itself to the tuples of interest.
I realize that, after a fashion, I'm using this backwards. But when I'm
doing temporal stuff, that tends to be the pattern:
Yes. The index is a btree index on a normal column, so range types can't
exactly help with that directly -- except maybe as a rewrite like you
say.
One thing you might try is a functional index on (range(whensit)) and
then do: where '...' @ range(whensit).
Does that work for you?
That doesn't appear to actually help:
rangetest@localhost- create index i2 on some_data (range(whensit));
CREATE INDEX
rangetest@localhost- explain analyze select * from some_data where
'[2010-01-01,2010-02-01)'::daterange @ range(whensit);
QUERY PLAN
-
Seq Scan on some_data (cost=0.00..727.60 rows=12480 width=8) (actual
time=1.030..110.542 rows=390 loops=1)
Filter: ('[ 2010-01-01, 2010-02-01 )'::daterange @ range(whensit))
Total runtime: 110.585 ms
(3 rows)
In any case, I suggest that as a couple steps down the road thing, it
would be desirable to have that query rewrite. Seems like a reasonable
ToDo item to consider for the future, if not in the first deployment.
Maybe that's something to add in 9.2 CommitFest #3! :-)
--
There isn't any reason why Linux can't be implemented as an
enterprise computing solution. Find out what you've been missing
while you've been rebooting Windows NT. - Infoworld
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postponing some large patches to 9.2
sfr...@snowman.net (Stephen Frost) writes: * Robert Haas (robertmh...@gmail.com) wrote: - Range Types. This is a large patch which was submitted for the first time to the last CommitFest of the cycle, and the first version that had no open TODO items was posted yesterday, three-quarters of the way through that last CommitFest. Some good review has been done. While more is probably needed, I think we should feel good about what's been accomplished and mark this one Returned with Feedback. I don't agree w/ punting Range Types. Range Types were discussed as far back as the 2010 developer meeting, were discussed quite a bit again starting in October and throughout the fall, and Jeff has regularly been posting updates to it. Given how thorough Jeff is, my feeling is that this patch is more than ready for beta. My impression is also that it's not as invasive or destablizing as the others and while it wasn't being posted to the previous commit fests, it was clearly being worked on, updated, and improved. I generally mirror those thoughts. Range Types don't seem invasive or destabilizing, and the code base has been deployed for quite some time as an extension (not quite contrib). It would be disappointing to drop this one when it is mighty close. - synchronous replication. Based on some looking at this today, I am somewhat doubtful about the possibility of me or anyone else beating this completely into shape in time for 9.2, unless we choose to extend the deadline by several weeks. Simon said that he would have time to finish this in the next two weeks, but, as noted, the CommitFest is scheduled to be over in ONE week, and it looks to me like this is still pretty rough. However, there's a lot of good stuff in here, and I think it might be practical to get some portion of it committed even if we can't agree on all of it. I recommend we give this one a little more time to shake out before giving up on it. It really would be nice to have this, but I agree that it's pretty late in the game for it to be in the state is appears to be in. :/ It also seems to have been stalled for the past couple of months, which doesn't bode well for it, in my view. The stall troubles me, and doesn't bode terribly well for 9.1. -- Rules of the Evil Overlord #39. If I absolutely must ride into battle, I will certainly not ride at the forefront of my Legions of Terror, nor will I seek out my opposite number among his army. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postponing some large patches to 9.2
pg...@j-davis.com (Jeff Davis) writes: On Tue, 2011-02-08 at 06:57 -0500, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: - Range Types. This is a large patch which was submitted for the first time to the last CommitFest of the cycle, and the first version that had no open TODO items was posted yesterday, three-quarters of the way through that last CommitFest. Some good review has been done. While more is probably needed, I think we should feel good about what's been accomplished and mark this one Returned with Feedback. I don't agree w/ punting Range Types. Range Types were discussed as far back as the 2010 developer meeting, were discussed quite a bit again starting in October and throughout the fall, and Jeff has regularly been posting updates to it. Given how thorough Jeff is, my feeling is that this patch is more than ready for beta. I appreciate the sentiment, but in addition to some cleanup, any patch like this at least requires some discussion. It's a language change we'll be supporting for a long time. At minimum, we're a couple hundred emails shy of a real consensus on the naming ;) It's more than a bit sad... The RangeType change has the massive merit of enabling some substantial development changes, where we can get rid of whole classes of comparison clauses, and hopefully whole classes of range errors. That was my favorite would-be feature for 9.1. It'll take some time to get code changes into systems to use this; the sooner the feature's in a deployable version of Postgres, the earlier that kind of thing may start. -- (format nil ~S@~S cbbrowne gmail.com) http://www3.sympatico.ca/cbbrowne/internet.html Colorless green ideas sleep furiously. -- Noam Chomsky -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OpenVMS - an effort which needs guidance and support.
peder...@ccsscorp.com (Bill Pedersen) writes: I look forward to hearing from people in the PostgreSQL community as well as from others interested in this effort. To a number of us, it's academically interesting, though, as we don't have VMS systems, it's not likely to be super-easy to assist in the matter. It certainly would be interesting to see how easy or difficult the port would be. I suspect that's a more interesting port than, say, Digital UNIX, these days. -- (reverse (concatenate 'string ofni.sesabatadxunil @ enworbbc)) http://linuxfinances.info/info/slony.html On the other hand, you have different fingers. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep for 2011CF1
dp...@pgadmin.org (Dave Page) writes: On Mon, Feb 7, 2011 at 6:55 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one week, ... Really? I thought the idea for the last CF of a development cycle was that it kept going till we'd dealt with everything. Arbitrarily rejecting stuff we haven't dealt with doesn't seem fair. Uh, we did that with 8.4 and it was a disaster. The CommitFest lasted *five months*. We've been doing schedule-based CommitFests ever since and it's worked much better. Rejecting stuff because we haven't gotten round to dealing with it in such a short period of time is a damn good way to limit the number of contributions we get. I don't believe we've agreed at any point that the last commitfest should be the same time length as the others (when we originally came up with the commitfest idea, it certainly wasn't expected), and deciding that without giving people advanced notice is a really good way to piss them off and encourage them to go work on other things. If we're going to put a time limit on this - and I think we should - we should publish a date ASAP, that gives everyone a fair chance to finish their work - say, 4 weeks. Then, if we want to make the last commitfest the same length as the others next year, we can make that decision and document those plans. There *is* a problem that there doesn't seem to be enough time to readily allow development of larger features without people getting stuck fighting with the release periods. But that's not the problem taking place here. It was documented, last May, that the final CommitFest for 9.1 was to complete 2011-02-15, and there did seem to be agreement on that. It sure looks to me like there are going to be a bunch of items that, based on the recognized policies, need to get deferred to 9.2, and the prospects for Sync Rep getting into 9.1 don't look notably good to me. Looking at things statistically, the 9.1 commitfests have had the following numbers of items: #1 - 2010-09 - 52, of which 26 were committed #2 - 2010-11 - 43, of which 23 were committed #3 - 2011-01 - 98, of which 35 have been committed, and 10 are considered ready to commit. It may appear unfair to not offer everyone a fair chance to finish their work, but it's not as if the date wasn't published Plenty Long Ago. and well-publicized. But deferring the end of the CommitFest would be Not Fair to those that *did* get their proposed changes ready for the preceding Fests. We cannot evade unfairness. It's definitely readily arguable that fairness requires that: - Items not committable by 2011-02-15 be deferred to the 2011-Next fest There are around 25 items right now that are sitting with [Waiting for Author] and [Returned with Feedback] statuses. They largely seem like pretty fair game for next fest. - Large items that weren't included in the 2010-11 fest be considered problematic to try to integrate into 9.1 There sure seem to be some large items in the 2011-01 fest, which I thought wasn't supposed to be the case. We shouldn't just impose policy for the sake of imposing policy, but I do recall Really Long CommitFests being pretty disastrous. And there's *SO* much outstanding in this particular fest that it's getting past time for doing some substantial triage so that reviewer attentions may be directed towards the items most likely to be acceptable for 9.1. I hate to think that 9.1 won't include Simon's SR material, but that may have to be. -- http://www3.sympatico.ca/cbbrowne/slony.html It's a pretty rare beginner who isn't clueless. If beginners weren't clueless, the infamous Unix learning cliff wouldn't be a problem. -- david parsons -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: RangeTypes
pg...@j-davis.com (Jeff Davis) writes:
On Fri, 2011-01-28 at 09:17 -0800, David Fetter wrote:
For consistency, and in order not to continue our atrocious naming
tradition, I'd like to propose that the above be named timestamprange
(tsrange for short) and timestamptzrange (tstzrange for short).
No real objection, but I'd like to see if someone else will second it.
Also, I don't think aliases are very easy to define. They appear to all
be special cases in the backend code, without catalog support. Should I
use domains? If not, I think we'll have to stick to one name.
Somehow, rangets, rangetstz seem better to me, but that's not a deep
issue. I'm not certain of the basis for *truly* preferring an ordering
of the components (ts/timestamp, tz, range). As long as it's rational,
and not too terribly inconsistent with other prefix/suffix handlings,
I'm fine with it.
Mind you, timestamptzrange seems a mite *long* to me.
- INTRANGE (int4)
int4range/intrange and the missing bigintrange/int8range
I thought about adding int8range, and the first time around that's what
I tried. But then I realized that the literal 4 is interpreted as an
int4, meaning that range(1,10) would be interpreted as int4range, so
int8range was slightly annoying to use because you have to cast the
literals.
Also, the storage is not particularly efficient right now anyway, so if
you need int8range, you could probably use numrange instead.
I don't mind either way. If you think someone will use it, I'll add it.
Making sure it's consistent with int4, int8, bigint sure seems like a
good idea.
Should there also be a timerange and a timetzrange?
I thought about it, and I realized that I've never seen the time type
used. Again, I'll add it if someone will use it.
Keep in mind that it's fairly easy for people to add their own range
types. The most difficult part is defining the canonical function if
it is applicable, and the subtype_float function which is necessary
for GiST.
I don't see much use for time; it is *so* likely that you'll need date
overlaps that it's difficult for it to be useful without making it
extremely magical (e.g. - stowing a lot of logic inside that adds in
date information behind the scenes).
FYI, it's compiling and testing fine for me. This one strikes me as an
exciting change, once GIST is in place. Well, actually, even without it :-).
postgres@localhost- insert into foo (dr) values ('[2010-01-01,2011-12-31)');
INSERT 0 1
postgres@localhost- select * from foo;
id | dr
+
1 | [ 2010-01-01, 2011-12-31 )
(1 row)
--
let name=cbbrowne and tld=gmail.com in String.concat @ [name;tld];;
http://linuxfinances.info/info/rdbms.html
If vegetarians eat vegetables, what do humanitarians eat?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
and...@dunslane.net (Andrew Dunstan) writes: On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Now, our committers all seem to be pretty careful people, so I don't feel strongly about having extra security on this --- but if it's easy to do, it's probably a good idea. Pushing a local topic branch by mistake seems much more likely to me. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? There seems to be a non-zero amount of value to this; I accidentally pushed some private branches into the Slony repo this afternoon, briefly, by accident. It wasn't troublesome to clean it up, so I'm not sure there's *huge* value in pushing a bunch of infrastructure into place to prevent such. If a problem: a) Is readily fixed, b) Is readily noticed, c) Gets you smacked down if you leave it unfixed, then I'm not sure it warrants going to extreme measures to prevent such a problem. -- select 'cbbrowne' || '@' || 'linuxdatabases.info'; http://linuxdatabases.info/info/slony.html If all those psychics know the winning lottery numbers, why are they all still working? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... ADD FOREIGN KEY ... NOT ENFORCED
si...@2ndquadrant.com (Simon Riggs) writes: I just wanted to point out that the patch submitted here does not allow what is requested here for FKs (nor indexes). That's fine; I was trying to support the thought that there was something useful about this idea. Being able to expressly deactivate indices seems like a reasonable thing to add as a separate TODO item, and I'll see about doing so. -- MICROS~1 has brought the microcomputer OS to the point where it is more bloated than even OSes from what was previously larger classes of machines altogether. This is perhaps Bill's single greatest accomplishment. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: compact fsync request queue on overflow
robertmh...@gmail.com (Robert Haas) writes: On Mon, Jan 17, 2011 at 8:23 PM, Greg Smith g...@2ndquadrant.com wrote: Quite. It's taken me 12 days of machine time running pgbench to find the spots where this problem occurs on a system with a reasonably sized shared_buffers (I'm testing against 256MB). It's one of those things it's hard to reproduce with test data. Thanks for the thorough code review. I've got a clear test plan I'm progressing through this week to beat on the performance measurement aspects of the patch. Any update on this? I think the test results you've posted previously - particularly, the fact that when the queue fills up, there are always many duplicates - is pretty much sufficient for us to convince ourselves that this will provide a benefit in cases where that occurs. Agreed. This showed up eminently nicely when beating up the database using pgbench. I imagine it would be interesting to run it against a different test than pgbench, particularly one which involves a larger number of tables. From the behavior I have seen thus far, I'm expecting that the queue essentially gets compressed to the size indicating the number of active tables. With pgbench, there are 4 tables, and the queue kept getting compressed to 3 or 4 entries that nicely corresponds with that. And, in cases where the queue doesn't fill up, we'll never hit the test that triggers this code, so it seems pretty clear there won't be a negative impact there either. I don't want to rush your testing process, but if it's already fairly clear that this will have some benefit, I think it would be good to get it committed and move on to working on the parts we're less sure about, like sorting writes and spreading fsyncs, where we will probably need a lot more testing than here to be sure that we have the right behavior. I'm pretty happy with what I've seen thus far; I don't want to be over-antsy about getting it all dealt with Right Quick Instantly, but it seems like a change that doesn't have a terribly bad risk of a big downside. -- (reverse (concatenate 'string ofni.secnanifxunil @ enworbbc)) The statistics on sanity are that one out of every four Americans is suffering from some form of mental illness. Think of your three best friends. If they're okay, then it's you. -- Rita Mae Brown -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Review: compact fsync request queue on overflow
I have been taking a peek at the following commitfest item: https://commitfest.postgresql.org/action/patch_view?id=497 Submission: - I had to trim a little off the end of the patch to apply it, but that's likely the fault of how I cut'n'pasted it. It applied cleanly against HEAD. - I observe that it doesn't include any alterations to documentation or to regression tests. Both aspects seem apropos, as the behaviour is entirely internal to the backend. I wouldn't expect a GUC variable for this, or SQL commands to control it. Usability Review: Does the patch actually implement that? - Establishes a hash table - Establishes skip slot array - Walks through all BGWriter requests - Adds to hash table. (I observe that it wasn't all that obvious that hash_search() *adds* elements that are missing. I got confused and went looking for hash_add() or similar. It's permissible to say dumb Chris.) - If it's a collision, then mark collision in skip slot array, and add to count - After the walk - Clean up hash table - If nothing found, clean up skip slot array, and return - If collisions found, then clear them out. Question: Is there any further cleanup needed for the entries that got dropped out of BGWriterShmem-requests? It seems not, but a leak seems conceivable. Do we want that? Eliminating a bunch of fsync() calls that are already being induced by other backends seems like a good thing, yep. Do we already have it? Evidently not! Does it follow SQL spec, or the community-agreed behavior? That doesn't seem relevant; this is well outside the scope of what SQL spec should have to say. Does it include pg_dump support (if applicable)? Definitely not applicable. Are there dangers? Possibilities... - Mentioned in the patch is the possibility of processing the set of requests in reverse order, which might in principle reduce work. But there is some danger of this changing semantics, so that reversal is not done. - Concurrent access... Is there anything that can write extra elements to BGWriterShmem-requests while this is running? I wonder if we need to have any sort of lock surrounding this? Have all the bases been covered? It is a comparatively simple change, so I wouldn't think things are missing. Feature test: - Compiled and ran regression test; no problems found. Need to do... - Validate it works as advertised - Hook up pgbench - Turn on DEBUG1 level - Watch that compacted fsync request queue from %d entries to %d entries come up It was a little troublesome inducing it. I did so by cutting shared memory to minimum (128kB). I'd regularly get entries like the following: (Note that I changed the error level to WARNING to induce logging this without getting all sorts of other stuff). CONTEXT: writing block 1735 of relation base/11933/16396 WARNING: compacted fsync request queue from 16 entries to 3 entries - lost [13] entries CONTEXT: writing block 14 of relation base/11933/16387 WARNING: compacted fsync request queue from 16 entries to 3 entries - lost [13] entries CONTEXT: writing block 4 of relation base/11933/16387 WARNING: compacted fsync request queue from 16 entries to 3 entries - lost [13] entries CONTEXT: writing block 6 of relation base/11933/16387 WARNING: compacted fsync request queue from 16 entries to 3 entries - lost [13] entries CONTEXT: writing block 1625 of relation base/11933/16396 WARNING: compacted fsync request queue from 16 entries to 4 entries - lost [12] entries CONTEXT: writing block 880 of relation base/11933/16396 WARNING: compacted fsync request queue from 16 entries to 4 entries - lost [12] entries CONTEXT: writing block 133 of relation base/11933/16396 With higher shared memory, I couldn't readily induce compaction, which is probably a concurrency matter of not having enough volume of concurrent work going on. - Corner cases? It's already a corner case ;-). - Assertion failures? None seen thus far. Performance test - Does it slow down simple cases? It surely shouldn't; compaction is only considered if the fsync queue is larger than the number of shared buffers. That doesn't seem like a simple case to me! - Does it improve performance? I haven't been able to induce it at a level that would make the improvement visible. But a database that is busy enough to have a 'full' fsync queue should surely be helped by reducing the number of fsync requests. - Does it slow down other things? In principle, the only case where it should worsen performance is if the amount of time required to: - Set up a hash table - Insert an entry for each buffer - Walk the
Re: [HACKERS] [COMMITTERS] pgsql: Implement remaining fields of information_schema.sequences view
pete...@gmx.net (Peter Eisentraut) writes: Implement remaining fields of information_schema.sequences view Add new function pg_sequence_parameters that returns a sequence's start, minimum, maximum, increment, and cycle values, and use that in the view. (bug #5662; design suggestion by Tom Lane) Also slightly adjust the view's column order and permissions after review of SQL standard. http://git.postgresql.org/gitweb?p=postgresql.git;a=blobdiff;f=src/backend/catalog/information_schema.sql;h=5b8b9417701a06b423636fe8b6e4bff91f1aa563;hp=090c10c3220e6f8b41f60ad83135830206417de4;hb=39b88432968a2f4c01c20948f12bf9c8e388474d;hpb=e657b55e661577cf664949bce78068e2922f594f Is there a particular reason that the start/min/max/inc values are now being cast to character_data, rather than integer, as they have been? I have some views I'm using that just broke because of this. I'd rather not work around this if it's not necessary, and it doesn't seem to make sense for these values to be of other than a numeric type (and likely BIGINT, as that's what pg_sequence_parameters() returns). -- (reverse (concatenate 'string ofni.secnanifxunil @ enworbbc)) http://www3.sympatico.ca/cbbrowne/slony.html E.V.A., pod 5, launching... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] C++ keywords in headers
pete...@gmx.net (Peter Eisentraut) writes: On mån, 2010-12-27 at 12:33 -0500, Andrew Dunstan wrote: On a more general point, it would be useful to have some infrastructure for running quality checks like this and publishing the results. We should be way beyond the point where we rely on individuals doing this sort of stuff. I had a Hudson service set up for things like this, but the hosting was unreliable and then the thing faded away. I could try to revive it. Careful, Oracle has been trying to claim proprietary ownership of that... http://hudson-labs.org/content/whos-driving-thing -- ``God decided to take the devil to court and settle their differences once and for all. When Satan heard of this, he grinned and said, And just where do you think you're going to find a lawyer?'' -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql expanded auto
pete...@gmx.net (Peter Eisentraut) writes: I have often found myself wanting that psql automatically switch between normal and \x mode depending on the width of the output. Would others find this useful? I haven't tested the patch, but that *does* sound generally useful. It's no fun trying to get one's eyes to visually line up output that spans 3 lines... -- select 'cbbrowne' || '@' || 'gmail.com'; http://linuxdatabases.info/info/nonrdbms.html Very little is known about the War of 1812 because the Americans lost it. -- Eric Nicol -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... ADD FOREIGN KEY ... NOT ENFORCED
t...@sss.pgh.pa.us (Tom Lane) writes: Robert Haas robertmh...@gmail.com writes: ... On the other hand, there's clearly also a use case for this behavior. If a bulk load of prevalidated data forces an expensive revalidation of constraints that are already known to hold, there's a real chance the DBA will be backed into a corner where he simply has no choice but to not use foreign keys, even though he might really want to validate the foreign-key relationships on a going-forward basis. There may well be a case to be made for doing this on grounds of practical usefulness. I'm just voicing extreme skepticism that it can be supported by reference to the standard. Personally I'd prefer to see us look into whether we couldn't arrange for low-impact establishment of a verified FK relationship, analogous to CREATE INDEX CONCURRENTLY. We don't let people just arbitrarily claim that a uniqueness condition exists, and ISTM that if we can handle that case we probably ought to be able to handle FK checking similarly. I can point to a use case that has proven useful... Slony-I deactivates indices during the subscription process, because it is enormously more efficient to load the data into the tables sans-indices, and then re-index afterwards. The same would apply for FK constraints. I observe that the deactivation of indices is the sole remaining feature in Slony-I that still requires catalog access in a corruptive sense. (With the caveat that this corruption is now only a temporary one; the indexes are returned into play before the subscription process finishes.) That would be eliminated by adding in: ALTER TABLE ... DISABLE INDEX ... ALTER TABLE ... ENABLE INDEX ... For similar to apply to FK constraints would involve similar logic. -- output = reverse(moc.liamg @ enworbbc) http://linuxdatabases.info/info/rdbms.html The code should be beaten into submission -- Arthur Norman -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] would hw acceleration help postgres (databases in general) ?
j...@nasby.net (Jim Nasby) writes: On Dec 10, 2010, at 6:18 PM, Jeff Janes wrote: On Fri, Dec 10, 2010 at 3:09 PM, Hamza Bin Sohail hsoh...@purdue.edu wrote: Hello hackers, I think i'm at the right place to ask this question. Based on your experience and the fact that you have written the Postgres code, can you tell what a rough break-down - in your opinion - is for the time the database spends time just fetching and writing stuff to memory and the actual computation. The database is a general purpose tool. Pick a bottleneck you wish to have, and probably someone uses it in a way that causes that bottleneck to occur. A common bottleneck we run into is sorting of text data. Unfortunately, I doubt that a GPU would be able to help with that. Actually, that is a case where some successful experimentation has been done. http://www.cs.cmu.edu/afs/cs.cmu.edu/Web/People/ngm/15-823/project/Final.pdf Making it reliable to the point of being generally usable when someone installs Postgres via a generic packaging tool in default fashion may be somewhat more challenging! But it appears that sorting is a plausible application for GPUs. -- output = (cbbrowne @ linuxdatabases.info) The right honorable gentleman is reminiscent of a poker. The only difference is that a poker gives off the occasional signs of warmth. -- Benjamin Disraeli on Robert Peel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Anyone for SSDs?
loureir...@gmail.com (Daniel Loureiro) writes: You can believe whatever you want, that doesn't make it true. completely agree. Like yours, Its just my point of view, not the reality. I agree with some points here, but I wondering how many good ideas are killed with the thought: this will be a performance killer with so many random access, lets discarded it. An quicksort method in sequential disk its just awful to be thinking in a non SSD world, but its possible in an SSD. If in 80's the sequential access has more cost compared with random access will be the PostgreSQL in the same design that it have nowadays ? What turns out to be surprising is that the behaviours of new kinds of media not infrequently retrieve the usefulness of algorithms designed for elderly sorts of media. The entertaining one, on Postgres, has been that the behaviour of fairly large amounts of memory, as compared to the slower access rates for disk has led to retrieving tape-oriented algorithms from thought-to-be-obsolete literature. I don't think it's too likely that SSD changes this. But what is rather interesting is that the issue *isn't* one of how fast it is to sort things on disk - it is of how to sort in memory. It's quite feasible to load blocks of data into memory, sort in memory, via [some means], and then do tape merges to get the fully ordered result that is then sequentially written out to disk. Replacing [some means] with Quicksort is a plausible idea. I doubt it'd be an improvement on what is already there, but there's already room for it to work. -- let name=cbbrowne and tld=gmail.com in name ^ @ ^ tld;; http://linuxdatabases.info/info/nonrdbms.html Outside of a dog, a book is man's best friend. Inside of a dog, it's too dark to read. -Groucho Marx -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
t...@sss.pgh.pa.us (Tom Lane) writes: Kevin Grittner kevin.gritt...@wicourts.gov writes: Robert Haas robertmh...@gmail.com wrote: Simon Riggs si...@2ndquadrant.com wrote: Note that DB2 uses the table modifier VOLATILE to indicate a table that has a widely fluctuating table size, for example a queue table. the fact that DB2 uses that word to mean something entirely different is certainly a bit awkward It would be especially awkward should someone port their DB2 database to PostgreSQL without noticing the semantic difference, and then find their data missing. Not to mention that DB2 syntax tends to appear in the standard a few years later. And the term volatile has well-understood connotations that are analagous to those in DB2 in the C language and various descendants. http://en.wikipedia.org/wiki/Volatile_variable I'm not sure UNLOGGED is perfect... If TEMPORARY weren't already taken, it would be pretty good. Other possibilities include TRANSIENT, EPHEMERAL, TRANSIENT, TENUOUS. FLASH would be an amusing choice. PostgreSQL 9.1, now with support for FLASH! -- output = (cbbrowne @ acm.org) http://linuxdatabases.info/info/internet.html I've told you for the fifty-thousandth time, stop exaggerating. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Final(?) proposal for wal_sync_method changes
x...@thebuild.com (Christophe Pettus) writes: On Dec 7, 2010, at 2:43 PM, Josh Berkus wrote: Because nobody sane uses OSX on the server? The XServe running 10.5 server and 9.0.1 at the other end of the office takes your remark personally. :) I'd heard that Apple had cancelled XServe. [Poking back at that...] Yep, they won't be carrying anything for sale that's particularly rack-mountable after next January. Not precisely dead, but definitely moving on smelling funny... -- (format nil ~...@~s cbbrowne gmail.com) http://www3.sympatico.ca/cbbrowne/emacs.html Photons have mass? I didn't know they were catholic! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrent MERGE
robertmh...@gmail.com (Robert Haas) writes: On Thu, Aug 5, 2010 at 11:43 AM, Simon Riggs si...@2ndquadrant.com wrote: Looks like MERGE is progressing well. At 2010 Dev Mtg, we put me down to work on making merge work concurrently. That was garbled slightly and had me down as working on predicate locking which is the general solution to the problem. Do we still need me to work on concurrent MERGE, or is that included in the current MERGE patch (can't see it), or is that covered elsewhere (for example Kevin Grittner's recent work)? Still happy to do work as proposed, just checking still required. I suspect Kevin's patch will solve it if using a sufficiently high transaction isolation level, but something else might be needed otherwise. However, I confess to ignorance as to the underlying issues? Why is MERGE worse in this regard than, say, UPDATE? It's worse than UPDATE because - It could be an INSERT, if the data's new, but - If the data's there, it becomes an UPDATE, but - If some concurrent update has just DELETEd the data that's there, it becomes an INSERT again, but - Oops, that DELETE rolled bac, so it's an UPDATE again... Recurse as needed to make it more undecidable as to whether it's really an INSERT or an UPDATE :-). -- Rules of the Evil Overlord #208. Members of my Legion of Terror will attend seminars on Sensitivity Training. It's good public relations for them to be kind and courteous to the general population when not actively engaged in sowing chaos and destruction. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Two different methods of sneaking non-immutable data into an index
mmonc...@gmail.com (Merlin Moncure) writes: On Wed, Aug 4, 2010 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Aug 4, 2010 at 6:43 PM, Merlin Moncure mmonc...@gmail.com wrote: *) also, isn't it possible to change text cast influencing GUCs 'n' times per statement considering any query can call a function and any function can say, change datestyle? Shouldn't the related functions be marked 'volatile', not stable? This is just evil. It seems to me that we might want to instead prevent functions from changing things for their callers, or postponing any such changes until the end of the statement, or, uh, something. We can't afford to put ourselves in a situation of having to make everything volatile; at least, not if performance is anywhere in our top 50 goals. yeah -- perhaps you shouldn't be allowed set things like datestyle in functions then. I realize this is a corner (of the universe) case, but I can't recall any other case of volatility being relaxed on performance grounds... :-). Maybe a documentation warning would suffice? That would cause grief for Slony-I, methinks, and probably other things that behave somewhat similar. The logtrigger() function coerces datestyle to ISO, so that when dates get stored, they are stored in a canonical form, irrespective of an individual connection's decisions on datestyle, so we don't have to include datestyle information as part of the replicated data. -- output = reverse(moc.liamg @ enworbbc) http://linuxfinances.info/info/postgresql.html Chaotic Evil means never having to say you're sorry. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Two different methods of sneaking non-immutable data into an index
mmonc...@gmail.com (Merlin Moncure) writes: On Thu, Aug 5, 2010 at 12:59 PM, Chris Browne cbbro...@acm.org wrote: mmonc...@gmail.com (Merlin Moncure) writes: On Wed, Aug 4, 2010 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Aug 4, 2010 at 6:43 PM, Merlin Moncure mmonc...@gmail.com wrote: *) also, isn't it possible to change text cast influencing GUCs 'n' times per statement considering any query can call a function and any function can say, change datestyle? Shouldn't the related functions be marked 'volatile', not stable? This is just evil. It seems to me that we might want to instead prevent functions from changing things for their callers, or postponing any such changes until the end of the statement, or, uh, something. We can't afford to put ourselves in a situation of having to make everything volatile; at least, not if performance is anywhere in our top 50 goals. yeah -- perhaps you shouldn't be allowed set things like datestyle in functions then. I realize this is a corner (of the universe) case, but I can't recall any other case of volatility being relaxed on performance grounds... :-). Maybe a documentation warning would suffice? That would cause grief for Slony-I, methinks, and probably other things that behave somewhat similar. The logtrigger() function coerces datestyle to ISO, so that when dates get stored, they are stored in a canonical form, irrespective of an individual connection's decisions on datestyle, so we don't have to include datestyle information as part of the replicated data. hm -- interesting -- couldn't that cause exactly the sort of situation though where stability of statement is violated? It shouldn't... The data gets stored physically, on disk, in a canonical form. Why should it be unstable to capture data in a canonical form, when that's what gets stored on disk? -- (format nil ~...@~s cbbrowne gmail.com) The statistics on sanity are that one out of every four Americans is suffering from some form of mental illness. Think of your three best friends. If they're okay, then it's you. -- Rita Mae Brown -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SHOW TABLES
si...@2ndquadrant.com (Simon Riggs) writes: Just for the record, I've never ever met anyone that said Oh, this \d syntax makes so much sense. I'm a real convert to Postgres now you've shown me this. The reaction is always the opposite one; always negative. Which detracts from our efforts elsewhere. If we're opening up the code to change this, it makes a lot of sense to try to Do It Really Right so that we're not going over this again and again. I think we're seeing several things that suck, and I'm quite sure I have not yet heard an answer that resolves it all. Things that have become clear: 1. \d isn't exactly the most intuitive thing ever And it's pretty clear that we have been heading into some increasingly cryptic bits of fruit salad of \dfzb+-meta-bucky-alt-foo Having SHOW THIS and SHOW THAT which are a bit more readily guessed would be somewhat nice. 2. information_schema doesn't have some useful things that we'd like it to have Listing databases would be nice. Unfortunately, ANSI didn't define a way to do that, so we can't add it. Alas, I don't see a good way to improve on this :-( 3. The \? commands are *solely* for psql, and it would be nice to have the Improvement work on server side so it's not only usable with the one client. 4. It would be Mighty Useful for whatever extensions get defined server-side to also be relational so that they can be usefully scripted in ways NOT vulnerable to screen size, output hackery, and such. - I've seen too many QA scripts that do awk parsing of output of psql \d commands that are vulnerable to all kinds of awfulness. Add an updated-on column to the output, and suddenly everything breaks. - I'd sure like to be able to write queries that *don't* involve array smashing or using grep on \z output to analyze object permissions. - \? output is often *not* amenable to this, as it sometimes has extra bits of data hierarchy in it. And array aggregation. There's a certain risk of things being overspecified such that there's *no* solution, but I don't think that forcibly *has* to happen. But the answers I'm seeing thus far run slipshod across too many of these things, so I don't see that we have arrived at actual solutions yet. -- let name=cbbrowne and tld=gmail.com in String.concat @ [name;tld];; http://linuxfinances.info/info/linuxdistributions.html Rules of the Evil Overlord #77. If I have a fit of temporary insanity and decide to give the hero the chance to reject a job as my trusted lieutentant, I will retain enough sanity to wait until my current trusted lieutenant is out of earshot before making the offer. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Buildfarm + Git tryouts
I'm trying to start preparing buildfarm nodes for the upcoming Git migration, and have run into a few issues. I speculate that -hackers is one of the better places for this to get discussed; if it should be elsewhere, I'm sure Andrew Dunstan won't be shy to redirect this :-). What I was hoping to do was to run nodes for a little while against the repo at git://github.com/oicu/pg-cvs-mirror.git to validate that it all works against git. The best instructions available thus far... http://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto Unfortunately, I'm finding some anomalies in terms of differences between my environments and what that suggests. 1. git clone --mirror is apparently new in git version 1.6 http://kerneltrap.org/mailarchive/git/2008/8/2/2793244 My somewhat mouldy Ubuntu node is on git 1.5.4, which doesn't include this. There's a workaround, as --mirror is just syntactic sugar $ git clone --mirror $URL may be replaced by $ git clone --bare $URL $ (cd $(basename $URL) git remote add --mirror origin $URL) I see options: a) Require git 1.6 or so (latest stable is 1.7.1.1) b) Offer the alternative mirror approach 2. SCM.pl seems to expect the repository directory to be called pgsql (around line #373) I wound up hacking up the code to head to a specific directory, which isn't a terribly good hack, but apparently works for now. - chdir 'pgsql'; + chdir '/opt/build-farm/pgsql.git'; It feels as though something's not quite right about how the nearby Git repository is referenced. 3. Some problems checking status. i) Status Line: 491 bad ts parameter - [timestamp omitted] is in the future I know my clock's reasonable - ntp is reporting I'm within 0.25s of some stratum 2 nodes. Is it possible that the buildfarm server is ill-synced? ii) Status Line: 460 script version too low I just pulled a buildfarm build last week, so I'm not sure what this would relate to. That's where I've gotten thus far. Node caracara isn't reporting in just now because of this testing. I hope other people ought to be doing similar tryouts so that the revision effort to shift to Git doesn't wait to *start* until the development repo shifts. -- let name=cbbrowne and tld=gmail.com in String.concat @ [name;tld];; http://linuxfinances.info/info/linuxdistributions.html Rules of the Evil Overlord #77. If I have a fit of temporary insanity and decide to give the hero the chance to reject a job as my trusted lieutentant, I will retain enough sanity to wait until my current trusted lieutenant is out of earshot before making the offer. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Check constraints on non-immutable keys
mag...@hagander.net (Magnus Hagander) writes: I concur with the thought that the most useful solution might be a way to tell pg_restore to remove or disable check constraints. Uh, say what? Are you saying pg_restore should actually remove something from the database schema? And thus no longer be valid for taking database backups? Or are you just saying that it should have the constraints off, load the data, and then somehow create the constraint without having it validate the exinsting data (like the NOCHECK option in MSSQL?) Well, consider the way that foreign keys are handled by pg_dump. - Initially, it dumps out the table schema, NOT replete with foreign key constraints. - Data is loaded, *without* checking foreign keys. - Foreign keys are added in, afterwards. That's not a scenario where constraints are ignored - their evaluation is merely deferred. For constraints that involve dates, I can certainly see a potential for foot guns. It points me towards making sure that our apps don't do over-aggressive things like having constraints to prevent data from being inserted back-dated, as that would cause restores of backups to break. That's a dangerous kind of constraint. It's *possible* that it would be an idea to apply the check constraint late in the pg_dump, so that the ill effects might be imagined to be alleviated. That seems rather wishful. -- output = (cbbrowne @ gmail.com) http://linuxdatabases.info/info/languages.html HEADLINE: Suicidal twin kills sister by mistake! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Exposing the Xact commit order to the user
br...@momjian.us (Bruce Momjian) writes: Jan Wieck wrote: The point is not that we don't have that information now. The point is having a hint BEFORE wading through possibly gigabytes of WAL or log data. If getting that information requires to read all the log data twice or the need to read gigabytes of otherwise useless WAL data (as per Bruce's suggestion), we better not get it at all and just keep doing what we are doing now. I actually have a hard time understanding why people are so opposed to a feature that has zero impact at all unless a DBA actually turns in ON. What is the problem with exposing the commit order of transactions? If you want to fork Postgres and add it, go ahead, but if the community has to maintain the code and document it, we care. Are you caring or opposing? It seems rather uncharitable to imply that Jan doesn't care. I know *I'm* not interested in a forked Postgres for this - I would prefer to find out what things could be done that don't involve gross amounts of WAL file grovelling for data that mayn't necessarily even be available. -- select 'cbbrowne' || '@' || 'cbbrowne.com'; http://cbbrowne.com/info/internet.html MS apparently now has a team dedicated to tracking problems with Linux and publicizing them. I guess eventually they'll figure out this back fires... ;) -- William Burrow aa...@delete.fan.nb.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Exposing the Xact commit order to the user
gsst...@mit.edu (Greg Stark) writes:
On Wed, Jun 2, 2010 at 6:45 PM, Chris Browne cbbro...@acm.org wrote:
It would make it easy to conclude:
This next transaction did 8328194 updates. Maybe we should do
some kind of checkpoint (e.g. - commit transaction or such) before
working on it.
versus
This transaction we're thinking of working on had 7 updates. No
big deal...
I'm puzzled how you would define this value. How do you add 7 inserts,
7 deletes, and 7 updates? Is that 21 rows modified? Why are the 7
inserts and 7 deletes worth twice as much as the 7 updates when
they're basically the same thing? What if the inserts fired triggers
which inserted 7 more rows, is that 14? What if the 7 updates modified
2 TB of TOAST data but the 8238194 updates were all to the same record
and they were all HOT updates so all it did was change 8kB?
The presence of those questions (and their ambiguity) is the reason
why there's a little squirming as to whether this is super-useful and
super-necessary.
What this offers is *SOME* idea of how much updating work a particular
transaction did. It's a bit worse than you suggest:
- If replication triggers have captured tuples, those would get
counted.
- TOAST updates might lead to extra updates being counted.
But back to where you started, I'd anticipate 7 inserts, 7 deletes,
and 7 updates being counted as something around 21 updates.
And if that included 5 TOAST changes, it might bump up to 26.
If there were replication triggers in place, that might bump the count
up to 45 (which I chose arbitrarily).
In any case you'll have all the actual data from your triggers or
hooks or whatever so what value does having the system keep track of
this add?
This means that when we'd pull the list of transactions to consider,
we'd get something like:
select * from next_transactions('4218:23', 50);
[list of 50 transactions returned, each with...
- txid
- START timestamp
- COMMIT timestamp
- Approximate # of updates
Then, for each of the 50, I'd pull replication log data for the
corresponding transaction.
If I have the approximate # of updates, that might lead me to stop
short, and say:
That next update looks like a doozy! I'm going to stop and commit
what I've got before doing that one.
It's not strictly necessary, but would surely be useful for flow
control.
--
select 'cbbrowne' || '@' || 'cbbrowne.com';
http://cbbrowne.com/info/internet.html
MS apparently now has a team dedicated to tracking problems with
Linux and publicizing them. I guess eventually they'll figure out
this back fires... ;) -- William Burrow aa...@delete.fan.nb.ca
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Exposing the Xact commit order to the user
d...@csail.mit.edu (Dan Ports) writes: I'm not clear on why the total rowcount is useful, but perhaps I'm missing something obvious. It would make it easy to conclude: This next transaction did 8328194 updates. Maybe we should do some kind of checkpoint (e.g. - commit transaction or such) before working on it. versus This transaction we're thinking of working on had 7 updates. No big deal... -- (reverse (concatenate 'string ofni.secnanifxunil @ enworbbc)) http://linuxfinances.info/info/finances.html Rules of the Evil Overlord #189. I will never tell the hero Yes I was the one who did it, but you'll never be able to prove it to that incompetent old fool. Chances are, that incompetent old fool is standing behind the curtain. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Exposing the Xact commit order to the user
heikki.linnakan...@enterprisedb.com (Heikki Linnakangas) writes: On 24/05/10 19:51, Kevin Grittner wrote: The only thing I'm confused about is what benefit anyone expects to get from looking at data between commits in some way other than our current snapshot mechanism. Can someone explain a use case where what Jan is proposing is better than snapshot isolation? It doesn't provide any additional integrity guarantees that I can see. Right, it doesn't. What it provides is a way to reconstruct a snapshot at any point in time, after the fact. For example, after transactions A, C, D and B have committed in that order, it allows you to reconstruct a snapshot just like you would've gotten immediately after the commit of A, C, D and B respectively. That's useful replication tools like Slony that needs to commit the changes of those transactions in the slave in the same order as they were committed in the master. I don't know enough of Slony et al. to understand why that'd be better than the current heartbeat mechanism they use, taking a snapshot every few seconds, batching commits. I see two advantages: a) Identifying things on a transaction-by-transaction basis means that the snapshots (syncs) don't need to be captured, which is presently an area of fragility. If the slon daemon falls over on Friday evening, and nobody notices until Monday, the snapshot reverts to being all updates between Friday and whenever SYNCs start to be collected again. Exposing commit orders eliminates that fragility. SYNCs don't need to be captured anymore, so they can't be missed (which is today's problem). b) The sequence currently used to control log application ordering is a bottleneck, as it is a single sequence shared across all connections. It could be eliminated in favor of (perhaps) an in-memory variable defined on a per-connection basis. It's not a bottleneck that we hear a lot of complaints about, but the sequence certainly is a bottleneck. -- select 'cbbrowne' || '@' || 'cbbrowne.com'; http://cbbrowne.com/info/internet.html MS apparently now has a team dedicated to tracking problems with Linux and publicizing them. I guess eventually they'll figure out this back fires... ;) -- William Burrow aa...@delete.fan.nb.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add column if not exists (CINE)
robertmh...@gmail.com (Robert Haas) writes: On Wed, Apr 28, 2010 at 1:40 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Robert Haas robertmh...@gmail.com writes: Well, how would you define CREATE OR REPLACE TABLE? I think that doesn't make much sense, which is why I think CREATE IF NOT EXISTS is a reasonable approach. hand waving time The behavior I'd like to have would be to allow me to give a SELECT query to run for replacing what is there if there's something. If the query can not be run on the existing data set, error out of course. So you know the state for sure after the command, but it depends on your query being correct. And you can (de)normalize existing data using joins. The REPLACE keyword would here mean that there's a CTAS going under the hood, then we add the constraints and indexes and triggers etc. That would mean being able to express those entities changes too, but it seems important. Well, that may be not precise enough as a spec, but at least that's food for though I hope. This type of hand-waving convinces me more than ever that we should just implement CINE, and it should just C if it doesn't already E. This is what has been requested multiple times, by multiple people, including various people who don't normally poke their head into -hackers. I think the resistance to a straightforward implementation with easy-to-understand behavior is completely unjustifiable. It's completely unobvious to me that all of the above will work at all and, if it did, whether it would actually solve the problems that I care about, like being able to write schema-upgrade scripts that would work in a simple and predictable fashion. I tend to agree with you here. While yes, CINE is a simplification of COR (CREATE OR REPLACE), I'm not at all sure that it's reasonable to hope for the latter, in that it elides potentially grave problems that aren't reasonable to expect solved. Notably, the and what if a substantial data transformation is needed to accomplish this? CINE doesn't propose to try to do that transformation, which seems like the right choice to me. When I put my we've got things replicating using Slony-I hat on, CINE looks pretty preferable to me. It's unambitious - but it is certainly NOT doing a bunch of magic behind your back so as to make it tougher to predict what might happen in a trigger-replicated environment. In any case, CINE seems pretty useful to me. I'm prepared to listen to persuasion, but thus far, it looks like a +1 from me. An alternative that seems likable is COR, raising an exception if there's a type mismatch. Where there's certainly room to debate how much of a difference represents a mismatch. -- cbbrowne,@,gmail.com http://linuxfinances.info/info/wp.html Predestination was doomed from the start. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: Add JSON support
robertmh...@gmail.com (Robert Haas) writes: On Tue, Mar 30, 2010 at 8:58 PM, Josh Berkus j...@agliodbs.com wrote: I'd think that you could get quite a long ways on this, at least doing something like dbslayer without *necessarily* needing to do terribly much work inside the DB engine. There's actually an HTTP framework tool for Postgres which already does something of the sort. It was introduced at pgCon 2 years ago ... will look for. While it might be interesting to have/find/write a tool that puts an HTTP/JSON layer around the DB connection, it's pretty much entirely unrelated to the proposed project of creating a json type with PostgreSQL analagous to the xml type we already have, which is what the OP is proposing to do. Personally, I suspect that a JSON type is both a more interesting project to work on and a more useful result for this community. No disagreement here; I'd expect that a JSON type would significantly ease building such a framework. Indeed, that could be a demonstration of success... We then implemented an HTTP/JSON proxy in 27 lines of Python code... :-) -- Unless you used NetInfo. _Then_ changing network settings could often require torching of the existing system, salting of the ground it had rested on, and termination of anyone who used it. -- JFW jwi...@biff.com on comp.sys.next.advocacy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: Add JSON support
joeyadams3.14...@gmail.com (Joseph Adams) writes: I introduced myself in the thread Proposal: access control jails (and introduction as aspiring GSoC student), and we discussed jails and session-local variables. But, as Robert Haas suggested, implementing variable support in the backend would probably be way too ambitious a project for a newbie like me. I decided instead to pursue the task of adding JSON support to PostgreSQL, hence the new thread. Interesting... I had a discussion about much this sort of thing with a local LUG associate; he was interested in this from a doing CouchDB-ish things using PostgreSQL perspective. There were a couple perspectives there, which may be somewhat orthogonal to what you're trying to do. I'll mention them as they may suggest useful operations. 1. Buddy Myles pointed out a NYTimes project which does something pretty analagous... http://code.nytimes.com/projects/dbslayer This is a proxy that allows clients to submit requests via HTTP, returning responses in JSON form. Note that the HTTP request has the SQL query embedded into it. 2. CouchDB's interface is much the same, where clients submit HTTP requests and receive JSON responses back, but with the difference that the query is a stylized sorta-JSON form. I'd think that you could get quite a long ways on this, at least doing something like dbslayer without *necessarily* needing to do terribly much work inside the DB engine. Mapping a tuple, or a list of tuples, into a forest of JSON documents should be pretty straightforward; whether or not it's really desirable to operate a JSON-flavoured query inside PostgreSQL may be the difference between *this year's* GSOC and *next year's* :-). -- ...the Jedi learned early on what language the universe was programmed in. Then they took advantage of an accident of language to obscure this fact from the unwashed. They all affected an inverted lisp. so, a Jedi to be, you the Forth must use. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] An idle thought
si...@2ndquadrant.com (Simon Riggs) writes: On Tue, 2010-03-16 at 15:29 +, Greg Stark wrote: big batch delete Is one of the reasons for partitioning, allowing the use of truncate. Sure, but it would be even nicer if DELETE could be thus made cheaper without needing to interfere with the schema. The concurrency issue might be resolved (*might!*) by the following complication... - A delete request is looking at a page, and concludes, oh, all the tuples here are now marked dead!. - It flags the page as *possibly* dead. Almost what Greg suggests for the visibility map, but this is just marking it as proposed dead. - It throws the page number, along with xid, into a side map. When something wants to do something with the page (e.g. - vacuum), it sees that it's possibly dead, and looks at the side map for the list of xids that wanted to mark the page dead. for each xid: if xid is still active do nothing with it else remove xid entry from the map if all xids were failed remove flag from page if any xid committed empty the page; the tuples are all dead I'm less confident about that last clause - I *think* that if *any* page-clearing XID is found, that means the page is well and truly clear, doesn't it? The extra map mayn't be a nice thing. It's food for thought, anyways. -- let name=cbbrowne and tld=linuxfinances.info in String.concat @ [name;tld];; The real problem with the the year 2000 is that there are too many zero bits and that adversely affects the global bit density. -- Boyd Roberts b...@france3.fr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL compatibility reminder: MySQL vs PostgreSQL
francois.pe...@free.fr (François Pérou) writes: * I am very surprised by the SQL level of Php developers. The example Drupal developers trying to rewrite SQL queries dynamically adding DISTINCT clause is just an example. So don't expect them to understand the difference between MySQL and PostgreSQL. It is out of reach. They focuse on Php code. If they refuse to contemplate suggestions as to how to write more portable SQL queries that would work with databases other than MySQL, then I don't know what constructive discussion there can be about this. I believe that PostgreSQL should support more MySQLisms in order to BEAT MySQL. Why, when the MySQLisms in questions are divergences from the SQL standards, and the issue seems to bite all the other databases in more or less the same way? It doesn't seem to me that the answer to Drupal developers refuse to consider writing portable SQL is to try to impose MySQL's divergences from SQL onto all the *other* DBMSes. -- Have you noticed that, when we were young, we were told that `everybody else is doing it' was a really stupid reason to do something, but now it's the standard reason for picking a particular software package? -- Barry Gehm -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Anyone know if Alvaro is OK?
scrawf...@pinpointresearch.com (Steve Crawford) writes: Marc G. Fournier wrote: Is there a higher then normal amount of earthquakes happening recently? haiti, japan just had one for 6.9, there was apparently one in illinos a few weeks back, one on the Russia/China/N.Korean border and now Chile? Random events come in bunches - something I always stop to remind myself of whenever there is a sudden bunch of quakes, celebrity deaths, plane crashes, etc. Especially with relatively unusual events like great-quakes and plane crashes, it can be tough to see if there is any signal in the noise - a job I have to leave to experienced statisticians. I'll nit pick a little bit... Random events are often *noticed* when there is some reason to think it's an unusually large batch. Nobody really notices the carnage on the highways, because, stochastically, there are such a large number of events, both positive and negative (e.g. - millions of people making it home safely, and a tiny number that don't) that it's difficult for there to be a sufficiently large number of adverse events to notice. People are a lot more worried about terrorists than about car accidents, even though the latter are *enormously* more likely to cause one's demise, by a *huge* factor. (This mismeasurement irritates me a lot, particularly when I visit airports!) 2010 has had more news about earthquakes than other nearby years, but as you say, it is not obvious that there is any signal to be found in the noise. http://en.wikipedia.org/wiki/List_of_deadly_earthquakes_since_1900 is an interesting list. Very few quakes are listed for 2009; I wonder if this results from events not being reported yet? Preceding years consistently have quite a lot of deadly earthquakes, dating back for many years. From that list, Chile has been seeing pretty potent earthquakes on a regular basis since 1905. I'm mighty glad to hear that Alvaro is OK, and that things weren't too disastrously shaken up, for him. -- (format nil ~...@~s cbbrowne gmail.com) http://linuxfinances.info/info/lsf.html Sponges grow in the ocean. I wonder how much deeper the ocean would be if that didn't happen. -- Steven Wright -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OpenVMS?
rocr...@gmx.de (Robert Doerfler) writes: On Tue, 16 Feb 2010, Bruce Momjian wrote: Marc G. Fournier wrote: On Tue, 16 Feb 2010, Bruce Momjian wrote: I hate to pour cold water on this, but why is it worth adding support for a platform that has such marginal usage. Because someone feels like dedicating their resources to it ... ? Well, there is going to be impact on the community too --- patches, testing, etc. The community effort is small, but isn't zero. But its totally worth it. Do we have a patch yet? If it's small and easy, then that points to you being right. If ugly and hairy... then not so much... -- (format nil ~...@~s cbbrowne gmail.com) http://www3.sympatico.ca/cbbrowne/nonrdbms.html I think you ought to know I'm feeling very depressed -- Marvin the Paranoid Android -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OpenVMS?
scra...@hub.org (Marc G. Fournier) writes: On Tue, 16 Feb 2010, Bruce Momjian wrote: I hate to pour cold water on this, but why is it worth adding support for a platform that has such marginal usage. Because someone feels like dedicating their resources to it ... ? But adding it in would require *some* degree of thought on the part of committers as to what might break VMS builds? If someone wants to make a branch to run on VMS, that would be a well and fine thing. It is quite likely that once we're on Git, that might be easily managed by having a repo which feeds off official releases, modifying only in those places where VMS-specific changes are required. It might even turn out to be the case that the patches are sufficiently small and undemanding that it would turn out to be easy to merge into the official release, so as to make it an official platform. I'd not want to assume that up front, though. -- output = reverse(moc.liamg @ enworbbc) http://linuxdatabases.info/info/slony.html As long as each individual is facing the TV tube alone, formal freedom poses no threat to privilege. --Noam Chomsky -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LISTEN/NOTIFY and notification timing guarantees
t...@sss.pgh.pa.us (Tom Lane) writes: Merlin Moncure mmonc...@gmail.com writes: On Tue, Feb 16, 2010 at 10:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: 2. Add an extra lock to serialize writers to the queue, so that messages are guaranteed to be added to the queue in commit order. As long as fwiw, I think you're definitely on the right track. IMO, any scenario where an issued notification ends up being deferred for an indefinite period of time without alerting the issuer should be avoided if at all possible. Just to clarify though, does your proposal block all notifiers if any uncommitted transaction issued a notify? It will block other notifiers until the transaction releases its locks, which should happen pretty promptly --- there are no user-accessible reasons for it to wait. I have heard of reasons to want to be able to have some actions run at COMMIT time. You probably recall Jan's proposal of a commit time timestamp. The particular implementation may have fallen by the wayside, but the reasons to want such things do continue to be. Indeed an on commit trigger hook would be a mighty valuable thing to support things like (but not restricted to) commit timestamps. It's conceivable that clustering issues might introduce some somewhat more user-accessible hooks that could cost something here. Certainly not true today, but plausibly foreseeable... -- select 'cbbrowne' || '@' || 'linuxfinances.info'; http://www3.sympatico.ca/cbbrowne/lsf.html Beauty is the first test: there is no permanent place in the world for ugly mathematics. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
kevina...@hotmail.com (Kevin Ar18) writes: Of course all of this is from the perspective of Python users. Of course, you have your own features that you want from your end (from PostgreSQL's perspective). Perhaps this info would help you to know which avenue to pursue. No, those seem like fine ways of getting a good perspective on the issue. I happen not to use Python much, so there's a certain aspect of don't care on my part... but that doesn't imply that my PostgreSQL perspective would tend to override yours. Instead, I think that the Python users' perspective *is* a mighty important thing. The interface needs aspects of cleanness on both sides of the interface... - On the Python side, it needs to play well in a variety of ways that you seem to have described nicely, some technical, some licensing oriented. Some relating to interfacing to further bits of Python and to applications and frameworks written in Python. - On the PostgreSQL side, there's certainly a preference for licensing simplicity. Note that most of the issues there really lie on the Python side, which underlines the importance of Python users' perspective. Further, the ideal and issues/problems that you point out all seem reasonable. The good seems good and the bad seems like things that do indeed need to be accepted as consequences of the good. It will doubtless help guide assistance. -- output = reverse(moc.liamg @ enworbbc) ...as a robotics designer once told me, you don't really appreciate how smart a moron is until you try to design a robot... -- Jerry Pournelle -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
bada...@gmail.com (Alex Hunsaker) writes: On Fri, Jan 8, 2010 at 02:03, Magnus Hagander mag...@hagander.net wrote: You can always create your own branch with just the .gitignore files and merge that into whatever you're working on :) The only thing annoying about that is if you generate diffs ala git diff origin/master.. you get your .gitignore in it. What I do is have a .gitignore that is gitignored. That way its not committed, its on any branch i switch to or make and I don't accidentally commit it. I'd put that in $GITHOME/.git/info/exclude That's specifically what that file's for... -- output = reverse(moc.liamg @ enworbbc) http://www3.sympatico.ca/cbbrowne/slony.html DSK: STAN.K; ML EXIT -- FILE NOT FOUND -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
robertmh...@gmail.com (Robert Haas) writes:
On Fri, Jan 8, 2010 at 10:12 AM, Dave Page dp...@pgadmin.org wrote:
I have long spoken against making Windows a second class citizen. But I
don't think David is going to do that (and I'll hound him if he does). But
that doesn't mean it has to be fully supported from day one.
I'm not saying it should be supported from day 1, but I think the
initial plan will make it very difficult to add Windows support later
without a great deal of rewriting/redesign. It's lack of forward
planning I was objecting to.
I personally suspect that the client is not the most important part of
this project. I think the value of CPAN is for searching, more than
auto-installing. Personally, I never use the auto-install feature
because I always want more control than you get that way. I just use
the site to find possible modules and browse the docs, and then if I
find something I like I check with I can pull it from the Red Hat
repos with rpm, and if not I download it and look it over to see if it
DWIW, and then if so I usually make a private SRPM for it and install
from that. I'd be happy if we just had a good search-and-download
site.
If PGAN leads to us having:
a) A database containing a useful set of metadata about a large set of
extensions, and
b) A way for PostgreSQL developers and binary distribution makers (who
*do* have GCC / XCode / MingW / Visual Studio / ... available to
them) to easily:
- build
- test
- try out
- think about how to package
that large set of extensions
then we've got a Big Win of the same sort as CPAN, Ruby Gems, and PyPI.
It does NOT need to include installers for every known kind of computer;
that is a *second* problem, which actually requires a series of
solutions for:
a) Fedora
b) Debian (hence derivatives like Ubuntu)
c) BSD Ports
d) Yes, Windows
e) I think Solaris has something new for packaging...
If David gets it to the point where it's easy to build and install
extensions into a PostgreSQL installation, then turning that into
packages for specific targets should be a not-insurmountable problem
that may be treated separately.
That having been said, we should consider our filesystem layout
carefully however to make sure that if we want to provide things like
Windows installers in the future, we have a clean way to do that.
If the extensions get installed in a way that is scalable in the sense
that it's not a particularly big deal to write a script that pulls 250
extensions and installs them on a particular host for a particular PG
installation, then I'd think that the exercise has been a successful
one.
That leads, naturally enough, to an Extension BuildFarm :-).
I'd be somewhat surprised if the use of Windows was a material factor in
the matter.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','acm.org').
http://www3.sympatico.ca/cbbrowne/postgresql.html
Laugh-a while you can, Monkey Boy. -- Dr. Lizardo - Buckaroo Banzai
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Thoughts on statistics for continuously advancing columns
j...@commandprompt.com (Joshua D. Drake) writes: On the other hand ANALYZE also: 1. Uses lots of memory 2. Lots of processor 3. Can take a long time We normally don't notice because most sets won't incur a penalty. We got a customer who has a single table that is over 1TB in size... We notice. Granted that is the extreme but it would only take a quarter of that size (which is common) to start seeing issues. I find it curious that ANALYZE *would* take a long time to run. After all, its sampling strategy means that, barring having SET STATISTICS to some ghastly high number, it shouldn't need to do materially more work to analyze a 1TB table than is required to analyze a 1GB table. With the out-of-the-box (which may have changed without my notice ;-)) default of 10 bars in the histogram, it should search for 30K rows, which, while not free, doesn't get enormously more expensive as tables grow. -- cbbrowne,@,gmail.com http://linuxfinances.info/info/linuxdistributions.html Rules of the Evil Overlord #179. I will not outsource core functions. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Has anyone used CLANG yet?
age...@themactionfaction.com (A.M.) writes: [Much of interest elided... Cool to see that clang clearly *can* compile PostgreSQL...] You are probably running configure with gcc, no? I was *attempting* to run configure using clang: CC=/usr/bin/clang ./configure --prefix=/home/chris/dbs/postgresql-git-head I know it's using clang, as some of the early tests indicate that specifically. checking types of arguments for accept()... configure: error: could not determine argument types It's worth noting that the problem is NOT fundamentally any Pascal-parm-passing-style issue; that's a red herring. The trouble is that it's not finding a function signature for accept(), and a number of the attempts (well, half of them...) happen to try to use Pascal parm-passing conventions. Actually, there's a little more mystery to it... I pulled out the C code from config.log that corresponds with my favorite /usr/include/sys/socket.h accept() signature, and clang is happy to compile it, even though configure logs, in config.log, that there was a mismatch. So, for some reason, configure had no problem running clang a bunch of times against *other* C fragments, but somehow didn't like how it ran this one. Presumably there's some dang GNU magic going on ;-). Thanks for verifying that the notion of compiling PostgreSQL using clang is something that in principle ought to be able to work. Perhaps this first Debian packaging of it has some deficiency, or my workstation hates me! :-). -- output = (cbbrowne @ gmail.com) The real problem with the the year 2000 is that there are too many zero bits and that adversely affects the global bit density. -- Boyd Roberts b...@france3.fr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Has anyone used CLANG yet?
This is a C front end for the LLVM compiler... I noticed that it entered Debian/Unstable today: http://packages.debian.org/sid/main/clang I thought it would be interesting to see if PostgreSQL compiles with this, as an alternative compiler that should presumably become more and more available on Linux et al. (And I suppose that the randomly selected .sig is supremely apropos!) configure blows up here at the following: conftest.c:75:28: error: invalid token after top level declarator extern unsigned int PASCAL accept (unsigned int, void *, void *); I suspect there's something about PASCAL that's a problem, as clang is nominally supposed to be a C compiler ;-). I haven't looked deeper, so haven't the remotest idea how deep the issue lies. At any rate, I should poke at this further soon, but if it seems interesting to others, well, CLANG is now an easy install on some number of systems! -- let name=cbbrowne and tld=gmail.com in name ^ @ ^ tld;; http://linuxdatabases.info/info/postgresql.html The problem with the cutting edge is that someone has to bleed. -- Zalman Stern -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New PostgreSQL Committers
dp...@pgadmin.org (Dave Page) writes: Congratulations! +1 Congratulations, indeed, to this worthy set of developers! -- output = reverse(moc.liamg @ enworbbc) http://linuxfinances.info/info/multiplexor.html Power tends to corrupt and absolute power corrupts absolutely. -- First Baron Acton, 1834 - 1902 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
t...@sss.pgh.pa.us (Tom Lane) writes: Robert Haas robertmh...@gmail.com writes: On Mon, Dec 7, 2009 at 9:48 AM, Bruce Momjian br...@momjian.us wrote: I wonder if we should rephrase this as, How hard will this feature be to add, and how hard will it be to remove in a few years if we decide we don't want it? Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, It's not the get it in part that scares me. The problem I have with it is that I see it as a huge time sink for future maintenance problems, most of which will be classifiable as security breaches which increases the pain of dealing with them immeasurably. Ah, yes, the importance of this is not to be underestimated... Once SE-Pg is added in, *any* bug found in it is likely to be considered a security bug, and hence a candidate for being a CERT Advisory. Some bad things are liable to happen: a) Such problems turn into a hue and cry situation requiring dropping everything else to fix the security problem. b) If everyone isn't using SE-Pg, then people won't be particularly looking for bugs, and hence bugs are likely to linger somewhat, with the consequence that a) occurs with some frequency. c) Having a series of CERT advisories issued is not going to be considered a good thing, reputation-wise! I feel about the same way about this as I did about the adding of native Windows support; I'm a bit concerned that this could be a destabilizing influence. I was wrong back then; the Windows support hasn't had the ill effects I was concerned it might have. I'd hope that my concerns about SE-Pg are just as wrong as my concerns about native Windows support. Hope doesn't make it so, alas... -- select 'cbbrowne' || '@' || 'gmail.com'; http://www3.sympatico.ca/cbbrowne/languages.html Just because it's free doesn't mean you can afford it. -- Unknown -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs chapters in our docs
pete...@gmx.net (Peter Eisentraut) writes:
On ons, 2009-11-25 at 16:27 +0100, Magnus Hagander wrote:
Attached is a patch which adds a chapter to git in our documentation,
around where we have several chapters about cvs today. It also removes
a few very out of date comments about cvs
I think this whole chapter could be removed and the relevant information
added to the web site or the wiki.
(Btw., it's spelled Git, not GIT.)
I think I'd rather see the documentation repaired in the CVS repository
where it happens to reside today.
Wikis have a habit of getting out of date in ways that make them even
more difficult to rectify, because the data is frequently structured in
a way that doesn't make it particularly easy to pull it out and
transform it into other forms.
Now, if someone knows a way of creating a Git repository[1] that tracks,
change-for-change, everything going on in a MediaWiki repository in a
textual form that would allow one to monitor everything going on, and
possibly even inject changes, that *would* be something.
(To *my* mind, the ultimate wiki platform that I have seen lately is
ikiwiki http://ikiwiki.info/, which manages the wiki in an SCM,
compiling the pages into HTML whenever things are changed. Should
cope with heavy query load rather well! But I digress...)
Footnotes:
[1] Or Darcs, Mercurial, SVN, or whatever...
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','linuxfinances.info').
http://www3.sympatico.ca/cbbrowne/
Dijkstra probably hates me
(Linus Torvalds, in kernel/sched.c)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify - what to do when the queue is full
g...@turnstep.com (Greg Sabino Mullane) writes: BTW, did we discuss the issue of 2PC transactions versus notify? The current behavior of 2PC with notify is pretty cheesy and will become more so if we make this change --- you aren't really guaranteed that the notify will happen, even though the prepared transaction did commit. I think it might be better to disallow NOTIFY inside a prepared xact. That's a tough one. On the one hand, simply stating that NOTIFY and 2PC don't play together in the docs would be a straightforward solution (and not a bad one, as 2PC is already rare and delicate and should not be used lightly). But what I really don't like the is the idea of a notify that *may* work or may not - so let's keep it boolean: it either works 100% of the time with 2PC, or doesn't at all. Should we throw a warning or error if a client attempts to combine the two? +1 from me... It should either work, or not work, as opposed to something nondeterministic. While it's certainly a nice thing for features to be orthogonal, and for interactions to just work, I can see making a good case for NOTIFY and 2PC not playing together. -- select 'cbbrowne' || '@' || 'gmail.com'; http://linuxfinances.info/info/slony.html Why isn't phonetic spelled the way it sounds? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
and...@dunslane.net (Andrew Dunstan) writes: Robert Haas wrote: I am personally quite tired of reviewing patches for people who don't in turn review mine (or someone's). It makes me feel like not working on this project. If we can solve that problem without implementing a policy of this type, that is good. I would much prefer to run by the honor system rather than having to threaten to drop patches, but only if the honor system actually works. Organizing contributors on a project like this is like herding cats. Threats and penalties are unlikely to be effective. This is essentially a charity where people give in ways that work for them, and you take whatever they have to give. I'm extremely uncomfortable with the idea of a prescriptive system. I've proposed them myself in the past, but I have since come to the realization that it will simply drive people away. Ah, but the thing is, what was proposed wasn't totally evilly draconian. There's a difference between: You haven't reviewed any patches - we'll ignore you forever! and Since you haven't reviewed any patches, we are compelled to defer your patches until the next CommitFest. It's enough pain to make people think, but it's not *totally* punitive. -- I really only meant to point out how nice InterOp was for someone who doesn't have the weight of the Pentagon behind him. I really don't imagine that the Air Force will ever be able to operate like a small, competitive enterprise like GM or IBM. -- Kent England -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
j...@commandprompt.com (Joshua D. Drake) writes: On Mon, 2009-11-16 at 11:31 -0500, Chris Browne wrote: Ah, but the thing is, what was proposed wasn't totally evilly draconian. There's a difference between: You haven't reviewed any patches - we'll ignore you forever! and Since you haven't reviewed any patches, we are compelled to defer your patches until the next CommitFest. It's enough pain to make people think, but it's not *totally* punitive. It is important to remember we are all volunteers here. Any increase to the barrier of contribution is a bad one. But this *isn't* a barrier to contribution, at least not notably more than the already existant issue that a paucity of reviewers is a barrier to contribution. It represents a policy for triaging review efforts with a bias in favor of those that *are* contributing to the reviewers' list. I don't think it's unjust for those that contribute to the review process to get more favorable scheduling of reviews to their patches. If we get so many reviewers that such triaging becomes unnecessary, then it may automatically *not* be a problem. -- (format nil ~...@~s cbbrowne acm.org) http://linuxfinances.info/info/slony.html Bother, said Pooh, as he deleted his root directory. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal - temporal contrib module
arta...@comcast.net (Scott Bailey) writes: Disk format - A period can be represented as [closed-closed], (open-open), [closed-open) or (open-closed] intervals. Right now we convert these to the most common form, closed-open and store as two timestamptz's. I mentioned this at the 2009 PGCon, and it was pointed out to me that PostgreSQL already has geometric types which already offer many of the semantics and operators that are likely to be desired. http://www.postgresql.org/docs/8.4/static/functions-geometry.html If direct analogy may be applied so that portions of the functionality are drawn from previously-accepted geometric contributions, it's likely to be a bit easier to get this into 8.5 (or so!) FYI, I *love* the idea of having the temporal types and operators. I'm a lot less certain about the merits of PK/FK constraints - it is a lot less obvious what forms of constraints will be able to be applied to particular applications. -- I really only meant to point out how nice InterOp was for someone who doesn't have the weight of the Pentagon behind him. I really don't imagine that the Air Force will ever be able to operate like a small, competitive enterprise like GM or IBM. -- Kent England -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] License clarification: BSD vs MIT
dp...@pgadmin.org (Dave Page) writes: As Tom says though, the effect this has on users is zero. The licence is still the same as its always been, regardless of what we say it is based on or looks like. There may be a fairly miniscule one... There do exist GPL zealots that bash, as not free (in the sense that people are doubtless well aware of), stuff licensed under a BSD license. There may be some non-zero advantage to saying MIT style, in that this changes coasts ;-) and takes a micro-step away from the political aspects of BSD vs GPL. But I'm not disagreeing with you, by any means! :-) -- select 'cbbrowne' || '@' || 'gmail.com'; http://linuxdatabases.info/info/emacs.html I really only meant to point out how nice InterOp was for someone who doesn't have the weight of the Pentagon behind him. I really don't imagine that the Air Force will ever be able to operate like a small, competitive enterprise like GM or IBM. -- Kent England -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rules: A Modest Proposal
sfr...@snowman.net (Stephen Frost) writes: * David Fetter (da...@fetter.org) wrote: On Sun, Oct 04, 2009 at 04:07:40PM -0400, Robert Haas wrote: The radical proposal was the RULE system. It's been tested now, and it's pretty much failed. You still haven't explained what actual benefit we'd get out of doing this. Removing land mines is a benefit. Removing useful functionality without replacing it is definitely worse. Well, I think we can start here with the premise that there is disagreement on this... Position #1: Rules are land mines; in effect, an anti-feature. Position #2: Rules represent useful functionality. I'd tend more towards #1, myself, and with that as a premise, replacement isn't, per se, necessary. The one and only rule I have in the sizable app I'm working on is there because of the absence of updatable views. If we could put triggers on views, then I wouldn't need the rule, and that seems like a reasonable use case to have drawn into the modest proposal... -- select 'cbbrowne' || '@' || 'gmail.com'; http://linuxdatabases.info/info/emacs.html I really only meant to point out how nice InterOp was for someone who doesn't have the weight of the Pentagon behind him. I really don't imagine that the Air Force will ever be able to operate like a small, competitive enterprise like GM or IBM. -- Kent England -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 TODO: Add comments to output indicating version of pg_dump and of the database server
pete...@gmx.net (Peter Eisentraut) writes: On Fri, 2009-09-25 at 16:59 -0400, Tom Lane wrote: shakahsha...@gmail.com shakahsha...@gmail.com writes: From pg_dump/pg_restore section (9.2 of the Todo page on the PostgreSQL Wiki), is the following item Add comments to output indicating version of pg_dump and of the database server simply asking for a change to the pg_dump header from: I think so, but what's not clear is whether this is a good idea to do in the default output. It might only be appropriate in verbose mode, so as not to introduce unnecessary diffs between logically identical dumps. Well, a diff of the same database made by different (major) versions of pg_dump will already be different in most situations, so adding the pg_dump version number in it is essentially free from this perspective. What is the use case for adding the server version? I can imagine something like wanting to know exactly where the dump came from, but then host name and such would be better. (And then you can infer the server version from that.) I added this ToDo because we had a case where we were spelunking through some old pg_dumps, and the provenance was sufficiently distant that we couldn't readily infer what PostgreSQL version was involved. If pg_dump reported something like: -- pg_dump version: 8.5_devel -- postgres server version: 8.4.17 then it would be trivial to ascertain the information. Actually, I have no argument with your point; perhaps a whole header section is the right answer: -- pg_dump version: 8.5_devel -- postgres server version: 8.4.17 -- dump began at: 2010-07-01 14:22:27 EDT -- server name: wolfe -- more, maybe? Another issue is that it's not all that clear what to do or how to do it for archive dumps --- do you then want both pg_dump and pg_restore to tell you about themselves? I don't see a good reason for pg_restore to get involved. Agreed. This isn't needed for pg_restore to do anything better; it's so that humans can do better archaeology. -- let name=cbbrowne and tld=acm.org in name ^ @ ^ tld;; http://linuxfinances.info/info/languages.html Rules of the Evil Overlord #187. I will not hold lavish banquets in the middle of a famine. The good PR among the guests doesn't make up for the bad PR among the masses. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
s...@samason.me.uk (Sam Mason) writes: On Sun, Jul 26, 2009 at 01:42:32PM +0900, KaiGai Kohei wrote: Robert Haas wrote: In some cases, the clearance of infoamtion may be changed. We often have dome more complex requirements also. OK, so there is some other trusted entity that has unfettered access to both databases and its job is to manage these requirements. No, that's not what this implies. What this implies is along the following lines... If a user at the more secret level updates some data that had been classified at a lower level, then that data gets reclassified at the higher level. If this sort of outcome is problematic, then that suggests that maybe the attempt at sharing wasn't such a good idea. Thus, it is necessary a capability to store and manage data objects with different security labeles in a single database instance here. (If we don't want to use commercial solutions instead.) SE-PG is about doing the above in one database and allowing more rigorous checks to be done? I don't think the issue is so much about more rigorous; it's about having mandatory labelling that is handled consistently. -- (reverse (concatenate 'string ofni.sesabatadxunil @ enworbbc)) http://linuxdatabases.info/info/rdbms.html The people's revolutionary committee has decided that the name e is retrogressive, unmulticious and reactionary, and has been flushed. Please update your abbrevs. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New types for transparent encryption
gsst...@mit.edu (Greg Stark) writes: However I have a different concern which hasn't been raised yet. Encrypting lots of small chunks of data with the same key is a very dangerous thing to do and it's very tricky to get right. Yeah, that's exactly the sort of thing that would be Most Useful for someone trying to do differential cryptanalysis. http://en.wikipedia.org/wiki/Differential_cryptanalysis It would provide an *exact* vector for differential attack if the attacker has the ability to add in a series of bits of data of their choosing before capturing the thus-encrypted dump. If you add some more-or-less-randomish salt, ala SSHA, that could be of some tiny help, maybe, arguably, but I doubt that's usable :-(. http://www.openldap.org/faq/data/cache/347.html -- let name=cbbrowne and tld=acm.org in String.concat @ [name;tld];; http://linuxfinances.info/info/sap.html Why do scientists call it research when looking for something new? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [pgsql-www] commitfest.postgresql.org
robertmh...@gmail.com (Robert Haas) writes: I suspect both are true, but in the unlikely event that we decide on some massive change to the system, we can either run the DBs in parallel as Tom suggests, or dump out the older data in Wiki markup and post it on there. But I can't imagine what we'd want to do that would even make us consider such drastic steps. Your example would not be a difficult migration, for instance. We had an ancient version of Bugzilla in use for quite a while; it was *SO* vastly different from modern versions that it wasn't remotely plausible to port the data from the old instance to a new one. I went and ran wget to pull all the contents of the old instance, turning that into a series of static web pages. No longer updatable, but certainly browsable. Once a CommitFest is complete, I could easily see making a summary of it, as a series of static web pages. No need for a database anymore altogether ;-). -- select 'cbbrowne' || '@' || 'linuxfinances.info'; http://linuxdatabases.info/info/spreadsheets.html I'd give my right arm to be ambidextrous! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New types for transparent encryption
a...@esilo.com (Andrew Chernow) writes:
Would the IV be regenerated every time the plaintext is updated, to
avoid using it twice? For instace: update t set text = 'abc' where id
= 1 . ISTM that the IV for OLD.text should be thrown away.
Where would the key come from? Where would it be stored? What cipher is
used?
LDAP authentication systems tend to use SSHA these days...
http://www.openldap.org/faq/data/cache/347.html
With SSHA, the key used for hashing passwords is picked randomly;
often by grabbing a few bytes from /dev/random. It's not important
that it be cryptographically secure, as it is presented directly as
part of the stored password.
In python, SSH hashes thus:
You need two inputs:
1. password, which is the value that is to be hidden
2. salt, a seed value.
The point isn't for salt to need to be super-secure, just for it to
not be frequently repeated. Fairly random seems to be generally
good enough.
import sha from base64
import b64encode
ctx = sha.new( password )
ctx.update( salt )
hash = {SSHA} + b64encode( ctx.digest() + salt )
Sort-of-aside:
FYI, I tried implementing SSHA in pl/pgsql, with mixed results.
It interoperated fine with other SSHA implementations as long as the
salt values were plain text.
The SSHA implementation in OpenLDAP (slappasswd) uses 4 byte binary
values (I think it grabs them from /dev/random or /dev/urandom);
unfortunately that wouldn't play OK with my pl/pgsql implementation.
I think having that work would be pretty keen, could share code if
anyone is interested...
--
output = reverse(ofni.secnanifxunil @ enworbbc)
http://linuxdatabases.info/info/unix.html
Rules of the Evil Overlord #145. My dungeon cell decor will not
feature exposed pipes. While they add to the gloomy atmosphere, they
are good conductors of vibrations and a lot of prisoners know Morse
code. http://www.eviloverlord.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
gsst...@mit.edu (Greg Stark) writes: I would like to propose a different strategy. Instead of always tackling all the smaller patches and leaving the big patches for last, I would suggest we start with Hot Standby. In fact I would suggest as Hot Standby has already gotten a first pass review that we consider applying it on day 1. That gets it into everyone's development trees so they can see any suspicious code or effects it has in their peculiar environments. It may not be perfect but if we apply it now there's plenty of time to make improvements. Then we can have a regular commitfest a month or so later. Hopefully any followon changes to Hot Standby would actually get into that commitfest if they're relatively minor. I could see going either way on this, either: a) Doing an as-early-as-possible CommitFest to knock off easy items that have been waiting a while, and having the *second* Fest be the one where we expect all the large, controversial items to get added (e.g. - stuff like hot standby, SEPostgreSQL), or b) Focusing on the likely-hard ones (hot standby, SE PostgreSQL) first, and deferring others to Fest #2. -- select 'cbbrowne' || '@' || 'acm.org'; http://cbbrowne.com/info/linuxdistributions.html Everything should be built top-down, except the first time. -- Alan J. Perlis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pre-proposal: permissions made easier
and...@dunslane.net (Andrew Dunstan) writes: Jeff Davis wrote: On Mon, 2009-06-29 at 12:55 -0400, Tom Lane wrote: I think it has to be looked at in comparison to more general prospective-permissions schemes; When I searched google for prospective permissions, all I found were links to messages in this thread ;) Can you refer me to a general prospective-permissions scheme that is more widely accepted? Being more widely accepted also has the benefit that users will feel more comfortable with the behavior. Think of MySQL's wildcard permissions. They apply to any object whether that object is created before or after the rule is set, AIUI. That means the wildcard pattern is applied at the time the permission rule is referenced, rather than when the rule is created, thus applying it prospectively. It's a feature many users would like to have, although, as Tom rightly points out, it can be a bit of a footgun if used carelessly. I'll point out, for posterity, that way back in yesteryear, TOPS-10 (introduced in 1967) had a declarative permissioning system for file access that resembles this. The best description I'm aware of is the following: http://lkml.org/lkml/1999/2/5/2 A FILDAE config file for a particular user might look like the following: # anything in a directory named private is off limits */private/*:*:*:*: # people in group foo get full (create, delete, read, write, # execute) access to everything in the foo project directory ~/projects/foo/*:*:foo:*:cdrwx # people playing mygame can update the high score file ~/mygame/score.dat:*:*: ~/mygame/bin/mygame:rw # some friends have access to the RCS files for mygame ~/mygame/src/RCS/*:dennis,kevin,josh:*: /usr/bin/ci:rw ~/mygame/src/RCS/*:dennis,kevin,josh:*: /usr/bin/co:rw # I'll put stuff I want everyone to read in my ~/public directory # I'll make the public directory 744, so no one will actually have # to check .access_list, but I'll still put in this entry for completeness ~/public/*:*:*:*:r# anything left over gets no access*:*:*:*: This obviously isn't notably SQL-like, but that's not the point :-). -- cbbrowne,@,cbbrowne.com http://www3.sympatico.ca/cbbrowne/lisp.html Ubuntu is an ancient African word, meaning can't configure Debian -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] conditional dropping of columns/constraints
robertmh...@gmail.com (Robert Haas) writes: On Mon, May 4, 2009 at 10:10 AM, Andres Freund and...@anarazel.de wrote: Would a patch adding 'IF EXISTS' support to: - ALTER TABLE ... DROP COLUMN - ALTER TABLE ... DROP CONSTRAINT possibly be accepted? Having it makes the annoying task of writing/testing of schema-upgrade scripts a bit easier. Can't speak for the committers, but I've wished for this a time or two myself. For constraints, it's easy enough to treat that as idempotent; it's no big deal to drop and re-add a constraint. For columns, I'd *much* more frequently be interested in ALTER TABLE ... ADD COLUMN IF NOT EXISTS ... Note that this is distinctly NOT the same as: ALTER TABLE ... DROP COLUMN IF EXISTS ... ALTER TABLE ... ADD COLUMN ... -- (format nil ~...@~s cbbrowne linuxdatabases.info) http://linuxdatabases.info/info/lisp.html Signs of a Klingon Programmer - 10. A TRUE Klingon Warrior does not comment his code! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] License Issue
t...@sss.pgh.pa.us (Tom Lane) writes: Korry Douglas korry.doug...@enterprisedb.com writes: Seems like the easy solution is to rip out the AIX files in your server deployments ... or are you actually intending to support AIX? AIX itself now offers the functions found in src/backend/port/ dynloader/aix.c so I think that file may be obsolete as of (at least AIX 5.3). aix.c was required long ago because AIX did not offer the popular dlopen(), dlsym(), dlclose(), and dlerror() functions - you had to write them yourself. Well, we've got AIXen in the buildfarm, shall we rip out those files and see what happens? Go nuts :-). From our perspective (at Afilias), I wouldn't see any problem with stripping out obsolete AIXisms like this, as long as we're validating the change against buildfarm, which we're running regularly. We got rid of our last AIX 5.1 a couple of years ago. However, there have been some reports out there on the InterWeb of people running PostgreSQL on AIX 4.3 and such, and those people might be displeased by this stuff getting ripped out. But that being said, I can't see it being a particularly sensible to be keeping PostgreSQL up to date on a platform where the OS has been out of maintenance for *years*. -- let name=cbbrowne and tld=acm.org in name ^ @ ^ tld;; http://linuxfinances.info/info/linuxdistributions.html Rules of the Evil Overlord #151. I will not set myself up as a god. That perilous position is reserved for my trusted lieutenant. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A renewed plea for inclusion of zone.tab
and...@tao11.riddles.org.uk (Andrew Gierth) writes: The usual conversation goes something like this (generally following on from some discussion of how to do timezone conversions): Q: how do I get the list of available zone names? A: see pg_timezone_names Q: but there's 1650/1400/560/452 [delete as applicable] entries in there! how do I know which one to use for any given user? Can I work it out from the user's location? A: Some locations have timezones that vary by county level, so it's hard to automate unless you have a street address and detailed maps/database of Indiana and other awkward places. Best bet is to ask the user themselves, once you know what country they're in. Q: How do you know what zones are in what countries? A: that info is in zone.tab, which you can find either from your OS's timezone directory or from the postgres source for your postgres version. Put that data in a table or something and use it to prompt the user; it has text to help disambiguate the obscure cases. Q: ... wtf? why is that not installed anywhere? I can confirm having recently hit something rather like this... We wanted to indicate which timezone people were in, but in the absence of having something like zone.tab conveniently available, people were starting to talk about designing their own ad-hoc, buggy version of something looking like about half of zone.tab. Tom Any such application is far more likely to be looking at the Tom system tzdata files. Only if it's using the system TZ functions to do conversions rather than doing them inside pg, which certainly isn't how _I'd_ recommend an app writer do it. Right. Having zone.tab data available in the DB would be *way* more convenient, especially in that it is not at all obvious that it's always available on systems, let alone in a consistent place. This isn't an argument for it must be considered standard, yesterday. But it's plenty valuable to look to having a way to parse zone.tab and turn it into a table or two. -- let name=cbbrowne and tld=cbbrowne.com in name ^ @ ^ tld;; http://linuxdatabases.info/info/linuxxian.html The only thing better than TV with the sound off is Radio with the sound off. -- Dave Moon -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
XML only working in UTF-8 - Re: [HACKERS] 8.4 open items list
j...@agliodbs.com (Josh Berkus) writes: This one is also really bad, but probably only Doc-patchable. However, can SQL/XML really be said to be core functionality if it only works in UTF-8? * BUG #4622: xpath only work in utf-8 server encoding Well, much of the definition of XML assumes the use of Unicode, so I don't feel entirely badly about there being such a restriction. It seems likely to me that opening its use to other encodings has a considerable risk of breaking due to a loss of, erm, closure, in the mathematical sense. Or, alternatively, opening a Pandora's Box of needing to do translations to prevent mappings from breaking. -- let name=cbbrowne and tld=linuxfinances.info in String.concat @ [name;tld];; http://cbbrowne.com/info/languages.html If you add a couple of i's to Microsoft's stock ticker symbol, you get 'misfit'. This is, of course, not a coincidence. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multi calendar system for pgsql
m.alimom...@gmail.com (Mohsen Alimomeni) writes:
I want to try to add a multi calendar system for pgsql. I want to
know if it will be accepted as a patch to pgsql?
I would expect there to be nearly zero chance of such, at least in the
form of a change to how dates are stored.
As long as there is commonality in epochs and some continuity of
calculations of dates relative to epochs, there shouldn't be any
fundamental problem in adding on functions to do the following sorts
of things:
- Calculate what the UNIX date is for a given date in a given
calendar
- Output a UNIX date in the form indicated by a given calendar
You should avail yourself of the book, _Calendrical Calculations_, by
Edward M Reingold and Nachum Deerschowitz; it presents details of
calculations and conversions of dates between the following calendars:
- Gregorian
- Julian
- Coptic
- Ethiopic
- ISO
- Islamic
- Hebrew
- Ecclesiastical Calendars, for dates of Christian holidays such as Easter
- Old Hindu
- Modern Hindu
- Mayan
- Balinese Pawukon
- Persian
- Baha'i
- French Revolution
- Chinese
It would seem a whole lot preferable to create functions like (and
there may be better names!):
create function parse_date (locale, text) returns timestamp
create function output_date (local, timestamp) returns text
Thus, you might expect the following:
select parse_date('Islamic', 'Miharram 1, AH 1');
parse_date
-
622-07-16 00:00:00
Or
select output_date('Persian', '622-03-19'::timestamp);
output_date
-
1 Farvardin AH 1
(It is entirely likely that I'm fracturing spellings of things!
Apologies if I am!)
http://emr.cs.uiuc.edu/home/reingold/calendar-book/index.shtml
--
(reverse (concatenate 'string moc.enworbbc @ enworbbc))
http://cbbrowne.com/info/x.html
Thank you for calling PIXAR! If you have a touch tone phone, you can
get information or reach anybody here easily! If your VCR at home is
still blinking '12:00', press '0' at any time during this message and
an operator will assist you.
-- PIXAR'S toll-free line (1-800-888-9856)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Simple postgresql.conf wizard
[EMAIL PROTECTED] (Dave Page) writes: On Fri, Nov 14, 2008 at 8:10 AM, Simon Riggs [EMAIL PROTECTED] wrote: On Fri, 2008-11-14 at 02:21 +, Gregory Stark wrote: On the other hand what does occur to me in retrospect is that I regret that I didn't think about how I was disparaging the importance of mental illness and hope nobody took offense for that reason. Your comments surprise me because you mentioned to me privately that you disliked on-list bullies. It hardly seems like bullying to me - a tongue-in-cheek humorous remark to someone many of us, including Greg, have known and worked with for years. If it were made to a newbie to the community, that would be another matter. It's pretty much like the you'd have to be on crack to do that!, erm, crack. -- let name=cbbrowne and tld=cbbrowne.com in String.concat @ [name;tld];; http://linuxfinances.info/info/rdbms.html Zaphod's just zis guy, you know? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL5 budget
[EMAIL PROTECTED] (Alvaro Herrera) writes: David Rowley escribió: Or is sponsoring a feature paying money to people that already plan to implement something? Nobody on their mind would plan to implement the features being proposed here ... I didn't look very far but it seems mainly nonsense. Oh, dear... In searching back to the archives, it becomes evident that I shouldn't have been as polite as I was, and that it is indeed mainly nonsense. There *would* be merit in supporting the work on recursive queries, in 8.4, as that will help when trying to build queries that are to return hierarchical result sets. But that's not what he noticed; what he's looking for is to somehow use XML in lieu of, um, either SQL, PHP, libpq, or something like that. (And the fact that those are 4 rather different things reveals how confused the matter is.) Apologies; I didn't realize how fragmented the kettle was[1]... [1] Or did I misspell cracked and pot? -- (reverse (concatenate 'string ofni.secnanifxunil @ enworbbc)) http://linuxfinances.info/info/ I still maintain the point that designing a monolithic kernel in 1991 is a fundamental error. Be thankful you are not my student. You would not get a high grade for such a design :-) -- Andrew Tanenbaum to Linus Torvalds -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL5 budget
[EMAIL PROTECTED] (Jonah H. Harris) writes: On Sun, Nov 9, 2008 at 7:41 PM, Decibel! [EMAIL PROTECTED] wrote: I think you're barking up the wrong tree here; the community can't really do hacking for hire. If you want to pay for something to be implemented (which is great!), you'll need to talk to companies that do Postgres consulting. You can find examples on the website and through google. You could also try posting to pgsql-jobs. I would suggest submitting it to pgsql-jobs. Except that it's not a job offer - it's more of a solicitation for implementation of something, so I'm not sure that's the right place either. Some of the things suggested seem interesting, but many seem (to me) to be overly tied to an all XML all the time view. I really rather have PostgreSQL head towards being more relational (in the 3rd Manifesto sense) than take the all singing, all dancing XML road. -- select 'cbbrowne' || '@' || 'linuxfinances.info'; http://www3.sympatico.ca/cbbrowne/sgml.html Rules of the Evil Overlord #114. I will never accept a challenge from the hero. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL/MED compatible connection manager
[EMAIL PROTECTED] (Martin Pihlak) writes: Tons of details have been omitted, but should be enough to start discussion. What do you think, does this sound usable? Suggestions, objections? Slony-I does some vaguely similar stuff in its handling of connection paths; here's the schema: create table @[EMAIL PROTECTED] ( pa_server int4, pa_client int4, pa_conninfo text NOT NULL, pa_connretryint4, CONSTRAINT sl_path-pkey PRIMARY KEY (pa_server, pa_client), CONSTRAINT pa_server-no_id-ref FOREIGN KEY (pa_server) REFERENCES @[EMAIL PROTECTED] (no_id), CONSTRAINT pa_client-no_id-ref FOREIGN KEY (pa_client) REFERENCES @[EMAIL PROTECTED] (no_id) ) WITHOUT OIDS; comment on table @[EMAIL PROTECTED] is 'Holds connection information for the paths between nodes, and the synchronisation delay'; comment on column @[EMAIL PROTECTED] is 'The Node ID # (from sl_node.no_id) of the data source'; comment on column @[EMAIL PROTECTED] is 'The Node ID # (from sl_node.no_id) of the data target'; comment on column @[EMAIL PROTECTED] is 'The PostgreSQL connection string used to connect to the source node.'; comment on column @[EMAIL PROTECTED] is 'The synchronisation delay, in seconds'; I wouldn't be surprised to find there being some value in using something like SQL/MED. One detail I'll point out, that I'm noticing from an application I'm working on right now. We might want to have something like a db connection data type; here's a prototype I put together: slonyregress1=# create type dbconn as (port integer, dbname text, username text, password text, ssl boolean); CREATE TYPE slonyregress1=# create table dbconns (id serial primary key, db dbconn); NOTICE: CREATE TABLE will create implicit sequence dbconns_id_seq for serial column dbconns.id NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index dbconns_pkey for table dbconns CREATE TABLE slonyregress1=# insert into dbconns (db) values ((5432, 'slonyregress1', 'slony', 'secret!', 'true')); INSERT 0 1 slonyregress1=# select * from dbconns; id | db +-- 1 | (5432,slonyregress1,slony,secret!,t) (1 row) I'm not certain that this is forcibly the right representation, but I think it is possible that we'd want a finer-grained representation than merely a connection string. -- (reverse (concatenate 'string ofni.sesabatadxunil @ enworbbc)) http://linuxdatabases.info/info/finances.html DTDs are not common knowledge because programming students are not taught markup. A markup language is not a programming language. -- Peter Flynn [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL future ideas
[EMAIL PROTECTED] (Jonah H. Harris) writes: On Fri, Sep 26, 2008 at 11:52 AM, Andrew Dunstan [EMAIL PROTECTED] wrote: Speaking of language choice, no one said that _all_ the source code would need to be rewritten. It would be nice, for example, if PostgreSQL rewrote the current GUC system with a glue language like Lua (which is also very C-like). No it wouldn't. All it would mean is that you'd need developers fluent in both languages. Having done quite a bit of internals work with SAP DB (which is an amalgamation of C, C++, and Pascal), I completely agree. The entire system, if possible, should be in a single language. Note that this actually *isn't* possible; PostgreSQL is implemented in a number of languages already: a) C, obviously b) m4 and some autoconf macrology c) GNU make d) There's some awk e) Shell script f) Flex g) Bison And I'm not sure that's all there is :-). -- let name=cbbrowne and tld=cbbrowne.com in String.concat @ [name;tld];; http://www3.sympatico.ca/cbbrowne/lisp.html If a hole in the street is a manhole, is a hole in a man a streethole? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL future ideas
[EMAIL PROTECTED] (Gevik Babakhani) writes: Advantage of C++ is that it reduce lot of OO code written in C in PostgreSQL, but it is so big effort to do that without small gain. It will increase number of bugs. Do not forget also that C++ compiler is not so common (so good) on different platforms. If somebody interesting in that yes but like a fork ( PostgreSQL++ :-). Reducing OO code that is written in C is one of my major interests. After some investigating myself it appears that having the codebase fully (rewritten in C++ will have an impact on the performance. So I guess such an effort will result the code being more C++ish and fully OO, being a mixture in C with some OO taste. I'm not convinced that it would a good idea at all to make the system fully OO, nor that C++ would be a meaningful tool to use to that end. After all, C++ can certainly be used in decidedly non-OO ways. For instance, STL is NOT an OO framework, and the author of STL, obviously something of a fan of C++, characterizes OO as almost as much of a hoax as Artificial Intelligence. http://en.wikipedia.org/wiki/Object-oriented_programming#Criticism I tend to agree with that characterization. Further, C++ suffers from the same not OO at its base problem of Java, which contributes complexity as well as hurting the OO-ness of it. Better idea is to start to use C99 in PostgreSQL ;-). I have not investigated this yet. But I am very interested to know what the advantages would be to upgrade the code to C99 standards. It would give us heartburn on any platforms where the preferred compiler doesn't grok C99, for sure. As much as I'm ok with using GCC, it would seem unfortunate to force people into using GCC everywhere, and preclude using other compilers. (And actually, I'm more ambivalent about GCC than that; I'm not totally happy with how GCC has gone, but that's another tale for another day...) -- (format nil [EMAIL PROTECTED] cbbrowne acm.org) http://linuxdatabases.info/info/linux.html NT 5.0 is the last nail in the Unix coffin. Interestingly, Unix isn't in the coffin... It's wondering what the heck is sealing itself into a wooden box 6 feet underground... -- Jason McMullan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL future ideas
[EMAIL PROTECTED] (Andrew Dunstan) writes: A.M. wrote: Speaking of language choice, no one said that _all_ the source code would need to be rewritten. It would be nice, for example, if PostgreSQL rewrote the current GUC system with a glue language like Lua (which is also very C-like). No it wouldn't. All it would mean is that you'd need developers fluent in both languages. I expect it would be both a little better *and* a little worse than that. On the better side, I don't expect that, in this instance, there would be terribly much need for anything but the shallowest understanding of Lua. If this were all there was to it, I'd contend that there's little to object to. However, there's a pretty considerable worse side, namely that developers would need to understand the API for interfacing between the bits of C that are the Lua 'external interface' and how that gets plumbed into PostgreSQL. *That's* got very little to do with language, per se; it has to do with the implementation of the language. -- let name=cbbrowne and tld=acm.org in String.concat @ [name;tld];; http://linuxdatabases.info/info/oses.html Real concurrency---in which one program actually continues to function while you call up and use another---is more amazing but of small use to the average person. How many programs do you have that take more than a few seconds to perform any task? -- New York Times, 4/25/89 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql: Make LC_COLLATE and LC_CTYPE database-level settings.
[EMAIL PROTECTED] (Heikki Linnakangas) writes: Log Message: --- Make LC_COLLATE and LC_CTYPE database-level settings. Collation and ctype are now more like encoding, stored in new datcollate and datctype columns in pg_database. This is a stripped-down version of Radek Strnad's patch, with further changes by me. pgsql/src/bin/pg_dump: pg_dump.c (r1.500 - r1.501) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/bin/pg_dump/pg_dump.c?r1=1.500r2=1.501) pg_dumpall.c (r1.106 - r1.107) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/bin/pg_dump/pg_dumpall.c?r1=1.106r2=1.107) Let me report a problem with this... Running pg_dump encounters the following: [EMAIL PROTECTED]:Slony-I/CMD/slony1-HEAD/tests pg_dump -s slonyregress1 pg_dump: column number -1 is out of range 0..7 pg_dump: column number -1 is out of range 0..7 zsh: segmentation fault pg_dump -s slonyregress1 It's not a Slony-I-related issue; I get identical behaviour with other databases that do not have Slony-I in place. Things seem to work if I roll back this patch to pg_dump.c. So something's up... -- (reverse (concatenate 'string moc.enworbbc @ enworbbc)) http://www3.sympatico.ca/cbbrowne/emacs.html Being really good at C++ is like being really good at using rocks to sharpen sticks. -- Thant Tessman -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL future ideas
[EMAIL PROTECTED] (Gevik Babakhani) writes: It might look like an impossible goal to achieve.. But if there is any serious plan/idea/ammo for this, I believe it would be very beneficial to the continuity of PG. Actually, I imagine that such a rewrite would run a very considerable risk of injuring the continuity of PostgreSQL VERY BADLY, to the point of causing community fractures and forks of the codebase. When you write something in C++, you have to pick a subset of the language that is supported fairly identically (in semantics) by all of the compilers that you wish to support. Seeing as how PostgreSQL is already a mature system written in C, a rewrite into C++, *which is a different language* that is NOT simply a superset of C functionality, would require substantial effort, lead to fractious disagreements, and would, without ANY doubt, fracture the code base into *AT LEAST* two versions, namely: a) The existing C code base, and b) One (possibly more) C++ rewrites This does not strike me as a particularly useful exercise. If I intended such a rewrite, I'd much rather consider using something *interestingly* different from C, like Erlang or Eiffel or Haskell. -- cbbrowne,@,linuxdatabases.info http://linuxfinances.info/info/sgml.html For a good prime call: 391581 * 2^216193 - 1 -- [EMAIL PROTECTED] (Szymon Rusinkiewicz) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Transaction Snapshots and Hot Standby
[EMAIL PROTECTED] (Heikki Linnakangas) writes: Simon Riggs wrote: Taking snapshots from primary has a few disadvantages ... * snapshots on primary prevent row removal (but this was also an advantage of this technique!) That makes it an awful solution for high availability. A backend hung in transaction-in-progress state in the slave will prevent row removal on the master. Isolating the master from queries done performed in the slave is exactly the reason why people use hot standby. And running long reporting queries in the standby is again a very typical use case. I agree that this is a demerit to this approach. Whether or not, on balance, it makes it an 'awful solution for high availability' is much more in the eye of the beholder, and NOT obvious on the face of it. -- let name=cbbrowne and tld=linuxdatabases.info in name ^ @ ^ tld;; http://linuxfinances.info/info/sgml.html Question: How many surrealists does it take to change a light bulb? Answer: Two, one to hold the giraffe, and the other to fill the bathtub with brightly colored machine tools. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Fast REVERSE() function?
I've got a case where I need to reverse strings, and find that, oddly enough, there isn't a C-based reverse() function. A search turns up pl/pgsql and SQL implementations: create or replace function reverse_string(text) returns text as $$ DECLARE reversed_string text; incoming alias for $1; BEGIN reversed_string = ; for i in reverse char_length(incoming)..1 loop reversed_string = reversed_string || substring(incoming from i for 1); end loop; return reversed_string; END $$ language plpgsql; CREATE OR REPLACE FUNCTION reverse(TEXT) RETURNS TEXT AS $$ SELECT array_to_string( ARRAY ( SELECT substring($1, s.i,1) FROM generate_series(length($1), 1, -1) AS s(i) ), ''); $$ LANGUAGE SQL IMMUTABLE; Unfortunately, neither is particularly fast. This should be blinding-quick in C, in comparison; reversing a set of bytes should be able to be done mighty quick! (Aside: presumably we could walk thru the string destructively, in-place, swapping bytes; I think that would be theoretically quickest...) I could probably add this in as an SPI() function; is there a good reason to try to avoid doing so? -- output = reverse(ofni.sesabatadxunil @ enworbbc) http://www3.sympatico.ca/cbbrowne/sgml.html Consistency is the single most important aspect of *ideology.* Reality is not nearly so consistent. - [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fast REVERSE() function?
[EMAIL PROTECTED] (hubert depesz lubaczewski) writes: On Mon, Sep 08, 2008 at 11:20:18AM -0400, Chris Browne wrote: I've got a case where I need to reverse strings, and find that, oddly enough, there isn't a C-based reverse() function. A search turns up pl/pgsql and SQL implementations: just for completenes - there is also pl/perl and c versions freely available: http://www.depesz.com/index.php/2007/07/30/indexable-field-like-something/ (pl/perl) http://blog.frosties.org/post/2007/08/28/Fonction-reverse-C-avec-PostgreSQL (c) I hadn't thought about the Unicode issue (mentioned elsewhere in the thread); that's a good reason why the method I mentioned *wouldn't* be a good one! I'm NOT interested in pl/perl as an option; building and deploying all of Perl is a mighty expensive way to get *ONE* function (and I don't think that fundamentally changes if it's 10 functions!). In the long run, I'd be keen on there being a REVERSE function available in pg_catalog, which is why I'm asking about the C version, as that would be the way to put it into the core. -- cbbrowne,@,linuxdatabases.info http://www3.sympatico.ca/cbbrowne/sap.html DSK: STAN.K; ML EXIT -- FILE NOT FOUND -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Planner creating ineffective plans on LEFT OUTER joins
[EMAIL PROTECTED] (Simon Riggs) writes: On Wed, 2008-06-25 at 23:34 -0400, Robert Haas wrote: I can predict that Tom will say that the planning time it would take to avoid this problem isn't justified by the number of queries that it would improve. That's possible, but it's unfortunate that there's no way to fiddle with the knobs and get the planner to do this kind of thing when you want it to. I don't think we should invent a new parameter for each new optimisation. We would soon get swamped. IMHO we should have a single parameter which indicates how much planning time we consider acceptable for this query. e.g. optimization_level = 2 (default), varies 1-3 Most automatic optimisation systems allow this kind of setting, whether it be a DBMS, or compilers (e.g. gcc). We should agree a simple framework so that each new category of optimization can be described as being a level X optimisation, or discarded as being never worth the time. We do this with error messages, so why not do this with something to control planning time? Is there something more parametric that we could use to characterize this? That is, to attach some value that *does* have some numeric interpretation? I don't quite have a for instance, but here's some thoughts on modelling this... - If there is some query optimization option/node that clearly adds to planning cost in a linear (or less) fashion, then it would be meaningful to mark it as linear, and we'd be fairly certain to validate any linear options. - There would also be options/nodes that have a multiplicative effect on planning time. - Thirdly, there are options/nodes (particularly when considering cases of multiple joins) where there is a polynomial/exponential effect on query planning. I could see: a) Evaluating which roads to consider from a linear/multiplicative/exponential perspective, which would look a lot like level 1, level 2, level 3. b) Estimating values, and, in effect, trying to model the amount of planning effort, and dropping out sets of routes that are expected to make the effort exceed [some value]. Sane? Silly? -- cbbrowne,@,linuxfinances.info http://www3.sympatico.ca/cbbrowne/nonrdbms.html STATED REASON DOES NOT COMPUTE WITH PROGRAMMED FACTS... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Overhauling GUCS
[EMAIL PROTECTED] (Greg Smith) writes: On Fri, 6 Jun 2008, Heikki Linnakangas wrote: Or perhaps we should explicitly mark the settings the tool has generated, and comment out: #shared_buffers = 32MB # commented out by wizard on 2008-06-05 shared_buffers = 1024MB # automatically set by wizard on 2008-06-05 What I would like to do is make the tool spit out a revision history in the same way I find all big IT shops handling this already: by putting a revision history style commentary above the current setting. Here's a sample: # 2008-03-02 : 32MB : postgres : Database default # 2008-05-02 : 512MB : pg_autotune : Wizard update # 2008-05-15 : 1024MB : gsmith : Increased after benchmark tests shared_buffers = 1024MB If the first tuning tool that comes into existance used this format, and the format was reasonable, I think it would be possible to get people making manual edits to adopt it as well. My first reaction to this is... And why is it that you seem to imagine it a good idea to recreate RCS, badly? While that may be an unkind reaction, I don't think it's a particularly wrong one! Getting into comment formatting certainly *does* seem like an attempt to do, poorly, what RCS does. And I am pointing at a tool from 1985 as opposed to modern ones to point out that there were better approaches to managing versioning of simple text documents *THIRTY YEARS AGO*. Indeed, CVS, which we are still using doesn't fundamentally do anything, in this context, that RCS didn't. I kind of wish that there was some sort of library for doing RCS; even in the absence of that, it would be perfectly logical for a tool that manages this to DEMAND that there be a diff utility or some equivalent available. In the absence of that, let me suggest that it would be preferable to follow VMS's lead and, if you automagically make any changes, create a version called postgresql.conf;[n] where [n] is either: a) 1, if postgresql.conf;1 does not exist, or b) 1 + the largest integer found. Replace ; with some other character as need be. Add a line (or a few lines) at the top of the current version that indicates some bit of commented-out-metadata that is obviously formatted and easily recognizable such as: ### [pgauto] postgresql.conf-autotuner - ran 2008-07-02 ### [pgauto] little bit of safely-turfable header ### [pgauto] further little bit of safely-turfable header In effect, what it does is thus: - Keep the old copy as postgresql.conf;24 (because we have some 24 elder versions) - Strips off any leading lines that begin with ### [pgauto] - Puts any interesting comments in a set of leading lines that begin with ### [pgauto] Merits, over some wacky rewrite-comment-formatting: - This requires only the thinnest bit of special handling of comments. - It doesn't force in a dependancy on [pick-someone's-pet-SCM that you happen to hate] - Even in the absence of an SCM, it keeps versioning information around - If you have a diff, it'll work perfectly well, and be reasonably informative - It doesn't throw ANY old data away! Sorry, that's more than enough rant for today! :-) -- cbbrowne,@,linuxdatabases.info http://cbbrowne.com/info/rdbms.html Rules of the Evil Overlord #220. Whatever my one vulnerability is, I will fake a different one. For example, ordering all mirrors removed from the palace, screaming and flinching whenever someone accidentally holds up a mirror, etc. In the climax when the hero whips out a mirror and thrusts it at my face, my reaction will be ``Hmm...I think I need a shave.'' http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
[EMAIL PROTECTED] (Andreas 'ads' Scherbaum) writes: On Thu, 29 May 2008 23:02:56 -0400 Andrew Dunstan wrote: Well, yes, but you do know about archive_timeout, right? No need to wait 2 hours. Then you ship 16 MB binary stuff every 30 second or every minute but you only have some kbyte real data in the logfile. This must be taken into account, especially if you ship the logfile over the internet (means: no high-speed connection, maybe even pay-per-traffic) to the slave. If you have that kind of scenario, then you have painted yourself into a corner, and there isn't anything that can be done to extract you from it. Consider: If you have so much update traffic that it is too much to replicate via WAL-copying, why should we expect that other mechanisms *wouldn't* also overflow the connection? If you haven't got enough network bandwidth to use this feature, then nobody is requiring that you use it. It seems like a perfectly reasonable prerequisite to say this requires that you have enough bandwidth. -- (reverse (concatenate 'string ofni.secnanifxunil @ enworbbc)) http://www3.sympatico.ca/cbbrowne/ There's nothing worse than having only one drunk head. -- Zaphod Beeblebrox -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
[EMAIL PROTECTED] (Tom Lane) writes: As I said originally, we have no expectation that the proposed features will displace the existing replication projects for high end replication problems ... and I'd characterize all of Robert's concerns as high end problems. We are happy to let those be solved outside the core project. I expect that one of the effects of having an improved PITR as a 'core' replication solution will be that Slony-I will become less popular amongst naive users, and that is *not* a bad thing. I am quite sure that there are a lot of cases today where users would be better served by what falls out of this. It will mean that those that are left will, despite being fewer, be more cognizant that they are getting what they require. We should see somewhat fewer expectation gaps, and that should allow more people to be more satisfied with what they are getting. -- let name=cbbrowne and tld=cbbrowne.com in name ^ @ ^ tld;; http://linuxdatabases.info/info/linux.html Lisp stoppped itself FEP Command: -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
[EMAIL PROTECTED] (Florian Weimer) writes: * Thomas Mueller: What do you think about it? Do you think it makes sense to implement this security feature in PostgreSQL as well? Can't this be implemented in the client library, or a wrapper around it? A simple approximation would be to raise an error when you encounter a query string that isn't contained in some special configuration file. This could be implemented in a client library, but that means that you're still entirely as vulnerable; any client that chooses not to use that library won't be protected. It would be a mighty attractive thing to have something at the server level to protect against the problem. -- let name=cbbrowne and tld=linuxfinances.info in String.concat @ [name;tld];; http://linuxdatabases.info/info/lsf.html If you add a couple of i's to Microsoft's stock ticker symbol, you get 'misfit'. This is, of course, not a coincidence. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
[EMAIL PROTECTED] (Alvaro Herrera) writes: Darren Reed wrote: Because interacting with the database is always through an action that you do and if you're being half way intelligent about it, you are always checking that each action succeeded before going on to the next. Hmm, it won't be pretty for the drivers that do PQexec(COMMIT; BEGIN). The driver will think that it's in a transaction when in fact the second command in the string has been ignored, and so it's not ... We have worked pretty hard around here to expunge use of drivers that do this sort of thing. (Cough, cough, dbrow...) Recent versions of PostgreSQL don't suffer too badly, but back in the 7.2/7.4 days, we had applications that left transactions open IDLE in transaction for days at a time (if a user quit using the web app without expressly logging out), with _atrocious_ results. Andrew Sullivan recently had some choice words about the merits of ENUM; I think the same applies to drivers that do PQexec(COMMIT;BEGIN)... -- output = (cbbrowne @ linuxfinances.info) http://www3.sympatico.ca/cbbrowne/advocacy.html :FATAL ERROR -- ILLEGAL ERROR -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] we don't have a bugzilla
[EMAIL PROTECTED] (Andrew Dunstan) writes: Raphaël Jacquot wrote: would seem like a good idea, no ? http://www.murrayc.com/blog/permalink/2008/04/25/postgresql-has-no-bugzilla/ Before you come trolling on this (or any other) subject, please read the voluminous debates that have taken place about it. Apparently you think it's something we have never considered, which in light of the product we maintain would be more than remarkable. Having done that, please endeavour to make an actual contribution to the discussion. This seems more than a tad unfair. Yes, it is certainly fair to observe that there have been voluminous debates. But it will take a whole lot of trolling around in the archives to figure out the shape of the *conclusions* of those debates. Seeming relevant conclusions: - Yes, there probably ought to be some sort of structured bug tracker. - HOWEVER, there are many who prefer to use email for their work, as opposed to being forced into some sort of webby thing. - That being said, experimentation is taking place for the commitfests with using a wiki to track statuses of patches and discussions. - It is hoped that out of this experimentation, patterns surrounding what is *truly* needful to structure will emerge. There is reason to be hopeful about this, as the wiki-based process has been showing some useful structures to impose already. -- select 'cbbrowne' || '@' || 'cbbrowne.com'; http://www3.sympatico.ca/cbbrowne/spreadsheets.html I find it hard to believe a professional programmer could support the concept of Open Source. -- David J. Owens [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] MERGE Specification
[EMAIL PROTECTED] (Decibel!) writes: On Apr 22, 2008, at 1:17 PM, Gregory Stark wrote: Simon Riggs [EMAIL PROTECTED] writes: As I've said elsewhere, we could have it lock each row, its just more overhead if we do and not necessary at all for bulk data merging. I'll presume we want locking as an option, unless people say otherwise. It's not so simple. If you look for a row to merge into and don't find one there's no row to lock. What unique constraints do is hold the lock on the index page where the entry would have to be added. That's the trick that plpgsql cannot implement. That's why users are forced to loop and retry until they manage to do an update successfully or insert successfully. Yeah, hopefully there's a better way to do this other than row locks. But no matter how this is done, I think we need to handle the race conditions, and handle them by default. If people *really* know what they're doing, they can disable the row locking (perhaps one way to do this would be to grab an explicit lock on the table and have merge check for that...). I agree that handling the race conditions by default is preferable. Consider: An excellent reason to prefer MERGE is if it handles race conditions that would otherwise require application code be more carefully and cleverly written to avoid the race conditions. If MERGE solves it automatically, and eliminates hand-written code, that's TWO benefits, that quite likely outweigh any performance costs. I know we've had cases where race conditions [that a well-done MERGE would probably solve easily] bit us badly, requiring considerable development effort, and the addition of extra table columns and such. There are some possibilities of worst case MERGE being pretty slow; it's likely to be faster than the alternatives we used in its absence ;-). -- (reverse (concatenate 'string ofni.secnanifxunil @ enworbbc)) http://www3.sympatico.ca/cbbrowne/wp.html I once went to a shrink. He told me to speak freely. I did. The damn fool tried to charge me $90 an hour. -- [EMAIL PROTECTED] (Jim Moore Jr) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Problem with server/utils/snapmgr.h
[EMAIL PROTECTED] (Alvaro Herrera) writes: Chris Browne wrote: If I use: AC_CHECK_HEADER(utils/snapmgr.h, HAVE_SNAPMGR=1) this turns out to fail. Apparently autoconf wants to compile the #include file to validate that it's an OK #include file. GCC barfs on it, thus: [EMAIL PROTECTED]:~/Slony-I/CMD/slony1-HEAD gcc -I/opt/OXRS/dbs/pgsql84-beta/include/server /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: Hmm. It works for me if I forcefully include postgres.h: gcc -I/pgsql/install/00head/include/server -include postgres.h /pgsql/install/00head/include/server/utils/snapmgr.h Our header file rule says that a header must include any header it needs to compile, but never include postgres.h, which must be the first include in all the .c files. So I'm not sure the fix for this. What does Slony-I need snapmgr.h for, anyway? This code is in a state of a flux right now -- there are pending patches which are likely to change the horizon a bit. Perhaps it does not make sense for Slony to adjust to a state that's expected to be short-lived. Well, one of the Sun guys observed this... http://www.slony.info/bugzilla/show_bug.cgi?id=46 And I was trying to build against CVS HEAD (for both projects ;-)) and observed it: [EMAIL PROTECTED]:CMD/slony1-HEAD/src/backend make gcc -g -O2 -Wall -Wmissing-prototypes -Wmissing-declarations -I../.. -fpic -I/opt/OXRS/dbs/pgsql84-beta/include/ -I/opt/OXRS/dbs/pgsql84-beta/include/server/ -c -o slony1_funcs.o slony1_funcs.c slony1_funcs.c: In function '_Slony_I_createEvent': slony1_funcs.c:142: error: 'SerializableSnapshot' undeclared (first use in this function) slony1_funcs.c:142: error: (Each undeclared identifier is reported only once slony1_funcs.c:142: error: for each function it appears in.) The definition for SerializableSnapshot has moved from server/utils/tqual.h to server/utils/snapmgr.h I agree that the code seems in flux; it seems quite likely that there will be further changes between now and release of 8.4. That being said, it's useful to observe these problems *early*, and have workarounds, so that we can validate that Slony-I is generally compatible with 8.4 throughout its development cycle. As a workaround, for now, I'll see if Tom's counsel on this works out well; I expect so. The point of the exercise wasn't so much to ask How do I work around this? as it was to point out that there's *something* up with the header file, in that the autoconf AC_CHECK_HEADER function barfs on it. *THAT* (the fact that AC_CHECK_HEADER breaks) appears to be a bad thing, irrespective of any Slony-I issues. -- let name=cbbrowne and tld=linuxdatabases.info in name ^ @ ^ tld;; http://linuxdatabases.info/info/x.html Signs of a Klingon Programmer - 1. Defensive programming? Never! Klingon programs are always on the offense. Yes, offensive programming is what we do best. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO, FAQs to Wiki?
[EMAIL PROTECTED] (Bruce Momjian) writes: I am impressed at the state of the May wiki patch queue: http://wiki.postgresql.org/wiki/CommitFest:May It is even tracking the psql wrap patch I am working on now. Aside: I have made a few little changes that oughtn't be too controversial: 1. Added [[Category:CommitFest]] to all of the pages that are related, thus generating a central 'index page' about this. http://wiki.postgresql.org/index.php?title=Category:CommitFest This makes it unnecessary to have so many direct interlinks between the commitfests. 2. Added a = CommitFest (March|May) 2008 = header to the respective Fests, as it is not inconceivable that there might be a CommitFest:May2009 or CommitFest:March2010. I think I want to learn a bit more about this templating thing, used in the May 'fest; that looks like that might be a real good way to simplify some of the wiki work we're doing internally. Seems like a slick, slick way to make it easier to build pretty structures whilst avoiding hard-coding tables. I'm getting more and more impressed at MediaWiki... -- let name=cbbrowne and tld=cbbrowne.com in name ^ @ ^ tld;; http://cbbrowne.com/info/spreadsheets.html Rules of the Evil Overlord #107. Even though I don't really care because I plan on living forever, I will hire engineers who are able to build me a fortress sturdy enough that, if I am slain, it won't tumble tothe ground for nogood structural reason. http://www.eviloverlord.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Problem with server/utils/snapmgr.h
There's a new #include file that it turns out we need for Slony-I to reference, namely include/server/utils/snapmgr.h I tried adding an autoconf rule to Slony-I to check for its existence (goal then is to do a suitable #define so that we can #ifdef the #include, so that we #include this only with versions of PostgreSQL that have the file). If I use: AC_CHECK_HEADER(utils/snapmgr.h, HAVE_SNAPMGR=1) this turns out to fail. Apparently autoconf wants to compile the #include file to validate that it's an OK #include file. GCC barfs on it, thus: [EMAIL PROTECTED]:~/Slony-I/CMD/slony1-HEAD gcc -I/opt/OXRS/dbs/pgsql84-beta/include/server /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: /opt/OXRS/dbs/pgsql84-beta/include/server/storage/block.h:31: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'BlockNumber' /opt/OXRS/dbs/pgsql84-beta/include/server/storage/block.h:55: error: expected specifier-qualifier-list before 'uint16' In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/storage/off.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemid.h:46: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'ItemOffset' /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemid.h:47: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'ItemLength' In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: /opt/OXRS/dbs/pgsql84-beta/include/server/storage/off.h:24: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'OffsetNumber' In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:17, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:40: error: expected specifier-qualifier-list before 'OffsetNumber' /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:143: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'ItemPointerEquals' /opt/OXRS/dbs/pgsql84-beta/include/server/storage/itemptr.h:144: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'ItemPointerCompare' In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: /opt/OXRS/dbs/pgsql84-beta/include/server/storage/relfilenode.h:44: error: expected specifier-qualifier-list before 'Oid' In file included from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapshot.h:16, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/tqual.h:18, from /opt/OXRS/dbs/pgsql84-beta/include/server/utils/snapmgr.h:16: /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:106: error: expected specifier-qualifier-list before 'TransactionId' /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:118: error: expected specifier-qualifier-list before 'int32' /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:142: error: expected specifier-qualifier-list before 'uint16' /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:442: error: expected specifier-qualifier-list before 'uint32' /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:498: error: expected specifier-qualifier-list before 'uint32' /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:623: error: expected specifier-qualifier-list before 'uint16' /opt/OXRS/dbs/pgsql84-beta/include/server/access/htup.h:671:
Re: [HACKERS] MERGE SQL Statement
[EMAIL PROTECTED] (Simon Riggs) writes: Should there be a new rule option? ie. ON MERGE rules ? Maybe, but not as part of this project. That seems to warrant a bit of elaboration... If we're running a MERGE, and it performs an INSERT or UPDATE of a particular tuple in(to) a particular table, will it fire the ON INSERT/ON UPDATE trigger? I'd hope so... -- let name=cbbrowne and tld=cbbrowne.com in String.concat @ [name;tld];; http://linuxfinances.info/info/multiplexor.html Canada, Mexico, and Australia form the Axis of Nations That Are Actually Quite Nice But Secretly Have Nasty Thoughts About America -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lessons from commit fest
[EMAIL PROTECTED] (Tom Lane) writes: Chris Browne [EMAIL PROTECTED] writes: Would it be a terrible idea to... - Draw the indent code from NetBSD into src/tools/pgindent I am not real eager to become maintainers of our own indent fork, which is what you propose. (Just for starters, what will we have to do to make it run on non-BSD systems?) We are presently at the extreme position where pgindent is run once in a very long time (~ once a year), at pretty considerable cost, and with the associated cost that a whole lot of indentation problems are managed by hand. Yeah. One reason for that is that the typedef problem makes it a pretty manual process. As I hear more about the typedef problem, a part of me gets more and more appalled... It seems like we're creating some problem for ourselves in that the typedefs don't seem to be able to be consistent. I don't have an answer, but it's looking like a sore tooth that clearly needs attention. The main problem I see with pgindent early and often is that it only works well if everyone is using exactly the same pgindent code (and exactly the same typedef list). Otherwise you just get buried in useless whitespace diffs. It's bad enough that Bruce whacks around his copy from time to time :-(. I would say that the single greatest annoyance for maintaining our back branches is that patches tend to not back-patch cleanly, and well over half the time it's because of random reformattings done by pgindent to code that hadn't changed at all, but it had formatted differently the prior year. For the same reason, my take on your random whitespace changes are acceptable theory is not no but hell no. It's gonna cost us, permanently, in manual patch adjustments if we allow the repository to get cluttered with content-free diffs. I don't want to be cavalier about it; I'm hoping that in the discussion, some more stable answer may fall out. Though with the typedef issues that have emerged, I'm not entirely sanguine... -- (reverse (concatenate 'string gro.mca @ enworbbc)) http://linuxfinances.info/info/internet.html HEADLINE: Suicidal twin kills sister by mistake! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get rid of psql welcome message
[EMAIL PROTECTED] (Stephen Frost) writes: * Peter Eisentraut ([EMAIL PROTECTED]) wrote: Around http://archives.postgresql.org/pgsql-patches/2008-01/msg00089.php it was proposed to truncate the psql welcome screen. What do you think about that? I'd recommend an option in .psqlrc to disable it, if possible. That would be in line with what alot of other splash-screen type things do. Shorten: Welcome to psql 8.1.9 (server 8.1.8), the PostgreSQL interactive terminal. Type: \copyright for distribution terms \h for help with SQL commands \? for help with psql commands \g or terminate with semicolon to execute query \q to quit To: psql 8.1.9 (server 8.1.8) - PostgreSQL interactive terminal Type: \h for SQL help, \? for psql help, \q to quit which removes 3/4 of the bloat, whilst only losing info about \copyright and \g. That's close enough to an 80% improvement for me. That *would* be a big win in doing cut'n'paste of psql sessions, and while the experienced user may not care about \h, \?, and \q, I'd miss getting the version information. There's enough room still there, by the way, that one might cleverly add in the port number without forcing the addition of an extra line, which could be useful material, even in a cut'n'paste... -- (format nil [EMAIL PROTECTED] cbbrowne linuxfinances.info) http://www3.sympatico.ca/cbbrowne/spiritual.html Editing is a rewording activity. -- Alan J. Perlis [And EMACS a rewording editor. Ed.] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lessons from commit fest
[EMAIL PROTECTED] (Tom Lane) writes: Magnus Hagander [EMAIL PROTECTED] writes: I think pg_indent has to be made a lot more portable and easy to use before that can happen :-) I've run it once or twice on linux machines, and it comes out with huge changes compared to what Bruce gets on his machine. Yeah, I've had no luck with it either. Every so often there are discussions of going over to GNU indent instead. Presumably that would solve the portability problem. The last time we tried it (which was a long time ago) it seemed to have too many bugs and idiosyncrasies of its own, but it would be worth a fresh round of experimenting IMHO. Well, GNU indent is now on version 2.2.9, and has evidently addressed *some* problems with it. Unfortunately, the pgindent README does not actually specify what any of the actual problems with GNU indent are, thus making it pretty much impossible to evaluate whether or not any of the subsequent releases might have addressed any of those problems. I doubt that the pgindent issues have been addressed. -- output = reverse(ofni.sesabatadxunil @ enworbbc) http://linuxfinances.info/info/sgml.html In elementary school, in case of fire you have to line up quietly in a single file line from smallest to tallest. What is the logic? Do tall people burn slower? -- Warren Hutcherson -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lessons from commit fest
[EMAIL PROTECTED] (Bruce Momjian) writes: Magnus Hagander wrote: And I think adopting surrounding naming, commeting, coding conventions should come naturally as it can aide in copy-pasting too :) I think pg_indent has to be made a lot more portable and easy to use before that can happen :-) I've run it once or twice on linux machines, and it comes out with huge changes compared to what Bruce gets on his machine. Other times, it doesn't :-) So yeah, it could be that it just needs to be made easier to use, because I may certainly have done something wrong. Agreed, pgindent is too cumbersome to require patch submitters to use. One idea would be to allow C files to be emailed and the indented version automatically returned via email. Would it be a terrible idea to... - Draw the indent code from NetBSD into src/tools/pgindent - Build it _in place_ inside the code tree (e.g. - don't assume it will get installed in /usr/local/bin) - Thus have the ability to run it in place? -- let name=cbbrowne and tld=linuxfinances.info in name ^ @ ^ tld;; http://cbbrowne.com/info/lisp.html It worked about as well as sticking a blender in the middle of a lime plantation and hoping they'll make margaritas out of themselves. -- Frederick J. Polsky v1.0 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lessons from commit fest
[EMAIL PROTECTED] (Bruce Momjian) writes: Chris Browne wrote: [EMAIL PROTECTED] (Bruce Momjian) writes: Magnus Hagander wrote: And I think adopting surrounding naming, commeting, coding conventions should come naturally as it can aide in copy-pasting too :) I think pg_indent has to be made a lot more portable and easy to use before that can happen :-) I've run it once or twice on linux machines, and it comes out with huge changes compared to what Bruce gets on his machine. Other times, it doesn't :-) So yeah, it could be that it just needs to be made easier to use, because I may certainly have done something wrong. Agreed, pgindent is too cumbersome to require patch submitters to use. One idea would be to allow C files to be emailed and the indented version automatically returned via email. Would it be a terrible idea to... - Draw the indent code from NetBSD into src/tools/pgindent - Build it _in place_ inside the code tree (e.g. - don't assume it will get installed in /usr/local/bin) - Thus have the ability to run it in place? Yes, but it bloats our code and people still need to generate the typedefs and follow the instructions. The other problem is if they run it on a file they have modified, it is going to adjust places they didn't touch, thereby making the patch harder to review. The bloat is 154K, on a project with something around 260MB of code. I don't think this is a particlarly material degree of bloat. If it is included in src/tools/pgindent, you can add in a Makefile such that it is automatically built, so the cost of running it goes way down, so that it could be run all the time rather than once in a great long while. If it was being run *all* the time, would we not expect to find that we would be seeing relatively smaller sets of changes coming out of it? We are presently at the extreme position where pgindent is run once in a very long time (~ once a year), at pretty considerable cost, and with the associated cost that a whole lot of indentation problems are managed by hand. If we ran pgindent really frequently, there would admittedly be the difference that there would be a lot of little cases of changes-from-pgindent being committed along the way, but [1] might it not be cheaper to accept that cost, with the concomittant benefit that you could tell patchers Hey, run pgindent before submitting that patch, and that'll clean up a number of of the issues. Yes, it doesn't address code changes like typedef generation, but that never was an argument against running pgindent... [1] In cases where the differences primarily fall from differences in indentation, cvs diff -u can drop out those differences when reading the effects of a patch... -- let name=cbbrowne and tld=cbbrowne.com in name ^ @ ^ tld;; http://linuxdatabases.info/info/sap.html A study in the Washington Post says that women have better verbal skills than men. I just want to say to the authors of that study: Duh. -- Conan O' Brien -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lessons from commit fest
[EMAIL PROTECTED] (Bruce Momjian) writes: Chris Browne wrote: Would it be a terrible idea to... - Draw the indent code from NetBSD into src/tools/pgindent - Build it _in place_ inside the code tree (e.g. - don't assume it will get installed in /usr/local/bin) - Thus have the ability to run it in place? Yes, but it bloats our code and people still need to generate the typedefs and follow the instructions. The other problem is if they run it on a file they have modified, it is going to adjust places they didn't touch, thereby making the patch harder to review. The bloat is 154K, on a project with something around 260MB of code. I don't think this is a particlarly material degree of bloat. You mean 37kb vs 13MB, right? That is the tarball sizes I see. Hmm. I was apparently badly counting the size of the sources. I ran a find | wc command that seemed to report 260MB of code. A du on a make distclean tree gives me 104MB. At any rate, that's only out by a bit more than a binary order of magnitude ;-). I was thinking about the 154K of source code sitting in CVS, not the (yes, lower) cost of it in a tarball. Seems immaterial either way... If we ran pgindent really frequently, there would admittedly be the difference that there would be a lot of little cases of changes-from-pgindent being committed along the way, but [1] might it not be cheaper to accept that cost, with the concomittant benefit that you could tell patchers Hey, run pgindent before submitting that patch, and that'll clean up a number of of the issues. Yes, it doesn't address code changes like typedef generation, but that never was an argument against running pgindent... That is much a more radical use of pgindent than it has had in the past but it is certainly possible. Well, supposing you're cleaning up a patch after someone has generated it in bad style, it would seem like rather less work to use pgindent to impose style policy right away rather than simulating its effects manually. I'm not proposing anything *really* radical, like migrating away from CVS, after all! ;-) -- let name=cbbrowne and tld=linuxfinances.info in String.concat @ [name;tld];; http://linuxfinances.info/info/lisp.html There was a young lady of Crewe Whose limericks stopped at line two. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
