[HACKERS] Time zone database
Hackers, Today on the ietf-applications list, I saw this: http://www.ietf.org/mail-archive/web/apps-discuss/current/msg02301.html If there are particular issues with respect to the time zone database stuff that you all have struggled with and want highlighted, feel free to send them to me and I'll try to figure out whether they're relevant to this Internet Draft and the new plans. I will be at the IETF meeting in Prague, though I won't plan to attend the session in question unless someone tells me I ought. If none of this is relevant to Postgres, sorry for the noise. I just saw it in passing and remember some of the annoyances that happened in the past. Also, if you want me to see what you have to say, send your mail directly to me or cc: me. I can't really keep up with the volume on this list, and I'm likely to miss it if it's only here. Best, A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Time zone database
On Thu, Mar 03, 2011 at 09:27:58PM +0200, Heikki Linnakangas wrote: Yes, we use David Olson's tz database. According to this proposal, David Olson is retiring, and they propose that IETF takes over maintainership of the tz database. Yeah, I guess I ought to have summarized. That is indeed the plan. The idea seems to be that the work to keep the database and tz code current would continue like before, just in the hands of different people, so I don't think this affects us in any way. There is the possibility that the IETF will be somehow less quick to cope with changes. (The IETF is not a speedy way to get anything done.) I think that's the biggest reservation I've heard expressed. Anyway, as long as nobody's worried, I can stand mute :) Thanks for the reply. A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 release planning
it will work; but to my way of thinking, it's a mindset foreign to the principles of RDBM system design. That could be why some of us react to the proposal with perplexed looks. A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to get SE-PostgreSQL acceptable
On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: use. The people that need them understand the ramifications of row filtering and the absence of inaccessible rows won't be surprising. I wish there was even a little bit of evidence that users of a security system may be relied upon to understand its implications and effects. In my experience, however, they often don't. you have to have them seems fairly weak, certainly not strong enough to justify the costs. We have already touched on some ways that you can The costs are nil for people who don't want this feature. That's also false, because developers who don't care about the feature have to continue to maintain it as part of the system. If maintenance were free, I suspect nobody would be objecting to the feature. But this feature will in fact constrain future development and will impose maintenance requirements on the programmers of the system. Those maintenance requirements in turn amount to a cost that every user has to pay, because time spent addressing issues that result from this feature (or accommodating it in new development) is time that is not spent on other problems users might face. If I imagined I were project manager of the PostgreSQL project (a preposterous supposition, let me be clear), then I'd be very worried that this feature, which is apparently poorly understood by at least one big contributor to the project, would amount to a significant drag on future development work. In that case, I'd have to ask why having this feature as part of the main line of PostgreSQL is a good trade-off. Happily, I'm not someone who has to make that determination, so I can't say whether it _is_ a good trade-off. But I think that's what the resistance to the feature is all about, so you'll need to make the case that the trade-off is a good one. A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 release planning
On Tue, Jan 27, 2009 at 12:41:36PM -0500, Stephen Frost wrote: * Gregory Stark (st...@enterprisedb.com) wrote: It does seem weird to simply omit records rather than throw an error and require the user to use a where clause, even if it's something like WHERE pg_accessible(tab). […] do row-level security and security labels. Requiring a where clause or you throw an error would certainly make porting applications that depend on that mechanism somewhat difficult, and doesn't really seem like it'd gain you all that much... Throwing an error would entail a side-channel leak that would not be acceptable to the security community, I bet. That said, I have reservations, along the lines of Peter E's, that the early design-level objections to the approach were never answered. I certainly never got any real answer to questions I asked, for what it's worth. I will note that I tried to have a look at the literature on this topic. As I started to read, it became obvious that it was copious, but pretty well-determined. What bothered me most about the answers I got was that there never seemed to be an answer to please outline the design principles except for it's what SE-Linux does. The OS-level control rules seemed to me to be totally foreign to the database world, precisely because ACID is a problem in databases in a way it isn't for filesystems under the traditional UNIX model. I formed the impression -- only an impression, mind, that there was a poor fit between SE-Linux and database systems, and that the proponents had decided that enough caulk (in the form of don't do that) would seal the gap. I haven't (obviously) been paying much attention to this topic since, but I detect something of the same sort of response in the recent discussion. Maybe the goal isn't explicit enough. If the goal is compliance with some set of well-defined tests, what are they? If the goal is buzzword compliance, what are the tests of that (to the extent there ever are some)? If the goal is just security enhancement, I confess that I am still unable to understand the definitions of security and enhancement such that I think we have some operationalization of what the patch is supposed to provide. I know there are people who think this is cool. I guess, if I were running the circus, I'd want to know what's cool about it, and why. Then maybe the project would be in a position to understand whether that kind of cool is the way it wants to be. But without a clear problem statement, and a roadmap of how the patches solve the problem, I'm at a loss. And last I checked (which was, admittedly, not today), the project pages didn't have that information. A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pluggable Indexes
None of this is Any of My Business any more, but On Wed, Jan 21, 2009 at 03:44:15PM +, Simon Riggs wrote: The patch takes special care to allow calls to the rmgr functions only from the startup process. The APIs are exactly like the indexams and *are* called only in specific ways, at specific times. At your earlier request I put in filters to prevent WAL inserts for plugins that didn't exist, ensuring that all WAL writes were crash recoverable. I haven't even started to think about looking at the code, but I buy Simon's argument here. The Pg project is at big pains to point out how the extensible PL support and custom datatypes are such big deals. So why is pluggable index support not also a good thing? I take no position on the merits of the proposed patch, which I do not pretend to understand. But it'd be nice to see opponents distinguish beteween bad idea in principle and bad idea in this case. If you're arguing the former, clarifying why the analogies aren't relevant would be helpful. A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Well done, Hackers
On Sat, Nov 01, 2008 at 04:21:30PM -0400, Robert Haas wrote: It looks to me like there are at least half a dozen patches submitted in the last week that are pretty half-baked and fall into the category of Let's submit something before the deadline for CommitFest, in the hopes of being allowed to finish it later. Supposing this is true, one way to prevent that in future commitfest-driven releases is to send them back as not ready this time, and tell them they'll be 1st in line in the next go-round. It's worth remebering that this is the first release using the commitfest model, so there will be things to learn from the 1.0 attempt. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL + Replicator developer meeting 10/28
On Wed, Oct 29, 2008 at 12:02:20PM +0200, Hannu Krosing wrote: A good goal. But why would anybody _need_ 50 slaves ? They might have a contractual responsibility for extremely wide geographic distribution. Or they might be building an application that needs extremely wide network-topological distribution to avoid large loads on any one network. For instance, I can imagine building a network of nameservers in which you peered the nameservers, colocated in every ISP you could think of. If you were backing the nameserver with Postgres, this would work. To be clear, this is _not_ the case with any product I've ever built, but it is a design I have seen deployed. That design was supposed to be on top of Oracle. There were well over 50 slaves. I don't really believe they had that many Oracle-using slaves, though. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql Feature request \set query
On Wed, Oct 22, 2008 at 04:14:11PM -0700, Joshua Drake wrote: True enough, but a car doesn't roll without at least four wheels. I'm not sure I agree: http://en.wikipedia.org/wiki/Image:1885Benz.jpg (Sorry, I couldn't resist.) -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unicode escapes in literals
On Thu, Oct 23, 2008 at 06:04:43PM +0300, Peter Eisentraut wrote: Man that's ugly. Why the ampersand? Yeah, excellent question. It seems completely unnecessary, but it is surely there in the syntax diagram. Probably because many Unicode representations are done with U+ followed by 4-6 hexadecimal units, but + is problematic for other reasons (in some vendor's implementation)? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SSL cleanups/hostname verification
On Tue, Oct 21, 2008 at 08:47:35AM -0400, Tom Lane wrote: Um, IIRC what it's checking there is the server's key signature, which has nada to do with certificates. That depends on whether you used an X.509 certificate to authenticate the original signature. Just about nobody does, but AIUI, there's a way to do so. Anyway, in the strict sense you're right, but the comparison is wrong anyway. SSH doesn't pretend to be authenticating over SSL. It's authenticating using the SSH protocol, which has its own RFCs describing it. If I understand the description of the current behaviour, I have to agree with those who say the current behaviour is almost worse than nothing. In the presence of DNS forgery (and I'll bet a pretty good lunch most people aren't using DNSSEC), it's not hard to send a client to the wrong server. If the ssl-using client will blithely proceed if it can't authenticate the server, it's pretty hard to see in what sense this is a conforming use of anything I know as SSL. SSL is supposed to provide both encryption and authentication (the self-signed certificate nonsense is actually breakage that everyone in the protocol community wails about whenever given the opportunity, because of the results in user behaviour. It was a compromise that people made back in the period when Verisign had a lock on the market and would charge you an arm and a leg for a cert). A [Actually, to be pedantic, it might be better to call the authentication method TLS, so as not to conflate it with the Netscape-defined SSL. But this is maybe straying into a different topic.] -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
On Fri, Oct 10, 2008 at 01:09:48PM +0900, KaiGai Kohei wrote: 4. Metadata-level access controls. None of the proposals so far seem to provide a complete set of access controls for the system details -- schemas, databases, c. Such controls are often requested, so I wonder about that. We are already have GRANT/REVOKE on databases, schemaes and so on as a core facility. This optional facility does not need to provide it again. I think I wasn't clear enough. One of the requests we hear all the time -- indeed, somone just posted an RFQ looking for coders for it -- is a request to prevent users who haven't any permission on a database to learn anything about it at all. In a shared hosting environment, for instance, the idea is that two customers can have databases in the same back end, and not be able to learn anything about one another _including that they are there_. I am pretty sure I first heard someone wishing for something like that when was using PostgreSQL 6.something, so it's a long-standing irritant. Anyway, I'm not trying to suggest, You should do this. I'm just trying to point out that what are the obvious areas of access control from one point of view are not even interesting from another. This is why I think a fairly complete analysis is needed (and why I think it hasn't been done yet). A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
On Fri, Oct 10, 2008 at 01:44:49PM +0900, KaiGai Kohei wrote: Andrew Sullivan wrote: I want to focus on this description, because you appear to be limiting the problem scope tremendously here. We've moved from general security policy for database system to security policy for database system as part of a web-application stack. The general security policy for database system is an incorrect term. SELinux does not cover database system only. It covers operating sytem and application managing objects (like database object, X window, ...). Thus, it should be talked as general security policy for operating system, database system and so on. Ok, then let's use the broader case, which is general security policy for entire computing system including a RDBM subsystem (call this GSPECS+DB, say). This shows up even more the issue that considering primarily the application stack does not actually cover all the cases. I'm not suggesting, even a little bit, that securing an application stack as you propose is a waste of time. It could be, actually, that this more modest goal is the more appropriate one, and that SE-PostgreSQL would be a killer feature in this space (because it would, if it worked, solve a lot of problems that other systems have, as you have pointed out). But it is not GSPECS+DB, because of all the corner case problems whose behaviour still needs working out. Since I don't have to do any of the work to maintain the system in future in the face of the proposed new code, I can be indifferent as to whether the achievement of the goal is worth the cost. But plainly, others who need to look after the code will want to know what the exact goal is before committing themselves to future maintenance. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
Hi, On Sat, Sep 27, 2008 at 12:18:45PM +0900, KaiGai Kohei wrote: As I repeated several times, SE-PostgreSQL applies the seuciry policy of SELinux to achieve consistency in access controls. I get this. The problem as I see it is that the SELinux security policy is designed around a very different set of assumptions about concurrency and consistency than any database system has to provide. My admittedly hurried glance at some abstracts in the literature suggests to me that others have looked at the paradoxes that come out of this kind of security policy consistency when you apply them to database systems. I think that clearly stating which of the trade-offs are the ones to be accepted is all that's needed. In my vision, Apache assigns its contents handler an individual security context based on HTTP authentication, source IP address and so on just before web application invoked. Because web applications works with individual least privilege set, its accesses on filesystem are restricted by the security policy. In a similar way, its accesses on databases are also restricted via SE-PostgreSQL by same policy, by same privilege set. I want to focus on this description, because you appear to be limiting the problem scope tremendously here. We've moved from general security policy for database system to security policy for database system as part of a web-application stack. I suggest that the range of practically available behaviours of the DBMS working as part of a web-application stack is a subset of the practically available behaviours of the DBMS overall. This could be the source of some of the difficulty. For instance, it seems that some scenarios people are worried about are really the sort of scenario relevant to online users of Postgres, rather than to people seeing results filtered through a web application. If we just don't care about the online, interactive users, then maybe some of those concerns go away. (I'll be honest, though, that I have a hard time getting excited about a security system that doesn't really work as advertised outside of a narrow context.) All that said, I think there are definite practical uses for the work you're undertaking, and I want to emphasise that I think the general goal is probably a good one. I am suggesting that some additional work clarifying the specific goals of the work is all that's really needed. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
On Wed, Oct 08, 2008 at 11:36:19AM +0900, KaiGai Kohei wrote: Yes, unfortunatelly. No one replied to my proposed design: http://marc.info/?l=pgsql-hackersm=12470930544w=2 FWIW, I didn't know what to say about that proposal because I still don't know what problems any of this is trying to solve. Perhaps I'm just obtuse (and people should feel free to ignore me, since I'm not volunteering any code anyway); but this entire discussion seems to me to lack clear statements of what the goals of the effort are. I know, consistent access control; that still doesn't tell me enough, I think. I haven't given it a great deal of thought, but it seems to me that there are at least a few different goals people have in mind. These are the ones I've thought of, but I don't pretend this is an exhaustive list. It certainly isn't based on a serious review of the literature, which seems to be plentiful: 1. Single-policy access control for all objects available in a system. I'm guessing that this is the point of SE-PostgreSQL. It also appears to me that the SE-Linux approach may not have completely defined how these things work in a database context, but that there has been academic work in this area. I think a clear description of what the costs and benefits of this approach are would go a long way to helping evaluation. For instance, what sort of side channels does this expose? Are there database design techniques that help? c. 2. Granular row-level access control based on database-level ACLs. I have formed the impression that there is some support of this sort needed anyway to implement (1), but maybe not. I guess this is the proposal you mention. What is the goal here? Simply administrator-defined controls on data access inside the database? Is there a model we're following, or are we making one up? If the latter, are we sure we're solving all the use cases? What are they? What are the problems here: for instance, this has exactly the same sorts of foreign-key issues that swamped the discussion of the SE-PostgreSQL patches, and I don't see that the new proposal addresses any of that. 3. Column-level access controls. This is ruled out of the current discussion in the proposal mentioned above. Surely you need column-level access controls to make (1) work, though? If not, then I'm even more confused than I thought. 4. Metadata-level access controls. None of the proposals so far seem to provide a complete set of access controls for the system details -- schemas, databases, c. Such controls are often requested, so I wonder about that. Please understand that I'm not trying to be obstructive, but at the moment I don't understand what the proposals aim to do. I suggest that, without some clear statements of what things are trying to do, and what the intended limitations are, it will always be impossible for anyone to review the implementation of such a big feature and say whether it does what it intends to do. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
On Thu, Sep 25, 2008 at 08:57:46PM -0400, Tom Lane wrote: Another point is that the proposed behavior leaks quite a lot of information, since it will fail operations on the basis of tuples that supposedly aren't visible to the invoking user. While I admit that it's hard to see an alternative if we're to preserve FK integrity, I have to worry that this definition isn't going to satisfy the tin-foil-hat brigade that are supposed to be the main users of SEPostgres. If the goal is you don't know the row is there, this doesn't seem to meet it. The above point, and other similar ones in every discussion of the proposed functionality, makes me think once again either that the requirements for this feature aren't understood by everyone, or else that they're not actually explicit enough. I have a feeling it's the latter. Certainly, I've not yet read a complete security analysis of a data system security plan that outlines why the proposed model is correct. What I think is really happening with this development is that the SE-Linux understanding of security enhancement has been taken as the correct analysis for how one secures an information system. That deep assumption appears to me to be informing much of the development of SE-PostgreSQL. In particular, that deep assumption includes an assumption that consistency of access control trumps all. The Postgres developers who are questioning the SE approach are (I think) coming at this from the point of view of data systems developers, where consistency of the data set trumps all. I suspect that the tension between these approaches will not be reconciled without a fairly complete outline of possible security models for data systems, their relationship to what the OS security people have decided is the right thing to do, and the trade-offs necessary to make different priorities work. Some of the trade offs may include things like violate traditional understanding of data set consistency and possible disclosure of existence of datum. I think this will be a lot of work, and I'm not volunteering to do it. I nevertheless think that without it, the SE-PostgreSQL features will continue to be a very awkward fit. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
On Thu, Sep 25, 2008 at 10:32:24PM -0400, Tom Lane wrote: I can't escape the lurking suspicion that some bright folk inside the NSA have spent years thinking about this and have come up with some reasonably self-consistent definition of row hiding in a SQL database. But have they published it where we can find it? I have a couple contacts in the security world who might be able to help with references. I'm asking them now. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
Dear colleagues, I said earlier I'd ask around about some of the literature on security controls vs. databse accessibility and side channels. I did, and I heard back. One person told me that this conference often has things on this topic: http://www.ieee-security.org/TC/SP-Index.html From my brief glimpse of the TOCs from the proceedings, as well as some spelunking in the ACM guide, it seems to me that some people have already worked out what ought to happen in many of these cases, and all we need to do is write down what we think ought to happen for the various use cases. I note in particular that an awful lot of work seems to be coming out of the health care sector in this area. That strikes me as at least as good a guide as national security concerns, and anything that one might want to do probably ought to be able to cope with at least those two caricatures of use cases. I also found a 2007 doctoral thesis by Azhar Rauf, Colorado Technical University, _A tradeoff analysis between data accessibility and inference control for row, column, and cell level security in relational databases_. The title and abstract make me think it might be worth looking at. Hope this is helpful, A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches
On Fri, Sep 26, 2008 at 06:15:46PM -0400, Bruce Momjian wrote: I am confused how knowing that a sequence number used for a primary key exists or doesn't exist is leaking _meaningful_ information. This sort of side-channel intelligence is _exactly_ how certain kinds of security exploits work: I'm not supposed to know that _x_ exists; but by knowing key-of-_x_, I learn that _x_ exists. From existence, I can infer something, and from that inference I construct an attack that was supposed to be forestalled by the access controls. I am by no means a security expert, but I know enough about the area to know that it is very hard to get right, and that seemingly insignificant flaws in design turn out to be major vulnerabilities. To speak about something I do know about, when DNS was designed, nobody could have imagined that the widespread availability of recursion would turn out to be a flaw. Today, it turns out that open recursion can be used in an attack that magnifies the attacker's outbound traffic by many orders of magnitude. This sort of surprise side effect is why I am so anxious that something advertised as a security system fit really well with the proposed use cases. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Do we really need a 7.4.22 release now?
On Thu, Sep 18, 2008 at 03:25:10PM -, Greg Sabino Mullane wrote: Frankly, the whole pg_dump mess is what keeps many people on older versions, somtimes including 7.4. This isn't my experience. The reasons people stay on older releases are manifold. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
On Thu, Sep 04, 2008 at 01:26:44AM +0300, Hannu Krosing wrote: So Andrews opinion was that Mb (meaning Mbit) is different from MB (for megabyte) and that if someone thinks that we define shared buffers in megabits can get confused and order wrong kind of network card ? I know it's fun to point and laugh instead of giving an argument, but the above is not what I said. What I said is that there is a technical difference between at least some of these units, and one that is relevant in some contexts where we have good reason to believe Postgres is used. So it seems to me that there is at least a _prima facie_ reason in favour of making case-based decisions. Your argument against that appears to be, Well, people can be sloppy. Alvaro's suggestion seems to me to be a better one. It is customary, in servers with large complicated configuration systems, for the server to come with a tool that validates the configuration file before you try to load it. Postfix does this; apache does it; so does BIND. Heck, even NSD (which is way less configurable than BIND) does this. Offering such a tool provides considerable more benefit than the questionable one of allowing people to type whatever they want into the configuration file and suppose that the server will by magic know what they meant. I can understand Alvaros stance more readily - if we have irrational constraints on what can go into conf file, and people wont listen to reason Extending your current reasoning, it's irrational that all the names of the parameters have to be spelled correctly. Why can't we just accept log_statement_duration_min? It's _obvious_ that it's the same thing as log_min_duration_statement! It's silly to expect that harried administrators have to spell these options correctly. Why can't we parse all the file, separating each label by _. Then if any arrangements of those labels matches a real configuration parameter, select that one as the thing to match and proceed from there? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
On Thu, Sep 04, 2008 at 07:01:18AM -0700, Steve Atkins wrote: Settings in postgresql.conf are currently case-insensitive. Except for the units. And, of course, filenames when you are using a case-sensitive filesystem. Because these are things that are defined by some convention other than the ones the PGDG made up. Since units fall into that category, it seems to me that we're stuck with using external conventions. one right now. If the answer to that is something along the lines of we don't support megaabits for shared_buffers, and never will because nobody in their right mind would ever intend to use megabits to set their shared buffer size... that's a useful datapoint when it comes to designing for usability. And you are going to establish this worldwide convention on what someone in right mind would do how, exactly? For instance, I think nobody in right mind would use KB to mean kilobytes. I suppose you could get a random sample of all current Postgres users to decide what makes sense, but then you'd have the problem of knowing whether you had a random sample, since the population isn't obviously identifiable. Or, we could just stick with the convention that we already have, and write a tool that captures this an other issues. Maybe even one that could later form the basis for an automatic tuning advisor, as well. The problem with appeals to common sense always turns out to be that different people's common sense leads them to different conclusions. (We had a devastating government in Ontario some years ago that claimed to be doing things that were just common sense; the Province is still cleaning up the mess.) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
On Wed, Sep 03, 2008 at 06:37:29PM +0300, Hannu Krosing wrote: On Wed, 2008-09-03 at 08:20 -0700, Joshua D. Drake wrote: There is no arguing that MB != Mb; The whole point of this discussion is, that mostly people expect MB == Mb = mb == mB, especially if they see weird constructs like kB used (k for Kilo, or actually Kibi). Note that in the networked computer world, MB and Mb are importantly different. The latter is relevant for the speed of your network interface, for instance. People often get this wrong when speaking carelessly, but a mistake of this sort is a serious one, given the orders of magnitude difference. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
On Wed, Sep 03, 2008 at 01:48:18PM -0400, Alvaro Herrera wrote: I think the energy wasted in this discussion would be better spent in working a the check-the-config-file feature. That would equally solve this problem, as well as many others. This seems like a good idea to me. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: plan invalidation vs stored procedures
On Wed, Aug 20, 2008 at 03:12:43PM +0300, Asko Oja wrote: - If there is nothing that can be done in 8.3 at least warning should be added into the documentation. It will be just one more don't in our long list don'ts for our developers. I am in favour of that change in the 8.3 branch. ERROR: cache lookup failed for function. - Could the plan be marked as invalid so it would fail only once so the next call to the function would get replanned and work again. At least it would be better than losing parts of application for indeterminate time. That seems to me to be a behaviour change, not a bug fix. I agree that the current behaviour is pretty annoying. That is not the same thing as a bug except in the loosest sense. The system works as specified, and therefore it's not a bug. If the specification is wrong, you need a new specification; that's a bug fix that is usually pronounced major release. - Could some less dangerous looking mechanism be added to 8.3 that wouldn't make users not used to PostgreSQL limitations gasp for air when they see the workarounds :) I think it a very bad idea even to suggest that we start undertaking things like adding mechanisms to minor releases, even with smileys at the end of the sentence. I appreciate (possibly more than many hackers) the limitations that are imposed on users by some of the decisions historically taken by developers in some of the previous major releases. But I very strongly agree with Dimitri: the super-conservative approach to maintenance releases that this project takes is a really big benefit to users, and is ultra important in mission critical environments. Otherwise, it becomes practically impossible to get minor releases into production. If you have to worry about the possibility of major changes between minor versions, you will have to treat every release as a major release. I don't think we have sufficient commercial integration support yet that we can follow the lead of the Linux kernel, where the system vendor has the effective obligation to make sure your kernel actually works. In addition, if someone wants to develop back-patches for 8.3 that give it new functionality otherwise planned for 8.4, I see nothing wrong with them doing so. That's the advantage offered by having the source. But the idea that the new functionality should be patched back by the project because one is impatient is not on. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: plan invalidation vs stored procedures
On Wed, Aug 20, 2008 at 05:03:19PM +0300, Asko Oja wrote: Lets get on with 8.4 Oh, I shoulda mentioned that, too -- I completely support doing this work for 8.4. (I can think of more than one case where this feature alone would be worth the upgrade.) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: plan invalidation vs stored procedures
On Tue, Aug 19, 2008 at 02:47:13PM -0400, Tom Lane wrote: Whether (and how far) to backpatch has always been a best-judgment call in the past, and we've gotten along fine with that. I think having a formal policy is just likely to lead to even more complaints: I completely agree with this. If you formalise the back-patch policy, then it will be necessary to invent classifications for bug severity to determine whether to back patch. This will inevitably lead to some sort of false objectivity measure, where bugs get a severity number that actually just means we have already decided to back-patch. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: plan invalidation vs stored procedures
On Tue, Aug 19, 2008 at 12:42:29PM -0700, Joshua Drake wrote: Generally speaking we adhere to the following guideline for patches. * Security fixes are applied to all applicable branches. * Bugfixes are applied to all applicable branches * Note: A patch that addresses a known limitation is generally not backpatched * New features are always applied to -HEAD only. This is not a policy as much as a legend for developers to consider before they submit their patch. But it's meaningless. Bugfixes are applied to all applicable branches, is either false or trivially true. It's trivially true if you interpret applicable branches to mean the ones that get the patch. It's false if you mean bugfix to mean every patch that fixes a bug. I can think of bugs that we have lived with in older releases because fixing them was too risky or because the bug was so tiny or unusual as to make the risk greater than the reward. A formal policy that's any more detailed than what's in the FAQ today is a solution in search of a problem. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Should creating a new base type require superuser status?
On Wed, Jul 30, 2008 at 06:07:53PM -0400, Alvaro Herrera wrote: I do agree that creating base types should require a superuser though. It too seems dangerous just on principle, even if today there's no actual hole (that we already know of). I agree. -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] plproxy v2
On Mon, Jul 21, 2008 at 09:32:57PM -0400, Tom Lane wrote: Marko Kreen [EMAIL PROTECTED] writes: 2. If cluster connection strings do not have 'user=' key, ' user=' || current_username() is appended to it. Cool, I missed that. At minimum the documentation has to explain this point and emphasize the security implications. Is it a good idea to allow user= in the cluster strings at all? I wondered about this myself. Is there anything at all preventing me from doing 'user=' for some other user? If not. . . Also, plroxy does _nothing_ with passwords. That means the password for remote connection must be in postgres user's .pgpass, That seems *exactly* backwards, because putting the password in postgres user's .pgpass is as good as disabling password auth altogether. . . .this means that any user on system1 for which there is at least one user on system2 with plproxy access automatically also has that access on system2. (Plus what Tom noted). We regularly get beat up about any aspect of our security apparatus that isn't secure by default. This definitely isn't, and from a PR point of view (if nothing else) that doesn't seem a good idea. I'm less worried about the PR, and more worried about the truck-sized hole this opens in any authentication controls. It seems to me that it's a fairly serious problem. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Load spikes on 8.1.11
On Sat, Jul 19, 2008 at 07:09:46AM +0530, Gurjeet Singh wrote: Will try this option, at least in the next schema upgrade or when setting up Slony. As I've already suggested, however, if you try to set up slony on a loaded database, you're going to see all manner of problems. Slony takes some heavy-duty locks when it does its setup work. It's designed that you should have an application outage for this sort of work. Please see previous discussion on the Slony mailing list. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?
On Mon, Jul 21, 2008 at 01:17:39PM -0700, David E. Wheeler wrote: pgFoundry ain't the CPAN, alas. Maybe that's the problem that really needs solving? One of the big Postgres features is its extensibility. I agree that the extensions can sometimes be hard to find, but surely the answer to that is not an infinitely large source tarball? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Load spikes on 8.1.11
On Tue, Jul 22, 2008 at 02:41:55AM +0530, Gurjeet Singh wrote: I am aware of the heavy locking involved with Slony, which should mean that it blocks the application connections; that's be completely acceptable, given all the warnings in the Slony docs. But what I am concerned about and trying to hunt down is why IDLE backend processes are all consuming up all of CPU (!!!) so much so that I am unable to fire up any new process! Ah, well, then, yes, the spinlock improvements probably will help you. But you should disabuse yourself of the idea that IDLE processes have no cost. You still have to talk to all those connections when doing schema changes. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Load spikes on 8.1.11
On Fri, Jul 18, 2008 at 10:41:36AM +0530, Gurjeet Singh wrote: Just started INIT cluster Slonik command and that spiked too.. for more than 10 minutes now!! Are you attempting to do Slony changes (such as install Slony) on an active database? I strongly encourage you to read the Slony manual. Slony, frankly, sucks for this use case. The manual says as much, although in more orotund phrases than that. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Load spikes on 8.1.11
On Fri, Jul 18, 2008 at 10:05:33AM +0530, Gurjeet Singh wrote: I just ran DROP SCHEMA _slony schema CASCADE; and it spiked again, on a very low loaded box!! Ah, well, if slony is involved, then you have possible locking problems in the database _also_ to contend with, along with the spinlock problems. This will for sure cause spikes. You need to tell us more about what you're doing. And I bet some of it belongs on the slony lists. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A new take on the foot-gun meme
On Mon, Jun 30, 2008 at 12:57:03AM -0400, Tom Lane wrote: So is that a golf club gun? Maybe a gun club? A footclub? A foot-gun that can create 18 holes (36 for professionals)? Anyway, with recent USian legal rulings, I anticipate we'll have to stop talking about feet-guns. People will complain we are interfering with their right to bare feet. Or something. (Apologies, everyone. I guess I better go have more coffee.) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO Item: Allow pg_hba.conf to specify host names along with IP addresses
On Sun, Jun 15, 2008 at 11:53:57PM +0200, Peter Eisentraut wrote: Isn't that what a local DNS caching-only server would accomplish? Only if you looked up the DNS name at auth time :) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO Item: Allow pg_hba.conf to specify host names along with IP addresses
On Sun, Jun 15, 2008 at 11:56:35PM +0200, Peter Eisentraut wrote: It would probably be a good idea to check how other programs deal with hostname lookups during authentication. Programs like SSH, Apache, and Squid come to mind. There is actually a great deal of controversy about most of this hostname-based authentication, particularly in the absence of DNSSEC. If anyone implementing this is interested in the controversy, I have a huge mail archive of it (because I'm the current editor of the IETF working group document on this, and therefore have received much hate mail on the topic). I think it's all summarised in the draft[1] I mentioned upthread. Since that's possibly about to go to IETF last call, it'd be a good time for someone planning to implement something to look at that document, and report on whether it provides any useful guidance at all. I'd be keenly interested in hearing the verdict. A [1] http://tools.ietf.org/wg/dnsop/draft-ietf-dnsop-reverse-mapping-considerations/ -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO Item: Allow pg_hba.conf to specify host names along with IP addresses
On Mon, Jun 16, 2008 at 11:47:21AM +0200, Peter Eisentraut wrote: I'm a bit curious how useful in practice this would actually be. Obviously, you want to use host names to simplify the management of hosts, currently being done with IP addresses. But how widely useful is it really to authenticate a bunch of hosts in different ways? I'd say the standard case is localhost vs everything else. Or perhaps localhost vs LAN vs rest of the Internet. In neither of these cases , using host names helps much. The obvious case for it (which is why I'm not arguing against using it as such) is network renumbering. If you renumber a network, right now you have to update these files. Today this isn't a big deal, but as IPv4 addresses get scarcer and IPv6 addresses come online, this is going to become a problem people have more often. In large deployments with a lot of postmasters and many people's hands involved, one fewer change to manage would be a boon. Moreover, in managed networks, you don't always control when your IPs will change or how. Doing this by hostname could have advantages for reliability, at the possible cost of startup performance. One reason to use DNS names rather than static /etc/host entries or IP addresses is this resilience in the face of a changing network infrastructure. We have people here concerned about security of DNS, rightly so. But what about relying on IP addresses or, by extension, MAC addresses for security; is that safe? Well, there's one fewer thing that can be subverted. But authenticating from a host address is pretty weak authentication. I'd think the Morris worm teaches us that. Without DNSSEC, subverting the DNS is very close to trivial. But, in the presence of competent DNSSEC deployment, subverting the DNS becomes just shy of impossible, so this might become a strategy approximately as strong as authentication by host address. You're still trusting a connection on the basis of who it claims to be and where it's coming from; that's hardly strong authentication. I agree with Andrew Dunstan that for any real world wide-scale uses, you want to use some sort of strong authentication. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO Item: Allow pg_hba.conf to specify host names along with IP addresses
On Fri, Jun 13, 2008 at 12:47:22PM -0400, Andrew Dunstan wrote: The reason it wasn't done years ago was that there was disagreement on the way it should work. And the TODO actually lists several alternatives: Host name lookup could occur when the postmaster reads the pg_hba.conf file, or when the backend starts. Another solution would It needs to happen at authentication time. I'm not sure whether reads the pg_hba.conf or backend starts is the right way to say that, but it must happen only when you're actually authenticating the host entry. This is because DNS RRs have a TTL on them, so looking up the host at any moment other than when you're actually doing the authentication is prone to error. be to reverse lookup the connection IP and check that hostname against the host names in pg_hba.conf. We could also then check that the host name maps to the IP address. There is, curiously, an existing Internet Draft currently in WGLC at the dnsop working group at the IETF that warns explicitly against using hostname forward and reverse matching checks as a security mechanism, without having other options. So if the mechanism is going to force matching forward and reverse data, then I urge whoever implements this to make it possible to turn that matching check off, because it won't work reliably. The draft is available from http://tools.ietf.org/wg/dnsop/draft-ietf-dnsop-reverse-mapping-considerations/. By the way, in the context of DNSSEC, a matching check might not add anything, but a check for existing signed reverse data may. That is, if you have authenticated forward zone data and you have authenticated reverse zone data, you can be confident that you have the right hostname even if the forward and reverse hostnames don't match. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO Item: Allow pg_hba.conf to specify host names along with IP addresses
On Fri, Jun 13, 2008 at 08:51:19PM +0100, Simon Riggs wrote: The best of both ideas would be to have an option inside pg_hab.conf to indicate when lookup occurs. Some parts of a network are static, others are not, so a global option would not be useful. We would point and laugh at people who thought that something was static inside PostgreSQL, and depended on that for something critical without some pretty heavy-duty locks. Are we really proposing to offer an authentication mechanism that depends on something as flimsy as hostname lookups in the DNS, and then not insist that the bare minimum of integrity check (I checked this DNS lookup at connection time) is the rule? DNS is a distributed database. Surely the least we can demand is that the lookup happen when the naive think it will (i.e., at the time the connection from that hostname happens). If the user knows a portion of their network is static, If there were the slightest evidence that users historically believed in such knowledge correctly, then I might have some sympathy for this. The fact is that DNS (at least without DNSSEC) is one of the areas in which sysadmins have the worst record of trust to this day. I think we'd be fools to encourage such trust. If you don't look up at _least_ at connection time, this feature should be rejected on the grounds that it opens a new authentication hole a mile wide. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Wed, Jun 04, 2008 at 09:24:20AM +0200, Markus Schiltknecht wrote: Are the slides of your PgCon talk available for download somewhere? There weren't any slides, really (there were 4 that I put up in case the cases I was discussing needed back-references, but they didn't). Joshua tells me that I'm supposed to make the paper readable and put it up on Command Prompt's website, so I will soon. BTW: up until recently, there was yet another mailing list: [EMAIL PROTECTED] It was less focused on hooks and got at least some traffic. :-) Are those mails still archived somewhere? Unless whoever was operating that list moved it to pgfoundry, I doubt it (except on backups somewhere). A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Sun, Jun 01, 2008 at 01:43:22PM -0400, Tom Lane wrote: power to him. (Is the replica-hooks-discuss list still working?) But Yes. And silent as ever. :-) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Case-Insensitve Text Comparison
On Sun, Jun 01, 2008 at 10:13:07PM -0700, David E. Wheeler wrote: What locale is right? If I have a Web app, there could be data in many different languages in a single table/column. I think the above amounts to a need for per-session locale settings or something, no? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Table rewrites vs. pending AFTER triggers
On Sun, Jun 01, 2008 at 11:51:29PM -0700, Robert Hodges wrote: If you are saying that DDL should be auto-commit, yes, this really does limit some use cases. I agree. Transactional DDL is a big feature I'd hate to see go away. Oracle DBAs I know look with envy on this feature of Postgres. with the database that causes a command to fail. SLONY is an example of such a generator where transactional DDL would be helpful though I don¹t know for a fact that SLONY uses it. It sort of does, in that all work under Slony is performed in a transaction. But Slony attempts to isolate the DDL in a sync at just the right point, and not mix schema and data changes in syncs. Still, one of the important best practices in preparing your DDL changing scripts for Slony is to try running that script inside a transaction (and then rolling back) on every node, to ensure that it will in fact work on every node. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Case-Insensitve Text Comparison
On Mon, Jun 02, 2008 at 10:29:30AM -0700, Jeff Davis wrote: What if you had a CHECK constraint that was locale-sensitive? Would the constraint only be non-false (true or null) for records inserted under the same locale? That's not very useful. It would seem that this is one of the important cases that needs to be worked out. I wasn't suggesting that per-session locale (or whatever we want to call it) is _easy_ or, for that matter, even possible; just that it would solve a large number of the problems that people complain about. In fact, I suspect that what we really need is something a little more like in-database locale or something. I think if you want some special treatment of text for some users, it should be explicit. Yes. Also, not just text. Think of currency, numeric separators, c. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Thu, May 29, 2008 at 01:58:34PM -0700, David Fetter wrote: If people on core had come to the idea that we needed to build in replication *before* 8.3 came out, they certainly didn't announce it. Now is a great time to mention this because it gives everybody time to: 1. Come to a consensus on what the out-of-the-box replication should be, and 2. Build, test and debug whatever the consensus out-of-the-box replication turns out to be. None of that is an argument for why this has to go in 8.4. I argued in Ottawa that the idea that you have to plan a feature for _the next release_ is getting less tenable with each release. This is because major new features for Postgres are now often big and complicated. The days of big gains from single victories are mostly over (though there are exceptions, like HOT). Postgres is already mature. As for the middle-aged person with a mortgage, longer-term planning is simply a necessary part of life now. There are two possibilities here. One is to have huge releases on much longer timetables. I think this is unsustainable in a free project, because people will get bored and go away if they don't get to use the results of their work in a reasonably short time frame. The other is to accept that sometimes, planning and development for new features will have to start a long time before actual release -- maybe planning and some coding for 2 releases out. That allows large features like the one we're discussing to be developed responsibly without making everything else wait for it. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Thu, May 29, 2008 at 12:11:21PM -0400, Brian Hurt wrote: Being able to do read-only queries makes this feature more valuable in more situations, but I disagree that it's a deal-breaker. Your managers are apparently more enlightened than some. ;-) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Thu, May 29, 2008 at 07:20:37PM +0300, Marko Kreen wrote: So you can do lossless failover. Currently there is no good solution for this. Indeed. Getting lossless failover would be excellent. I understand David's worry (having had those arguments more times than I care to admit), but if people don't want to spend the money on the extra machine that can't be queried, they can use another solution for the time being. The big missing piece is lossless failover. People are currently doing it with DRBD, various clustering things, c., and those are complicated to set up and maintain. (As I've told more than one person looking at it, there is a risk that you'll actually make your installation complicated enough that you'll make it _less_ reliable. I have some bitter personal experiences with this effect, and I know some others on this list do as well.) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Thu, May 29, 2008 at 12:05:18PM -0700, Robert Hodges wrote: people are starting to get religion on this issue I would strongly advocate a parallel effort to put in a change-set extraction API that would allow construction of comprehensive master/slave replication. You know, I gave a talk in Ottawa just last week about how the last effort to develop a comprehensive API for replication failed. I had some ideas about why, the main one of which is something like this: Big features with a roadmap have not historically worked, so unless we're willing to change the way we work, we won't get that. I don't think an API is what's needed. It's clear proposals for particlar features that can be delivered in small pieces. That's what the current proposal offers. I think any kind of row-based approach such as what you're proposing would need that kind of proposal too. That isn't to say that I think an API is impossible or undesirable. It is to say that the last few times we tried, it went nowhere; and that I don't think the circumstances have changed. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] replication hooks
On Thu, May 29, 2008 at 11:05:09PM +0300, Marko Kreen wrote: There is this tiny matter of replicating schema changes asynchronously, but I suspect nobody actually cares. I know that Slony's users call this their number one irritant, so I have my doubts nobody cares. But maybe nobody cares enough. - The task cannot even be clearly defined (on technical level - how the events should be represented). Really? I've been in discussions where different people had clear (but, alas, different) ideas of how to represent them. - Any schema changes need to be carefully prepared anyway. Whether to apply them to one or more servers does not make much difference. One problem that designers of replication systems have is that they're already thinking in the Serious Database Application world. But I have recently had the pleasure of being reminded how many users of database systems neither know nor care to know any of the details of the underlying system. They already know how to make schema changes: log into database, and start typing ALTER TABLE. . . You or I agreeing that more careful preparation than that is important will not change their mind. This is part of the reason MySQL looks so good: you can just do these things. If it doesn't work out later, well, you don't know that when your ALTER TABLE just works. - Major plus of async replica is ability to actually have different schema on slaves. I agree. - People _do_ care about exact schema on single place - failover servers. Yeah, but not only there. One of the things I was hoping to have nailed down in the hooks discussion was, in fact, the use cases. Half the time, people have such a clear idea of what _they_ want from their replication that they come to believe replication means that. Another thing I like about the current proposal is that it is very clear about what it is (and isn't) aiming for. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Thu, May 29, 2008 at 01:39:29PM -0700, David Fetter wrote: I think the consensus in the core team was that having synchronous log shipping in 8.4 would already be a worthwhile feature by itself. If that was in fact the consensus of the core team, and what I've been seeing from several core members in this thread makes that idea unclear, it's out of step with the stated goal of the feature. Having some kind of half-way, doesn't-actually-quite-work-out-of-the-box replication will make things worse and not better. I don't agree with that. It seems to me that the stated goal is fourfold: 1. Get something that is simple (out of the box). 2. Get something that is generally useful, even if not universally useful. 3. Get something that is implementable in stages, with each stage representing an increase in functionality. 4. Aim for 8.4, but be realistic about chances. It seems to me that sync. replication to a write-only standby actually meets those goals. I can think of more than one case where I have spent lots of time, money, or both attempting to achieve this goal with various add-on hardware and software systems that mostly worked except when they didn't. These additions were complicated to set up, painful to use, and documented by sadists who couldn't write. I agree that there are plenty of people unwilling to use a system that requires additional hardware that mostly can't be used for other things. But those people already have alternatives (maybe even more, and simple ones, soon). The synchronous-needing crowd currently have nothing. The proposed feature would be a huge improvement. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Core team statement on replication in PostgreSQL
On Thu, May 29, 2008 at 07:02:56PM -0400, Tom Lane wrote: People want the bits to go from point A to point B; they don't want to have to research, design, test, and administer their own solution for moving the bits. I agree with this. I think I probably know as well as most people -- perhaps less well than maybe two people in the world -- where most of the bodies in Slony are hidden, and I still find it a pain to administer. Other systems are only somewhat less troublesome; and I really do know about the concepts involved. I'm not tripping on important things. It's just some work to set up. Other systems hide that work. Given that (for instance) psql is really very easy to use once you know a few things, the ongoing pain of simple replication in Postgres is a big wart. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting a pre-existing index as a primary key
On Sat, May 10, 2008 at 11:55:29AM -0400, Tom Lane wrote: IMHO a utility command should do one easily-explained thing. The fewer options the better. Sticking to that principle makes for a better-maintained system. I agree. If we want to point out, You might rename your index afterwards to make it look like other default primary keys, I have no objection. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [0/4] Proposal of SE-PostgreSQL patches
On Wed, May 07, 2008 at 12:01:21AM -0400, Greg Smith wrote: It may be the case that clean row and column filtering at the SQL layer are pre-requisites for a clean SELinux implementation, where the only difference is that the permission checks are handled by asking SELinux instead of looking in the catalog. That strikes me as an approach more likely to be fruitful. I get the point about imposing all the restrictions at the SELinux layer. But the way to do that, I think, is to make the individual policies possible to implement in PostgreSQL simpliciter, and then have some interface to the SELinux permissions system so that it becomes possible to set those definitions outside Postgres. (I know that the latter raises all sorts of nasty DoS scenarios. That's clearly one of the problems that will need addressing.) Again, I support the effort in principle; I'm just not sure that the current proof-of-concept work is what will be needed to address the design goals. I do think that somewhat clearer scope definitions would be a big help in deciding which modifications are really needed, and where. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [0/4] Proposal of SE-PostgreSQL patches
On Tue, May 06, 2008 at 02:56:41PM -0400, Tom Lane wrote: AFAICS the only thing left that really needs to be discussed more during this commit-fest is the business about whether it's sane to be trying to apply selinux restrictions in simple_heap_update and friends. I don't have any opinion about the patches, obviously, but I'm wondering whether there is, somewhere, an outline of what the _goals_ of this system are. If we have requirements that we can measure the implementation against, we can know whether to throw away a particular goal as being basically incompatible with the design of Postgres (i.e. not an enhancement but a reconception) or else as being implementable with another approach. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [0/4] Proposal of SE-PostgreSQL patches
On Tue, May 06, 2008 at 03:28:25PM -0400, Tom Lane wrote: The only documentation I've seen is http://code.google.com/p/sepgsql/wiki/WhatIsSEPostgreSQL which contains only examples of enforcing restrictions on *user* queries and tables. I agree that, having just read that, anything that involves itself with the system catalogues and such is way overstepping the stated design goal. There is an issue in most high-security systems having to do with side-channel leakage of supposedly sensitive data. So, the mere exsistence of certain tables, columns, or users might be regarded as security-sensitive data. I'm not sure I see how to get around that without mucking in the areas that are causing some of the trouble. But I think before we get into that discussion, a fairly clear statement of exactly which problems are going to be in scope is needed. FWIW, I support and think important the row- and column- level access controls this seems to be proposing, at least in principle. Whether that's a support that will extend to 2x overhead on everything is rather a different matter. Also, I am more than prepared to trade away some cases in order to get a broadly useful functionality (so if you can't hide the existence of a table, but all efforts to learn its contents don't work, I might be willing to support that trade-off). A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Odd timezone backend output
On Thu, May 01, 2008 at 09:58:09PM -0400, Andrew Chernow wrote: The more I think about it, I personally like the display behavior of NTFS file times over something like EXT3. When I am in EDT, it is useful to have all display times in that zone (regardless of whether that time falls within winter or summer) ... just as they would if I switched to PST. There's a difference between the two cases you're thinking of, though. In the case of you being in EDT, you didn't actually switch time zone. In Eastern time zones, the offset is -05 from UTC for part of the year, and -04 from UTC from part of the year. That's what it _means_ to be in Eastern time: you didn't really switch time zone at all. I'm in America/Toronto in January and in June. My display should show the time correct to my time zone, not according to the offset from UTC. If you really want that, change your time zone to be an offset from UTC rather than a particular zone. (We used to have to differentiate between EST and EDT during the summer months, because Indiana didn't switch. As far as I know, there is no longer a single jurisdiction where the summer time change doesn't happen in Eastern time. If not for historical reasons, I'd argue the name should be changed to Eastern time.) If you switch to Pacific time (and why is it that people say EDT but PST? I don't know of any Pacific time jurisdictions that don't switch, either. Arizona doesn't switch mostly, but they're in Mountain time), the same thing happens. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Odd timezone backend output
On Fri, May 02, 2008 at 09:54:20AM -0400, Andrew Chernow wrote: I prefer offset from UTC, the timezone abbrevs are ambiguos and confusing. So did you read the manual on SET TIME ZONE? TIME ZONE SET TIME ZONE value is an alias for SET timezone TO value. The syntax SET TIME ZONE allows special syntax for the time zone specification. Here are examples of valid values: 'PST8PDT' The time zone for Berkeley, California. 'Europe/Rome' The time zone for Italy. -7 The time zone 7 hours west from UTC (equivalent to PDT). Positive values are east from UTC. INTERVAL '-08:00' HOUR TO MINUTE The time zone 8 hours west from UTC (equivalent to PST). A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
On Fri, May 02, 2008 at 03:58:01PM -0400, Chris Browne wrote: Andrew Sullivan recently had some choice words about the merits of ENUM; I think the same applies to drivers that do PQexec(COMMIT;BEGIN)... Oh, heaven. I can at least think of ways to use ENUM such that you can justify the trade-off. I can think of no excuse whatever for PQexec(COMMIT; BEGIN). That's just lazy and sloppy. Note also that more recent releases, concurrent with the improvements to the drivers, also reduce the impact of this sort of database misuse slightly. But really, people who are doing that sort of thing have no excuse for themselves. They should be relegated to the same circle of hell as people who think it's a good plan to write a crappy schema the first time, because you can always optimise later. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
On Wed, Apr 30, 2008 at 05:33:38PM -0400, Tom Lane wrote: you're at risk of some clients being secure and some not. I thought what we were discussing was a server-side GUC parameter that would disallow more than one SQL statement per PQexec. That was certainly what I was intending, yes. The _principal_ trick with SQL injection is to fool the application into somehow handing a ; followed by an arbitrary SQL statement. There are of course other things one can do, but most of them are constrained to abuse of statements your application already performs. This injection problem, on the other hand, allows an attacker to do whatever they want. Obviously, if the server simply throws an error whenever one tries to do this, the attack will be foiled. It sounded to me like a patch that implemented this was already rejected. I agree that it's a bit filthy, and I'd way prefer that people build their applications such that these vectors aren't open in the first place. But given the prevalence of quick and dirty development with code one hasn't always completely vetted, this might be a nice feature in some environments. As long as it's possible to turn it off (we'd probably need to make it require a server restart to make it really effective), I think it could be useful. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Odd timezone backend output
On Thu, May 01, 2008 at 09:53:41AM -0400, Andrew Chernow wrote: I am confused about the below results. The backend is in EDT but it is converting timestamps into EST ... excluding NOW(). Regardless of the timezone provided, the backend is dishing out EST. First, this doesn't really belong on hackers, which is for hacking of the back end. If you're going to follow up, please take it to -general or something. Your client thinks it's in America/Toronto or something similar. You can set the time zone you're in by SET TIME ZONE (or timezone). There's a lot more about this in the manual at http://www.postgresql.org/docs/8.3/interactive/datatype-datetime.html#DATATYPE-TIMEZONES postgres=# select now(); now - Thu May 01 09:28:53.164084 2008 EDT May is in Daylight time in (AFAIK all) Eastern zones. postgres=# select '1997-01-29 12:31:42.92214 EDT'::timestamptz; timestamptz Wed Jan 29 11:31:42.92214 1997 EST postgres=# select '1997-01-29 12:31:42.92214 PST'::timestamptz; timestamptz Wed Jan 29 15:31:42.92214 1997 EST January is in Standard time in Eastern zones. Note that you asked for 1997-01-29 12:31:42.92214 EDT, and got back what time that would be _for your actual timezone_. Same thing for the Pacific case. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
On Thu, May 01, 2008 at 11:26:21AM -0400, Tom Lane wrote: 1. Inexpensive to implement; 2. Unlikely to break most applications; 3. Closes off a fairly large class of injection attacks. The cost/benefit ratio looks pretty good (unlike the idea that started this thread...) That's a much more elegant way of putting what I thought. Thanks, Tom. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
On Thu, May 01, 2008 at 06:33:07PM +0200, PFC wrote: But it's true that preventing multi-statements adds a layer of idiot-proofness... a rather thin layer... As I already said in a previous remark in this thread, I don't really like partial security solutions. What the no multi-statement SQL switch adds is a complete protection against _one class_ of injection attacks. What is nice about it is that it completely eliminates that class of attacks, so they are no longer something one needs to worry about. They do not, of course, prevent every kind of injection attack. I think the thread has already had ample evidence that such complete prevention is either impractical to implement, too costly to existing applications, too limiting, not actually effective (i.e. not really complete prevention), or some combination of the above. That's not an argument that the simple change that is effective for only one class of attacks is a bad idea. Making the battlefield smaller is one thing one can do to decrease one's exposure to attack. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
On Tue, Apr 29, 2008 at 09:02:30PM -0400, Gregory Stark wrote: Did you guys miss Tom's comment up-thread? Postgres already does this if you use PQExecParams(). I did, yes. Thanks for the clue. OTOH, I do see the OP's point that it'd be nice if the DBA could enforce this rule. Maybe a way of insisting on PQExecParams() instead of anything else? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
On Tue, Apr 29, 2008 at 04:33:01PM -0400, Andrew Dunstan wrote: Moreover, it seems unlikely that it will even cover the field. A partial cloak might indeed be worse than none, in that it will give some developers an illusion of having security. I think this is a really important point, and one that isn't getting enough attention in this discussion. Half a security measure is almost always worse than none at all, exactly because people stop thinking they have to worry about that area of security at all. I think without a convincing argument that the proposal will even come close to covering most SQL injection cases, it's a bad idea. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Protection from SQL injection
[I know, I know, bad form] On Tue, Apr 29, 2008 at 04:55:21PM -0400, Andrew Sullivan wrote: thinking they have to worry about that area of security at all. I think without a convincing argument that the proposal will even come close to covering most SQL injection cases, it's a bad idea. To be perfectly clear, I also think that the reverse is true: if a fairly complete design was demonstrated to be possible such that it covered just about every case, I'd be all for it. (I sort of like the suggestion up-thread, myself, which is to have a GUC that disables multi-statement commands. That'd probably cover a huge number of cases, and combined with some sensible quoting rules in client libraries, would quite possibly be enough.) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] we don't have a bugzilla
On Sun, Apr 27, 2008 at 11:58:01AM -0400, Alvaro Herrera wrote: So the proper thing to do is complain to the writer of the GUI app so that it has an option for showing the list headers, perhaps adding a menu entry when they are found. At least in the case of Thunderbird, you already have an option: http://www.juergen-ernst.de/addons/mailinglistheader.html According to the manual, claws has a whole Message/Mailing-List submenu with built-in support for this. (This took me approximately 10 seconds to find through Google. The claws manual looks pretty good, I have to say.) I was unable to find the way to to this in Lookout! I don't use the Mac OS Mail client, but a quick look didn't yield anything obvious. However, these both allow you to look at the raw mail message, which has the header. Given that List-Archive was standardized in RFC2369, which came out in 1998, and that virtually every list server software shipping since about 2000 has it built in and turned on by default, I fail completely to see how using something as free-form as a signature footer is supposed to be an improvement. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] we don't have a bugzilla
On Sat, Apr 26, 2008 at 08:54:46AM -0700, Joshua D. Drake wrote: How would he know to search at the archives? * There is no archives signature at the bottom of -hackers lists Maybe because there's a perfectly functional archive link in the mail headers? And because there's an RFC that tells us how such headers are supposed to work? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commit fest queue
On Fri, Apr 11, 2008 at 06:46:18PM +0100, Gregory Stark wrote: As an aside, you've reminded me about another thing that bothers me about Bugzilla and RT. In both cases they seem to put a lot of focus around the idea of searching bugs. I don't really get why. To be fair to RT, it's really designed as a general-purpose trouble-ticket system. If you've ever worked a help desk, you'll have no trouble knowing why searching is a valuable function. But yes, you can (with some pain, like everything else in RT) completely rejigger the access screens to eliminate this. A -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 05:09:14PM -0400, Andrew Dunstan wrote: Standard Modules. Maybe we could rename the directory modules. IIRC This seems like an easy and practical answer. A -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [pgsql-www] New email list for emergency communications
On Mon, Mar 24, 2008 at 06:39:25PM -0400, Andrew Dunstan wrote: perusing a mailbox again. We have unfortunately been badly underprepared for this. Surely that there is an emerging consensus to that effect means that it's not as unfortunate as it might be? I seem to recall the original announcement suggesting this was an experiment. I wouldn't expect the first couple rounds to go without a hitch; as long as there is procedural improvement the next time, that's a good thing, right? A -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x
On Thu, Feb 28, 2008 at 08:58:01AM -0800, Josh Berkus wrote: Well, I guess the question is: if we don't offer some builtin way to render non-standard formats built into company products, will those companies fix their format or just not use PostgreSQL? Well, there is an advantage that Postgres has that some others don't: you can extend Postgres pretty easily. That suggests to me a reason to be conservative in what we build in. This is consistent with the principle, Be conservative in what you send, and liberal in what you accept. A ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Including PL/PgSQL by default
On Thu, Feb 21, 2008 at 10:43:27AM -0800, Joshua D. Drake wrote: often. It is poor implementation and proof that the theoretical security implications that are being brought up in this thread are far from the practical reality. We have this hole over here for historical reasons, so let's maybe open a new one over there? A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Including PL/PgSQL by default
On Tue, Feb 19, 2008 at 08:37:51PM -0500, Andrew Dunstan wrote: The way I intended to do it would indeed allow it to be undone simply by executing 'drop language plpgsql' in template1. Why isn't it enough that administrators can do CREATE LANGUAGE plpgsql in template1? I think this is completely unneeded, given the ease with which this can be enabled. It seems to me the source distribution of the code ought to be minimalist. Moreover, given that the trend in daemons is to turn everything off by default, just in case, I'm puzzled why we want to do the opposite here. Note that packagers are in a different boat entirely; I see no reason why packages might not turn this on by default. But they have a narrower target of users. I'd be more persuaded by a convenience package of things to enable by default that ships with the code, and can be run by the installing party. We'd at least then have an argument to the security community that we require explicit administrator action to enable the features. A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] VACUUM FULL out of memory
On Tue, Jan 08, 2008 at 09:50:07AM +0100, Michael Akinde wrote: stack size (kbytes, -s) 8192 Perhaps this is the issue? (I don't know.) Also, this _is_ for the postgres user, right? That's the relevant one: the one that's actually running the back end process. Also, are you sure there's nothing else in the way? I don't remember what OS you're using. On AIX, for instance, there's some _other_ dopey setting that allows you to control user resource consumption as well, and it means that ulimit's answers are not the full story. (I learned this through painful experience, and confess it's one of the many reasons I think AIX should be prounounced as one word, rather than three letters.) Andrew Sullivan wrote: Something is using up the memory on the machine, or (I'll bet this is more likely) your user (postgres? Whatever's running the postmaster) has a ulimit on its ability to allocate memory on the machine. If one looks at the system resources while the VACUUM FULL is going up, its pretty obvious that its a postgres process going on a memory allocation rampage that eats up all the resources. Of course VACUUM FULL is eating up as much memory as it can: it's moving a lot of data around. But is it in fact exhausting memory on the machine? There are only two possibilities: either there's something else that is preventing that allocation, or else you've run into a case so unusual that nobody else has ever seen it. The data you're talking about isn't that big: I've run similar-sized databases on my laptop without pain. Or in this case: if VACUUM FULL is never required (except in very special circumstances), it might be a good idea not to have VACUUM recommend running it (cf. the VACUUM I ran before New Year on a similar size table). The suggestion you see there, though, is in fact one of the cases where you might in fact want to run it. That is, WARNING: relation pg_catalog.pg_largeobject contains more than max_fsm_pages pages with useful free space HINT: Consider using VACUUM FULL on this relation or increasing the configuration parameter max_fsm_pages. what it is saying is that a regular vacuum can no longer recover all the dead pages in the table, and if you want that space back and marked usable on your disk, you have to run VACUUM FULL (or, in fact, CLUSTER, or else dump and reload the table. But one of these). Note that I said that, if you have things configured _correctly_, you shouldn't have to run VACUUM FULL except in unusual circumstances. That doesn't mean never. The problem here is an historical one: you have a hangover from previous missed maintenance or sub-optimal vacuum scheduling. In those cases, you may want to perform VACUUM FULL, provided you understand the potential side effects (like possibly slower inserts initially, and some possible index bloat). A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] VACUUM FULL out of memory
On Tue, Jan 08, 2008 at 05:27:16PM +0100, Michael Akinde wrote: Those are the ulimits of the db_admin account (i.e., the user that set up and runs the DB processes). Is Postgres limited by other settings? Are you sure? On one system I used many years ago, /bin/sh wasn't what I thought it was, and so the ulimit that I got when logged in was not what the postmaster was starting under. Took me many days to figure out what was up. A ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps
On Tue, Jan 08, 2008 at 01:08:52AM +0100, Markus Schiltknecht wrote: Uh, which key are you talking about? AFAIU Simon's proposal, he suggests maintaining min/max values for all columns of the table. Right, but I think that's just because that approach is automatable. Only some use cases are going to be approproate to this. Yeah, and if only *one* tuple in the 1G segment has: some_date = '1998-12-31' OR some_date = '2001-01-01' Segment Exclusion can't exclude that segment. That's all I'm saying. Correct. Huh? I'm certainly not the one asking for it. Quite the opposite, I'm warning from over-estimating the use of SE. Right; I think one should be clear that there are many -- maybe most -- uses of PostgreSQL where the proposal will be of no use. I just think we need to be clear that for the areas where it _can_ be useful, it could be very useful indeed. A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps
On Tue, Jan 08, 2008 at 02:12:28AM +, Gregory Stark wrote: Yes: it doesn't solve the problem I have, which is that I don't want to have to manage a whole bunch of tables. I want one table, and I want to be able to say, That section is closed. That's not your problem, that's the solution you're looking for. You're assuming a particular solution in your problem statement. Probably in that one, yes. I'm still waiting for permission to post my original problem statement; I suspect it's not going to be forthcoming by next Monday, so it's not going to happen. But I did outline something like what I'm talking about elsewhere in this thread. For my case, I'm thinking of the sort of data that builds up over time, and most of which happens probably not to be useful at any moment, but all of which _might_ be useful over the long haul. So what I wanted, originally, was to be able to set arbitrary ranges of tuples to be read-only, and to be able to set them offline if I wanted. Pseudo-DDL: ALTER TABLE foo SET read_only='t' WHERE created_on '2007-01-01'; ALTER TABLE foo SET tuple_offline='t' WHERE created_on '2006-01-01'; Now, the second segment is marked offline. If I query the table for things in that range, I get an ERROR telling me there might be data in the range, but it's not mounted at the moment. If I try to update records in the read-only (first) range, I get an error telling me the tuple is marked read only. The idea then is that these older tuples can be put off into long-term storage (wave hands here about the management of that stuff), and this keeps my online system compact but yet allows me, for just the cost of mounting a backup tape and reading the segments back, to go back and query those old ranges. The case I was particularly aiming at originally was for a case of data that cannot cost more than fractions of pennies to store, but that might represent a hugely expensive liability if the answer is not always right. Driving down that storage cost was mostly what I was aiming at, but people gradually convinced me that slightly more generic implementations might be useful. Simon's proposal came along, and it seems to me to be something like the generic implementation that others already convinced me was needed. I think Simon's proposal loses the very feature that makes partitioning useful. The DBA doesn't have a thing to describe, he has to define what parts of the table he's describing for every operation. And if you define a whole new object to name these things I think you'll end up with something that looks a lot like tables. I don't see how that's the case at all. In fact, I have the feeling it's the opposite, so perhaps I've misunderstood something. A ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] VACUUM FULL out of memory
On Tue, Jan 08, 2008 at 05:53:28PM +, Sam Mason wrote: What about a stored procedure in a language that allows you to do system(3) calls? PL/bash? (I think there is something like this). But surely the ulimit before start is much easier! A ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps
On Sat, Jan 05, 2008 at 08:02:41PM +0100, Markus Schiltknecht wrote: Well, management of relations is easy enough, known to the DBA and most importantly: it already exists. Having to set up something which is *not* tied to a relation complicates things just because it's an additional concept. But we're already dealing with some complicated concepts. There isn't anything that will prevent current-style partitioning strategies from continuing to work in the face of Simon's proposal. But let me see if I can outline the sort of cases where I see real value in what he's outlined. There is a tendency in data systems to gather all manner of data that, in retrospect, _might_ turn out to be be valuable; but which, at the time, is not really valuable at all. Moreover, the value later on might be relatively low: if you can learn something much later from that data, and do so easily, then it will be worth doing. But if the work involved passes some threshold (say 1/2 a day), it's suddenly not worth it any more. It's simple economics: below a certain cost, the data is valuable. Above a certain cost, you simply shouldn't keep the data in the first place, because the cost of using it is higher than any value you'll likely be able to extract. Simon's proposal changes the calculations you have to do. If keeping some data online longer does not impose administrative or operational overhead (you have it marked read only, so there's no I/O for vacuum; you don't need to do anything to get the data marked read only; c.), then all it costs is a little more disk, which is relatively cheap these days. More importantly, if the longer-term effect of this strategy is to make it possible to move such data offline _without imposing a big cost_ when moving it back online, then the value is potentially very high. Without even trying, I can think of a dozen examples in the past 5 years where I could have used that sort of functionality. Because the cost of data retrieval was high enough, we had to decide that the question wasn't worth answering. Some of those answers might have been quite valuable indeed to the Internet community, to be frank; but because I had to pay the cost without getting much direct benefit, it just wasn't worth the effort. The thing about Simon's proposal that is beguiling is that it is aimed at a very common use pattern. The potential for automatic management under such a use pattern makes it seem to me to be worth exploring in some detail. Agreed. I'd say that's why the DBA needs to be able to define the split point between partitions: only he knows the meaning of the data. I think this is only partly true. A casual glance at the -general list will reveal all manner of false assumptions on the parts of administrators about how their data is structured. My experience is that, given that the computer has way more information about the data than I do, it is more likely to make the right choice. To the extent it doesn't do so, that's a problem in the planning (or whatever) algorithms, and it ought to be fixed there. A ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] VACUUM FULL out of memory
On Mon, Jan 07, 2008 at 10:40:23AM +0100, Michael Akinde wrote: As suggested, I tested a VACUUM FULL ANALYZE with 128MB shared_buffers and 512 MB reserved for maintenance_work_mem (on a 32 bit machine with 4 GB RAM). That ought to leave more than enough space for other processes in the system. Again, the system fails on the VACUUM with the following error (identical to the error we had when maintenance_work_mem was very low. INFO: vacuuming pg_catalog.pg_largeobject ERROR: out of memory DETAIL: Failed on request of size 536870912 Something is using up the memory on the machine, or (I'll bet this is more likely) your user (postgres? Whatever's running the postmaster) has a ulimit on its ability to allocate memory on the machine. It strikes me as somewhat worrying that VACUUM FULL ANALYZE has so much trouble with a large table. Granted - 730 million rows is a good deal - No, it's not really that big. I've never seen a problem like this. If it were the 8.3 beta, I'd be worried; but I'm inclined to suggest you look at the OS settings first given your set up. Note that you should almost never use VACUUM FULL unless you've really messed things up. I understand from the thread that you're just testing things out right now. But VACUUM FULL is not something you should _ever_ need in production, if you've set things up correctly. A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps
On Mon, Jan 07, 2008 at 07:16:35PM +0100, Markus Schiltknecht wrote: Does anything speak against letting the DBA handle partitions as relations? Yes: it doesn't solve the problem I have, which is that I don't want to have to manage a whole bunch of tables. I want one table, and I want to be able to say, That section is closed. Sure, there's value in Simon's proposal. But it has pretty strict requirements. IMO, it's pretty hard to say, if it would have helped at all for your cases. Any of them still available to check? No, but one of your worries doesn't bother me: Remember the requirements: no single tuple in the segment may be significantly out of the average bounds. Otherwise, the min/max gets pretty useless and the segment can never be excluded. The segment can never be excluded in a search on that key, yes. But consider the likely cases we're looking at: WHERE some_date = '1999-01-01' AND some_date '2001-01-01'; WHERE sequence_field BETWEEN 3000 AND 30; c. These are the two obvious cases: you're searching for data in a given date range or for primary (sometimes artificial) identifiers in a range, and the source data increases (almost) monotonically. You have to do this now anyway, because there's _some_ basis on which you're partitioning your data; but today, you do this with a lot of fooling around with views and nasty triggers that push data into the right table, assuming someone doesn't screw it up. need to maintain CLUSTERed ordering, aren't there better ways? For example, you could use binary searching on the relation directly, much like with indices, instead of sequentially scanning on the CLUSTERed relation. That would even give us some sort of indices with visibility. I think this is a nice idea too :) Well, Postgres doesn't automatically create indices, for a counter example. Yes, and it has no data-use analyser tools that automatically suggest indexes, either. That's the sort of thing people coming from other (err, Other ;-) products complain about, in fact. definitely has more information available, than the computer. A DBA (hopefully) knows future plans and emergency strategies for the storage system, for example. Perhaps my jaundice comes from too much time spent in operational trenches, but while good DBAs have some ideas about that, large numbers of them are harried and overwhelmed just by the piles of work they already have. Nevertheless, while what you say is true, I'm not sure what it has to do with the present case. I don't think the current proposal is to address partitioning across table spaces. It's to do with the way certain segments of a table are interpreted by the system. It's undoubtedly true that this strategy is of questionable utility for many kinds of use of PostgreSQL. But it seems to offer very significant advantages for one use-pattern that is very common. That said, I am not trying to argue it should be adopted without poking at its weaknesses. I just think it unfair to ask the proposal to address problems it's not really aimed at. A ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps
On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: Agreed. Just a minor note: I find marked read-only too strong, as it implies an impossibility to write. I propose speaking about mostly-read segments, or optimized for reading or similar. I do want some segments to be _marked_ read-only: I want attempted writes to them to _fail_. A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Index performance
On Thu, Jan 03, 2008 at 07:11:07AM +0200, Brian Modra wrote: Thanks, I think you have me on the right track. I'm testing a vacuum analyse now to see how long it takes, and then I'll set it up to automatically run every night (so that it has a chance to complete before about 6am.) Note that VACUUM ANALYSE and ANALYSE are not identical: the former also performs vacuum. On a table that is not updating that often but that is expanding rapidly, you may not need that extra I/O. Analyse on its own can perform just the statistical sampling. If you're not creating dead tuples with UPDATE, DELETE, or ROLLBACK, that might be enough most of the time. A ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] SSL over Unix-domain sockets
On Fri, Jan 04, 2008 at 02:37:03PM -0500, Bruce Momjian wrote: The problem with adding SSL to local sockets is this slippery slope where we only do part of the job, but it isn't clear where to draw the line. I don't think part of the job for a patch is a slippery slope. It's what you do with patches for issues discovered too late to make full release: pick the least invasive thing you can do, release that patch, and implement the full feature later. A ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps
On Fri, Jan 04, 2008 at 10:26:54PM +0100, Markus Schiltknecht wrote: I'm still puzzled about how a DBA is expected to figure out which segments to mark. I think that part might be hand-wavy still. But once this facility is there, what's to prevent the current active segment (and the rest) from also getting this mark, which would mean the table is read only? A ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Slow count(*)
On Wed, Jan 02, 2008 at 09:29:24AM -0600, Abraham, Danny wrote: We are looking for a patch that will help us count using the indexes. Is this for SELECT count(*) FROM table; or SELECT count(1) FROM table WHERE. . . The latter _will_ use an index, if the index is correct, the statistics are right, and the index selectivity is worth the cost of reading the index. The former will not use an index at all, because the answer depends on visibility, and you can't know that without reading the table. If you're counting how many rows are in the table (for, for instance, display purposes), you probably need to do something else. Our product is about 20 times slower on Postgres compared to MS SQL Server. Any ideas? Not without the queries, the EXPLAIN ANALYZE plans, and some information about the database. A ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Index performance
On Wed, Jan 02, 2008 at 05:53:35PM +0200, Brian Modra wrote: This table is added to in real time, at least 10 rows per second. [. . .] If I do a select which uses the pkey index, where equal to the ID column, and greater than one of the values, which should return about 1500 rows, it sometimes takes 1/2 minute to return, and other times takes only seconds. Is it the number of rows being added in real time, that is maybe causing the index to be locked? No, it's probably a bad plan. A minimum 10 rows/second is probably just making the statistics for the table look bad. You likely want to SET STATISTICS wider on the 1st (~150 distinct values) column, and then run ANALYSE on the table very frequently. Are you updating or deleting at all? If so, that will also affect things: you need to perform very frequent VACUUM on that table in that case. Aside from that generic advice, it's impossible to say more without EXPLAIN ANALYSE output for the slow and fast examples. A ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Spoofing as the postmaster
On Sat, Dec 29, 2007 at 02:09:23AM +1100, Naz Gassiep wrote: In the web world, it is the client's responsibility to ensure that they check the SSL cert and don't do their banking at www.bankofamerica.hax0r.ru and there is nothing that the real banking site can do to stop them using their malware infested PC to connect to the phishing site. The above security model is exactly how we got into the mess we're in: relying entirely on the good sense of a wide community of users is how compromises happen. Strong authentication authenticates both ways. For instance, the web world you describe is not the only one. Banks who take security seriously have multiple levels of authentication, have trained their users how to do this, and regularly provide scan tools to clients in an attempt (IMO possibly doomed) to reduce the chances of input-device sniffing. A ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spoofing as the postmaster
On Fri, Dec 28, 2007 at 07:48:22AM -0800, Trevor Talbot wrote: I don't follow. What are banks doing on the web now to force clients to authenticate them, and how is it any different from the model of training users to check the SSL certificate? Some banks (mostly Swiss and German, from what I've seen) are requiring two-token authentication, and that second token is really the way that the client authenticates the server: when you install your banking application, you're really installing the keys you need to authenticate the server and for the server to authenticate you. There's a fundamental problem that you can't make someone else do authentication if they don't want to, and that's exactly the situation clients are in. Right, but you can train users to expect authentication of the server. One way to do that is to require them to use an intrusive enough system that they end up learning what to look for in a phish attack. That said, I tend to agree with you: if we had dnssec everywhere today, it's totally unclear to me what client applications would do in the event they got a bogus resolution. A ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Spoofing as the postmaster
On Mon, Dec 24, 2007 at 12:04:16AM +0100, Tomasz Ostrowski wrote: Not at all, as it won't run as root, it'll just start as root and then give up all root privileges. The only thing it would have after being root is just an open socket. If you think that is complete protection against privilege escalation, I encourage you to read some more bugtraq archives. The answer to MITM attacks is not superuser-reserved ports anyway. The privileged port idea was a bad one in retrospect. The answer is strong authentication. A ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Spoofing as the postmaster
On Sun, Dec 23, 2007 at 09:52:14PM +0100, Magnus Hagander wrote: My point is that all these other server products have the exact same issue. And that they deal with it the exact same we do - pretty much leave it up to the guy who configure the server to realize that's just how things work. The problem with that approach is that, in the computer security world, taking that approach is increasingly regarded as negligent. And pointing out that others are similarly negligent is not a response. Note that I am explicitly not subscribing to or disagreeing with that view. A ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Spoofing as the postmaster
On Sun, Dec 23, 2007 at 01:45:14AM -0500, Tom Lane wrote: The primary reason things work like that is that there are boatloads of machines that are marginally misconfigured. For instance, userland thinks there is IPv6 support when the kernel thinks not (or vice versa). Not only marginally misconfigured, but broken as shipped, in the case of some OSes. And in those cases, you can't even fix it. A ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] function body actors (was: [PERFORM] viewing source code)
On Fri, Dec 21, 2007 at 12:09:28AM -0500, Merlin Moncure wrote: Maybe a key management solution isn't required. If, instead of strictly wrapping a language with an encryption layer, we provide hooks (actors) that have the ability to operate on the function body when it arrives and leaves pg_proc, we may sidestep the key problem (leaving it to the user) and open up the doors to new functionality at the same time. I like this idea much better, because the same basic mechanism can be used for more than one thing, and it doesn't build in a system that is fundamentally weak. Of course, you _can_ build a weak system this way, but there's an important difference between building a fundamentally weak system and making weak systems possible. A ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] function body actors (was: [PERFORM] viewing source code)
On Fri, Dec 21, 2007 at 12:40:05AM -0500, Tom Lane wrote: whether there is a useful policy for it to implement. Andrew Sullivan argued upthread that we cannot get anywhere with both keys and encrypted function bodies stored in the same database (I hope that's an adequate summary of his point). It is. I'm not a security expert, but I've been spending some time listening to some of them lately. The fundamental problem with a system that stores the keys online in the same repository is not just its potential for compromise, but its brittle failure mode: once the key is recovered, you're hosed. And there's no outside check of key validity, which means attackers have a nicely-contained target to hit. I'm not convinced that he's right, but that has to be the first issue we think about. The whole thing is a dead end if there's no way to do meaningful encryption --- punting an insoluble problem to the user doesn't make it better. Well, one thing you could do with the proposal is build a PKCS#11 actor, that could talk to an HSM. Not everyone needs HSMs, of course, but they do make online key storage much less risky (because correctly designed ones make key recovery practically impossible). So the mechanism can be made effectively secure even for very strong cryptographic uses. Weaker cases might use a two-level key approach, with a data-signing key online all the time to do the basic encryption and validation, but a key-signing key that is always offline or otherwise unavailable from within the system. The key signing key only authenticates (and doesn't encrypt) the data signing key. You could use a different actor for this, to provide an interface to one-way functions or something. This gives you a way to revoke a data-signing key. You couldn't protect already compromised data this way, but at least you could prevent new disclosures. Yes, I'm being hand-wavy now, but I can at least see how these different approaches are possible under the suggestion, so it seems like a possibly fruitful avenue to explore. The more I think about it, actually, the more I like it. A ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] function body actors (was: [PERFORM] viewing source code)
On Fri, Dec 21, 2007 at 01:57:44PM -0500, Tom Lane wrote: Merlin Moncure [EMAIL PROTECTED] writes: ISTM the main issue is how exactly the authenticated user interacts with the actor to give it the information it needs to get the real key. This is significant because we don't want to be boxed into an actor implementation that doesn't allow that interaction. We don't? What purpose would such a setup serve? I would think that for the applications we have in mind, the *last* thing you want is for the end user to hold the key. The whole point of this is to keep him from seeing the function source code, remember? Hmm; this may be exactly part of the problem, though. It seems there are two possible cases in play: 1. Protect the content in the database (in this case, function bodies) from _all_ users on a given server. This is a case where you want to protect (say) your function body from your users, because you have a closed-source application. 2. Protect the content of a field from _some_ users on a given system, based on the permissions they hold. This is roughly analagous to others not being able to look in the table I created, because I haven't GRANTed them permission. (2) is really a case for column-level access controls, I guess. But if we're trying to solve this problem too, then user passwords or something make sense. A ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] function body actors (was: [PERFORM] viewing source code)
On Fri, Dec 21, 2007 at 04:19:51PM -0500, Tom Lane wrote: 2. Protect the content of a field from _some_ users on a given system, I would argue that (2) is reasonably well served today by setting up separate databases for separate users. I thought actually this was one of the use-cases we were hearing. Different people using the same database (because the same data), with rules about the different staff being able to see this or that function body. I can easily imagine such a case, for instance, in a large organization with different departments and different responsibilities. It seems a shame that the only answer we have there is, Give them different databases. I actually think organizations that think keeping function bodies secret like this to be a good idea are organizations that will eventually make really stupid mistakes. But that doesn't mean they're not under the legal requirement to do this. For instance, my current employer has (externally-mandated) organizational conflict of interest rules that require all disclosure to be done exclusively as need to know. Under the right (!) legal guidance, such a requirement could easily lead to rules about function-body disclosure. From my point of view, such a use case is way more compelling than function-body encryption (although I understand that one too). A ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Testing mail list
On Wed, Dec 19, 2007 at 11:15:37AM -0500, Tom Lane wrote: hoping to draw responses from careless people? I've heard of web comment-spammers who try to get other people to decode captchas for them this way. Yes. This is the latest spammer trick. They get people all over the globe to decode the captchas. It's way easier than programming to decode the captchas (which itself isn't that hard -- there are plenty of toolkits out there that will decode such things for you). A ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Negative LIMIT and OFFSET?
On Fri, Dec 14, 2007 at 06:42:24PM -0500, Tom Lane wrote: How do people feel about applying this to 8.3, rather than holding it? To me, this is a feature change, and therefore should be held. A ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
