[HACKERS] Re: Escaping strings for inclusion into SQL queries
On Wed, Aug 22, 2001 at 05:16:44PM +, Florian Weimer wrote: > We therefore suggest that a string escaping function is included in a > future version of PostgreSQL and libpq. A sample implementation is > provided below, along with documentation. I use Perl, which (through DBD::Pg) has a "quote" function available, but I think this is a very good idea to include in the library. I only have one issue - the SQL standard seems to support the use of '' to escape a single quote, but not \'. Though PostgreSQL has an extended notion of character string literals, I think that the usual policy of using the standard interface when possible should apply. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] refusing connections based on load ...
The whole argument over how to get load averages seems rather silly, and it's moot if the idea of using the load information to alter PG behavior is rejected. I personally have no use for it, but I don't think it's a bad idea in general. Particularly given future redundancy/load sharing features. On the other hand, I think almost all of this stuff can and should be done outside of postmaster. Here is the 0-change version, for rejecting connections, and for operating systems that have built-in firewall capability, such as FreeBSD: a standalone daemon that adds a reject rule for the Postgres port when the load gets too high, and drops that rule when the load goes back down. Now here's the small-change version: add support to Postgres for a SET command or similar way to say "stop accepting connections", or "set accept/transaction delay to X". Write a standalone daemon which monitors the load and issues commands to Postgres as necessary. That daemon may need extra privileges, but it is small, auditable, and doesn't talk to the outside world. It's probably better to include in the Postgres protocol support for accepting (TCP-wise) a connection, then closing it with an error message, because this daemon needs to be able to connect to tell it to let users in again. It's probably as simple as always letting the superuser in. The latter is nicer in a number of ways. Persistent connections were already mentioned - rejecting new connections may not be a good enough solution there. With a fancier approach, you could even hang up on some existing connections with an appropriate message, or just NOTICE them that you're slowing them down or you'd like them to go away voluntarily. >From a web-hosting standpoint, someday it would be nifty to have per-user-per-connection limits, so I could put up a couple of big PG servers and only allow user X one connection, which can't use more than Y amount of RAM, and passes a scheduling hint to the OS so it shares CPU time with other economy-class users, which can be throttled down to 25% of what ultra-mega-hosting users get. Simple load shedding is a baby step in the right direction. If nothing else, it will cast a spotlight on some of the problem areas. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Re: No printable 7.1 docs?
On Tue, Apr 17, 2001 at 12:07:26AM +, Thomas Lockhart wrote: > > They're not ready yet. > > Since they were deemed non-essential for this release, and since the > release schedule is not built around their creation, I no longer feel > obligated to have them finished on the release date. A nice change from > the deadlines I've been working on for the last three years or so :) > > This is the first release with the "no hardcopy" policy, and user > feedback is certainly desirable and appreciated. My feedback at this time is mostly the desire to know a bit better what prevents the hardcopy docs from being built automatically. I am currently having some trouble compiling jadetex, so I can't take a look at the generated PDF yet, but I assume there's something wrong with it. That seems like a big deficiency in the doc tools, which suprises me, given that they're rather large projects that have been used by other large projects for quite a while. My interest is partly to be able to compile the docs on my own, and partly research - I'm involved in the development of an application that has some hefty documentation requirements, and I was hoping that SGML + free software would come to the rescue. If it's just a matter of time and effort, this may be an big enough area of overlap with work that I can spend Official Time and/or Official Money on it. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
[HACKERS] No printable 7.1 docs?
On Sun, Apr 15, 2001 at 01:11:34PM -0400, Mitch Vincent wrote: > The "Current Release Docs" on the PostgreSQL website still look 7.0.Xish.. I can't finh the 7.1 PS docs anywhere. The stuff in the doc directory on the FTP sites is from last year, and the docs in the tar files are all SGML and HTML. I could really use a printable copy, but last time I tried to generate it from the SGML, it was a nightmare (and afterward, I discovered that the release docs are apparently made by hand anyway). Am I missing something? -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Re: Changing the default value of an inherited column
On Fri, Mar 30, 2001 at 12:10:59PM -0500, Tom Lane wrote: > Comments? I'm going to implement and commit this today unless I hear > loud squawks ... I like it in general and I think it opens some interesting possibilities. I don't know much about how the inheritance system is implemented, so I will put out this scenario in case it makes a difference. We recently decided to refactor our schema a bit, using inheritance. All of our tables have a primary key called "seq" along with some other common fields such as entry time, etc. We realized that moving them into a "base" table allowed us to create functions on "base" that would work on every derived table. The main problem was that we needed fields like "seq" to have distinct sequences, which was not possible without the ability to override the default value in each derived table. It seems like this would be easily doable with this change. Another thing that seems kind of interesting would be to have: CREATE TABLE base (table_id CHAR(8) NOT NULL [, etc.]); CREATE TABLE foo (table_id CHAR(8) NOT NULL DEFAULT 'foo'); CREATE TABLE bar (table_id CHAR(8) NOT NULL DEFAULT 'foo'); Then a function on "base" could look at table_id and know which table it's working on. A waste of space, but I can think of uses for it. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] FAQ: Current state of replication ?
On Mon, Mar 19, 2001 at 11:00:20AM -, Peter Galbavy wrote: > 1. One "writer", many "reader" PostgreSQL servers. We will want to write > provisioning / configuration information centrally and can tolerate a > "writer" failuer for a time. > 2. Consitency at the transaction level. All changes to the "writer" server > will be wrapped in transactions, and there will be foreign key consistency > checking in many tables. > 3. Delays from "writer" through to consistent state on "readers" can be > tolerated to within a few minutes or even more. All read-servers must be in > the same state when answering requests. > > Our objective is to acheive performance and some fault tolerance as the data > is going to be used for near-real time configuration of various other > backend systems in an almost traditional 'net environment. Your application sounds like a perfect fit for LDAP. In other words, keep your database in Postgres, but export views of it through for clients to query through LDAP. Rely on LDAP replication, since it has the model you need and works today. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Proposed WAL changes
Please forgive me if I'm misunderstanding something of these rather complex issues. But I think this is an important question from the perspective of a sytem administrator responsible for the safety and uncorruptedness of his users' data. If I understand correctly, it is possible, through power failure, Postgres or OS crash, or disk failure, to end up with a situation where Postgres cannot put the database in a consistent state. Rather than failing to start at all, something will be put in place that allows PG to partially recover and start up, so that you can get to your data. I think that leaves DBAs wondering two things: First, how will I know that my database is corrupted? I may not be present to witness the power failure, unattended reboot, and automatic restart/quasi-recovery. If the DB has become inconsistent, it is critical that users do not continue to use it. I'm all for having Postgres throw up its hands and refuse to start until I put it in disaster-dump mode. Secondly, since disaster-dump seems to be a good term for it, is there some way to know the extent of the damage? I.e. in the typical case of power failure, is the inconsistent part just the "recent" data, and would it be possible to find out which records are part of that damaged set? -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] performance modality in 7.1 for large text attributes?
On Tue, Dec 19, 2000 at 03:03:43PM +, Thomas Lockhart wrote: > o WAL fsync() log commits and cleanup (aggregate throughput is great, > but every once in a while someone waits while the paperwork gets done. > Waiting may be due to processor resource competition) > > o Underlying file system bookkeeping from the kernel. e.g. flushing > buffers to disk etc etc. I was going to suggest the same, but it's interesting that it happens on reads as well. I can't tell for sure from the graph, but it looks like it happens fairly consistently - every Nth time. I'd be curious to see how this changes if you artificially slow down your loop, or adjust your OS's filesystem parameters. It may give some more clues. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/
Re: [HACKERS] Who is a maintainer of GiST code ?
On Sun, Dec 17, 2000 at 11:30:23PM -0500, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > I think > > there are some papers at Berkeley or a web site that goes into it in > > detail. > > I imagine there's some GiST stuff at the Berkeley papers repository > http://s2k-ftp.CS.Berkeley.EDU:8000/postgres/papers/ > but I'd be surprised if it's more than an overview... Well, there's this: http://gist.cs.berkeley.edu/ and this: http://gist.cs.berkeley.edu/pggist/ -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/
