Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT
On Thu, 2004-11-11 at 23:05 -0500, Bruce Momjian wrote: Andrew McMillan wrote: -- Start of PGP signed section. On Wed, 2004-11-10 at 11:45 -0500, Tom Lane wrote: Andrew McMillan [EMAIL PROTECTED] writes: When tracking down gnarly problems in heavily multi-user applications enabling higher log levels at selective points has the potential to help _a lot_ with diagnostic detail, without smothering you in _every_ detail. Sure. As I pointed out in the other thread, if you want to allow an app to do this, you can make available a SECURITY DEFINER function that performs the desired SET on its behalf. By setting execute permissions on the function and/or including restrictions in the function's code, you can make this as tight or as loose a loophole as you like. So it's certainly possible to do what you want in any case. I think the issue at hand is what's appropriate to provide as hard-wired functionality. That sounds excellent - I hadn't realised that this workaround would be possible, and indeed with this in place that will provide even better control over the facility. OK, here is one vote for the ALTER USER/remove USERLIMIT croud, and you were the person who originally mentioned the problem. You don't think the function creation is hard. Perhaps that's the way to go then. Yes, I agree - it seems good. Also, I don't see that this function would need to be written under stress as ISTR you suggested elsewhere - any analysis like this is going to be following on from review of other statistics - I think it would normally be a well-planned process. Cheers, Andrew. - Andrew @ Catalyst .Net .NZ Ltd, PO Box 11-053, Manners St, Wellington WEB: http://catalyst.net.nz/PHYS: Level 2, 150-154 Willis St DDI: +64(4)803-2201 MOB: +64(272)DEBIAN OFFICE: +64(4)499-2267 How many things I can do without! -- Socrates - signature.asc Description: This is a digitally signed message part
Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT
Andrew McMillan wrote: That sounds excellent - I hadn't realised that this workaround would be possible, and indeed with this in place that will provide even better control over the facility. OK, here is one vote for the ALTER USER/remove USERLIMIT croud, and you were the person who originally mentioned the problem. You don't think the function creation is hard. Perhaps that's the way to go then. Yes, I agree - it seems good. Also, I don't see that this function would need to be written under stress as ISTR you suggested elsewhere - any analysis like this is going to be following on from review of other statistics - I think it would normally be a well-planned process. OK, Tom please go ahead with the patch. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] Relation does not exist
Title: Relation does not exist Our customer reported a problem with Postgres v.7.3.2 server on Solaris 8 I asked them to run a quick query and it looks like the database is corrupted: select count (1) from t_revisioncontrol ERROR: Relation 17236 does not exist What can I do next? Thanks, Mike
[HACKERS] src/tools/make_keywords
Does this 'script' belong in the tree anymore? It relies on the existence of src/tools/SQL_keywords -- which no longer exists. gavin ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Romanian translation
On Mon, 08 Nov 2004 11:45:34 +0800, Christopher Kings-Lynne [EMAIL PROTECTED] wrote: I'd like to translate postgres in Romanian, if nobody's doing this already. If you like, we'd love it if you translate phpPgAdmin as well :) Where do I start? :) Alin Vaida ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] CVS should die
On Sat, 06 Nov 2004 11:53:13 +0100, Thomas Hallgren wrote: Andrew McMillan wrote: Switching to Arch is more work, but it also offers a lot more benefits - including the opportunity for individuals to maintain their own trees, and be able to work out which patchsets from someone else's tree have not been applied. If anything is going to become the open-source BitKeeper it will be this, I think. For those interested in SVN versus arch, I found the following from Tom Lord (the guy behind Arch) http://web.mit.edu/ghudson/thoughts/diagnosing and a reply from Greg Hudson (SVN developer) http://web.mit.edu/ghudson/thoughts/undiagnosing. There is a fairly detailed comparison in the GNU Arch wiki as well. URL: http://wiki.gnuarch.org/moin.cgi/SubVersionAndCvsComparison Note: if you're a postgres committer you may have more luck seeking out your nearest SCM advocate -- almost all of them would regard Postgres migrating to their preferred SCM as a 'win' -- let them work for you by walking you through things. Cheers, Anand ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Relation does not exist
Brusser, Michael [EMAIL PROTECTED] writes: Our customer reported a problem with Postgres v.7.3.2 server on Solaris 8 I asked them to run a quick query and it looks like the database is corrupted: select count (1) from t_revisioncontrol ERROR: Relation 17236 does not exist If you are lucky, the problem is just one of corrupted indexes on the system catalogs (most likely pg_class_oid_index) and can be fixed by REINDEXing. What happened to lead up to this? Usually this sort of thing doesn't appear out of the blue. Any system crashes or anything? regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] CVS should die
Tom Lane wrote: ... There aren't any alternatives that are enough better than CVS to be worth the changeover effort. I've done some research over the last couple of days for a fairly big project where we face the challenges of breaking up a monolith into modules and consequently will be forced to move a lot of files. I now second Tom's opinion. Here's why: Subversion doesn't move files. They copy and delete. So if you have parallel work on a file that is moved, you are headed for problems. See threads: Question about rename on [EMAIL PROTECTED] news://news.gmane.org:119/[EMAIL PROTECTED] and Misinforming the user on rename with local changes [EMAIL PROTECTED] news://news.gmane.org:119/[EMAIL PROTECTED] What I find especially intriguing is that although Subversion have version controlled directories, they still identify the content of the files using the location in the repository rather than using a globally unique identifier. Didn't they anticipate files being moved around and perhaps linked? This thread started due to CVS problems with moving files and Subversion will perhaps get there eventually but IMHO they are certainly not there yet. GNU-Arch seems promising in some respects. It really can rename files and track them using an id, but it doesn't run on Windows without Cygwin (and even then not too well it seems). Personally I dislike the fact that the author seems somewhat religious about free software and hostile towards Windows instead of focusing on delivering a portable solution. In my case, the fact that GNU-Arch is not portable is reason enough to discard it as a viable alternative and I think it would be unfortunate if PostgreSQL locked Windows users out from repository access. The other Open Source alternatives are, IMHO not mature enough to be considered for serious projects yet. I wish ClearCase was fast, free, and suitable for distributed development :-) Unfortunately it's slow, expensive, and extremely network intensive. My approach will be to wait and perhaps contribute to Subversion if I get some time left. They really need a great database backend. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS should die
Thomas Hallgren wrote: GNU-Arch seems promising in some respects. It really can rename files and track them using an id, but it doesn't run on Windows without Cygwin (and even then not too well it seems). Personally I dislike the fact that the author seems somewhat religious about free software and hostile towards Windows instead of focusing on delivering a portable solution. In my case, the fact that GNU-Arch is not portable is reason enough to discard it as a viable alternative and I think it would be unfortunate if PostgreSQL locked Windows users out from repository access. s/unfortunate/totally unacceptable/ cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS should die
On Nov 13, 2004, at 6:20 PM, Thomas Hallgren wrote: Question about rename on [EMAIL PROTECTED] news://news.gmane.org:119/[EMAIL PROTECTED] Thomas (Hallgren): Unfortunately, my efforts to get Thunderbird to do something useful with that URL have been unsuccessful and I can't find the thread on the (usable) mailing list archive ( don't use the tigris.org archive; use http://svn.haxx.se/ ). Misinforming the user on rename with local changes [EMAIL PROTECTED] news://news.gmane.org:119/[EMAIL PROTECTED] Might be easier to read with a browser here: http://svn.haxx.se/dev/archive-2004-11/index.shtml Yes, looks like it could be a potential problem/inconvenience if a file is both moved and altered simultaneously. I see the risk of problems as similar to those that if two people edit the same section of the same file at the same time, complicated conflicts could emerge. You could avoid this problem using Lock-Modify-Unlock rather than Copy-Modify-Merge. Some people do. Others (many CVS users) choose the productivity gains by using Copy-Modify-Merge and then deal with the eventual problem when/if it shows up. It is too bad the Subversion design didn't anticipate this such that it's not a problem at all, but in many environments, it may not be much of an issue. This thread started due to CVS problems with moving files and Subversion will perhaps get there eventually but IMHO they are certainly not there yet. It is worth noting that there is already huge improvement in this area. There's a baby in that bathwater. :-) It's made my life much easier, particularly for Java development where refactoring requires renames and moves of files and directories more often than with some other languages like C/C++. I'm not really advocating a switch if you don't think it's worth it. Just hoping to contribute constructively to the discussion where the worth of the costs/benefits are measured by others. Back to playing with the 8 beta for me, :-) --Travis ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] IpcSemaphoreLock/Unlock and proc_exit on 7.2.6
I have an underpowered server running 7.2.6 that backs a website which occasionally gets hit by a bunch of traffic and starts firing off FATAL 1: Sorry, too many clients already messages. This is all as expected, but sometimes it just crashes. I had no clue what was going on until I checked the stderr log (because I had set it up to use syslog). In there I find a whole bunch of these: IpcSemaphoreLock: semop(id=-1) failed: Invalid argument IpcSemaphoreLock: semop(id=-1) failed: Invalid argument IpcSemaphoreLock: semop(id=-1) failed: Invalid argument IpcSemaphoreLock: semop(id=-1) failed: Invalid argument IpcSemaphoreUnlock: semop(id=-1) failed: Invalid argument IpcSemaphoreLock: semop(id=-1) failed: Invalid argument IpcSemaphoreUnlock: semop(id=-1) failed: Invalid argument IpcSemaphoreLock: semop(id=-1) failed: Invalid argument Looking at the source I see proc_exit as the failure path for these two functions (IpcSemaphoreLock, IpcSemaphoreUnlock). I've read the comments around the code, but must admit that I can't really follow what's going on. Could anyone shed some light on what is going on? Certainly the semId of -1 looks a little suspicious. This is on freebsd 4.5 Kris Jurka ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [Pgsphere-dev] GIST index concurrency concern
I expect my site to sustain something around 1000-3000 new user acquisitions per day, all of which will account for an insert into 3 GIST indices. Most people when they talk about a large load on a DBMS system talk about transactins per second. As in 100 per second Even if we only assume 12 hour days, 3000 per day is only one transaction every 14 seconds. That's a triveal rate that could be handled on an older Pentium II PC. Assume the system runs for five years at 3000/day. That's only only about 500,000 rows. In database terms that's not much. Don't worry you have a problem well within the limits of a small PC runnig PostgreSQL. You want to of course place the intire process of adding a new user inside a begin/commit transaction. This will provide the type of queue you want. All of the inserts will get done when the commit happens. Also you will likely want to run the user interface in its own process or thread. Those two things will be all you need as long as your average transaction rate remains so low. If there are ANY locks done in your code you need to remove them and re-think the design. Everyone always thinks they have a large database project. Even a 200,000 row table is small enough that it and its index files can be cached in RAM. Where you might run into the kinds of problems you are thinking about is if you had automated sensor systems (looking either down at the Earth or up at the sky) and software to automatically extract features and catlog those in to a DBMS. Then if you have several of those sensors running you get to the high rates that drive concurrentcy issues. But if you only have four or five users each doing a transaction per second it's not an issue. After you get past the 100 transacton per second rates you are looking at Ocacle on Sun hardware and terrabyte sized disk arrays Like we have down in the lab here. BUt belleive my you need automated data collection systems to gemerate enough data to get you into trouble But I run low-end stuff on my very old 500Mhz PIII = Chris Albertson Home: 310-376-1029 [EMAIL PROTECTED] Cell: 310-990-7550 Office: 310-336-5189 [EMAIL PROTECTED] KG6OMK __ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [Pgsphere-dev] GIST index concurrency concern
Oleg, Daniel and I have both been collaborating on this structure for a while now. We are aware that GiST reads work very fast. But won't they be paralyzed when there are writes? Both of us are working on dating sites, and the main problem that concerns us is a very heavy traffic load. At this point I am planning to queue all changes to a GiST index and commit them every 10-15 minutes. Is that really necessary? It's realistic to assume here that if there is a problem with locking the table for writes, it will be a problem in this situation because this structure is going to be hit VERY hard (and Daniel's situation is on an even larger scale). We hope that we can alleviate that with a transaction queue, but this is not a simple fix. Have you seen any projects that were under a heavy load using a GiST index, and were they able to avoid being paralyzed somehow? Thanks in advance, Patrick On Tuesday 09 November 2004 22:08, Oleg Bartunov wrote: Oleg Bartunov [EMAIL PROTECTED] Daniel, concurrency is a big issue of current implementation of GiST. But it should don't bite you for READ ops ! -hackers mailing list is a very relevant mailing list for GiST discussions. It's pity we several times claimed to work on GiST concurrency and recovery, but never got a chance :) I see Neil become interested in GiST concurrency, though. Oleg On Tue, 9 Nov 2004, Daniel Ceregatti wrote: Hi, It's recently come to my attention that GIST indices suffer from concurrency issues. I have already developed a dating sites using GIST for use with attributes using the intarray contrib, and for Earth distance/radius calculations using pg_sphere. I'm wondering if I haven't shot myself in the foot here. So far, I understand that a GIST index will be locked by a backend for any DML. Basically I'm concerned that my database will not scale in the manner that I was hoping, because the sites that access the database are to be used by many multiple concurrent users, doing some DML. I expect my site to sustain something around 1000-3000 new user acquisitions per day, all of which will account for an insert into 3 GIST indices. Additionally there will be people that will be updating their attributes and locations as well, but this will probably only account for a small fraction of the DML. We don't allow people to delete stuff. My concern now is this concurrency issue. My question is: Is there anyone out there using a GIST index on a database where there's a lot of DML? Should I be concerned with this issue at all? If so, what can be done to minimize the impact of heavy DML on a GIST index? I've pondered rolling all DML into queues via triggers and then de-queuing them in one transaction every so often, like 15 minutes, via cron. Any other suggestions? I'm posting to this list because I understand that both Oleg and Teodor read it, and I found no other relevant list. If I've misposted, please accept my apology and please direct me to the appropriate list. Thanks, Daniel ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] pgxs regression
I'm not sure exactly what has changed, nor at the moment how to fix it, but I'm finding that pgxs no longer works for PL/R or dblink. Error as follows: make: *** No rule to make target `/usr/local/pgsql-dev/lib/pgxs/src/makefiles/../../src/port/pg_config_paths.h', needed by `all-static-lib'. Stop. The problem is related specifically to Makefiles using MODULE_big. I tested a few contribs that use MODULES and they seem to work fine under pgxs. Joe ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
