Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
Redmond crowd should be able to figure out that recycling process IDs instantly would be a stupid idea...) Can you explain more of this? IMHO, if we rely on feature like this, the difference is unstable-every-day vs. unstable-every-year. The mere existence of the kill()

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Tue, Apr 04, 2006 at 11:02:11PM -0400, Tom Lane wrote: Peter Brant [EMAIL PROTECTED] writes: I think I've found the cause (or one of the causes) why stats collection is unreliable on Windows and I'm wondering about the best way to go about fixing it. The problem is that process IDs on

Re: [HACKERS] I have changed employers

2006-04-05 Thread Devrim GUNDUZ
Hi, On Tue, 2006-04-04 at 20:06 -0400, Bruce Momjian wrote: FYI, I have left SRA and am now working for EnterpriseDB: Congratulations Bruce! It is good to see many community members working in PostgreSQL-related companies. Regards, -- The PostgreSQL Company - Command Prompt, Inc.

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Tue, Apr 04, 2006 at 11:17:49PM -0400, Tom Lane wrote: Qingqing Zhou [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] wrote Redmond crowd should be able to figure out that recycling process IDs instantly would be a stupid idea...) Can you explain more of this? IMHO, if we rely on

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Martijn van Oosterhout
On Wed, Apr 05, 2006 at 03:20:47AM -0400, [EMAIL PROTECTED] wrote: TerminateProcess takes a HANDLE, not a process identifier. Yes, they provide the kill primitive, but only as a compatibility measure. A good Windows process, should maintain a HANDLE to the process, and kill the process using

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 09:30:06AM +0200, Martijn van Oosterhout wrote: On Wed, Apr 05, 2006 at 03:20:47AM -0400, [EMAIL PROTECTED] wrote: TerminateProcess takes a HANDLE, not a process identifier. Yes, they provide the kill primitive, but only as a compatibility measure. A good Windows

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Martijn van Oosterhout
On Wed, Apr 05, 2006 at 03:38:28AM -0400, [EMAIL PROTECTED] wrote: Once upon a time, when I played with this stuff (I mostly use UNIX, not Windows), I concluded to myself that HANDLE was process-local, and that it was allocated. Meaning - it won't be re-used until you CloseHandle(). It's best

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
Once upon a time, when I played with this stuff (I mostly use UNIX, not Windows), I concluded to myself that HANDLE was process-local, and that it was allocated. Meaning - it won't be re-used until you CloseHandle(). It's best then, to think of HANDLE as a opaque object.

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
Redmond crowd should be able to figure out that recycling process IDs instantly would be a stupid idea...) Can you explain more of this? IMHO, if we rely on feature like this, the difference is unstable-every-day vs. unstable-every-year. The mere existence of the kill()

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 09:58:54AM +0200, Martijn van Oosterhout wrote: On Wed, Apr 05, 2006 at 03:38:28AM -0400, [EMAIL PROTECTED] wrote: Once upon a time, when I played with this stuff (I mostly use UNIX, not Windows), I concluded to myself that HANDLE was process-local, and that it was

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 10:31:37AM +0200, Magnus Hagander wrote: TerminateProcess takes a HANDLE, not a process identifier. Yes. You get the handle by doing OpenProcess() with PROCESS_TERMINATE access. The functions used to enumerate processes all return the process id, not a HANDLE.(See

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Martijn van Oosterhout
On Wed, Apr 05, 2006 at 06:03:31AM -0400, [EMAIL PROTECTED] wrote: It's no different from a file descriptor on UNIX. Neither UNIX nor Windows promise that a process identifier is valid beyond the life of the process. UNIX avoids it from happening, as it is necessary to avoid races with

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 12:20:49PM +0200, Martijn van Oosterhout wrote: On Wed, Apr 05, 2006 at 06:03:31AM -0400, [EMAIL PROTECTED] wrote: It's no different from a file descriptor on UNIX. Neither UNIX nor Windows promise that a process identifier is valid beyond the life of the process.

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
It's no different from a file descriptor on UNIX. Neither UNIX nor Windows promise that a process identifier is valid beyond the life of the process. UNIX avoids it from happening, as it is necessary to avoid races with system calls such as kill(). Windows does not have this

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
Yes, they provide the kill primitive, but only as a compatibility measure. They do? Where is it? Certainly not in the documented SDK that I can see. I believe it is called KillProcess(). No such function documented on my MSDN, other than one in SQL Server DMO. Are you referring

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
This is a little bit of a distraction though, as any system that requires the user to be able to kill broken backends, is only indicative of a broken backend. We're talking about how to deal with a broken process, after the process owner (PostgreSQL) has forgotten about it. Don't confuse

[HACKERS] commit callback, request

2006-04-05 Thread Pavel Stehule
Hello Is possible make transaction commit trigger without patching code now? I finding way , but all usable interfaces are static. I remember on diskussion about it and about changes in LISTEN/NOTIFY implementation. Is there any progress? I need it for simulation of Oracle dbms_alert.signal

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: HANDLE is process local? That is worse then, because then there's no guarentee that each process will see a different identifier. HANDLE is process local. What you need to do is run DuplicateHandle() on it specifying it should also be valid for

Re: [HACKERS] Tru64/Alpha problems

2006-04-05 Thread Andrew Dunstan
I wrote: I have done some more digging on this. The buildfarm member had a couple of configuration issues which I have remedied, and which almost certainly account for the float test errors we saw. However, we still get an error when we try to start the installed s/w with the default

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
HANDLE is process local? That is worse then, because then there's no guarentee that each process will see a different identifier. HANDLE is process local. What you need to do is run DuplicateHandle() on it specifying it should also be valid for process Y (for which you need a

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: What happens if process Y goes away between the time you obtain a handle for it and the time you try to run this DuplicateHandle call? I can put together some quick test-code for this if you need me to? Nah, it was just a rhetorical question meant

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread stephen joseph butler
2006/4/5, Tom Lane [EMAIL PROTECTED]: AFAICS, don't-reuse-PIDs-too-quick has exact analogs that Windows hasto solve by ensuring it doesn't reuse HANDLEs too quick.There's a disconnect here. handles aren't process identifiers: they're reference counted pointers to the kernel structures for the

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
What happens if process Y goes away between the time you obtain a handle for it and the time you try to run this DuplicateHandle call? I can put together some quick test-code for this if you need me to? Nah, it was just a rhetorical question meant to poke a hole in the claim that

Re: [HACKERS] commit callback, request

2006-04-05 Thread Pavel Stehule
Pavel Stehule [EMAIL PROTECTED] writes: Is possible make transaction commit trigger without patching code now? You can get pretty close with a deferred trigger. I don't think there's any way to have a guaranteed at commit trigger --- as soon as (1) there are two of them and (2) one can get an

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
There's a disconnect here. handles aren't process identifiers: they're reference counted pointers to the kernel structures for the process. If you are holding a handle (ie: from CreateProcess or OpenProcess) that handle cannot and will not be reclaimed until you call CloseHandle (or

[HACKERS] recover without $PGDATA/global

2006-04-05 Thread patrick
here's a resume of what i am doing. note that i don't have the global direcotry of $PGDATA. first of all reinstall a new hard-drive and installed the same version of postgresql (8.0.4) with the same configuration. now doing a initdb somewhere on the new hd. then createdb

Re: [HACKERS] float8 regression test failure in head

2006-04-05 Thread Patrick Welche
On Mon, Mar 22, 2004 at 11:03:38PM -0500, Neil Conway wrote: Christopher Kings-Lynne wrote: This has not yet been fixed... Attached is a patch for this issue. I've checked with Chris, and this patch allows the regression tests to pass on his machine. I also updated float8-exp-three-digits

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
Peter Brant [EMAIL PROTECTED] writes: I added some strategic printfs to pgstat.c. Attached is the output when a little program is run which, in a loop, makes 10 connections, sleeps 3 seconds, closes them, sleeps another 3 seconds. My workstation (Windows XP) was otherwise idle. Search for

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
Search for is known to be dead, ignoring to find the re-used process IDs. Things start out clean, but after a few cycles anywhere between 1 and 5 backends are being missed. Looking at the pgstats code, I notice that once it makes an entry in the dead-backends hashtable, it keeps

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: After reviewing my own patch I just sent in (stats write delay), I realised I had just upped that from 10 seconds to 5 minutes. Which is then even longer than forever :) Which prompted me to consider suggesting just this - do we really need the

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Jim Nasby
If nothing else, any of the 'beginner todo' items are likely candidates, though I suspect none of them individually are enough work for an entire summer. If no one beats me to it, I'll try and compile a list of likely TODOs for this. On Apr 5, 2006, at 12:16 AM, Josh Berkus wrote:

Re: [HACKERS] WAL Bypass for indexes

2006-04-05 Thread Martin Scholes
Title: Converted from Rich Text I wrote: I will run multiple tests and post the actual numbers. I did run more extensive tests and did not bother writing down the numbers, and here's why: the unmodified Pg ran pgbench with a whopping average of 6.3% time in IO wait. If I was able

Re: [HACKERS] WAL Bypass for indexes

2006-04-05 Thread Tom Lane
Martin Scholes [EMAIL PROTECTED] writes: On a related note, we currently have some indexes that are unsafe during recovery (GIST and Hash come to mind). In the spirit of making Pg safe at any speed, would it make sense to put some code in the recovery section that rebuilds all indexes whose

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 10:05:56AM -0400, Tom Lane wrote: What happens if process Y goes away between the time you obtain a handle for it and the time you try to run this DuplicateHandle call? Think of Windows HANDLE like UNIX fd, but Windows HANDLE works for everything - not just sockets,

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 10:30:36AM -0400, Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: What happens if process Y goes away between the time you obtain a handle for it and the time you try to run this DuplicateHandle call? I can put together some quick test-code for this if

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Peter Brant
I'm going to rip out the destroy code and see how it goes. Patch will be forthcoming if things turn out well. Pete Tom Lane [EMAIL PROTECTED] 04/05/06 4:49 pm So I'm sort of thinking that the destroy delay has outlived its usefulness. regards, tom lane

[HACKERS] Control File

2006-04-05 Thread Bruno Almeida do Lago
Quick (real) story to illustrate situation: Some time ago we used to have a server with 2 disk arrays attached to it. They were /mnt/array1 and /mnt/array2. PostgreSQL (8.0) had tablespaces on both. In one cold dark night, one SCSI controller from array2 stopped, and manufacturer was called to

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Nathan Buchanan
A list of simpler TODOs would be great. I might be interested in doing something (probably w/o the summer of code because I have a summer job). We'll see after exams finish. Please post something about where we can find this TODO list when it is available. Thanks, NathanOn 4/5/06, Jim Nasby

Re: [HACKERS] WAL Bypass for indexes

2006-04-05 Thread Simon Riggs
On Wed, 2006-04-05 at 09:40 -0700, Martin Scholes wrote: I will run multiple tests and post the actual numbers. I did run more extensive tests and did not bother writing down the numbers, and here's why: the unmodified Pg ran pgbench with a whopping average of 6.3% time in IO wait. If

[HACKERS] request: muting notice CREATE TABLE will create implicit sequence

2006-04-05 Thread Pavel Stehule
Hello I am working on general functions accessable from console too. I create tempory tables from functions. Is necessary print notice about using serial type? I think actually we don't need print it, becouse DROP TABLE use dependencies. Regards Pavel Stehule

Re: [HACKERS] commit callback, request, SOLVED

2006-04-05 Thread Pavel Stehule
Refered triggers works well, better than I expected. It's not equal NOTIFY, but it works. Thank You Pavel Stehule CREATE OR REPLACE FUNCTION dbms_alert._defered_signal() RETURNS trigger AS $$ BEGIN PERFORM dbms_alert._signal(NEW.event, NEW.message); DELETE FROM ora_alerts WHERE id=NEW.id;

Re: [HACKERS] First Aggregate Funtion?

2006-04-05 Thread Bruno Wolff III
On Fri, Mar 31, 2006 at 15:02:47 -0600, Tony Caduto [EMAIL PROTECTED] wrote: Has there ever been any talk of adding a first aggregate function? It would make porting from Oracle and Access much easier. Note, that without special support those functions aren't going to run very fast. So you

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Robert Treat
Summer of Code projects dont have to revolve around the core project... for example drupal got like 11 projects last year and bricolage got a few too; I got a small list of items that could be looked at that are sort of 3rd party projects, should we attempt to collaborate on putting up a list

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Josh Berkus
Robert, Summer of Code projects dont have to revolve around the core project... for example drupal got like 11 projects last year and bricolage got a few too; I got a small list of items that could be looked at that are sort of 3rd party projects, should we attempt to collaborate on putting

[HACKERS] Rigth toolset to compile under mingw?

2006-04-05 Thread Juan Manuel Diaz Lara
I have tried to compile 8.1.3 with mingw but I am receiving come errors, I do not know with versions are the correct one.Could some body tell me the right versions ? and url's to download from ? please. I searched the mainling lists but it is not clear.Thanks.Atte.Juan Manuel Díaz Lara

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Marc G. Fournier
On Wed, 5 Apr 2006, Josh Berkus wrote: Robert, Summer of Code projects dont have to revolve around the core project... for example drupal got like 11 projects last year and bricolage got a few too; I got a small list of items that could be looked at that are sort of 3rd party projects, should

Re: [HACKERS] Speaking of pgstats

2006-04-05 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: While we're talking about pgstats... There was some talk a while back about the whole bufferer/collector combination perhaps being unnecessary as well, and that it might be a good idea to simplify it down to just a collector. I'm not 100% sure what

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread John DeSoi
On Apr 5, 2006, at 5:04 PM, Marc G. Fournier wrote: Sure, although the important part is to find students. I'm not sure how we do that. Do we have any professors online? I'm not one, but I know some. If there is a link with details and perhaps a list of possible projects, I'll be

Re: [HACKERS] Speaking of pgstats

2006-04-05 Thread Agent M
The general idea would be to still use UDP backend-stats but get rid of the pipe part (emulated by standard tcp sockets on win32), so we'd still have the lose packets instead of blocking when falling behind. Right. Please correct me if I am wrong, but using UDP logging on the same computer

[HACKERS] Anyone want to finish BEFORE COMMIT triggers?

2006-04-05 Thread Josh Berkus
Folks, Anyone want to finish this work? http://gorda.di.uminho.pt/community/pgsqlhooks/ ... specifically the commit triggers? I may have funding for creating BEFORE COMMIT triggers. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of

Re: [HACKERS] WAL Bypass for indexes

2006-04-05 Thread Martin Scholes
Title: Converted from Rich Text Simon, The WAL becomes more of a hotspot as you scale up numbers of CPUs. I tend to agree and the original idea came when I was working on a Sun quad-CPU system for a highly parallelized web application. Each page was broken into several dynamic

Re: [HACKERS] psql \c error

2006-04-05 Thread Philip Yarra
On Mon, 3 Apr 2006 06:13 am, Neil Conway wrote: I've committed a patch to HEAD that should improve this behavior. Let me know if the current behavior is still unsatisfactory. Yes, thanks, it fixes the stuff that bugged me: [EMAIL PROTECTED] pgsql]$ psql -p5434 -dpyarra [snip opening car chase]

Re: [HACKERS] Speaking of pgstats

2006-04-05 Thread Tom Lane
Agent M [EMAIL PROTECTED] writes: Please correct me if I am wrong, but using UDP logging on the same computer is a red herring. Any non-blocking I/O would do, no? If the buffer is full, then the non-blocking I/O send function will fail and the message is skipped. Uh, not entirely. We'd

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Jim Nasby
One idea that comes to mind is to come up with a list of popular OSS projects that we'd like to see add PostgreSQL support and have students work on those... As for finding students, I believe a call on -general and -announce would probably produce results. I know there's some professors

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Jim Nasby
From the main website, hit developers, roadmap and then the TODO link on that page. On Apr 5, 2006, at 1:44 PM, Nathan Buchanan wrote: A list of simpler TODOs would be great. I might be interested in doing something (probably w/o the summer of code because I have a summer job). We'll see

Re: [HACKERS] commit callback, request

2006-04-05 Thread Christopher Kings-Lynne
The only solution I know if is this patch: http://gorda.di.uminho.pt/community/pgsqlhooks/ Chris Pavel Stehule wrote: Hello Is possible make transaction commit trigger without patching code now? I finding way , but all usable interfaces are static. I remember on diskussion about it and

Re: [HACKERS] Speaking of pgstats

2006-04-05 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: While we're talking about pgstats... There was some talk a while back about the whole bufferer/collector combination perhaps being unnecessary as well, and that it might be a good idea to simplify it down to just a collector. I'm not 100% sure what

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Marc G. Fournier
On Wed, 5 Apr 2006, Jim Nasby wrote: One idea that comes to mind is to come up with a list of popular OSS projects that we'd like to see add PostgreSQL support and have students work on those... As nice an idea as this is, we'd also need to quickly co-ordinate with those projects to make

Re: [HACKERS] Summer of Code Preparation

2006-04-05 Thread Qingqing Zhou
Josh Berkus josh@agliodbs.com wrote Sure, although the important part is to find students. I'm not sure how we do that. I noticed two email domains are @mit.edu and @cs.toronto.edu but I am afraid both of them are not students any more :-) Regards, Qingqing

[HACKERS] domain_in performance considerations

2006-04-05 Thread Tom Lane
Neil Conway wrote some pretty nice things here: http://www.advogato.org/person/nconway/diary.html?start=26 but commented It would be worthwhile to investigate whether this results in a performance regression, though: there's no easy way to cache the executor machinery needed to evaluate a CHECK