Re: [HACKERS] Roadmap for a Win32 port
Hi Bruce, You obviosuly missed my recent posting advertising the homepage of Konstantin Knizhnik? Make sure to have a look: http://www.garret.ru/~knizhnik/ You find there -everything- concerning multiplatform IPC, threading and even some extraordinary, complete database backends that are superior to the database backends previously available as open source (including PostgreSQL, I'm afraid...). The licensing of all of this stuff is -public domain-. I think this should really be worth a look/discussion/mentioning. Here an excerpt of my last email, describing the furios list of features abailable in GOODS: Some core features of the GOODS backend (as they come to my mind): -> full ACID transaction support, incl. distributed transactions -> Multiple stoarge servers distributed over a TCP/ID network -> multible reader/single writer (MVCC) -> dual client side object cache -> online backup (snapshot backup AND permanent backup) -> nested transactions on object level -> transaction isolation levels on object level -> object level shared and exclusive locks -> excellent C++ programming interface -> WAL -> garbage collection for no longer reference database objects (online VACUUM) -> fully thread safe client interface -> JAVA client API -> very high performance as a result of a lot of fine tuning (better perfomance than berkeley db in my benchmarks!!!) -> asyncrous event notification on object instance modification -> extremly high code quality -> a one person effort, hence a very clean design -> the most relevant platforms are supported out of the box -> complete build is done in less than a minute on my machine -> it's documented -> it's tested and found to be working for a while now ... kind regards, Robert Schrem On Thursday 06 June 2002 04:57, you wrote: > Here is a summary of the responses to my Win32 roadmap. I hope this > will allow further discussion. > > --- > > INSTALLER > - > Cygwin Setup.exehttp://cygwin.com > Nullsofthttp://www.nullsoft.com/free/nsis/ > > GUI > --- > pgAdmin2 > http://pgadmin.postgresql.org/pgadmin2.php?ContentID=1 pgaccess >http://pgaccess.org/ > Java admin (to be written) > Dev-C++ admin (to be written) > http://sourceforge.net/projects/dev-cpp/ > > BINARY > -- > > > FORK() > > cygwin fork() http://cygwin.com > CreateProcess() and copy global area > > THREADING > > Posix threads > Gnu pth http://www.gnu.org/software/pth/ > ST > http://state-threads.sourceforge.net/docs/st.html (single-session > multi-threading possible) > (Posix AIO is possible) > > IPC > > Cygwin http://cygwin.com > MinGW http://www.mingw.org/ > ACE > http://www.cs.wustl.edu/~schmidt/ACE.html APR > http://apr.apache.org/ > Our own > > ENVIRONMENT > > Cygwin http://cygwin.com > UnxUtilshttp://unxutils.sourceforge.net/ > Write own initdb > > > IMPLEMENTATIONS > --- > PostgreSQLe > http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html Dbexperts > http://www.dbexperts.net/postgresql Connx > http://www.connx.com/ > gborg > http://gborg.postgresql.org/project/winpackage/projdisplay.php Interbase > > http://community.borland.com/article/0,1410,23217,00.html ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Straight-from-the-horses-mouth dept
On Thu, 2002-06-06 at 07:18, Tom Lane wrote: > I've been having a lot of fun here at the SIGMOD annual conference, > attaching faces to names like Stonebraker, Hellerstein, Aoki, > Seltzer (if these do not ring a bell, you ain't read enough Postgres > source code lately). I felt I had to pass along this gem from Joe > Hellerstein, right after he observed that he knew the PG sources > quite well, and he'd noticed MySQL was a lot smaller: > > "Postgres is bloatware by design: it was built to house PhD theses." > > immediately followed by > > "The current maintainers [he's looking right at me while he says this] > have done a great job of trimming the fat. I know that *my* thesis > is gone entirely." I hope it is removed in such a clean way that it can be put back _as an installable module_ if needed. Bloatware or not, one of the main advantages of PG is that it is designed to be extensible. One thing I think we have stripped too much is time travel. I hope that there will be possibility to put back hooks for the following: 1) logging dead tuples as they are removed,either to text file or archive table/database depending on installed logging function) 2) have VACUUM delete only tuples dead before transaction N (I'd prefer some timestamp, but that would need logging transaction times or moving transaction iods to 64bits so that they can embed time) 3) some way to tell postgres to get data as it was at transaction N - it would be a great way for recovering accidentally deleted data (I send out my quite crappy python code for retrieving dead tuples from data files 1-2 times a month :) SELECT * FROM MYTABLE AS OF TRANSACTION(123456) would be great. 4) some sparse logging of transaction times, say log current trx nr every 5 minutes, making 3) more usable. --- Hannu, anxiousy wating for more stuff the alleged horses have to say ;) ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [SQL] PostgreSQL on AIX
On June 5, 2002 12:33 pm, Bruce Momjian wrote: > D'Arcy J.M. Cain wrote: > > On May 13, 2002 12:50 am, Rajesh Kumar Mallah. wrote: Catching up on an old mailbox, Bruce? :-) > > Now if only I could get IBM to understand that. They still claim that my > > problem is that PostgreSQL (an "unsupported" application) is doing > > something to catch SIGKILL. > > First, an application can't catch SIGKILL. It never arrives to > applications. It is supposed to pull the process with no warning. > > However, there are things processes can do to wedge themselves in a > system call so they don't see the SIGKILL. Of course, as soon as they > return from the system call, they die. Exactly. What IBM was saying was was that we were "catching" SIGKILL and I could not convince the (supposedly technical) IBMers that they were talking out their ass. Anyway, I am pretty sure that PostgreSQL is not the culprit here. As it happens this project is back on the table for me so it is interesting that your email popped up now. I just compiled the latest version of PostgreSQL on my AIX system and it generated lots of errors and then completed and installed fine. Makes me sort of nervous. We'll see how it goes. Anyone have any horror/success stories about PostgreSQL on AIX for me? Changed subject and mailing list. -- D'Arcy J.M. Cain| Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [SQL] PostgreSQL on AIX
I've been using PosgreSQL 7.2 on AIX 4.3.3 with no probelms at all. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of D'Arcy J.M. Cain Sent: Thursday, June 06, 2002 6:35 AM To: Bruce Momjian; [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [SQL] PostgreSQL on AIX On June 5, 2002 12:33 pm, Bruce Momjian wrote: > D'Arcy J.M. Cain wrote: > > On May 13, 2002 12:50 am, Rajesh Kumar Mallah. wrote: Catching up on an old mailbox, Bruce? :-) > > Now if only I could get IBM to understand that. They still claim that my > > problem is that PostgreSQL (an "unsupported" application) is doing > > something to catch SIGKILL. > > First, an application can't catch SIGKILL. It never arrives to > applications. It is supposed to pull the process with no warning. > > However, there are things processes can do to wedge themselves in a > system call so they don't see the SIGKILL. Of course, as soon as they > return from the system call, they die. Exactly. What IBM was saying was was that we were "catching" SIGKILL and I could not convince the (supposedly technical) IBMers that they were talking out their ass. Anyway, I am pretty sure that PostgreSQL is not the culprit here. As it happens this project is back on the table for me so it is interesting that your email popped up now. I just compiled the latest version of PostgreSQL on my AIX system and it generated lots of errors and then completed and installed fine. Makes me sort of nervous. We'll see how it goes. Anyone have any horror/success stories about PostgreSQL on AIX for me? Changed subject and mailing list. -- D'Arcy J.M. Cain| Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] smime.p7s Description: application/pkcs7-signature
Re: [HACKERS] Roadmap for a Win32 port
Bruce Momjian wrote: > > Let me throw out an idea. I have been mentioning full fork, light > fork(copy globals only), and threading as possible solutions. > > Another idea uses neither threading nor copying. It is the old system > we used before I removed exec() from our code. We used to pass the > database name as an argument to an exec'ed postgres binary that > continued with the database connection. > > We removed the exec, then started moving what we could into the > postmaster so each backend didn't need to do the initialization. > > One solution is to return to that for Win32 only, so instead of doing: > >initialization() >want for connection() >fork backend() > > we do for Win32: > >want for connection() >exec backend() >initialization() Summarizes pretty much what we discussed Monday on the phone. Except that the postmaster still has to initialize the shared memory and other stuff. It's just that the backends and helper processes need to reinitialize themself (attach). > It wouldn't be hard to do. We would still do CreateProcess rather than > CreateThread, but it eliminates the fork/threading issues. We don't > know the database before the connection arrives, so we don't do a whole > lot of initialization. All I see so far is the reading of the postgresql.conf, the pg_hba.conf and the password files. Nothing fancy and the postmaster could easily write out a binary content only file that the backends then read, eliminating the parsing overhead. The bad news is that Tom is right. We did a terrible job in using the new side effect, that the shared memory segment is at the same address in all forked processes, after removing the need to reattach. In detail the XLog code, the FreeSpaceMap code and the "shared memory" hashtable code now use pointers, located in shared memory. For the XLog and FreeSpace code this is understandable, because they where developed under the fork() only model. But the dynahash code used offsets only until v7.1! All three (no claim that that's all) make it impossible to ever have someone attaching to the shared memory from the outside. So with these moves we made the shared memory a "Postmaster and children" only thing. Raises the question, why we need an IPC key at all any more. Anyhow, looks as if I can get that fork() vs. fork()+exec() feature done pretty soon. It'll be controlled by another Postmaster commandline switch. After cleaning up the mess I did to get it working quick, I'll provide a patch for discussion. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Roadmap for a Win32 port
Added to the list. Thanks. --- Robert Schrem wrote: > Hi Bruce, > > You obviosuly missed my recent posting advertising the homepage > of Konstantin Knizhnik? > > Make sure to have a look: http://www.garret.ru/~knizhnik/ > > You find there -everything- concerning multiplatform IPC, > threading and even some extraordinary, complete database > backends that are superior to the database backends > previously available as open source (including PostgreSQL, > I'm afraid...). The licensing of all of this stuff is -public domain-. > I think this should really be worth a look/discussion/mentioning. > > Here an excerpt of my last email, describing the furios list > of features abailable in GOODS: > > Some core features of the GOODS backend (as they come to my mind): > -> full ACID transaction support, incl. distributed transactions > -> Multiple stoarge servers distributed over a TCP/ID network > -> multible reader/single writer (MVCC) > -> dual client side object cache > -> online backup (snapshot backup AND permanent backup) > -> nested transactions on object level > -> transaction isolation levels on object level > -> object level shared and exclusive locks > -> excellent C++ programming interface > -> WAL > -> garbage collection for no longer reference database objects (online VACUUM) > -> fully thread safe client interface > -> JAVA client API > -> very high performance as a result of a lot of fine tuning (better >perfomance than berkeley db in my benchmarks!!!) > -> asyncrous event notification on object instance modification > -> extremly high code quality > -> a one person effort, hence a very clean design > -> the most relevant platforms are supported out of the box > -> complete build is done in less than a minute on my machine > -> it's documented > -> it's tested and found to be working for a while now > ... > > kind regards, > Robert Schrem > > > On Thursday 06 June 2002 04:57, you wrote: > > Here is a summary of the responses to my Win32 roadmap. I hope this > > will allow further discussion. > > > > --- > > > > INSTALLER > > - > > Cygwin Setup.exehttp://cygwin.com > > Nullsofthttp://www.nullsoft.com/free/nsis/ > > > > GUI > > --- > > pgAdmin2 > > http://pgadmin.postgresql.org/pgadmin2.php?ContentID=1 pgaccess > >http://pgaccess.org/ > > Java admin (to be written) > > Dev-C++ admin (to be written) > > http://sourceforge.net/projects/dev-cpp/ > > > > BINARY > > -- > > > > > > FORK() > > > > cygwin fork() http://cygwin.com > > CreateProcess() and copy global area > > > > THREADING > > > > Posix threads > > Gnu pth http://www.gnu.org/software/pth/ > > ST > > http://state-threads.sourceforge.net/docs/st.html (single-session > > multi-threading possible) > > (Posix AIO is possible) > > > > IPC > > > > Cygwin http://cygwin.com > > MinGW http://www.mingw.org/ > > ACE > > http://www.cs.wustl.edu/~schmidt/ACE.html APR > > http://apr.apache.org/ > > Our own > > > > ENVIRONMENT > > > > Cygwin http://cygwin.com > > UnxUtilshttp://unxutils.sourceforge.net/ > > Write own initdb > > > > > > IMPLEMENTATIONS > > --- > > PostgreSQLe > > http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html Dbexperts > > http://www.dbexperts.net/postgresql Connx > > http://www.connx.com/ > > gborg > > http://gborg.postgresql.org/project/winpackage/projdisplay.php Interbase > > > > http://community.borland.com/article/0,1410,23217,00.html > > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PostgreSQL and Windows2000 and defunct processes
Hi. On Thu, 06 Jun 2002 16:23:29 +0930 Steven Vajdic <[EMAIL PROTECTED]> wrote: > How to get read of those "defunct" processes? What version of cygipc and cygwin do you use? --- Yutaka tanida<[EMAIL PROTECTED]> $BFf$N(BWebsite http://www.hi-net.zaq.ne.jp/yutaka/ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Roadmap for a Win32 port
Jan Wieck wrote: > > One solution is to return to that for Win32 only, so instead of doing: > > > >initialization() > >want for connection() > >fork backend() > > > > we do for Win32: > > > >want for connection() > >exec backend() > >initialization() > > Summarizes pretty much what we discussed Monday on the phone. > Except that the postmaster still has to initialize the shared > memory and other stuff. It's just that the backends and > helper processes need to reinitialize themself (attach). Yes, obviously I simplified, and I do believe our optimizations are helping on Unix. It is just that I think for Win32 the fork is more harmful than removing those optimizations. One thing that may not have been clear is that we don't need to play with globals at all. We just pass whatever info we want to the child via command-line arguments, rather than shared memory. > > It wouldn't be hard to do. We would still do CreateProcess rather than > > CreateThread, but it eliminates the fork/threading issues. We don't > > know the database before the connection arrives, so we don't do a whole > > lot of initialization. > > All I see so far is the reading of the postgresql.conf, the > pg_hba.conf and the password files. Nothing fancy and the > postmaster could easily write out a binary content only file > that the backends then read, eliminating the parsing > overhead. Yes, that is clearly possible. Another option is to just write out a no-comments, no-whitespace version of each file and just have the backends read those. The advantage is that we can use the same code to read them, and I don't think it would be any slower than a binary file. > The bad news is that Tom is right. We did a terrible job in > using the new side effect, that the shared memory segment is > at the same address in all forked processes, after removing > the need to reattach. > > In detail the XLog code, the FreeSpaceMap code and the > "shared memory" hashtable code now use pointers, located in > shared memory. For the XLog and FreeSpace code this is > understandable, because they where developed under the fork() > only model. But the dynahash code used offsets only until > v7.1! > > All three (no claim that that's all) make it impossible to > ever have someone attaching to the shared memory from the > outside. So with these moves we made the shared memory a > "Postmaster and children" only thing. Raises the question, > why we need an IPC key at all any more. Well, we could force shmat() to bind to the same address, but I suspect that might fail in some cases. > Anyhow, looks as if I can get that fork() vs. fork()+exec() > feature done pretty soon. It'll be controlled by another > Postmaster commandline switch. After cleaning up the mess I > did to get it working quick, I'll provide a patch for > discussion. Yes, very little impact. We then need someone to do some Win32 timings to see if things have improved. As Tom mentioned, we need some hard numbers for these things. In fact, I would like a Win32 test that takes our code and compares fork(), then exit(), with CreateProcess(), exit(). It doesn't have create a db session, but I would like to see some timings to know what we are gaining. Heck, time CreateThread too and let's see what that shows. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PostgreSQL and Windows2000 and defunct processes
Steven, I found the following snippet in MSDN under CreateProcess: "The created process remains in the system until all threads within the process have terminated and all handles to the process and any of its threads have been closed through calls to CloseHandle. The handles for both the process and the main thread must be closed through calls to CloseHandle. If these handles are not needed, it is best to close them immediately after the process is created." If this is your case, what is unknown is why it is "not" happening in win98?? Can you run a test and close the PROCESS_INFORMATION HANDLEs for the main thread and the process itself and see if that makes a difference (and any other HANDLEs that you might have gotten through OpenProcess() calls...)? Mike Shelton -Original Message- From: Steven Vajdic [mailto:[EMAIL PROTECTED]] Sent: Thursday, June 06, 2002 12:53 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: [HACKERS] PostgreSQL and Windows2000 and defunct processes Problem: Win2000 and Cygwin/postgresql-7.1 produce "defunct" processes after each run which can ONLY be killed by Win Task Manager. If you allow too many "defunct" processes your database requests slow down and your number of connections increases, i.e. if you have 32 connections specified in your postgresql.conf file and and 32 "defunct" processes and try to run again the connection will be refused. When you are killing "defunct" processes and reach the one which was first formed the postmaster restarts postgresql. How to get read of those "defunct" processes? Much obliged. Steven. P.S. You do not have these "defunct" processes under Win98!!!??? ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Straight-from-the-horses-mouth dept
Hannu Krosing <[EMAIL PROTECTED]> writes: > > One thing I think we have stripped too much is time travel. Actually, I was just discussing that at last night's dinner with someone whose name I forget at the moment (I have his card, but not on me). He claimed to know how to support time travel as an optional feature --- ie, you don't pay for it if you don't need it. I'm hoping to hear more about this after the conference is over... regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] patch for contrib/intarray (7.2 and 7.3) (fwd)
Patch to contrib/intarray is attached to this message. Please apply it to 7.2 and CVS Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 -- Forwarded message -- Date: Mon, 3 Jun 2002 19:45:03 +0300 (GMT) From: Oleg Bartunov <[EMAIL PROTECTED]> To: Pgsql Hackers <[EMAIL PROTECTED]> Subject: [HACKERS] patch for contrib/intarray (7.2 and 7.3) Please apply attached patch to contrib/intarray (7.2, 7.3). Fixed bug with '=' operator for gist__int_ops and define '=' operator for gist__intbig_ops opclass. Now '=' operator is consistent with standard 'array' type. Tnanks Achilleus Mantzios for bug report and suggestion. Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 patch_intarray.gz Description: Binary data ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] non-standard escapes in string literals
Lincoln Yeoh writes: > However raw control characters can still cause problems in the various > stages from the source to the DB. I still don't see why. You are merely speculating about implementation fallacies that aren't there. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Roadmap for a Win32 port
Bruce Momjian writes: > Lots of our code requires a unix shell and utilities. Will we continue > using cygwin for this? We should probably get rid of using shell scripts for application programs altogether, for a number of reasons besides this one, such as the inability to properly handle input values with spaces, commas, etc. (we probably don't handle very long values either on some platforms), the inability to maintain open database connections so that createlang needs to prompt for the same password thrice, general portable scripting headaches, and the lack of internationalization facilities. I'd even volunteer to do this. Comments? -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] pgaccess.org - invitation for a working meeting
Folks, If I might just say ... I think it's phenominal that somebody has started working on the PGAccess code again and I really look forward to testing (and documenting!) your work. If you don't read the Novice list, new Postgres users have been dying for a PGAccess update for the last year. Your contribution will be widely appreciated by Postgres users everywhere. -Josh Berkus PostgreSQL/Techdocs Writer ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pgaccess.org - invitation for a working meeting
Thanks Josh, Good, encouraging words. There are two mailing lists - please feel free to make them popular - [EMAIL PROTECTED] [EMAIL PROTECTED] Both are run on qmail/ezmlm, for help send a blank e-mail to - [EMAIL PROTECTED] [EMAIL PROTECTED] The lists are not moderated. Please, ONLY people who develop or would like to develop subscribe to 'developers'. On 'users' we'll be happy to read what people expect form pgaccess. There will be bugzilla soon on bugzilla.pgaccess.org The current stage is - we are about the merge the three major (known) patches to pgaccess done by Bartus, Chris and Boyan with the latest known version of pgaccess. Tacho ([EMAIL PROTECTED]) will be the release engineer in the beginning. This should happen this week or so (Chris is about to submit his work). To repeat again - we all use, like and admire PostgreSQL. And pgaccess. The only reason to set up a separate community for it is because we believe that it needs a bit more air. pgaccess has no meaning outside PostgreSQL. We do not try to steal it or take it over,... One day I should write a page how the whole things started, so that I do not have to explain it every time. All best to everyone, Iavor PS Josh, testing and documenting are two good activities. What we urgently need is a dedicated release engineer - Tacho will do this in the beginning but he can not promise long term commitment. And Teo is busy right now. > -Original Message- > From: Josh Berkus [mailto:[EMAIL PROTECTED]] > Sent: Mittwoch, 05. Juni 2002 17:35 > To: Bruce Momjian; Iavor Raytchev > Cc: Tom Lane; Thomas Lockhart; Stanislav Grozev; Ross J. Reedstrom; > Nigel J. Andrews; Marc G. Fournier; Constantin Teodorescu; Cmaj; Boyan > Filipov; Boyan Dzambazov; Bartus. L; Brett Schwarz; pgsql-hackers > Subject: Re: [HACKERS] pgaccess.org - invitation for a working meeting > > > Folks, > > If I might just say ... I think it's phenominal that somebody has > started working on the PGAccess code again and I really look forward to > testing (and documenting!) your work. > > If you don't read the Novice list, new Postgres users have been dying > for a PGAccess update for the last year. Your contribution will be > widely appreciated by Postgres users everywhere. > > -Josh Berkus > PostgreSQL/Techdocs Writer ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pgaccess.org - invitation for a working meeting
Iavor Raytchev wrote: > pgaccess has no meaning outside PostgreSQL. We do not try to steal it or > take it over,... One day I should write a page how the whole things started, > so that I do not have to explain it every time. I have not heard --- where are you going to keep the master CVS? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Cooperation
Dann Corbit wrote: > I apologize for my English language message. I am unable to speak > Japanese. We do have a native Japanese speaker here, who could be > called upon if necessary. There is no need to aplogize writing an e-mail in English. It's global standards, but some portion is a bit difficult to understand. Anyhow, we must firstly express our thanks for your interest in our project, though we are facing also hard obstacles as listed on the Web site. > The PostgreSQL team is planning to do a native Win32 port. Perhaps you > would like to help with the effort. In that way, your changes will get > propagated back up the source code tree and you can gain the benefits > from future development efforts without performing any work. It is nice to hear that the PostgreSQL development team has also working on this subject. Will you please illustrate the procedure more clearly how to we contribute our effort to your project. The last four words in the above clause mean that once we supply you with the changed source, then everything afterwords could be handled by the team? How the copy right will be dealt with? The development has been continued by the volunteer developers here, however, we have to admit that businesses (companies) are also involved to support those people providing time to work on the development, not to commercialization purpose but expecting some return, e.g. earning company's prestige. So, we have to regulate those backgrouds first based upon your proposal. We are positive to help you with our effort anyway, if things goes well. > We did a port to Win32 also, but your approach seems much better. We > have very fat executables and you have a marvelous DLL approach. > Probably, the way that you perform the operations is much better. Thanks. Toshi ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [GENERAL] Re : Solaris Performance - 64 bit puzzle
Dann Corbit wrote: > > TODO reverted to be: > > > > Add BSD-licensed qsort() for Solaris > > > > My guess is that your test case didn't tickle the bug. > > I am the author of several special sort functions [I wrote the sorting > chapter in this book: http://users.powernet.co.uk/eton/unleashed/]. > > I would be happy to contribute sort routines to the project under the > Berkeley style license. > > Likely (if C++ is allowed) a large efficiency gain can be had through > the use of templates. Thanks. I think we will go with Free/NetBSD code because they are already tested. Thanks. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Roadmap for a Win32 port
Peter Eisentraut wrote: > Bruce Momjian writes: > > > Lots of our code requires a unix shell and utilities. Will we continue > > using cygwin for this? > > We should probably get rid of using shell scripts for application programs > altogether, for a number of reasons besides this one, such as the > inability to properly handle input values with spaces, commas, etc. (we > probably don't handle very long values either on some platforms), the > inability to maintain open database connections so that createlang needs > to prompt for the same password thrice, general portable scripting > headaches, and the lack of internationalization facilities. > > I'd even volunteer to do this. Comments? I know I have discouraged it because I think shell script language has a good toolset for those applications. I have fixed all the spacing issues. What language where you thinking of using? C? Also, it seems Win32 doesn't need these scripts, except initdb. PostgreSQLe didn't use the, it just did initdb, and the rest were done using a GUI. However, initdb would remain a problem. PostgreSQLe wrote its own. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] non-standard escapes in string literals
Yes it's speculation. The implementation at the DB isn't there, neither are the associated DBD/JDBC/ODBC drivers for it. Basically if the fallacies aren't in postgresql _if_ the decision is to implement it, I'd be happy. I was just noting (perhaps superfluously) that backspaces and friends (nulls) have been useful for exploiting databases (and other programs). Recently at least one multibyte character (0x81a2) allowed potential security problems with certain configurations/installations of Postgresql. Would switching to the standard cause such problems to be less or more likely? Would making it an option make such problems more likely? Cheerio, Link. p.s. Even +++AT[H](remove square brackets and = carriage return) as data can cause problems sometimes - esp with crappy modems. Once there was a site whose EDI metadata had lots of +++ and they were experiencing "bad connections" ... At 07:10 PM 6/6/02 +0200, Peter Eisentraut wrote: >Lincoln Yeoh writes: > > > However raw control characters can still cause problems in the various > > stages from the source to the DB. > >I still don't see why. You are merely speculating about implementation >fallacies that aren't there. > >-- >Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Straight-from-the-horses-mouth dept
On Thu, 2002-06-06 at 21:13, Tom Lane wrote: > Hannu Krosing <[EMAIL PROTECTED]> writes: > > > > One thing I think we have stripped too much is time travel. > > Actually, I was just discussing that at last night's dinner with someone > whose name I forget at the moment (I have his card, but not on me). > He claimed to know how to support time travel as an optional feature > --- ie, you don't pay for it if you don't need it. I'm hoping to hear > more about this after the conference is over... I guess that we could do something similar to oracle (yes, they have some limited time travel starting from ver 9i ;-p ) 1. They log transaction times at some rather coarse interval - this is the cheap part if done relatively seldom. 2. then they have gone through much of trouble to get historic data from the logs. The part that could make it cheap for us is that we don't need to go to logs, just having an option to tell the executor to assume it is in some other transaction in some part of the tree would be enough (and to ignore the new dead-tuple bit in indexes now that it is there) - this should be possible at very low extra cost. so we could resurrect old tuples by selecting them into new table: CREATE TABLE salvage_mytable AS SELECT oid as oldoid,tmin as oldtmin, ..., * FROM mytable AS OF YESTERDAY; - Hannu ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PostgreSQL and Windows2000 and defunct processes
Hi Michael/Yutaka, Thanks for your reply. I discovered the cause of the problem (I think) - I just do not know why it happens. PROBLEM (again): DEFUNCT processes under Win2K not seen under Win98. I am running cygwin-2.125.2.10 (postgresql 7.1) and cigipc-1.11-1 installed in Jan 2002. Prior to that I struggled with prior cygwin version - no proper postgresql run at all. In April/May I tried then new version of cygwin (postgresql 7.2) and cigipc BUT cygwin environment was very poor and without some essential unix commands and I gave up. I can try again with now latest versions if that will help (!!!???). The problem is in closing pgSQL database. Somehow the process does not finish. Under Win98 cygwin does not show (ps -ef) defunct processes BUT if you run a program which opens/closes pgSQL 2 times in succession the second time the connection will be refused - like it was not properly closed the first time!!!??? Under Win2K (same version og cygwin/cigipc) this does not happen BUT a defunct process remains and accumulates/multiplies after each run I'll try with Michael's suggestion RE: threads and process closure but I do not think I'll discover anything new - the "postgresql" process does not finish for some reason - the question is why: bad cygwin/cigipc, or bad postgresql.conf parameters or bad postmaster/postgresql run or .. . Please, do not ask why I run under Win2K/Win98 - I normally run my HTML/PHP/pgSQL application under linux (redhat, mandrake, SuSe) and it works fine BUT Win is a much more used OS and one has to take that into account. More advice/help would be appreciated. Cheers, Steven. -- *** Steven Vajdic (BSc/Hon, MSc) Senior Software Engineer Motorola Australia Software Centre (MASC) 2 Second Avenue, Technology Park Adelaide, South Australia 5095 email: [EMAIL PROTECTED] email: [EMAIL PROTECTED] Ph.: +61-8-8168-3543 Fax:+61-8-8168-3501 Front Office (Ph): +61-8-8168-3500 mobile: +61 (0)419 860 903 AFTER WORK email: [EMAIL PROTECTED] Home address: 6 Allawah Av., Glen Osmond SA 5064, Australia *** ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Redhat 7.3 time manipulation bug
Tom Lane wrote: > Thomas Lockhart <[EMAIL PROTECTED]> writes: > >> Why should we rely on broken glibc and the standard? Why don't we make > >> our own mktime() and use it on all platforms. > > > The downside to doing that is that we then take over maintenance of the > > code and, more importantly, the timezone database. > > > But it might be the best thing to do. > > I've been sorta thinking the same thing. We could get out from under > the Y2038 issue, and also eliminate a whole lot of platform > dependencies. Not to mention sillinesses like being unable to recognize > a bad timezone name when it's fed to us. > > Exactly how much work (and code bulk) would we be taking on? I've > never looked at how big the timezone databases are... I am not really excited about distributing a timezone database as part of PostgreSQL, and it wouldn't match the OS's timezone. (We do need a 64-time time_t, but I think we can wait to get closer to 2038.) Can we detect if glibc is being used for the compile (easy), and substitute a non-broken mktime in the link path ahead of glibc's mktime? Seems that would be the easiest solution. Of course, pre-1970 dates then wouldn't match the OS on glibc systems, but that seems like a win. :-) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Redhat 7.3 time manipulation bug
Trond Eivind Glomsrød wrote: > On Tue, 21 May 2002, Lamar Owen wrote: > > > On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote: > > > I see. This behavior is consistent with the fact that mktime is > > > supposed to return -1 on error, but then is broken in every other Unix > > > implementation that I know. > > > > > Any other workaround than downgrade or install FreeBSD? > > > > Complain to Red Hat. Loudly. However, as this is a glibc change, other > > distributors are very likely to fold in this change sooner rather than > > later. > > Relying on nonstandardized/nondocumented behaviour is a program bug, not a > glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at > it, but glibc is not the component with a problem. No one has really answered the question --- if the way PostgreSQL is using mktime() for pre-1970 dates is wrong, why do timezone databases have pre-1970 timezone information? I assume Linux does or the old mktime() wouldn't have worked for pre-1970 dates. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] revised sample SRF C function; proposed SRF API
Tom Lane wrote: > Definitely better. I'd suggest also thinking about whether the > same/similar macros can support functions that return a set of a > scalar (non-tuple) datatype. In my mind, the cleanest design would > be some base macros that support functions-returning-set (of anything), > and if you want to return a set of scalar then you just use these > directly (handing a Datum to FUNC_RETURN_NEXT). If you want to return > a set of tuples then there are a couple extra steps that you need to > do to build a tupdesc, build a tuple, and convert the tuple to Datum > (which at the moment you do by putting it into a slot, but I think we > ought to change that soon). If it were really clean then the macros > supporting these extra steps would also work without the SRF macros, > so that you could use 'em in a function returning a single tuple. > Sorry for the long delay. I just got back to this today, and I've run into an interesting question. I have a proposal and patch almost ready which I think pretty much meets the above design requirements. I also wanted to incorporate a built-in function for returning guc variables (varname text, varval text), consistent with previous posts. This is both useful and a good test of the Composite & SRF function API (the API includes functions/macros to facilitate returning composite types, and an independent set of functions/macros for returning sets, whether composite or scalar). The question is how to best bootstrap this new function. In order to create the pg_proc entry I need the return type oid. If I understand correctly, in order to get a composite return type, with a known oid, I would need to create a bootstrapped relation and the corresponding bootstrapped pg_type entry. Is there any alternative? It seems ugly to bootstrap so many objects for every (future) builtin function which returns a composite type. Thanks, Joe ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] SASL, compression?
Tom Lane wrote: > Bear Giles <[EMAIL PROTECTED]> writes: > > 1) add SASL. This is a new standards-track protocol that is often > >described as "PAM" for network authentication. PostgreSQL could > >remove *all* protocol-specific authentication code and use > >standard plug-in libraries instead. > > To me, "new standards-track protocol" translates as "pie in the sky". > When will there be tested, portable, BSD-license libraries that we > could *actually* use? I'm afraid this really would end up meaning > writing and/or supporting our own SASL code ... and I think there > are more important things for the project to be doing. > > IMHO we've got more than enough poorly-supported authentication options > already. Unless you can make a credible case that using SASL would > allow us to rip out PAM, Kerberos, MD5, etc *now* (not "in a few releases > when everyone's switched to SASL"), I think this will end up just being > another one :-(. > > (It doesn't help any that PAM support was sold to us just one release > cycle back on the same grounds that it'd be the last authentication > method we'd need to add. I'm more than a tad wary now...) I agree with Tom on this one. "Plugin" sounds so slick, but it really translates to "abstraction", and as if our authentication stuff isn't already confusing enough for users to configure, we add another level of abstraction into the mix, and things become even more confusing. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Use of /etc/services?
Peter Eisentraut wrote: > Since we now have an official entry in /etc/services, shouldn't we be able > to make use of it, by using getservbyname() if a nonnumeric port number is > specified? Is any OS actually shipping us in /etc/services? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Per tuple overhead, cmin, cmax, OID
Tom Lane wrote: > Manfred Koizar <[EMAIL PROTECTED]> writes: > > No comment on a planned 7.3 timeframe? :-( > > I think we are planning to go beta in late summer (end of August, say). > Probably in July we'll start pressing people to finish up any major > development items, or admit that they won't happen for 7.3. So we've > still got a couple months of full-tilt development mode before we > start to worry about tightening up for release. I am concerned about slowing down too early. We did that in previous releases and didn't get the beta focus we needed, and it was too paralyzing on people to know what is to be slowed and what to keep going. I think a slowdown two weeks before beta would be fine. Basically, I think the slowdown lengthened the time we were not doing something productive. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Use of /etc/services?
On Fri, 7 Jun 2002, Bruce Momjian wrote: > Peter Eisentraut wrote: > > Since we now have an official entry in /etc/services, shouldn't we be able > > to make use of it, by using getservbyname() if a nonnumeric port number is > > specified? > > Is any OS actually shipping us in /etc/services? NetBSD will be, as of 1.7, though the 1.7 release is a while away yet. (Sorry, I didn't find out about this in time to get it into for 1.6, which is just about to be released.) cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Per tuple overhead, cmin, cmax, OID
Tom Lane wrote: > Manfred Koizar <[EMAIL PROTECTED]> writes: > > what about WITHOUT OIDS? I know dropping the OID from some tables and > > keeping it for others is not trivial, because t_oid is the _first_ > > field of HeapTupleHeaderData. I'm vaguely considering a few possible > > implementations and will invest more work in a detailed proposal, if > > it's wanted. > > Yeah, I had been toying with the notion of treating OID like a user > field --- ie, it'd be present in the variable-length part of the record > if at all. It'd be a bit tricky to find all the places that would need > to change, but I think there are not all that many. > > As usual, the major objection to any such change is losing the chance > of doing pg_upgrade. But we didn't have pg_upgrade during the 7.2 > cycle either. If we put together several such changes and did them > all at once, the benefit might be enough to overcome that complaint. I think it is inevitable that there be enough binary file changes the pg_upgrade will not work for 7.3 --- it just seems it is only a matter of time. One idea is to allow alternate page layouts using the heap page version number, but that will be difficult/confusing in the code. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Use of /etc/services?
Curt Sampson wrote: > On Fri, 7 Jun 2002, Bruce Momjian wrote: > > > Peter Eisentraut wrote: > > > Since we now have an official entry in /etc/services, shouldn't we be able > > > to make use of it, by using getservbyname() if a nonnumeric port number is > > > specified? > > > > Is any OS actually shipping us in /etc/services? > > NetBSD will be, as of 1.7, though the 1.7 release is a while away yet. > (Sorry, I didn't find out about this in time to get it into for 1.6, > which is just about to be released.) Sure, then let's start using getservbyname(), if it works. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
