Re: [HACKERS] What is wrong with hashed index usage?
On the other hand, I like hints on how to do things better ;) David Thomas Lockhart wrote: >>So, is you vote for or against the elog(NOTICE)? >> >> > >OK, if we are still voting, then I'll mention that I generally dislike >the idea of notices of this kind. And would not like this notice in >particular. So would vote no with both hands ;) > > ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] cvs read lock
anoncvs, the lock cleared in the last hour. It didn't actually do any waiting, it aborted almost immediately. David Bruce Momjian wrote: >David Ford wrote: > > >>$ cvs -z3 up -d -P >>cvs [server aborted]: read lock failed - giving up >> >>Hmm, been this way for a while now, something need budging? :) >> >> > >I don't see the problem here. Is that CVS or anonCVS? Can you show us >the 'waiting' lines above this that show the lock location? > > > ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] cvs read lock
$ cvs -z3 up -d -P cvs [server aborted]: read lock failed - giving up Hmm, been this way for a while now, something need budging? :) -d ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PostGres Doubt
Gotcha. 'twas the first time I encountered it, I wasn't expecting it.
Thank you for the clarification. I hadn't paid attention to that
paragraph when I read over it.
David
Tom Lane wrote:
>David Ford <[EMAIL PROTECTED]> writes:
>
>
>>heakin=> \z
>>Access privileges for database "heakin"
>> Table | Access privileges
>>---+---
>> interviewers |
>>
>>
>
>
>
>>heakin=> grant select,insert,update on interviewers to heakin;
>>GRANT
>>heakin=> \z
>>Access privileges for database "heakin"
>> Table | Access privileges
>>---+
>> interviewers | {=,heakin=arwdRxt}
>>
>>
>
>I take it heakin is the owner of the table in question. As such,
>he implicitly has all privileges --- the initial null privilege list
>is a shorthand for what you see explicitly in the second case.
>
>The GRANT man page in current development sources has an example about
>this; see the Notes section of
>http://developer.postgresql.org/docs/postgres/sql-grant.html
>
> regards, tom lane
>
>
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PostGres Doubt
pg_auth=# select version(); version PostgreSQL 7.2 on i686-pc-linux-gnu, compiled by GCC 3.0.2 Which btw has a curious grant/revoke bug. create foo; grant select on foo to bar; results in all rights being granted. You must revoke and grant again in order to get the correct rights set. If this rights bug has been fixed, I'll upgrade, but I don't consider it a big problem since I am well aware of the bug. David Bruce Momjian wrote: >David Ford wrote: > > >>I'm using md5 in pg_hba.conf. That is the method, no? >> >>I'm writing a milter application which instantiates a private resource >>for each thread upon thread startup. I have priv->conn which I >>establish as priv->conn=PQconnectdb(connstr), connstr is const char >>*connstr="host=10.0.0.5 dbname=bmilter user=username password=password"; >> >>It segfaults depending on it's mood but it tends to happen about 50-70% >>of the time. I switched to PQsetdbLogin() which has worked perfectly. >> I don't really want to use that however, I would much prefer using my >>connstr. >> >> > >Wow, I am confused. md5 should be fine. Certainly sounds like there is >a thread problem with PQconnectdb(). Are you using 7.2.X? > > > ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] ATTN: Tom Lane
... while talking to sss.pgh.pa.us.: >> MAIL From:<[EMAIL PROTECTED]> >>> >>> <<< 550 5.7.1 Probable spam from 68.9.71.221 refused - see http://www.five-ten-sg.com/blackhole.php?68.9.71.221 554 5.0.0 Service unavailable Tom, if you block everyone on cable, dialup, dsl, and adsl, then you're probably blocking a lot of legitimate mail. I don't feel like paying some Big Company just so I can relay mail through them when I can do my company's own mail on my own networks. Big Company will get blacklisted soon enough for [inadvertently] allowing a spammer to send mail through them. Please don't punish the victim until they're proven guilty. David p.s. There isn't any contact address on the above URL for the requested updates. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PostGres Doubt
I'm using md5 in pg_hba.conf. That is the method, no? I'm writing a milter application which instantiates a private resource for each thread upon thread startup. I have priv->conn which I establish as priv->conn=PQconnectdb(connstr), connstr is const char *connstr="host=10.0.0.5 dbname=bmilter user=username password=password"; It segfaults depending on it's mood but it tends to happen about 50-70% of the time. I switched to PQsetdbLogin() which has worked perfectly. I don't really want to use that however, I would much prefer using my connstr. Am I missing something? Thanks, David Tom Lane wrote: >David Ford <[EMAIL PROTECTED]> writes: > > >>So reentrancy in libpq basically is put on hold until 7.3. >> >> > >Only if you insist on using "crypt", which is deprecated anyway. >md5 is the preferred encryption method. > >My feeling about the proposed patch was that crypt is now a legacy auth >method, and it's not clear that we should create platform/library >dependencies just to support making multiple connections simultaneously >under crypt auth. (Note that *using* connections concurrently is not >at issue, only whether you can execute the authentication phase of >startup concurrently.) > > regards, tom lane > > ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PostGres Doubt
I.e. "md5" in pg_hba.conf? This is rather disappointing. Here are a few references: http://lists.initd.org/pipermail/psycopg/2002-January/000673.html http://lists.initd.org/pipermail/psycopg/2002-January/000674.html http://archives.postgresql.org/pgsql-bugs/2002-03/msg00048.php And finally.. http://archives.postgresql.org/pgsql-bugs/2002-01/msg00153.php So reentrancy in libpq basically is put on hold until 7.3. David Dann Corbit wrote: >Are you using crypt on the connection? > >Unfortunately, crypt is not reentrant. > > > >>-Original Message- >>From: David Ford [mailto:[EMAIL PROTECTED]] >>Sent: Monday, June 10, 2002 6:16 PM >>To: Dann Corbit >>Cc: vikas p verma; [EMAIL PROTECTED] >>Subject: Re: [HACKERS] PostGres Doubt >> >> >>Is libpq/PQconnectdb() reentrant? I've tried repeatedly over >>time and >>it seems to incur segfaults every single time. >> >>-d >> >>Dann Corbit wrote: >> >> >> >>>The libpq functions are reentrant. These will be useful for >>> >>> >>just about >> >> >>>any project. >>> >>> >>> >>> >> >> ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PostGres Doubt
Is libpq/PQconnectdb() reentrant? I've tried repeatedly over time and it seems to incur segfaults every single time. -d Dann Corbit wrote: >The libpq functions are reentrant. These will be useful for just about >any project. > > ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] [patch] fe-connect.c doesn't handle EINTR correctly
Last year we had a drawn out discussion about this and I created a patch
for it. I never noticed that the patch didn't go in until I installed
7.2 the other day and realised that fe-connect.c never was fixed.
Here is the patch again. It is against CVS 3/16/2002. This time I only
rewrote the connect procedure at line 912, I leave it up to the regular
hackers to copy it's functionality to the SSL procedure just below it.
In summary, if a software writer implements timer events or other events
which generate a signal with a timing fast enough to occur while libpq
is inside connect(), then connect returns -EINTR. The code following
the connect call does not handle this and generates an error message.
The sum result is that the pg_connect() fails. If the timer or other
event is right on the window of the connect() completion time, the
pg_connect() may appear to work sporadically. If the event is too slow,
pg_connect() will appear to always work and if the event is too fast,
pg_connect() will always fail.
David
Index: src/interfaces/libpq/fe-connect.c
===
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.181
diff -u -r1.181 fe-connect.c
--- src/interfaces/libpq/fe-connect.c 2001/11/11 02:09:05 1.181
+++ src/interfaces/libpq/fe-connect.c 2002/03/16 05:17:47
@@ -909,29 +909,48 @@
* Thus, we have to make arrangements for all eventualities.
* --
*/
- if (connect(conn->sock, &conn->raddr.sa, conn->raddr_len) < 0)
- {
- if (SOCK_ERRNO == EINPROGRESS || SOCK_ERRNO == EWOULDBLOCK ||
SOCK_ERRNO == 0)
- {
- /*
-* This is fine - we're in non-blocking mode, and the
-* connection is in progress.
-*/
- conn->status = CONNECTION_STARTED;
- }
- else
- {
- /* Something's gone wrong */
- connectFailureMessage(conn, SOCK_ERRNO);
- goto connect_errReturn;
+ do {
+ int e;
+ e=connect(conn->sock, &conn->raddr.sa, conn->raddr_len)
+
+ if(e < 0) {
+ switch (e) {
+ case EINTR:
+ /*
+* Interrupted by a signal, keep trying. This
+handling is
+* required because the user may have turned
+on signals in
+* his program. Previously, libpq would
+erronously fail to
+* connect if the user's timer event fired and
+interrupted
+* this syscall. It is important that we
+don't try to sleep
+* here because this may cause havoc with the
+user program.
+*/
+ continue;
+ break;
+ case 0:
+ case EINPROGRESS:
+ case EWOULDBLOCK:
+ /*
+* This is fine - we're in non-blocking mode,
+and the
+* connection is in progress.
+*/
+ conn->status = CONNECTION_STARTED;
+ break;
+ default:
+ /* Something's gone wrong */
+ connectFailureMessage(conn, SOCK_ERRNO);
+ goto connect_errReturn;
+ break;
+ }
+ } else {
+ /* We're connected now */
+ conn->status = CONNECTION_MADE;
}
- }
- else
- {
- /* We're connected already */
- conn->status = CONNECTION_MADE;
- }
+
+ if(conn->status == CONNECTION_STARTED || conn->status ==
+CONNECTION_MADE)
+ break;
+ } while(1);
+
#ifdef USE_SSL
/* Attempt to negotiate SSL usage */
if (conn->allow_ssl_try)
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
>
>
>
>I think you are missing the point. I am not saying that we shouldn't
>deal with EINTR; rather I am raising what I think is a legitimate
>question: *what* is the most appropriate response? My reading of
>HP's gloss suggests that we could treat EINTR the same as EINPROGRESS,
>ie, consider the connect() to have succeeded and move on to the
>wait-for-connection-complete-or-failure-using-select() phase.
>If that works I would prefer it to repeating the connect() call,
>primarily because it avoids any possibility of introducing an
>infinite loop.
>
You wouldn't get an infinite loop, you'd get Exx indicating the
operation was in progress. Yes, you could spin on a select() waiting
for the end result. What I normally do is this:
connect()
while(select()) {
switch () {
case EINTR:
break;
case EINPROGRESS:
nanosleep();
break;
case ETIMEDOUT:
default:
/* handle timeout and other error conditions nicely for the
user */
break;
}
}
With EINTR, it's fine to immediately start working again because your
code was interrupted from outside this scope. We don't know where in
connect() we were interrupted, blocking or non-blocking. With
EINPROGRESS I sleep for a while to be nice to the system. Here we know
that things are moving along like they should be and we are in a proper
sleepable period.
That isn't to imply that things will break if we sleep from EINTR. Only
that connect() exited due to an interruption, not due to planning.
>
>No, we don't. If you think that, then you haven't studied the code
>sufficiently to be submitting patches for it. What we actually do
>is exactly what is recommended by the manpage you're quoting at me.
>It's just split across multiple routines.
>
I traced several calls and they run through a few functions which end
up in pqFlush. These code paths haven't checked the socket to see if it
is ready for RW operation yet. pqFlush calls send() [ignoring SSL].
Only after a lot of code has been traversed is pqWait run in which the
socket is checked for RW and EINTR. My point that I was bringing up
with Peter was that it's much more nice on the system to wait for the
socket to become usable before going through all that code. In the
previous email I suggested that with a sufficiently fast timer event,
you'd never get back through the PQconnect* before being interrupted
again and that's why I advocate putting the EINTR as close to the
connect() as possible. Tying this together is why it is possible to
fail, a good amount of code is traversed before you get back to dealing
with the socket. Anywhere inbetween this signal events can happen again.
That's what provoked this original patch. Unless I shut off my timer or
changed my timer to happen in the long distant future, I would never
have a successful connection established.
David
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
> > > >No, it should *not* look like that. The fe-connect.c code is designed >to move on as soon as it's convinced that the kernel has accepted the >connection request. We use a non-blocking connect() call and later >wait for connection complete by probing the select() status. Looping >on the connect() itself would be a busy-wait, which would be antisocial. > The fe-connect.c code moves on regardless of the completion of the connect() if it has been interrupted. To simplify, in a program without SIGALRM events, PQconnect* won't be interrupted. The connect() call will complete properly. In a program with SIGALRM events, the call is interrupted inside connect(). If SA_RESTART was disabled for connect() in POSIX semantics, the program would automatically jump right back into the connect() call. However by default POSIX code enables SA_RESTART which for SIGALRM means -don't- automatically restart the system call. This means the programmer needs to check for -1/errno=EINTR and jump back into connect() himself. There isn't a concern for busy wait/anti social code behavior, your program was in the middle of connect() when it was interrupted, you're simply jumping back to where you left off. It doesn't matter if it is a blocking connect or non-blocking connect, handling EINTR must be done if SIGALRM events are employed. A fast enough event timer with a non-blocking connect will also be susceptible to EINTR. EINTR is distinctly different from EINPROGRESS. If they were the same then there would be a problem. EINTR should be handled by jumping back into the connect() call, it is re-entrant and designed for this. Regardless, you don't wait for the connection to complete, the code following the connect() call returns failure for every -1 result from connect() unless it is EINPROGRESS or EWOULDBLOCK. select() is -not- used in fe-connect.c. It is possible with the current code for the connection to fail in non-blocking mode. Reason: you call connect() in non-blocking mode, break out of the section on EINPROGRESS, and continue assuming that the connection will be successful. EINPROGRESS The socket is non-blocking and the connection can not be completed immediately. It is possible to select(2) or poll(2) for completion by selecting the socket for writing. After select indicates writability, use getsockopt(2) to read the SO_ERROR option at level SOL_SOCKET to determine whether connect completed successfully (SO_ERROR is zero) or unsuccessfully (SO_ERROR is one of the usual error codes listed here, explaining the reason for the failure). The socket is not checked any further after the connect(). The code should not continue on into the SSL handling until you're sure that the socket is ready for operation. The reason why I am getting EINTR from a non-blocking connect is because my event timer happens to fire in the middle of the connect() call. Just because you set the socket to FIONBIO doesn't mean that connect() can't be interrupted. David ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
"The **SA_RESTART** flag is always set by the underlying system in POSIX mode so that interrupted system calls will fail with return value of -1 and the *EINTR* error in /errno/ instead of getting restarted." libpq's pqsignal.c doesn't turn off SA_RESTART for SIGALRM. Further, pqsignal.c only handles SIGPIPE and not to mention that other parts of libpq do handle EINTR properly. PQconnect* family does not handle EINTR. It does not handle the possible and perfectly legitimate interruption of a system call. Globally trying to disable system calls from being interrupted is a Bad Thing. Having a timer event is common, having a timer event in a daemon is often required. Timers allow for good housekeeping and playing nice with the rest of the system. Your reasonable behavior in the case of EINTR means repeatable and mysterious failure. There isn't a clean way to re-enter PQconnect* while maintaining state in the case of signal interruption and no guarantee the function won't be interrupted again. Basically if you have a timer event in your software and you use pgsql, then the following will happen. a) if the timer event always happens outside the PQconnect* call is completed your code will function b) if the timer event always fires during the PQconnect* call, your code will never function c) if your timer event sometimes fires during the PQconnect* call, your code will sometimes function There are no ifs, ands, or buts about it, if a timer fires inside PQconnect* as it is now, there is no way to continue. With a suitablly long timer period, you can try the PQconnect* call again and if the connect succeeds before the timer fires again you're fine. If not, you must repeatedly try. That said, there are two ways about it. a) handle it cleanly inside PQconnect* like it should be done, or b) have the programmer parse the error string for "Interrupted system call" and re-enter PQconnect. a) is clean, short, and simple. b) wastes a lot of CPU to attempt to accomplish the task. a) is guaranteed and b) is not guaranteed. David Peter Eisentraut wrote: David Ford writes: >Libpq doesn't deal with system calls being interrupted in the slightest. > None of the read/write or socket calls handle any errors. Even benign >returns i.e. EINTR are treated as fatal errors and returned. Not to >malign, but there is no reason not to continue on and handle EINTR. > Libpq certainly does deal with system calls being interrupted: It does not allow them to be interrupted. Take a look into the file pqsignal.c to see why. If your alarm timer interrupts system calls then that's because you have installed your signal handler to allow that. In my mind, a reasonable behaviour in that case would be to let the PQconnect or equivalent fail and provide the errno to the application. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
>
>
>This does not actually *say* that the appropriate behavior after EINTR
>is to retry, but reading between the lines one might infer that it will
>work like the nonblocking case, wherein a retry of connect tries to link
>to the existing connection attempt, not start a new one.
>
>What's more important is that a retry will expose the possibility of
>getting EALREADY or EISCONN. EALREADY certainly must be treated as
>success the same as EINPROGRESS (if it exists on a given platform ---
>better #ifdef it I think). Not so sure about EISCONN; does that imply
>"you moron, this socket's been open forever", or does it get returned on
>the first iteration that doesn't return EALREADY?
>
Not to worry. EINPROGRESS, etc, are normally used for select() and
return the current state of the connection attempt. connect() in
blocking mode will normally only return when it is interrupted or
complete. In non-blocking mode, it will return immediately with errno
set to EALREADY, in which case you should spin on the socket connection
attempt and wait until it returns a good status, i.e. return value=0, or
if you happen to make a mistake and drop that iteration, EISCONN.
Those of you who are interested in socket operations would probably
enjoy reading TCP/IP Illustrated, I believe the most recent version is
#3, sorry I don't have the ISBN handy but most libraries should have v2.
As to your last concern Tom, the cycle should be: return=-1 [repeats
until connection fails or succeeds], return=0 on success, or -1 on
failure w/ errno set to appropriate fault, and afterwards, either return
-1 with errno=EISCONN (already connected with prior success), or
appropriate fault.
The important part is that if the socket successfully connects, it will
return a 0. That should be the last iteration of your spin on that
connect call. If you have a mistake in your code and you keep spinning
on a successfully connected socket, yes you will get EISCONN, i.e.
"hello..the socket is still connected" :)
Thus a simplified loop should look like this:
do {
int r;
r=connect(...);
if(!r)/* we're connected, r=0. exit the loop */
break;
switch(errno) {/* connect returns 0 or -1, so this must be -1 */
case EINTR:/* we were interrupted by a signal */
continue;
default:
error_log(logfd, "Error connecting, kernel said: %s",
strerror(errno));
r=-2;/* jump out of the loop */
break;
}
} while(r==-1):
This will continue trying the connect call as long as it hasn't returned
a failure message and the errno is EINTR. All other -1 return values
will exit the loop. As soon as it has connected properly it will also
exit the loop.
David
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
> > >After further thought, though, it's unclear to me why this solves >David's problem. If he's got a repeating SIGALRM on a cycle short >enough to interrupt a connect(), seems like it'd just fail again >on the next try. > Ok, a few things. The connect() call is just an interface to the kernel. Sometimes a connect() to a socket may take a long time, even up to two minutes (or depending on your kernel's timeout), so it isn't unfeasible that the call can be interrupted. Next, the userland connect() is interrupted but the kernel isn't. The kernel keeps working it and eventually completes or aborts the connection attempt. It then sets the data structures and values so the next time userland comes alive it's ready for it. The connect() call doesn't restart at the beginning, it continues where it left off. David ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
> > > >It wouldn't surprise me in the least if this behavior is >platform-dependent. It may well be that David's kernel will allow >connect() to be interrupted by SIGALRM while yours won't. (Which >reminds me that neither of you specified what platforms you were >testing on. For shame.) Or maybe the difference depends on whether >you are trying to connect to a local or remote server. > >Unless someone can point out a situation where retrying connect() >after EINTR is actively bad, my inclination is to accept the patch. >It seems like a net improvement in robustness to me, with no evident >downside other than a line or two more code. > I didn't specify my OS because this sort of a thing is standard *nix etc design (well, m$ excluded of course). I use Linux. Every *nix that I know of can have system calls be interrupted. Please wait a day before applying the patch, I want to make it a bit more clean/readable and make sure I covered everything in fe-connect.c, I found that the SSL functions are traversed even if ssl is turned off in the config file and I have to handle that too. David ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
Many signals may be the cause of -EINTR. It depends on what the signal is as to how it's normally handled. sigalarm is the most common due to it being a timer event. Generate a timer that expires as fast as possible (not too fast to prevent code execution), and you should see things left and right return with -EINTR. I'm very much aware of why SIGALRM is happening, I generate it and I catch it. As per my original message on this thread, my program does maintenance on a scheduled basis. The period of that maintenance is many times per second. Sooo... :) Now let's get on with the story. Libpq doesn't deal with system calls being interrupted in the slightest. None of the read/write or socket calls handle any errors. Even benign returns i.e. EINTR are treated as fatal errors and returned. Not to malign, but there is no reason not to continue on and handle EINTR. David p.s. you cant use sleep() or alarm() functions and have a timer event as well. The only POSIX compliant function that doesn't trample signal timer events is nanosleep(). Brent Verner wrote: On 25 Oct 2001 at 17:08 (-0400), David Ford wrote: | I'm fresh in the code, but this has solved my issues with PQconnect* | failing when interrupted by signals. Some of it is sloppy and not to my | liking yet, but I'm still digging through to see if anything else needs | touched. Comments appreciated. Disclaimer: I may be wrong as hell ;-), but... I'm not sure this is correct. I've tried to /make/ a SIGALRM cause connect to errno==EINTR, but I can't cause this condition. I suspect you have another signal being raised that is causing your symptom. FTR, the textbook definition[1] of EINTR error for connect is: The attempt to establish a connection was interrupted by delivery of a signal that was caught; the connection will be established asynchronously. Please check the attached prog to see if it is representative of your code relating to the connect error you're seeing. If it is, please run it and see if you can get it to cause the EINTR error from connect. If you can't I'm more certain that you have a problem elsewhere. cheers. brent 1. http://www.opengroup.org/onlinepubs/7908799/xns/connect.html [snipped] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] [patch] helps fe-connect.c handle -EINTR more gracefully
I'm fresh in the code, but this has solved my issues with PQconnect*
failing when interrupted by signals. Some of it is sloppy and not to my
liking yet, but I'm still digging through to see if anything else needs
touched. Comments appreciated.
Honestly, I'm a bit surprised that this issue hasn't been encountered
before.
Summary:
* changes to connect() sections to handle errno=EINTR. this solves
libpq PQconnect* family problems if the connect is interrupted by a
signal such as SIGALRM.
* not all read/recv/write/send calls have been updated
David
--- src/interfaces/libpq/fe-connect.c.orig Wed Oct 24 17:43:52 2001
+++ src/interfaces/libpq/fe-connect.c Wed Oct 24 17:43:54 2001
@@ -912,21 +912,35 @@
* Thus, we have to make arrangements for all eventualities.
* --
*/
+
+ retry_socket:
if (connect(conn->sock, &conn->raddr.sa, conn->raddr_len) < 0)
{
- if (SOCK_ERRNO == EINPROGRESS || SOCK_ERRNO == EWOULDBLOCK ||
SOCK_ERRNO == 0)
- {
- /*
-* This is fine - we're in non-blocking mode, and the
-* connection is in progress.
-*/
- conn->status = CONNECTION_STARTED;
- }
- else
- {
- /* Something's gone wrong */
- connectFailureMessage(conn, SOCK_ERRNO);
- goto connect_errReturn;
+ switch (SOCK_ERRNO) {
+ case EINTR:
+ /*
+* interrupted by signal, keep trying
+*/
+goto retry_socket;
+break;
+
+ case 0:
+ case EINPROGRESS:
+ case EWOULDBLOCK:
+ /*
+* This is fine - we're in non-blocking mode, and the
+* connection is in progress.
+*/
+ conn->status = CONNECTION_STARTED;
+ break;
+
+ default:
+ /*
+* Something's gone wrong
+*/
+ connectFailureMessage(conn, SOCK_ERRNO);
+ goto connect_errReturn;
+ break;
}
}
else
@@ -2132,8 +2146,13 @@
"PQrequestCancel() -- socket() failed: ");
goto cancel_errReturn;
}
- if (connect(tmpsock, &conn->raddr.sa, conn->raddr_len) < 0)
+ while (connect(tmpsock, &conn->raddr.sa, conn->raddr_len) < 0)
{
+ /*
+* interrupted by a signal
+*/
+ if(errno==EINTR)
+ continue;
strcpy(conn->errorMessage.data,
"PQrequestCancel() -- connect() failed: ");
goto cancel_errReturn;
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] PQconnectStart() and -EINTR
I've got a bit of a problem. I added a fast SIGALRM handler in my
project to do various maintenance and this broke PQconnectStart().
Oct 23 21:56:36 james BlueList: connectDBStart() -- connect() failed:
Interrupted system call ^IIs the postmaster running (with -i) at
'archives.blue-labs.org' ^Iand accepting connections on TCP/IP port 5432?
PQstatus() returns CONNECTION_BAD, how can I reenter the connection
cycle or delay, more like, how do I differentiate between an actual
failure to connect and an interruption by signal? My alarm timer
happens much too frequently for this code to make a connection and
unfortunately I can't disable the alarm because it's used for bean
counting and other maintenance.
Thanks,
David
Code snippet:
...
/*
* play some tricks now, use async connect mode to find if the server
* is alive. once we've figured that out, disconnect and immediately
* reconnect in blocking mode. this mitigates the annoying hangs from
* using PQconnectdb which has no support for a timeout.
*/
conn=PQconnectStart(cstr);
if(!conn) {
dlog(_LOG_debug, "SQL conn is NULL, aborting");
return NULL;
}
do {
c++;
pgstat=PQstatus(conn);
switch (pgstat) {
case CONNECTION_STARTED:
dlog(_LOG_debug, "Connecting to SQL server...");
break;
case CONNECTION_MADE:
case CONNECTION_OK:
dlog(_LOG_debug, "Connected to SQL server in asynchronous
mode...");
break;
case CONNECTION_BAD:
dlog(_LOG_debug, PQerrorMessage(conn));
if(conn)
PQfinish(conn);
dlog(_LOG_warning, "failed to connect to server");
return NULL;
break;
default:
dlog(_LOG_debug, "pg conx state = %i", pgstat);
break;
}
if(pgstat==CONNECTION_MADE||CONNECTION_OK)
break;
if(c>15) {
if(conn)
PQfinish(conn);
dlog(_LOG_warning, "failed to connect to server, timed out");
return NULL;
}
req.tv_sec=1;
req.tv_nsec=0;
sleep(&req);
} while(1);
/*
* close it and reopen it in normal blocking mode
*/
PQfinish(conn);
conn=PQconnectdb(cstr);
...
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [GENERAL] To Postgres Devs : Wouldn't changing the selectlimit
Not possible to accept both forms at present and issue a notice that LIMIT m,n is deprecated? If LIMIT m,n is found, internally re-write it to LIMIT m OFFSET n and press on. This should appease everyone and still allow the 'proper' form to be implemented right now. There isn't just the question of when it appears in pgsql, but when it appears in everyone else's code that depends on postgres. If you delay LIMIT..OFFSET, then I too am affected in my code. If I use it today and my code is in beta (which it is), then when it goes release, I'll have to issue a change in the future for that. Granted it's not a big thing for me, but if I have 200,000 installations, that means eventually there will have to be 200,000 upgrades when they upgrade postgres. We all know that everyone updates their software frequently and in a timely manner to keep things running smoothly, right? *cough* David Tom Lane wrote: >Given the amount of noise being raised on the issue now, I think the >better part of valor is to revert to the 7.1 behavior and plan to >discuss it again for 7.3. But it's not like Bruce did this with no >warning or discussion. > ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [OT] [HACKERS] Re: List response time...
Vince Vielhaber wrote: >On Fri, 24 Aug 2001, David Ford wrote: > >>It's all in the configuration. I slam mails around dozens of machines >>in seconds using sendmail and I process a lot of mail. >> > >So have you patched for the latest of the many sendmail root exploits? > >Vince. > I keep my systems up to latest and greatest that passes the lab. Currently 8.12.0b19. Since I keep things up to date and read the documentation... I tend to avoid most security problems. Do keep in mind that most of the latest issues are symbiotic problems due to issues found in LK capabilities. -d ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Link to bug webpage
Tom Lane wrote: >>The last worthwhile item on this guy's list is changing ownership of a >>database. Well, I haven't yet had to do this: can we do this easily? >> >It could be better. See recent "Multiple Servers" thread over in >pg-admin, notably >http://fts.postgresql.org/db/mw/msg.html?mid=1031042 >(which the FTS server seems not to have linked into the thread for some >reason) > Here is where the Indexing by date/thread fails. If I were searching for changing ownership, how would I even begin to consider "Multiple Servers" in my searches? A bug db here would have been perfect. -d ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Re: List response time...
Vince Vielhaber wrote: >ooohh I've been raggin on >Marc on that one for well over a year, maybe two.. I started using >qmail when it was still in .7something beta and never looked back. The >folks at Security Focus have moved all of the lists to ezmlm (part of >qmail) and have had nothing but success... But don't tell Marc. > And ezlm is -ever- so quick to tell you your mail is bouncing when your link goes down for a few hours or is sporadic. I know of several others that simply send you the emails that are in queue. -d ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Re: List response time...
Ian Lance Taylor wrote: >Both qmail and postfix radically outperform sendmail for large mailing >list delivery on identical hardware. It seems strange to me to say >that there is no sendmail issue when sendmail itself is the issue. >The queuing structure sendmail uses is simply wrong when a single >message has many recipients. I've run moderately serious (1000 users, >dozens of messages per day) mailing lists using both sendmail and >qmail, and there really is no comparison. > Ian, please It's in the configuration. I run much more than the above and have no issues at all. -d ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Re: List response time...
> > >You are seeing sendmail's poorly designed queuing behaviour in action. >sendmail limits itself by outgoing messages, rather than outgoing >deliveries. This causes one slow delivery to hold up many fast >deliveries. > Again, all in the configurationrinse, repeat. Simply change your queue priority. David ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] List response time...
> > >All the delay seems to be in transferring the message from >postgresql.org to webmail.postgresql.org ... which are the same >machine, or at least the same IP address. What's up with that? > Looks like sendmail? Change your queue runs to be more aggressive. I have an mc file on http://blue-labs.org/clue/bluelabs.mc that has some aggressive queue definitions. David ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Re: List response time...
Ian Lance Taylor wrote: >>Mailing lists don't scale well to large numbers of subscribers. I see this >>delay constantly,on multiple lists. The bigger the list gets, the slower the >>list gets (and the more loaded the server gets, right Marc? :-)). >> > >Note that the postgresql.org mail server is still running sendmail. >In my personal experience with sources.redhat.com, qmail is a much >better choice to handle large mailing lists. When we switched from >sendmail to qmail, mailing list delays dropped from hours, or >sometimes even days, to seconds. > It's all in the configuration. I slam mails around dozens of machines in seconds using sendmail and I process a lot of mail. David ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Link to bug webpage
Tom Lane wrote: >Peter remarked that he wouldn't use a bug database unless it has some >input filtering to remove all the non-bug issues that currently clutter >the pgsql-bug archives. I tend to agree with him. A possible way to >handle that is to set up bug-input like a closed mailing list: only >accept mail from designated people (developers and people nominated to >help run the bug database). So, a bug database entry would start life >when some one of these people replies to an emailed bug report >confirming that there is a bug, or forwards the verified report to >bug-input, or whatever. > Here I respectfully disagree. If I have to wait on 'approval' to submit a bug or carry on a discussion about it, most of the time I'm going to silently drop it and find some other way to make my project work. I like Mozilla's bugzilla because I can instantly and with very little effort classify all sorts of things and describe my bug. Then along comes a person who can assign it to someone, confirm it, mark it up as clueless user, or whatever is needed. Everyone associated with this bug # gets a copy of every transaction that happens to this bug. You can easily cc this into the pgsql-bugs. A lot of projects grow and develop little things like 'it works for all of us so it's not a bug', I run into that now and then in an obscure issue...libtool comes to mind...and -nobody- has information on it except 4 other webpages in this universe where 1 person reports the problem, two people say the 1st person is shouldn't be using gcc 2.96 and the fourth person has a fix which meant gcc wasn't at fault in the first place. You can't have a really effective 100% bug database without allowing everyone to add to it. If I had to submit to mozilla "tables are one pixel off starting about Aug 21st" and get approval before it went in, I'd likely say screw it and simply make my tables one pixel bigger. As it is, I post the bug and 20 minutes later due to the magic of bugzilla, the right person put the fix in. I don't have to adapt my tables. :) -d ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Link to bug webpage
Tom Lane wrote: >Given a threaded index, you aren't wading through "a few hundred posts". >Agreed, a nice canned database entry might be easier to look at, but >who's going to expend the time to maintain the database? Unless someone >actively takes responsibility for keeping the DB up to date, it'll be >junk. So far I heard Philip say he'd be willing to check over some >fraction of the existing entries, but I don't hear anyone wanting to >take it on as a long-term commitment. > I've often had a hard time searching for results in email archives because the datum used for indexing changes. Different people change the subject line etc. You can't index by date a fair bit of the time because there will be lapses in the discussion. One of the better things about using a bugzilla [e.g.] is that it becomes a community responsibility, not a single person or small group. Anyone can now update the 'TODO' list. -d ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Link to bug webpage
Bruce Momjian wrote: >How do you communicate that to people looking at the content? Do you >put in big letters at the top, "This list is not complete." The fact an >items is missing from the list (new bug) is just as important as an item >appearing on the list. > How do you distinguish that from what we have now? I can't look at my pgsql email box and see how many and of what. A bugzilla is a more accurate representation of bugs and future features for the group. -d ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Link to bug webpage
Bruce Momjian wrote: >OK, what value does a bug database have over a TODO list? > history of a bug, entire discussion about that bug on the same page with hyperlinked patches and other attachments. ability of everyone to add to the bug documentation without submitting it to the TODO maintainer. categorization and "it works on X", "it's broken if Y" etc I really could go on and on. -d ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Link to bug webpage
Serguei Mokhov wrote: >Maybe a better solution for the short run would be >return the page where it was, and but links to the pgsql-bugs and >pgsql-hackers archives with some sort of exmplanatory saying that "this is >a *complete* (it must be complete of course) list of bugs, which are >being extensively discussed in the these lists and fixed. Please, visit/search >these mail archives for most up-to-date information blah blah blah" One can >invent some appropriate wording, I guess. But this atleast will show people >that there's work acually going on, and on daily basis. And also a good idea >to have "Last updated" time stamp on the page too... so it doesn't seem >to be forgotten for ages.. > The archives are a 'flat' database of bugs which require a lot of work for a researcher to figure out if a bug is already documented and what the status is. It is also not 100% accurate as not all bugs get reported there. -d ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Link to bug webpage
Bruce Momjian wrote: >>>That is the real question. Do we want to rely more heavily on a bug >>>database rather than the email lists? I haven't heard many say they >>>want that. >>> I'd very much like a bugzilla because I can do research on bugs past or present now as well as know the status of them. Right now if I had a bug I would have to dig through web page after web page or use wget and grep. -d ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Link to bug webpage
I vote for pgsql bugzilla. If I have a bug to report I'll file it. I file plenty of moz bugs and aid in resolving them. -d Bruce Momjian wrote: >>There are over 400 in the database. If that's a small percentage then >>so be it, but it's still over 400 bugs that appear to have been ignored. >>Having a place to look up possible problems and seeing if there was a >>solution seems to be a plus to me, but if you don't want it it doesn't >>bother me either way. The lookups are currently disabled, ball's in >>your court. >> > >It's up to the group to decide. If we have a database of bugs, I think >it has to be complete. I think a partial list is worse than no list at >all. > ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Link to bug webpage
Honestly I wasn't aware postgres had any bugs... tongue in cheek. What I mean is PG works very nicely for me and I haven't had any problems with it, so that means "no bugs". Yes there are bugs and things to be solved, but from my perspective it is already a pretty darn good piece of software. -d Philip Warner wrote: >At 08:32 21/08/01 -0400, Vince Vielhaber wrote: > >>Yes but noone was interested in it. It's still there but you're really >>the first to show interest in about a year. >> > >That's good (and depressing); where are they? > ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] OID unsigned long long
Tom Lane wrote: >[...] > >BTW, I think #ifdef would be a totally unworkable way to attack the >format-string problem. The code clutter of #ifdef'ing everyplace that >presently uses %u would be a nightmare; the impact on >internationalization files would be worse. And don't forget that %llu >would be the right thing on only some machines; others like %qu, and >DEC Alphas think %lu is just fine. The only workable answer I can see >is for the individual messages to use some special code, maybe "%O" for >Oid. The problem is then (a) translating this to the right >platform-dependent thing, and (b) persuading gcc to somehow type-check >the elog calls anyway. > You can ask gcc to typecheck format strings for printf type functions with something like the following: extern int my_printf (void *my_object, const char *my_format, ...) __attribute__ ((format (printf, 2, 3))); Ref: http://www.delorie.com/gnu/docs/gcc/gcc_77.html David ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Improving pg_hba.conf
What I mean is, do host lookups first in the text file, then if a pg_hostaccess table (example) exists, check it for host entries. This alleviates HUP or restart etc. to reload a config file. David Bruce Momjian wrote: >>> >>>I haven't heard of any more issues with pg_hba.conf so I will mark the >>>item as done. I did cleanup the comments in the file. I have also >>>added a TODO item: >>> >>> * Read pg_hba.conf only on postmaster startup or SIGHUP >>> >>If you do this, can you add an access table for secondary lookups? I >>tend to update host access frequently and personally I think it'd be >>dandy to have a table that can be updated for access as well as the >>primary pg_hba.conf file. >> > >Sorry, I don't understand what secondary lookups are. > ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Improving pg_hba.conf
> > >I haven't heard of any more issues with pg_hba.conf so I will mark the >item as done. I did cleanup the comments in the file. I have also >added a TODO item: > > * Read pg_hba.conf only on postmaster startup or SIGHUP > If you do this, can you add an access table for secondary lookups? I tend to update host access frequently and personally I think it'd be dandy to have a table that can be updated for access as well as the primary pg_hba.conf file. David ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] URGENT PROBLEM
How 'bout posting what version of pgsql you're running, and we'll start back at square one :) -d Bruce Irvine wrote: > Hi All, > > > > This is my first post, so I hope I'm in the right area and doing it > correctly. > > > > We are having MAJOR & URGENT problems with Postresql occaisonly > corrupting tables on insert. I had a quick look through your archive > and couldn't find anything. It seems to happen mostly on large inserts > (lots of data into one text field). This results in corrupting the > table and hanging the psql console whenever I try to INSERT, UPDATE, > DELETE, SELECT, etc. Doing an "EXPLAIN SELECT * FROM table" shows that > I have around 100 - 1000 extra rows. The problem is often fixed by > running VACUUM against the table, however VACUUM often hangs leaving > the table locked until I delete the lock file. > > > > Its only a basic INSERT statement into a basic table. > > > > Thanks in advance. > > > > Bruce Irvine. > > SportingPulse. > ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
