[HACKERS] bison
Hi, I talked to one of the bison guys and he told me where to find a beta version of bison 1.49. And this one translates the grammar without a problem, no more table overflow. So once they will release the new bison we should be able to expand our grammar. Michael -- Michael Meskes [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Redhat 7.3 time manipulation bug
On Thu, May 23, 2002 at 04:42:13AM +0200, Magnus Naeslund(f) wrote: Some answers on database sizes, if this is any help... I did du -sh /usr/share/zoneinfo/ on them all. OpenBSD 3.1, sparc64: 1.3M/usr/share/zoneinfo/ Linux, i686, oldish mandrake (6.x?), glibc 2.1.3: 478k/usr/share/zoneinfo Linux, i686, newish redhat 7.2, glibc 2.2.4: 4.9M/usr/share/zoneinfo What do they do with that di? Linux, alpha EV56, oldish redhat 6.2, glibc 2.1.3 1.4M/usr/share/zoneinfo Here's what my Debian Woody system says: 1.5M/usr/share/zoneinfo And this is with glibc 2.2.5. Of course this wouldn't be the first time that RedHat uses the same version number for a different version. Michael -- Michael Meskes [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] tuples gone?
Hi, I got an corrupted table,,, unfortunately with pretty important data :( VACUUM tells me: NOTICE: Rel relx: TID 2344/5704: OID IS INVALID. TUPGONE 1. NOTICE: Rel relx: TID 2344/5736: OID IS INVALID. TUPGONE 1. NOTICE: Rel relx: TID 2344/5768: OID IS INVALID. TUPGONE 1. (this, many times, then) pqReadData() -- backend closed the channel unexpectedly. This probably means the backend terminated abnormally before or while processing the request. I can read part (beginning?) of the relation with select or copy, but anything that touches this area dies badly :( Is there any way to recover this relation? Or at least as much data as possible? Oh, an this is 7.1.3 and I am probably running with too large oids :) DEBUG: NextTransactionId: 708172974; NextOid: 3480073772 Daniel ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Edge case problem with pg_dump
On May 22, 2002 10:28 am, you wrote:
Right now the only safe way to dump such a database is to use the
inserts-with-explicit-column-names option. Someone was working on
extending COPY to allow a column name list, and as soon as that gets
done I intend to change pg_dump to specify a column name list in
COPY commands. That should fix this problem.
Do you mean issue COPY commands with fields or COPY out the fields in a
specific order by using the extension in pg_dump? Seems like the latter
would be cleaner but the former is probably a lot simpler to do.
What would the new syntax of the COPY look like?
--
D'Arcy J.M. Cain darcy@{druid|vex}.net | Democracy is three wolves
http://www.druid.net/darcy/| and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner.
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] Redhat 7.3 time manipulation bug
On Wed, 2002-05-22 at 08:00, Thomas Lockhart wrote: ... If so, can we get them to champion changes which would comply with the standard but remove this arbitrary breakage? Unlikely. They already saw (and participated, at least Ulrich) a thread on this with Lamar. Their take is this is the standard, you should do what the standard says and not rely on undocumented, non-standardized sideeffects. OK. They must be new guys. :-) Very funny. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Redhat 7.3 time manipulation bug
On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 (same as Red Hat 7.3's version). This is a completely different version. Once Debian updates (in a few years) they'll get the same result. If you are misusing interfaces you get what you deserve. At no time was it correct to use these functions for general date manipulation. It always only was allowed to use them to represent system times and there was no Unix system before the epoch. Therefore you argumentation is completely wrong. If you need date manipulation write your own code which work for all the times you want to represent. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
[HACKERS] Security policy
It occurs to me that part of the problem with wasted and incomplete efforts can be fixed with a clear security policy. The part that I'm interested in is provided below, in a very truncated form. Secure Communications Channels -- Secure communications channels can be provided with Kerberos, GSS-API, and SSL, and Unix sockets for local communications. The goals of the secure commuications channel are: * Confidentiality Confidentiality means that the data is kept secret from all third parties. - Perfect Forward Security (PFS) Perfect Forward Security is the logical endpoint of confidentiality. It is a form of confidentiality where the data remains secret even if the static private keys used by the server (and client) are exposed. * Message Integrity Message integrity means that the message received is identical to the message sent. It is not possible for third parties to add, delete, or modify data midstream. * Endpoint Authentication Endpoint Authentication means that the identity of the other party can be firmly established. - Mutual Authentication Mutual Authentication is endpoint authentication by both parties. - Strong Authentication Strong Authentication means that the party has authenticated themselves with at least two of the following: something they know (e.g., password), something they have (e.g., private key, smart card), or something they are (e.g., biometrics). A mechanism to map channel-level authentication (Kerberos principal name, SSL distinguished name) to PostgreSQL userids is desirable, but not required. Initial support for all new protocols shall always include, at a minimum, all features present in the sample client and server provided with the respective toolkit. Any omissions must be clearly documented and justified. The development team shall maintain a matrix cross-referencing each protocol and the goals satisfied. Any omissions from normal practice for each protocol shall be clearly documented and provided to users. | L-SSL | L-KRB | SSL | GSS-API | SASL | Unix +---+---+-+-+--+-- Confidentiality | Y | N | Y |Y| Y | Y PFS | N | N | Y |?| ? | Y Message Integrity | N | N | Y |Y| Y | Y Authentication (server) | N(1) | ?(2) | Y |Y| Y | Y Authentication (mutual) | N | ?(2) | Y |Y| Y | Y +---+---+-+-+--+-- L-SSL legacy SSL L-KRB legacy Kerberos 4 5 SSL current SSL patches GSS-API GSS-API (Kerberos 5 reimplementation) SASLSASL with appropriate plug-ins UnixUnix sockets (1) a server certificate is required, but it is not verified by the client. (2) PostgreSQL provides some level of authentication via Kerberos 4 and Kerberos 5, but it may not have been properly implemented. As I mentioned in an earlier post on -patches, I'm not sure that the current Kerberos implementation is what people think it is. I may develop a GSS-API patch for evaluation purposes, but it will almost certainly require a different port. Bear ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] tuples gone?
Daniel Kalchev [EMAIL PROTECTED] writes: VACUUM tells me: NOTICE: Rel relx: TID 2344/5704: OID IS INVALID. TUPGONE 1. It's physically impossible to get 2344 tuples on a page. (If you're using 8k pages then the most you could have per page is less than 200.) So the above TID is obviously bogus, implying that you have pages with corrupted page headers --- probably pd_lower is much larger than it should be. You could try dumping out the contents of page 5704, eg dd bs=8k skip=5704 count=1 tablefile | od -x just to see what's there, but I bet you will find that it looks like it's been trashed. Is there any way to recover this relation? Or at least as much data as possible? If you can figure out what pd_lower should be on each of the trashed pages, you might be able to reset it to the correct value and recover the tuples, if there are any un-trashed. Otherwise zero out the trashed page(s). You should not expect to get everything back --- what you want is to make the table readable so that you can dump the contents of the undamaged pages. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Redhat 7.3 time manipulation bug
On Wed, 2002-05-22 at 11:23, Tom Lane wrote: Unix systems have *always* interpreted time_t as a signed offset from the epoch. No. This always was an accident if it happens. Do you really think that when Unixen were first built in the early 70s, there was no interest in working with pre-1970 dates? Hardly likely. There never were files or any system events with these dates. Yes. And just to educate you and your likes: the majority of systems on this planet use mktime this way. I hate using this as an argument, but beside major Unixes M$ systems also do this. But you will end up reverting this change due to pushback from users. Want to make a side bet? Sure. Especially not everybody is that stubborn. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Edge case problem with pg_dump
D'Arcy J.M. Cain [EMAIL PROTECTED] writes: On May 22, 2002 10:28 am, you wrote: Right now the only safe way to dump such a database is to use the inserts-with-explicit-column-names option. Someone was working on extending COPY to allow a column name list, and as soon as that gets done I intend to change pg_dump to specify a column name list in COPY commands. That should fix this problem. Do you mean issue COPY commands with fields or COPY out the fields in a specific order by using the extension in pg_dump? I intended that the dump scripts would say something like COPY mytab(field1,field2,field3) FROM STDIN; which would make it absolutely clear what the dump's field order is. We can't solve it by reordering the fields while we dump, which is what I think you mean by the other alternative: how is pg_dump to guess what schema you are going to load the data into? For example, it should work to do a data-only dump and then reload into the existing table structure. So the dump script really needs to work for either column ordering in the destination table, and that's why we need explicit labeling of the field order in the script. If we take this really seriously we might want to eliminate pg_dump's -d (simple INSERT) option, and have only two dump formats: COPY with field labels, or INSERT with field labels. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Redhat 7.3 time manipulation bug
On Wed, May 22, 2002 at 10:58:15AM -0700, Ulrich Drepper wrote: On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 (same as Red Hat 7.3's version). This is a completely different version. Once Debian updates (in a few years) they'll get the same result. Ulrich, how shall I understand this? I'm pretty sure Oliver does not use a Debian 2.2 system with glibc-2.1.3 but a pretty up-to-date one. The glibc version in the soon to be released Woody release is 2.2.5. This seems to be the very same version that RedHat uses. So what could/should Debian update? Besides, the in a few years comment looks like FUD to me. It may be a few years since we talked the last time, but I cannot imagine you changed that much that you spread FUD nowadays. So I probably misunderstood this sentence, but nevertheless would like to know what Debian should update. Or do you mean that once Debian updates to glibc 2.3 (or whatever the next release will be) it will show the same results? Does RedHat 7.3 already run on that new release? But then I would think they changed the version number. Michael -- Michael Meskes [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Redhat 7.3 time manipulation bug
On 22 May 2002, Ulrich Drepper wrote: On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 (same as Red Hat 7.3's version). This is a completely different version. Once Debian updates (in a few years) they'll get the same result. If you are misusing interfaces you get what you deserve. At no time was it correct to use these functions for general date manipulation. It always only was allowed to use them to represent system times and there was no Unix system before the epoch. Therefore you argumentation is completely wrong. If you need date manipulation write your own code which work for all the times you want to represent. We are Redhat, you will be assimilated ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Edge case problem with pg_dump
* Tom Lane [EMAIL PROTECTED] [020523 10:24]:
D'Arcy J.M. Cain [EMAIL PROTECTED] writes:
Do you mean issue COPY commands with fields or COPY out the fields in a
specific order by using the extension in pg_dump?
I intended that the dump scripts would say something like
COPY mytab(field1,field2,field3) FROM STDIN;
Cool. I assume that the (field1,field2,field3) would be optional for
backwards compatibility.
which would make it absolutely clear what the dump's field order is.
We can't solve it by reordering the fields while we dump, which is
what I think you mean by the other alternative: how is pg_dump to
guess what schema you are going to load the data into? For example,
Well, the issue now is that it creates the schema too but it is out of
sync with the data it spits out. I can see how figuring it out is a lot
more difficult though. The above works.
it should work to do a data-only dump and then reload into the existing
table structure. So the dump script really needs to work for either
column ordering in the destination table, and that's why we need
explicit labeling of the field order in the script.
That's nice. I have scripts that effectively do this in code now when
I have to dump from one schema and load into another.
If we take this really seriously we might want to eliminate pg_dump's
-d (simple INSERT) option, and have only two dump formats: COPY with
field labels, or INSERT with field labels.
Yah, I don't think that I have ever used -d. In fact, I bet I will
hardly ever use -D any more if we make the above change. The only
reason I ever used insert statements was to deal with loading into a
different schema.
So who was it that wanted to make this change. Perhaps I can help.
--
D'Arcy J.M. Cain darcy@{druid|vex}.net | Democracy is three wolves
http://www.druid.net/darcy/| and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner.
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] tuples gone?
I said: Daniel Kalchev [EMAIL PROTECTED] writes: NOTICE: Rel relx: TID 2344/5704: OID IS INVALID. TUPGONE 1. You could try dumping out the contents of page 5704, eg BTW, I got the ordering backwards: VACUUM prints TIDs as page number and then tuple number. So actually all these complaints are referencing a single page, 2344, suggesting that you've got just one trashed page header. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Redhat 7.3 time manipulation bug
--=-Z1lifK4QZqKV8kHqHfYT Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2= .5=20 (same as Red Hat 7.3's version). This is a completely different version. Once Debian updates (in a few years) they'll get the same result. If you are misusing interfaces you get what you deserve. At no time was it correct to use these functions for general date manipulation. It always only was allowed to use them to represent system times and there was no Unix system before the epoch. Therefore you argumentation is completely wrong. If you need date manipulation write your own code which work for all the times you want to represent. This is indeed a problem with dates on LIBC, because even if it is theoretically satisfactory to describe dates within some range within 2^31 seconds of 1970, that is certainly NOT satisfactory for describing all dates of interest unless you're being terribly parochial about what is to be considered of interest. My father's birth falls within 2^31 seconds of 1970, but the Battle of Agincourt doesn't. Any backup of any Unix system in history falls within 2^31 seconds of 1970, but there are people alive whose births don't. People get away with using Unix dates as a general date type when they don't have to work outside a limited range. If/when there is a move to 64 bit values, that will provide something with enough range to cover history back to ridiculously early times, relieving the limit. But anybody using Unix dates as general dates has leaped into exactly the same sort of trap that caused people to get so paranoid about Y2K. -- (concatenate 'string cbbrowne @acm.org) http://www.cbbrowne.com/info/oses.html Do Roman paramedics refer to IV's as 4's? -- (concatenate 'string aa454 @freenet.carleton.ca) http://www.ntlug.org/~cbbrowne/linuxxian.html So, when you typed in the date, it exploded into a sheet of blue flame and burned the entire admin wing to the ground? Yes, that's a known bug. We'll be fixing it in the next release. Until then, try not to use European date format, and keep an extinguisher handy. -- [EMAIL PROTECTED] (Tequila Rapide) ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Redhat 7.3 time manipulation bug
On 22 May 2002, Ulrich Drepper wrote: On Wed, 2002-05-22 at 11:23, Tom Lane wrote: Unix systems have *always* interpreted time_t as a signed offset from the epoch. No. This always was an accident if it happens. Do you really think that when Unixen were first built in the early 70s, there was no interest in working with pre-1970 dates? Hardly likely. There never were files or any system events with these dates. Yes. And just to educate you and your likes: the majority of systems on this planet use mktime this way. I hate using this as an argument, but beside major Unixes M$ systems also do this. M$ systems crashes regularly too ... is Redhat going to adopt that too? Harbison and Steele indicates that: Although the traditional return type of time is long, the value returned is usually of type unsigned long. That is NOT a Linux reference; it was published before Linus had started working on his kernel project. ANSI does not indicate that time_t is a signed int, signed long, or whatever. It is only defined to be an arithmetic type. It would not be a bug for GLIBC to define time_t to be an unsigned int, with an epoch beginning of January 1, 1999. That definition would conform to ANSI C. Since that definition can conform to ANSI C, and Unix systems have more particularly been known to use unsigned ints with epoch of 1970, it is NOT reasonable to assume that time_t can be used to express dates that are at all ancient in the past. Indeed, it is fairly _useful_ for different libc implementations to make different assumptions about things whose definitions are permitted to vary, as this makes it easier to call out as WRONG those systems that make up their own definitions. People will no doubt get defensive about their own non-standard implementations of things; it is certainly far easier to cry They're trying to play Microsoft! than it is to be honest and actually look at the standards. -- (concatenate 'string cbbrowne ntlug.org) http://www.cbbrowne.com/info/advocacy.html When confronted by a difficult problem, solve it by reducing it to the question, How would the Lone Ranger handle this? -- (reverse (concatenate 'string gro.mca enworbbc)) http://www.cbbrowne.com/info/linuxxian.html As of next Monday, COMSAT will be flushed in favor of a string and two tin cans. Please update your software. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Redhat 7.3 time manipulation bug
On 22 May 2002, Ulrich Drepper wrote: On Wed, 2002-05-22 at 11:23, Tom Lane wrote: Unix systems have *always* interpreted time_t as a signed offset from the epoch. No. This always was an accident if it happens. Do you really think that when Unixen were first built in the early 70s, there was no interest in working with pre-1970 dates? Hardly likely. There never were files or any system events with these dates. Yes. And just to educate you and your likes: the majority of systems on this planet use mktime this way. I hate using this as an argument, but beside major Unixes M$ systems also do this. M$ systems crashes regularly too ... is Redhat going to adopt that too? ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Really stupid question(?)
Bear Giles [EMAIL PROTECTED] writes: But the problem is that knowledgeable security administrators can replace the common hardcoded values with their own. How do you allow this to be easily done? Configuration parameters? One possibility that occured to me was that dynamic libraries would handle this nicely. There's even some support for dynamic libraries in the user-defined functions, so this wouldn't be a totally unprecedented idea. But this would be a new way of using dynamic libraries. You've lost me completely. What exactly are you suggesting? regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Really stupid question(?)
Bear Giles wrote: The really stupid question refers to some of the hardcoded fallback values in this code. The reason for having hardcoded values is to prevent downgrade attacks - you don't want to casually override the DBA, but you also don't want to make it easy for a knowledgeable attacker to fatally compromise the system in a way that your average DBA couldn't catch. But the problem is that knowledgeable security administrators can replace the common hardcoded values with their own. How do you allow this to be easily done? Would GUC variables work? Put in sensible defaults and let the more knowledgeable security admins override the defaults in postgresql.conf Joe ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Redhat 7.3 time manipulation bug
On Thu, 23 May 2002 [EMAIL PROTECTED] wrote: On 22 May 2002, Ulrich Drepper wrote: On Wed, 2002-05-22 at 11:23, Tom Lane wrote: Unix systems have *always* interpreted time_t as a signed offset from the epoch. No. This always was an accident if it happens. Do you really think that when Unixen were first built in the early 70s, there was no interest in working with pre-1970 dates? Hardly likely. There never were files or any system events with these dates. Yes. And just to educate you and your likes: the majority of systems on this planet use mktime this way. I hate using this as an argument, but beside major Unixes M$ systems also do this. M$ systems crashes regularly too ... is Redhat going to adopt that too? stuff deleted People will no doubt get defensive about their own non-standard implementations of things; it is certainly far easier to cry They're trying to play Microsoft! than it is to be honest and actually look at the standards. Just to clarify, if this was directed at my comment, I wasn't the one that brought up the fact that Redhat is trying to play Microsoft, Ulrich was the one that brought it into the argument ... I was just curious as to how far they planned on getting to what M$ systems do ... ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] ksqo?
On Wed, 22 May 2002 18:03:07 -0400 Tom Lane [EMAIL PROTECTED] wrote: Neil Conway [EMAIL PROTECTED] writes: The current KSQO code is currently #ifdef'ed out, and the 'ksqo' GUC variable does nothing. Is there a reason for keeping this code around? (or conversely, what was the original justification for disabling it?) I disabled it because I didn't have time to fix it properly when it got broken by the 7.1 rewrite of UNION/INTERSECT/EXCEPT. I've been waiting to see whether anyone notices that it's gone ;-). So far the demand for it has been invisible, so it hasn't gotten fixed. On the other hand I'm not quite convinced that it never will get fixed, so I haven't applied the coup de grace. Hmmm... Well, I'll take a look at it, but I'll probably just leave it be -- since the optimization might actually return invalid results, it doesn't seem like a very valuable thing to have, IMHO. Cheers, Neil -- Neil Conway [EMAIL PROTECTED] PGP Key ID: DB3C29FC ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] ksqo?
Neil Conway [EMAIL PROTECTED] writes: Hmmm... Well, I'll take a look at it, but I'll probably just leave it be -- since the optimization might actually return invalid results, it doesn't seem like a very valuable thing to have, IMHO. Yeah, I never cared for the fact that it altered the semantics of the query, even if only subtly. But I'm hesitant to rip out something that someone went to the trouble of writing and contributing ... regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
