I thought the intended way to change a GUC parameter permanently was to
edit data/postgresql.conf . No ?
What I've thought is to implement a new command to
change archdir under WAL's control.
If it's different from Vadim's plan I don't object.
Actually, I have no concrete plans for
I do not however see how the current solution fixes the original problem,
that we don't have a rollback for index modifications.
The index would potentially point to an empty heaptuple slot.
How? There will be an XLOG entry inserting the heap tuple before the
XLOG entry that updates
Before commit or rollback the xlog is not flushed to disk, thus you can loose
those xlog entries, but the index page might already be on disk because of
LRU buffer reuse, no ?
No. Buffer page is written to disk *only after corresponding records are flushed
to log* (WAL means Write-Ahead-Log -
Before commit or rollback the xlog is not flushed to disk, thus you can loose
those xlog entries, but the index page might already be on disk because of
LRU buffer reuse, no ?
No. Buffer page is written to disk *only after corresponding records are flushed
to log* (WAL means
I have just sent to the pgsql-patches list a rather large set of
Please send it to me directly - pgsql-patches' archieve is dated by Feb -:(
proposed diffs for the WAL code. These changes:
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to
Before commit or rollback the xlog is not flushed to disk, thus you can loose
those xlog entries, but the index page might already be on disk because of
LRU buffer reuse, no ?
No. Buffer page is written to disk *only after corresponding records are flushed
to log* (WAL means
Nevertheless, subj is rised. BTW, does anybody know results of kill -9
in Oracle/Informix/etc? Just curious -:)
Informix has no problem with it. Oracle dba's fear it, to say the least.
Andreas
---(end of broadcast)---
TIP 6: Have you
Consider the following scenario:
1. A new transaction inserts a tuple. The tuple is entered into its
heap file with the new transaction's XID, and an associated WAL log
entry is made. Neither one of these are on disk yet --- the heap tuple
is in a shmem disk buffer, and the WAL entry is
Vadim Mikheev wrote:
Nevertheless, subj is rised. BTW, does anybody know results of kill -9
in Oracle/Informix/etc? Just curious -:)
Progress has no problem with it that I have ever seen.
Regards,
Andrew.
--
The point is to make the allocation of XIDs and OIDs work the same way.
In particular, if we are forced to reset the XLOG using what's stored in
pg_control, it would be good if what's stored in pg_control is a value
beyond the last-used XID/OID, not a value less than the last-used ones.
If
On third thought --- we could still log the original page contents and
the modification log record atomically, if what were logged in the xlog
record were (essentially) the parameters to the operation being logged,
not its results. That is, make the log entry before you start doing the
mod
In short I do not think that the current implementation of
"physical log" does what it was intended to do :-(
Hm, wasn't it handling non-atomic disk writes, Andreas?
Yes, but for me, that was only one (for me rather minor) issue.
I still think that the layout of PostgreSQL pages was
Is anyone on this list in Hannover for CeBit? Maybe we could arrange a
meeting.
Michael
--
Michael Meskes
[EMAIL PROTECTED]
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!
---(end of broadcast)---
TIP 1: subscribe and
Hm, wasn't it handling non-atomic disk writes, Andreas?
Yes, but for me, that was only one (for me rather minor) issue.
I still think that the layout of PostgreSQL pages was designed to
reduce the risc of a (heap) page beeing inconsistent because it is
only partly written to an
Hello,
maybe I missed something, but in last days I was thinking how would I
write my own sql server. I got several ideas and because these are not
used in PG they are probably bad - but I can't figure why.
1) WAL
We have buffer manager, ok. So why not to use WAL as part of it and don't
log
Hm, wasn't it handling non-atomic disk writes, Andreas?
Yes, but for me, that was only one (for me rather minor) issue.
I still think that the layout of PostgreSQL pages was designed to
reduce the risc of a (heap) page beeing inconsistent because it is
only partly written to an
I thought the intended way to change a GUC parameter permanently was to
edit data/postgresql.conf . No ?
What I've thought is to implement a new command to
change archdir under WAL's control.
If it's different from Vadim's plan I don't object.
Actually, I have no concrete
Tom Lane wrote:
Christof Petig [EMAIL PROTECTED] writes:
We noticed that after upgrading to 7.1beta[245] the execution time for
some often used queries went up by a factor of 2 or more.
I get the desired plan after doing VACUUM ANALYZE ...
regards, tom lane
I
Hi Matthew,
I would love to get this stuff...
Could you send it to me or tell me where it is ?
TIA
On Tue, 6 Mar 2001, Matthew Hagerty wrote:
Greetings,
I wrote a few simple programs to log Apache access_log entries to pg. If
this is something anyone would be interested in or if there
Michal Maruka writes:
What about (optionally) printing the type of the column data?
io | tu | tipo |data
int | int | int2 |date
+---+--+
102242 | 26404 | 1203 | 2000-11-22
(1 row)
I've been meaning to implement this for a while. Now
Hiroshi Inoue [EMAIL PROTECTED] writes:
Do I have to send SIGHUP after changing postgresql.conf ?
In general, yes. I think that for the (still vaporware) archdir option,
you might not need to: archdir will only be looked at by the checkpoint
subprocess, and I think that a newly spawned backend
This was brought up a week ago, and I consider it an interesting idea.
The only problem is that we would no longer have control over which
pages made it to disk. The OS would perhaps write pages as we modified
them. Not sure how important that is.
Yes. As I work on linux kernel I know
"Vadim Mikheev" [EMAIL PROTECTED] writes:
I've asked if anyone can send crc64 impl to me and got only one from
Nathan Myers. Unfortunately, SWISS-PROT impl assumes that long long
is 8 bytes - is it portable?
No, it's not. I have written an implementation that uses uint64 if
available (per
Zeugswetter Andreas SB [EMAIL PROTECTED] writes:
I really think that xlog entries should be done by a layer below the
userland functions.
That seems somewhere between impractical and impossible: how will you
tie the functional xlog entries ("insert foo into index bar") to the
resulting page
"Vadim Mikheev" [EMAIL PROTECTED] writes:
So, I don't see why to remove archdir from pg_control now.
I didn't like the space consumption. I think it's important that the
pg_control data fit in less than 512 bytes so that it doesn't cross
physical sectors on the disk. This reduces the odds of
"Vadim Mikheev" [EMAIL PROTECTED] writes:
I have just sent to the pgsql-patches list a rather large set of
Please send it to me directly - pgsql-patches' archieve is dated by Feb -:(
Done under separate cover.
proposed diffs for the WAL code. These changes:
* Store two past checkpoint
Peter Eisentraut writes:
Michal Maruka writes:
What about (optionally) printing the type of the column data?
io | tu | tipo |data
int | int | int2 |date
+---+--+
102242 | 26404 | 1203 | 2000-11-22
(1 row)
I've
Bruce Momjian [EMAIL PROTECTED] writes:
The only problem is that we would no longer have control over which
pages made it to disk. The OS would perhaps write pages as we modified
them. Not sure how important that is.
Unfortunately, this alone is a *fatal* objection. See nearby
discussions
Bruce Momjian [EMAIL PROTECTED] writes:
The only problem is that we would no longer have control over which
pages made it to disk. The OS would perhaps write pages as we modified
them. Not sure how important that is.
Unfortunately, this alone is a *fatal* objection. See nearby
Tom Lane writes:
Hiroshi Inoue [EMAIL PROTECTED] writes:
Do I have to send SIGHUP after changing postgresql.conf ?
In general, yes. I think that for the (still vaporware) archdir option,
you might not need to: archdir will only be looked at by the checkpoint
subprocess, and I think that
On Wed, 7 Mar 2001, Vadim Mikheev wrote:
I have just sent to the pgsql-patches list a rather large set of
Please send it to me directly - pgsql-patches' archieve is dated by Feb -:(
Huh?
http://www.postgresql.org/mhonarc/pgsql-patches/2001-03/index.html
---(end
"Vadim Mikheev" [EMAIL PROTECTED] writes:
I feel that the fact that
WAL can't help in the event of disk errors
is often overlooked.
This is true in general. But, nevertheless, WAL can be written to
protect against predictable disk errors, when possible. Failing to
write a couple of
I reported this problem about 3 weeks ago or even more. The problem hasn't
disappeared yet. In 7.1beta4 if I use pg_dump with -a switch together, I
get each CREATE SEQUENCE twice. I suspected if this is an installation
problem at my place but now I think it maybe isn't.
You answered that noone
On Wed, 7 Mar 2001, Bruce Momjian wrote:
I have started coding a PostgreSQL performance monitor. It will be like
top, but allow you to click on a backend to see additional information.
It will be written in Tcl/Tk. I may ask to add something to 7.1 so when
a backend receives a special
I have just sent to the pgsql-patches list a rather large set of
Please send it to me directly - pgsql-patches' archieve is
dated by Feb -:(
Huh?
http://www.postgresql.org/mhonarc/pgsql-patches/2001-03/index.html
But there was nothing there yesterday -:)
(Seems archive wasn't
kovacsz wrote:
I reported this problem about 3 weeks ago or even more. The problem hasn't
disappeared yet. In 7.1beta4 if I use pg_dump with -a switch together, I
get each CREATE SEQUENCE twice. I suspected if this is an installation
problem at my place but now I think it maybe isn't.
But what can be done if fsync returns before pages flushed?
No, it won't. When fsync returns, data is promised by the OS to be on
disk.
Seems you didn't follow discussions about this issue.
Vadim
---(end of broadcast)---
TIP 3: if
On Wed, 7 Mar 2001, Mikheev, Vadim wrote:
But what can be done if fsync returns before pages flushed?
No, it won't. When fsync returns, data is promised by the OS to be on
disk.
-alex
---(end of broadcast)---
TIP 6: Have you searched our list
On Wed, Mar 07, 2001 at 11:09:25AM -0500, Tom Lane wrote:
"Vadim Mikheev" [EMAIL PROTECTED] writes:
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to the older checkpoint if the newer one
is unreadable. Also, a physical copy of the newest
On Wed, Mar 07, 2001 at 11:21:37AM -0500, Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
The only problem is that we would no longer have control over which
pages made it to disk. The OS would perhaps write pages as we modified
them. Not sure how important that is.
It is possible to build a logging system so that you mostly don't care
when the data blocks get written; a particular data block on disk is
considered garbage until the next checkpoint, so that you
How to know if a particular data page was modified if there is no
log record for that
How do people feel about adding a single handler to 7.1? Is it
something I can slip into the current CVS, or will it have to exist as a
patch to 7.1. Seems it would be pretty isolated unless someone sends
the signal, but it is clearly a feature addition.
Totally dead set against it
The Hermit Hacker [EMAIL PROTECTED] writes:
How do people feel about adding a single handler to 7.1?
Totally dead set against it ...
Ditto. Particularly a signal handler that performs I/O. That's going
to create all sorts of re-entrancy problems.
regards, tom lane
On Wed, Mar 07, 2001 at 12:03:41PM -0800, Mikheev, Vadim wrote:
Ian wrote:
I feel that the fact that
WAL can't help in the event of disk errors
is often overlooked.
This is true in general. But, nevertheless, WAL can be written to
protect against predictable disk errors,
"Mikheev, Vadim" [EMAIL PROTECTED] writes:
So, it's better to leave archdir in pg_control now - if we'll
decide that GUC is better place then we'll just ignore archdir
in pg_control. But if it will be better to have it in pg_control
then we'll not be able to add it there.
But what possible
Bruce Momjian [EMAIL PROTECTED] writes:
How do people feel about adding a single handler to 7.1? Is it
something I can slip into the current CVS, or will it have to exist as a
patch to 7.1. Seems it would be pretty isolated unless someone sends
the signal, but it is clearly a feature
Bruce Momjian [EMAIL PROTECTED] writes:
How do people feel about adding a single handler to 7.1? Is it
something I can slip into the current CVS, or will it have to exist as a
patch to 7.1. Seems it would be pretty isolated unless someone sends
the signal, but it is clearly a feature
At 20:48 7/03/01 +, Oliver Elphick wrote:
kovacsz wrote:
You answered that noone experienced anything like this. Here I get this
behaviour with the most simple table as well.
Is there a problem with the lists? I reveived Zoltan's message twice, and
now this one that seems to indicate
Tom Lane wrote:
"Mikheev, Vadim" [EMAIL PROTECTED] writes:
So, it's better to leave archdir in pg_control now - if we'll
decide that GUC is better place then we'll just ignore archdir
in pg_control. But if it will be better to have it in pg_control
then we'll not be able to add it
Philip Warner wrote:
At 20:48 7/03/01 +, Oliver Elphick wrote:
kovacsz wrote:
You answered that noone experienced anything like this. Here I get this
behaviour with the most simple table as well.
Is there a problem with the lists? I reveived Zoltan's message twice,
At 18:05 7/03/01 -0500, Bruce Momjian wrote:
All in all, I do not see this as an easy task that you can whip out and
then release as a 7.1 patch without extensive testing. And given that,
I'd rather see it done with what I consider the right long-term approach,
rather than a dead-end hack.
Hiroshi Inoue [EMAIL PROTECTED] writes:
But what possible reason is there for keeping it in pg_control?
AFAICS that would just mean that we'd need special code for setting it,
instead of making use of all of Peter's hard work on GUC.
I don't think it's appropriate to edit archdir by hand.
"Mikheev, Vadim" [EMAIL PROTECTED] writes:
I feel that the fact that
WAL can't help in the event of disk errors
is often overlooked.
This is true in general. But, nevertheless, WAL can be written to
protect against predictable disk errors, when possible. Failing to
Hi all,
Wouldn't another approach be to write a C function that does the
necessary work, then just call it like any other C function?
i.e. Connect to the database and issue a "select
perf_stats('/tmp/stats-2001-03-08-01.txt')" ?
Or similar?
Sure, that means another database connection which
At 11:33 8/03/01 +1100, Justin Clift wrote:
Hi all,
Wouldn't another approach be to write a C function that does the
necessary work, then just call it like any other C function?
i.e. Connect to the database and issue a "select
perf_stats('/tmp/stats-2001-03-08-01.txt')" ?
I think Bruce wants
Tom Lane wrote:
Hiroshi Inoue [EMAIL PROTECTED] writes:
But what possible reason is there for keeping it in pg_control?
AFAICS that would just mean that we'd need special code for setting it,
instead of making use of all of Peter's hard work on GUC.
I don't think it's appropriate to
Hiroshi Inoue [EMAIL PROTECTED] writes:
Tom Lane wrote:
Why not? How is this a critical parameter (more critical than, say,
fsync enable)?
I don't think 'fsync enable' is a critical parameter.
It's a dangerous parameter and it's not appropriate
as a GUC paramter either.
That's also
I like the idea of updating shared memory with the performance statistics,
current query execution information, etc., providing a function to fetch
those statistics, and perhaps providing a system view (i.e. pg_performance)
based upon such functions which can be queried by the administrator.
At 19:59 7/03/01 -0500, Mike Mascari wrote:
I like the idea of updating shared memory with the performance statistics,
current query execution information, etc., providing a function to fetch
those statistics, and perhaps providing a system view (i.e. pg_performance)
based upon such functions
Tom Lane wrote:
Hiroshi Inoue [EMAIL PROTECTED] writes:
Tom Lane wrote:
Why not? How is this a critical parameter (more critical than, say,
fsync enable)?
Does it have any meaning other than testing ? IMHO recovery system
doesn't allow any optimism and archdir is also a part of
But what can be done if fsync returns before pages flushed?
When you write out critical information, you keep earlier versions of
it. On startup, if the critical information is corrupt, you use the
earlier versions of it. This helps protect against the scenario I
mentioned: a few disk
"Mikheev, Vadim" [EMAIL PROTECTED] writes:
If fsync may return before data *actually* flushed then you may have
unlogged data page changes which breakes WAL rule and means corrupted
(inconsistent) database without ANY ABILITY TO RECOVERY TO CONSISTENT
STATE. Now please explain me how saving
* Store two past checkpoint locations, not just one, in pg_control.
On startup, we fall back to the older checkpoint if the newer one
is unreadable. Also, a physical copy of the newest
checkpoint record
And what to do if older one is unreadable too?
(Isn't it like using 2 x CRC32
"Mikheev, Vadim" [EMAIL PROTECTED] writes:
"two checkpoints" approach is not the best way to follow.
From my POV, scanning logs is much better - it doesn't require
doubling size of on-line logs and allows to *restart* if pg_control
was lost/corrupted:
If there is no pg_control or it's
I think Bruce wants per-backend data, and this approach would seem to only
get the data for the current backend.
Also, I really don't like the proposal to write files to /tmp. If we want a
perf tool, then we need to have something like 'top', which will
continuously update. With 40
At 18:05 7/03/01 -0500, Bruce Momjian wrote:
All in all, I do not see this as an easy task that you can whip out and
then release as a 7.1 patch without extensive testing. And given that,
I'd rather see it done with what I consider the right long-term approach,
rather than a dead-end
Mike Mascari's idea (er... his assembling of the other ideas) still
sounds like the Best Solution though.
:-)
+ Justin
+++
I like the idea of updating shared memory with the performance
statistics,
current query execution information, etc., providing a function to fetch
those statistics,
Yes, seems that is best. I will probably hack something up here so I
can do some testing of the app itself.
Mike Mascari's idea (er... his assembling of the other ideas) still
sounds like the Best Solution though.
:-)
+ Justin
+++
I like the idea of updating shared memory with the
If there is no pg_control or it's corrupted or points to
unexistent/corrupted checkpoint record then scan logs from
newest to oldest one till we find last valid checkpoint record
or oldest valid log record and than redo from there.
And how well will that approach work if the last
1) WAL
We have buffer manager, ok. So why not to use WAL as part of
it and don't log INSERT/UPDATE/DELETE xlog records but directly
changes into buffer pages ? When someone dirties page it has to
inform bmgr about dirty region and bmgr would formulate xlog record.
The record could be for
I wouldn't be at all surprised if you found a better approach - my
configuration above, to my mind at least, is not pretty. I hope you do find
a better approach - I know I'll be peeking at your code to see.
Yes, I have an idea and hope it works.
--
Bruce Momjian|
Was the following bug already fixed ?
Regards,
Hiroshi Inoue
Tom Lane wrote:
With current sources:
DEBUG: copy: line 629980, XLogWrite: new log file created - try to increase
WAL_FILES
DEBUG: copy: line 694890, XLogWrite: new log file created - try to increase
WAL_FILES
FATAL 2:
On Wed, Mar 07, 2001 at 04:17:09PM +0100, Olivier PRENANT wrote:
Hi Matthew,
I would love to get this stuff...
Could you send it to me or tell me where it is ?
TIA
On Tue, 6 Mar 2001, Matthew Hagerty wrote:
Greetings,
I wrote a few simple programs to log Apache access_log
73 matches
Mail list logo
