On Wed, Jan 4, 2012 at 7:24 AM, Heikki Linnakangas
wrote:
> Setting a latch that's already set is fast,
> but surely it's even faster to not even try.
Agreed. I think we should SetLatch() at the first point a backend
writes a dirty buffer because the bgwriter has been inactive.
Continually waki
On Tue, Jan 3, 2012 at 11:00 PM, Jim Nasby wrote:
> On Jan 3, 2012, at 4:21 PM, Kevin Grittner wrote:
>> (2) I'm not sure about doing this in three parts, to skip the
>> checksum itself and the hole in the middle of the page. Is this
>> because the hole might not have predictable data? Why woul
On Tue, Jan 3, 2012 at 10:21 PM, Kevin Grittner
wrote:
> I'm happy with how this looks, except (as noted in a code comment)
> that there seems to be room for optimization of the calculation
> itself. Details below:
...
> (3) Rather than having PageSetVerificationInfo() use memcpy,
> followed
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Is this perhaps by design?
Oy, this doesn’t look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
ERROR: server conn crashed?
ERROR: server conn crashed?
The connection to the server was lost. Attempting reset: Succeeded.
(pgxn@localhost:59
On Tuesday, January 03, 2012 11:21:42 PM Kevin Grittner wrote:
> (1) I like the choice of Fletcher-16. It should be very good at
> detecting problems while being a lot less expensive that an official
> CRC calculation.
I wonder if CRC32c wouldn't be a good alternative given more and more cpus
(
On Wed, Jan 4, 2012 at 9:20 AM, Andres Freund wrote:
> On Tuesday, January 03, 2012 11:21:42 PM Kevin Grittner wrote:
>> (1) I like the choice of Fletcher-16. It should be very good at
>> detecting problems while being a lot less expensive that an official
>> CRC calculation.
> I wonder if CRC32
On Tue, Jan 3, 2012 at 11:28 PM, Tom Lane wrote:
> Jim Nasby writes:
>> On Jan 3, 2012, at 12:11 PM, Simon Riggs wrote:
>>> This could well be related to the fact that DropRelFileNodeBuffers()
>>> does a scan of shared_buffers, which is an O(N) approach no matter the
>>> size of the index.
>
>> C
On Wed, Jan 4, 2012 at 3:49 AM, Robert Haas wrote:
> On Fri, Dec 30, 2011 at 11:58 AM, Jeff Janes wrote:
>> On 12/29/11, Ants Aasma wrote:
>>> Unless I'm missing something, double-writes are needed for all writes,
>>> not only the first page after a checkpoint. Consider this sequence of
>>> even
2012/1/4 Simon Riggs :
> On Wed, Jan 4, 2012 at 9:20 AM, Andres Freund wrote:
>
>> I wonder if CRC32c wouldn't be a good alternative given more and more cpus
>> (its in SSE 4.2) support calculating it in silicon.
>
> We're trying to get something that fits in 16bits for this release.
> I'm guessi
I have installed RAMdisk and pointed the parameter:
#stats_temp_directory = 'B:\pg_stat_tmp'
I also tried #stats_temp_directory = 'B:/pg_stat_tmp'
But, still there is no file created in the RAM disk.
The previous stat file is touched even after the change is made. (I have
restarted the service af
On 4 Leden 2012, 13:17, pratikchirania wrote:
> I have installed RAMdisk and pointed the parameter:
>
> #stats_temp_directory = 'B:\pg_stat_tmp'
> I also tried #stats_temp_directory = 'B:/pg_stat_tmp'
>
> But, still there is no file created in the RAM disk.
> The previous stat file is touched even
Simon Riggs wrote:
> We can either
>
> (1) report all errors on a page, including errors that don't change
> PostgreSQL data. This involves checksumming long strings of zeroes,
> which the checksum algorithm can't tell apart from long strings of
> ones.
What do you mean? Each byte which goes
Excerpts from Kevin Grittner's message of mié ene 04 04:12:43 -0300 2012:
> "Kevin Grittner" wrote:
>
> > if we define sum1 and sum2 as uint I don't see how we can get an
> > overflow with 8k byes
>
> I feel the need to amend that opinion.
>
> While sum1 only needs to hold a maximum of (BLC
Robert Haas wrote:
> Jeff Janes wrote:
>> But it doesn't seem safe to me replace a page from the DW buffer
>> and then apply WAL to that replaced page which preceded the age of
>> the page in the buffer.
>
> That's what LSNs are for.
Agreed.
> If we write the page to the checkpoint buffer j
Simon, all,
* Simon Riggs (si...@2ndquadrant.com) wrote:
> (1) report all errors on a page, including errors that don't change
> PostgreSQL data. This involves checksumming long strings of zeroes,
> which the checksum algorithm can't tell apart from long strings of
> ones.
Do we actually know whe
Simon Riggs wrote:
> My focus was on getting something working first, then tuning. If
> we're agreed that we have everything apart from the tuning then we
> can proceed with tests to see which works better.
Sure. I just think you are there already except for what I got into.
FWIW, moving th
Alvaro Herrera wrote:
> We don't support BLCKSZ higher than 32k anyway
Thanks for pointing that out. Then I think we should declare sum1 to
be uint and sum2 to be uint64. We can take out the "% 255" out from
where it sits in the v2 patch, and just add something like this after
the sums are g
Pavel Stehule wrote:
> here is new version of CHECK FUNCTION patch
>
> I changed implementation of interface:
>
> * checked functions returns table instead raising exceptions - it
> necessary for describing more issues inside one function - and it
> allow to use better structured data then Except
On Tue, Jan 3, 2012 at 9:22 PM, Robert Haas wrote:
> It seems to me that if you are happy with #2, you don't really need to
> enable sync rep in the first place.
>
> At any rate, even without multiple component failures, this
> configuration makes it pretty easy to lose durability (which is the
>
On 4 January 2012 07:24, Heikki Linnakangas
wrote:
> I think SetBufferCommitInfoNeedsSave() needs the same treatment as
> MarkBufferDirty(). And it would probably be good to only set the latch if
> the buffer wasn't dirty already. Setting a latch that's already set is fast,
> but surely it's even
On Wed, Jan 4, 2012 at 1:31 PM, Stephen Frost wrote:
> Simon, all,
>
> * Simon Riggs (si...@2ndquadrant.com) wrote:
>> (1) report all errors on a page, including errors that don't change
>> PostgreSQL data. This involves checksumming long strings of zeroes,
>> which the checksum algorithm can't te
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
This may be required in case we need information about more than one
checkpoint as pg_control can hold information of only recent checkpoint. But
I could not think of a case where more th
On 04.01.2012 08:42, Amit Kapila wrote:
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
I guess it wouldn't be strictly necessary...
This may be required in case we need information about more than one
checkpoint as pg_control can
On Wed, Jan 4, 2012 at 3:56 PM, Heikki Linnakangas
wrote:
> On 04.01.2012 08:42, Amit Kapila wrote:
>>
>> Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
>> same information in pg_control file?
>
>
> I guess it wouldn't be strictly necessary...
Apart from replicated stan
Andrew Dunstan writes:
> On 01/04/2012 12:47 AM, David E. Wheeler wrote:
>> Oy, this doesnt look good:
>> $ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
>> The connection to the server was lost. Attempting reset: Succeeded.
> Try
> elog(NOTICE, "$^V")
Isn't this a Perl bug? It seems to be c
On Jan 4, 2012, at 12:44 AM, Andrew Dunstan wrote:
> Try
>
> elog(NOTICE, "$^V")
Yeah, I used
elog(NOTICE, version->new($^V))
Which was fine. But still, it should’t segfault.
David
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscript
On Jan 4, 2012, at 8:15 AM, Tom Lane wrote:
> Isn't this a Perl bug? It seems to be crashing in SvPVutf8, which
> means that either Perl passed something that's not an SV to a function
> declared to accept SVs, or that SvPVutf8 fails on some SVs. Either
> way, Perl is failing to satisfy the POLA
On 01/04/2012 11:15 AM, Tom Lane wrote:
Andrew Dunstan writes:
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Oy, this doesn’t look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
The connection to the server was lost. Attempting reset: Succeeded.
Try
elog(NOTICE, "$^V")
Isn't th
Andrew Dunstan writes:
> The docs (perldoc perlvar) seem to suggest $^V isn't an SV (i.e. a
> scalar) but some other sort of animal:
Yeah, it's a version object, but I'd have thought that SvPV and friends
would automatically stringify such an object. Otherwise, practically
any kind of perl exte
On Wed, Jan 4, 2012 at 11:02 AM, Simon Riggs wrote:
> On Wed, Jan 4, 2012 at 3:56 PM, Heikki Linnakangas
> wrote:
>> On 04.01.2012 08:42, Amit Kapila wrote:
>>>
>>> Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
>>> same information in pg_control file?
>>
>>
>> I guess
On Wed, Jan 4, 2012 at 9:28 AM, Aidan Van Dyk wrote:
> I'ld love a "hook" script that was run if sync-rep state ever changed
> (heck, I'ld even like it if it just choose a new sync standby).
That seems useful. I don't think the current code quite knows its own
state; we seem to have each walsend
In http://archives.postgresql.org/pgsql-admin/2012-01/msg8.php
it's pointed out that recent versions of pg_restore fall over on
archives made with -Fc --inserts (or --column-inserts), but only when
restoring direct to database; if you ask for text output it's perfectly
fine. Investigation show
On Wed, Jan 4, 2012 at 8:31 AM, Kevin Grittner
wrote:
>> When we reach a restartpoint, we fsync everything down to disk and
>> then nuke the double-write buffer.
>
> I think we add to the double-write buffer as we write pages from the
> buffer to disk. I don't think it makes sense to do potential
On Fri, Dec 30, 2011 at 4:25 AM, Marko Kreen wrote:
>> I have the (hopefully wrong) impression that you're missing the fact
>> that it already exists, at least in 9.0.
>
> You are right, I missed it. For quite obvious reason:
>
> $ grep -ri aclexplode doc/
> $
>
> Is there a good reason why it'
Robert Haas wrote:
> we only fsync() at end-of-checkpoint. So we'd have to think about
> what to fsync, and how often, to keep the double-write buffer to a
> manageable size.
I think this is the big tuning challenge with this technology.
> I can't help thinking that any extra fsyncs are pre
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Jan 3, 2012 at 7:39 PM, Peter Geoghegan
>> wrote:
>>> Yes, I know that these only appeared in GCC 4.6+ and as such are a
>>> relatively recent phenomenon, but there has been some effort to
>>> eliminate them, and i
On 4 January 2012 18:44, Robert Haas wrote:
> On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
>> I'm also less than thrilled with the idea that whatever the gcc boys
>> decide to make a warning tomorrow will automatically become a MUST FIX
>> NOW for us.
>
> I'm not thrilled about that either. E
On Dec 23, 2011, at 2:23 PM, Kevin Grittner wrote:
> Jeff Janes wrote:
>
>> Could we get some major OLTP users to post their CLOG for
>> analysis? I wouldn't think there would be much
>> security/propietary issues with CLOG data.
>
> FWIW, I got the raw numbers to do my quick check using this
On 04.01.2012 20:44, Robert Haas wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
Robert Haas writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter Geoghegan wrote:
Yes, I know that these only appeared in GCC 4.6+ and as such are a
relatively recent phenomenon, but there has been some effort
On 01/04/2012 02:35 PM, Heikki Linnakangas wrote:
On 04.01.2012 20:44, Robert Haas wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
Robert Haas writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter
Geoghegan wrote:
Yes, I know that these only appeared in GCC 4.6+ and as such are a
relative
Jim Nasby wrote:
> Here's output from our largest OLTP system... not sure exactly how
> to interpret it, so I'm just providing the raw data. This spans
> almost exactly 1 month.
Those number wind up meaning that 18% of the 256-byte blocks (1024
transactions each) were all commits. Yikes. Tha
On Wed, Jan 4, 2012 at 1:32 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> we only fsync() at end-of-checkpoint. So we'd have to think about
>> what to fsync, and how often, to keep the double-write buffer to a
>> manageable size.
>
> I think this is the big tuning challenge with this technol
On 01/04/2012 12:56 PM, Tom Lane wrote:
Andrew Dunstan writes:
The docs (perldoc perlvar) seem to suggest $^V isn't an SV (i.e. a
scalar) but some other sort of animal:
Yeah, it's a version object, but I'd have thought that SvPV and friends
would automatically stringify such an object. Othe
Git master can already avoid rewriting the table for column type changes like
varchar(10) -> varchar(20). However, if the column in question is on either
side of a FK relationship, we always revalidate the foreign key. Concretely,
I wanted these no-rewrite type changes to also assume FK validity:
On Wed, Jan 4, 2012 at 13:13, Andrew Dunstan wrote:
>
>
> On 01/04/2012 12:56 PM, Tom Lane wrote:
>> I looked at that last night but it appeared that SvOK would be perfectly
>> happy. (Didn't actually try it, though, I was just eyeballing the flags
>> in gdb.)
>
> I tested it and you're right,
On Wed, Jan 4, 2012 at 3:02 PM, Kevin Grittner
wrote:
> Jim Nasby wrote:
>> Here's output from our largest OLTP system... not sure exactly how
>> to interpret it, so I'm just providing the raw data. This spans
>> almost exactly 1 month.
>
> Those number wind up meaning that 18% of the 256-byte bl
On Tue, Jan 3, 2012 at 9:38 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Jan 3, 2012 at 6:25 PM, Peter Eisentraut wrote:
>>> [ reasons ]
>
>> I agree with these reasons. We don't get charged $0.50 per GUC, so
>> there's no particular reason to contort things to have fewer of them.
>
> W
Further notes, from Andrew (RhodiumToad) on IRC about the cause of this crasher:
[12:03pm] RhodiumToad: what happens is this
[12:04pm] RhodiumToad: postquel_start know this statement doesn't
return the result, so it supplies None_Receiver as the dest-receiver
for the query
[12:04pm] RhodiumToad: h
I neglected to commit after revising the text of a few comments; use this
version instead. Thanks.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3b52415..9eba8e8 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
**
Robert Haas wrote:
> I think it may also be tricky to make sure that a backend that
> needs to write a dirty buffer doesn't end up having to wait for a
> double-write to be fsync'd.
This and other parts of your post seem to ignore the BBU write-back
cache. Multiple fsyncs of a single page can
Andrew Dunstan writes:
> On 01/04/2012 12:56 PM, Tom Lane wrote:
>> I looked at that last night but it appeared that SvOK would be perfectly
>> happy. (Didn't actually try it, though, I was just eyeballing the flags
>> in gdb.)
> I tested it and you're right, it doesn't help. I don't see what el
Robert Haas wrote:
> 2. The CLOG code isn't designed to manage a large number of
> buffers, so adding more might cause a performance regression on
> small systems.
>
> On Nate Boley's 32-core system, running pgbench at scale factor
> 100, the optimal number of buffers seems to be around 32. I'
Robert Haas writes:
> But, the OP makes me wonder: why can a standby only perform a
> restartpoint where the master performed a checkpoint? It seems like a
> standby ought to be able to create a restartpoint anywhere, just by
> writing everything, flushing it to disk, and update pg_control.
Perh
Paul Ramsey writes:
> Further notes, from Andrew (RhodiumToad) on IRC about the cause of this
> crasher:
> [12:31pm] RhodiumToad: there's no trivial fix
IMO the main bug here is that functions.c isn't expecting qd->dest to be
overwritten, so we could work around it by keeping a separate private
On Wed, Jan 4, 2012 at 3:51 PM, Kevin Grittner
wrote:
>> If double writes aren't going to give us anything "for free",
>> maybe that's not the right place to be focusing our
>> efforts...
>
> I'm not sure why it's not enough that they improve performance over
> the alternative. Making some other
On Wed, Jan 4, 2012 at 4:06 PM, Tom Lane wrote:
> Robert Haas writes:
>> But, the OP makes me wonder: why can a standby only perform a
>> restartpoint where the master performed a checkpoint? It seems like a
>> standby ought to be able to create a restartpoint anywhere, just by
>> writing everyt
On Jan 4, 2012, at 2:02 PM, Kevin Grittner wrote:
> Jim Nasby wrote:
>> Here's output from our largest OLTP system... not sure exactly how
>> to interpret it, so I'm just providing the raw data. This spans
>> almost exactly 1 month.
>
> Those number wind up meaning that 18% of the 256-byte blocks
On Wed, Jan 4, 2012 at 4:02 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>
>> 2. The CLOG code isn't designed to manage a large number of
>> buffers, so adding more might cause a performance regression on
>> small systems.
>>
>> On Nate Boley's 32-core system, running pgbench at scale factor
>>
On 01/04/2012 03:56 PM, Tom Lane wrote:
Andrew Dunstan writes:
On 01/04/2012 12:56 PM, Tom Lane wrote:
I looked at that last night but it appeared that SvOK would be perfectly
happy. (Didn't actually try it, though, I was just eyeballing the flags
in gdb.)
I tested it and you're right, it
Andrew Dunstan writes:
> On 01/04/2012 03:56 PM, Tom Lane wrote:
>> I think what's being passed *is* an SV --- at least, the contents look
>> reasonable in gdb --- but for some reason SvPVutf8 isn't coping with
>> this particular kind of SV. Googling suggests that SvPVutf8 used to
>> fail on READ
On 01/04/2012 01:13 PM, Tom Lane wrote:
In http://archives.postgresql.org/pgsql-admin/2012-01/msg8.php
it's pointed out that recent versions of pg_restore fall over on
archives made with -Fc --inserts (or --column-inserts), but only when
restoring direct to database; if you ask for text out
On 01/04/2012 05:05 PM, Tom Lane wrote:
Well, the crash is apparently solved by the following, which your
investigation suggested to me:
- cmsg = sv2cstr(msg);
+ cmsg = sv2cstr(newSVsv(msg));
That's kinda grotty ... and leaky ...
Of course it is. It wasn't meant as a solution
On Dec 20, 2011, at 11:29 PM, Tom Lane wrote:
> Robert Haas writes:
>> So, what do we do about this? The obvious answer is "increase
>> NUM_CLOG_BUFFERS", and I'm not sure that's a bad idea.
>
> As you say, that's likely to hurt people running in small shared
> memory. I too have thought about
On Dec 19, 2011, at 12:31 PM, Simon Riggs wrote:
> On Sun, Dec 18, 2011 at 4:22 PM, Jim Nasby wrote:
>> On Dec 18, 2011, at 2:28 AM, Gianni Ciolli wrote:
>>> I have written some notes about autonomous subtransactions, which have
>>> already been touched (at least) in two separate threads; please f
On Jan 4, 2012, at 2:48 PM, Andrew Dunstan wrote:
>> That's kinda grotty ... and leaky ...
>>
>
> Of course it is. It wasn't meant as a solution but as validation of your
> suspicions about the nature of the problem. (The leakiness could be solved,
> though.)
From #p5p on irc.perl.org:
[10:5
Andrew Dunstan writes:
> On 01/04/2012 01:13 PM, Tom Lane wrote:
>> Not entirely sure what to do about this. We could consider reverting
>> the aforesaid patch and trying to find another way of fixing that code's
>> failure to cope with standard-conforming strings, but I'm not sure that
>> there'
I wrote:
> Perhaps the intoRel stuff should be
> saving/restoring the original destreceiver instead of just blindly
> overwriting it.
I concluded that was the best fix, and have committed it.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgre
I agree with Pavel also. Putting these constants in the pg_catalog isn't
the cleanest solution.
Though one can make its own little lib in python, perl, whatever, to store
these constants, it would be better if through the compilation, these C
constants were copied in a way for PL/*.
I can't reall
On 01/04/2012 06:20 PM, Tom Lane wrote:
But we'd have to deal with
standard-conforming strings some way. The idea I had about that, since
we have an open database connection at hand, is to check the connected
backend's standard_conforming_strings state via PQparameterStatus. If
it doesn't hav
On Wed, Jan 04, 2012 at 04:58:08PM -0600, Jim Nasby wrote:
> Except AFAIR Oracle uses the term to indicate something that is
> happening *outside* of your current transaction, which is definitely
> not what the proposal is talking about.
That feature is commonly translated in PostgreSQL to a dblin
Andrew Dunstan writes:
> On 01/04/2012 06:20 PM, Tom Lane wrote:
>> But we'd have to deal with
>> standard-conforming strings some way. The idea I had about that, since
>> we have an open database connection at hand, is to check the connected
>> backend's standard_conforming_strings state via PQp
On 01/04/2012 06:15 PM, David E. Wheeler wrote:
[11:12pm]TonyC:theory: using sv_mortalcopy() instead of newSVsv()
should prevent the leak in that workaround, assuming there's no
FREETMPS between the call and use of the return value
That's the solution to leakiness I had in mind.
Tom said:
Andrew Dunstan writes:
> Tom said:
>> 2. A slightly cleaner fix for this should be to duplicate the SV and
>> then release the copy around the SvPVutf8 call, only if it's readonly.
>> "Fixing" it in do_util_elog is entirely the wrong thing.
> How do we tell if it's readonly?
SvREADONLY(sv) macro
Thanks, i missed that.
After doing these changes, following is the observation:
1. The size of the pgstat file is 86KB. Last edited was when i moved the
file location to RAMdisk.
2. The issue persists. I am seeing continuous logs:
2012-01-05 00:00:06 JST WARNING: pgstat wait timeout
2012-01-05
74 matches
Mail list logo
