On Thu, 11 Sep 2008, James Mansion wrote:
Most spinning disks now are nearer to 100MB/s streaming. You've talked
yourself into twice that, random access!
The point I was trying to make there is that even under impossibly optimal
circumstances, you'd be hard pressed to blow out the disk's rea
On Wed, Sep 10, 2008 at 11:21 PM, James Mansion
<[EMAIL PROTECTED]> wrote:
> Greg Smith wrote:
>>
>> Average seek time: 4ms
>> Seeks/second:250
>> Data read/seek:1MB(read-ahead number goes here)
>> Total read bandwidth:250MB/s
>>
> Most spinning disks now are nearer to
Greg Smith wrote:
Average seek time: 4ms
Seeks/second:250
Data read/seek:1MB(read-ahead number goes here)
Total read bandwidth:250MB/s
Most spinning disks now are nearer to 100MB/s streaming. You've talked
yourself into twice that, random access!
James
--
Sent
On Wed, 10 Sep 2008, Scott Carey wrote:
Ok, so this is a drive level parameter that affects the data going into the
disk cache? Or does it also get pulled over the SATA/SAS link into the OS
page cache?
It's at the disk block driver level in Linux, so I believe that's all
going into the OS pa
I suspect your table has index, or checkpoint_segments is small and lead PG
do checkpoint frequently.
If the table has index or constraint, drop it and copy it ,after copy
finished, do create index or constraint again.
If checkpoint_segments is small, enlarge it.
And also you can turn fsync off wh
Great info Greg,
Some follow-up questions and information in-line:
On Wed, Sep 10, 2008 at 12:44 PM, Greg Smith <[EMAIL PROTECTED]> wrote:
> On Wed, 10 Sep 2008, Scott Carey wrote:
>
> How does that readahead tunable affect random reads or mixed random /
>> sequential situations?
>>
>
> It stil
On Wed, Sep 10, 2008 at 11:16 AM, Bill Moran
<[EMAIL PROTECTED]> wrote:
> There's a program called pgloader which supposedly is faster than copy.
> I've not used it so I can't say definitively how much faster it is.
I think you are thinking of pg_bulkloader...
--
Sent via pgsql-performance maili
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Le 10 sept. 08 à 19:16, Bill Moran a écrit :
There's a program called pgloader which supposedly is faster than
copy.
I've not used it so I can't say definitively how much faster it is.
In fact pgloader is using COPY under the hood, and doing so v
On Wed, 10 Sep 2008, Scott Carey wrote:
How does that readahead tunable affect random reads or mixed random /
sequential situations?
It still helps as long as you don't make the parameter giant. The read
cache in a typical hard drive noawadays is 8-32MB. If you're seeking a
lot, you still
I am planning my own I/O tuning exercise for a new DB and am setting up some
fio profiles. I appreciate the work and will use some of yours as a
baseline to move forward. I will be making some mixed mode fio profiles and
running our own application and database as a test as well. I'll focus on
e
Kevin Grittner escribió:
> It sounds like the advice to the OP that running VACUUM FREEZE on all
> databases to clean up the files was off base?
His responses are not explicit enough to know.
--
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company -
And potentially to tune down the number kept by modifying the appropriate
freeze parameter for 8.1 (I'm not sure of the details), so that it keeps
perhaps 20MB or so rather than 50MB.
On Wed, Sep 10, 2008 at 10:47 AM, Kevin Grittner <
[EMAIL PROTECTED]> wrote:
> >>> Tom Lane <[EMAIL PROTECTED]> w
Correction --
2 hours to read the whole disk.
1. It won't make a load take 12 hours unless we're talking a load that is in
> total, similar to the size of the disk. A slow, newer SATA drive will read
> and write at at ~50MB/sec at minimum, so the whole 320GB can be scanned at
> 3GB per minute.
A single SATA drive may not be the best performer, but:
1. It won't make a load take 12 hours unless we're talking a load that is in
total, similar to the size of the disk. A slow, newer SATA drive will read
and write at at ~50MB/sec at minimum, so the whole 320GB can be scanned at
3GB per minute
>>> Tom Lane <[EMAIL PROTECTED]> wrote:
> The expected behavior (in 8.2 and newer) is to maintain about
> autovacuum_freeze_max_age transactions' worth of clog; which is to
say
> about 50MB at the default settings.
The active database I checked, where it didn't go all the way back to
, had
"Scott Marlowe" <[EMAIL PROTECTED]> writes:
> On Wed, Sep 10, 2008 at 8:58 AM, Kevin Grittner
> <[EMAIL PROTECTED]> wrote:
>> Some of my high-volume databases don't quite go back to , but this
>> does seem to be a problem. I have confirmed that VACUUM FREEZE on all
>> but template0 (which does
On Wed, Sep 10, 2008 at 9:26 AM, Scott Carey <[EMAIL PROTECTED]> wrote:
> How does that readahead tunable affect random reads or mixed random /
> sequential situations? In many databases, the worst case scenarios aren't
> when you have a bunch of concurrent sequential scans but when there is
> eno
On Wed, Sep 10, 2008 at 8:58 AM, Kevin Grittner
<[EMAIL PROTECTED]> wrote:
>> "Matt Smiley" <[EMAIL PROTECTED]> wrote:
>> Alvaro Herrera wrote:
>>> Move the old clog files back where they were, and run VACUUM FREEZE
> in
>>> all your databases. That should clean up all the old pg_clog files,
> if
Hi,
Le mercredi 10 septembre 2008, Ryan Hansen a écrit :
> One thing I'm experiencing some trouble with is running a COPY of a
> large file (20+ million records) into a table in a reasonable amount of
> time. Currently it's taking about 12 hours to complete on a 64 bit
> server with 3 GB memory a
In response to Ryan Hansen <[EMAIL PROTECTED]>:
>
> I'm relatively new to PostgreSQL but I've been in the IT applications
> industry for a long time, mostly in the LAMP world.
>
> One thing I'm experiencing some trouble with is running a COPY of a
> large file (20+ million records) into a table
On Wednesday 10 September 2008, Ryan Hansen <[EMAIL PROTECTED]>
wrote:
>Currently it's taking about 12 hours to complete on a 64 bit
> server with 3 GB memory allocated (shared_buffer), single SATA 320 GB
> drive. I don't seem to get any improvement running the same operation
> on a dual opteron
NEVERMIND!!
I found it. Turns out there was still a constraint on the table. Once
I dropped that, the time went down to 44 minutes.
Maybe I am an idiot after all. :)
-Ryan
--- Begin Message ---
Greetings,
I'm relatively new to PostgreSQL but I've been in the IT applications
industry for
Greetings,
I'm relatively new to PostgreSQL but I've been in the IT applications
industry for a long time, mostly in the LAMP world.
One thing I'm experiencing some trouble with is running a COPY of a
large file (20+ million records) into a table in a reasonable amount of
time. Currently it
How does that readahead tunable affect random reads or mixed random /
sequential situations? In many databases, the worst case scenarios aren't
when you have a bunch of concurrent sequential scans but when there is
enough random read/write concurrently to slow the whole thing down to a
crawl. Ho
> "Matt Smiley" <[EMAIL PROTECTED]> wrote:
> Alvaro Herrera wrote:
>> Move the old clog files back where they were, and run VACUUM FREEZE
in
>> all your databases. That should clean up all the old pg_clog files,
if
>> you're really that desperate.
>
> Has anyone actually seen a CLOG file get rem
On Tue, 9 Sep 2008, Mark Wong wrote:
I've started to display the effects of changing the Linux block device
readahead buffer to the sequential read performance using fio.
Ah ha, told you that was your missing tunable. I'd really like to see the
whole table of one disk numbers re-run when you
[EMAIL PROTECTED] ("Merlin Moncure") writes:
> I think the SSD manufacturers made a tactical error chasing the
> notebook market when they should have been chasing the server
> market...
That's a very good point; I agree totally!
--
output = reverse("moc.enworbbc" "@" "enworbbc")
http://www3.symp
27 matches
Mail list logo
