Good question. I'm in the process of completing more exhaustive tests with
the various disk i/o schedulers.
Basic findings so far: it depends on what type of concurrency is going on.
Deadline has the best performance over a range of readahead values compared
to cfq or anticipatory with concurren
On Thu, 11 Sep 2008, Scott Carey wrote:
Preliminary summary:
readahead | 8 conc read rate | 1 conc read rate
49152 | 311 | 314
16384 | 312 | 312
12288 | 304 | 309
8192 | 292 |
4096 | 264 |
2048 | 211 |
1024 | 162 | 302
512 | 108 |
256 | 81 | 300
8
On Fri, 12 Sep 2008, James Mansion wrote:
Scott Carey wrote:
Consumer drives will often read-ahead much more than server drives
optimized for i/o per second.
...
The Linux readahead setting is _definitely_ in the kernel, definitely uses
and fills the page cache, and from what I can gather, si
Scott Carey wrote:
Consumer drives will often read-ahead much more than server drives
optimized for i/o per second.
...
The Linux readahead setting is _definitely_ in the kernel, definitely
uses and fills the page cache, and from what I can gather, simply
issues extra I/O's to the hardware bey
On Thu, 11 Sep 2008, Alan Hodgson wrote:
LVM plus online resizable filesystems really makes multiple partitions
manageable.
I've seen so many reports blaming Linux's LVM for performance issues that
its managability benefits don't seem too compelling.
--
* Greg Smith [EMAIL PROTECTED] http:/
On Thu, Sep 11, 2008 at 4:33 PM, <[EMAIL PROTECTED]> wrote:
> On Thu, 11 Sep 2008, Scott Marlowe wrote:
>
>> On Thu, Sep 11, 2008 at 3:36 PM, <[EMAIL PROTECTED]> wrote:
>>>
>>> by even if it didn't, most modern drives read the entire cylinder into
>>> their
>>> buffer so any additional requests t
I also thought that LVM is unsafe for WAL logs and file system journals with
disk write cache -- it doesn't flush the disk write caches correctly and
build write barriers.
As pointed out here:
http://groups.google.com/group/pgsql.performance/browse_thread/thread/9dc43991c1887129
by Greg Smith
http
On Thu, 11 Sep 2008, Alan Hodgson wrote:
On Thursday 11 September 2008, [EMAIL PROTECTED] wrote:
while I agree with you in theory, in practice I've seen multiple
partitions cause far more problems than they have prevented (due to the
partitions ending up not being large enough and having to be
On Thursday 11 September 2008, [EMAIL PROTECTED] wrote:
> while I agree with you in theory, in practice I've seen multiple
> partitions cause far more problems than they have prevented (due to the
> partitions ending up not being large enough and having to be resized
> after they fill up, etc) so I
On Thu, 11 Sep 2008, Scott Marlowe wrote:
On Thu, Sep 11, 2008 at 3:36 PM, <[EMAIL PROTECTED]> wrote:
by even if it didn't, most modern drives read the entire cylinder into their
buffer so any additional requests to the drive will be satisfied from this
buffer and not have to wait for the disk
On Thu, Sep 11, 2008 at 3:36 PM, <[EMAIL PROTECTED]> wrote:
> On Thu, 11 Sep 2008, Scott Carey wrote:
>
>> Drives have their own read-ahead in the firmware. Many can keep track of
>> 2
>> or 4 concurrent file accesses. A few can keep track of more. This also
>> plays in with the NCQ or SCSI com
On Thu, 11 Sep 2008, Scott Carey wrote:
Drives have their own read-ahead in the firmware. Many can keep track of 2
or 4 concurrent file accesses. A few can keep track of more. This also
plays in with the NCQ or SCSI command queuing implementation.
Consumer drives will often read-ahead much m
Sorry, I forgot to mention the Linux kernel version I'm using, etc:
2.6.18-92.1.10.el5 #1 SMP x86_64
CentOS 5.2.
The "adaptive" read-ahead, as well as other enhancements in the kernel, are
taking place or coming soon in the most recent stuff. Some distributions
offer the adaptive read-ahead as a
Drives have their own read-ahead in the firmware. Many can keep track of 2
or 4 concurrent file accesses. A few can keep track of more. This also
plays in with the NCQ or SCSI command queuing implementation.
Consumer drives will often read-ahead much more than server drives optimized
for i/o pe
Greg Smith wrote:
The point I was trying to make there is that even under impossibly
optimal circumstances, you'd be hard pressed to blow out the disk's
read cache with seek-dominated data even if you read a lot at each
seek point. That idea didn't make it from my head into writing very
well
Hmm, I would expect this tunable to potentially be rather file system
dependent, and potentially raid controller dependant. The test was using
ext2, perhaps the others automatically prefetch or read ahead? Does it
vary by RAID controller?
Well I went and found out, using ext3 and xfs. I have a
On Thu, 11 Sep 2008, James Mansion wrote:
Most spinning disks now are nearer to 100MB/s streaming. You've talked
yourself into twice that, random access!
The point I was trying to make there is that even under impossibly optimal
circumstances, you'd be hard pressed to blow out the disk's rea
On Wed, Sep 10, 2008 at 11:21 PM, James Mansion
<[EMAIL PROTECTED]> wrote:
> Greg Smith wrote:
>>
>> Average seek time: 4ms
>> Seeks/second:250
>> Data read/seek:1MB(read-ahead number goes here)
>> Total read bandwidth:250MB/s
>>
> Most spinning disks now are nearer to
Greg Smith wrote:
Average seek time: 4ms
Seeks/second:250
Data read/seek:1MB(read-ahead number goes here)
Total read bandwidth:250MB/s
Most spinning disks now are nearer to 100MB/s streaming. You've talked
yourself into twice that, random access!
James
--
Sent
On Wed, 10 Sep 2008, Scott Carey wrote:
Ok, so this is a drive level parameter that affects the data going into the
disk cache? Or does it also get pulled over the SATA/SAS link into the OS
page cache?
It's at the disk block driver level in Linux, so I believe that's all
going into the OS pa
Great info Greg,
Some follow-up questions and information in-line:
On Wed, Sep 10, 2008 at 12:44 PM, Greg Smith <[EMAIL PROTECTED]> wrote:
> On Wed, 10 Sep 2008, Scott Carey wrote:
>
> How does that readahead tunable affect random reads or mixed random /
>> sequential situations?
>>
>
> It stil
On Wed, 10 Sep 2008, Scott Carey wrote:
How does that readahead tunable affect random reads or mixed random /
sequential situations?
It still helps as long as you don't make the parameter giant. The read
cache in a typical hard drive noawadays is 8-32MB. If you're seeking a
lot, you still
I am planning my own I/O tuning exercise for a new DB and am setting up some
fio profiles. I appreciate the work and will use some of yours as a
baseline to move forward. I will be making some mixed mode fio profiles and
running our own application and database as a test as well. I'll focus on
e
On Wed, Sep 10, 2008 at 9:26 AM, Scott Carey <[EMAIL PROTECTED]> wrote:
> How does that readahead tunable affect random reads or mixed random /
> sequential situations? In many databases, the worst case scenarios aren't
> when you have a bunch of concurrent sequential scans but when there is
> eno
How does that readahead tunable affect random reads or mixed random /
sequential situations? In many databases, the worst case scenarios aren't
when you have a bunch of concurrent sequential scans but when there is
enough random read/write concurrently to slow the whole thing down to a
crawl. Ho
On Tue, 9 Sep 2008, Mark Wong wrote:
I've started to display the effects of changing the Linux block device
readahead buffer to the sequential read performance using fio.
Ah ha, told you that was your missing tunable. I'd really like to see the
whole table of one disk numbers re-run when you
Hi all,
I've started to display the effects of changing the Linux block device
readahead buffer to the sequential read performance using fio. There
are lots of raw data buried in the page, but this is what I've
distilled thus far. Please have a look and let me know what you
think:
http://wiki.p
27 matches
Mail list logo
