Re: [PERFORM] Planning a new server - help needed

2008-03-29 Thread Craig Ringer
Greg Smith wrote:

>> Gigabyte should revamp their i-RAM to use ECC RAM of a larger
>> capacity... and longer lasting battery backup...
> 
> I saw a rumor somewhere that they were close to having a new version of
> that using DDR2 ready, which would make it pretty easy to have 8GB on
> there.

I'm hoping - probably in vain - that they'll also include a CF/SD slot
or some onboard flash so it can dump its contents to flash using the
battery backup.

For anybody wondering what the devices in question are, see:

http://www.anandtech.com/storage/showdoc.aspx?i=2480
http://www.gigabyte.com.tw/Products/Storage/Products_Overview.aspx?ProductID=2180

--
Craig Ringer

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Planning a new server - help needed

2008-03-29 Thread Greg Smith

On Sat, 29 Mar 2008, PFC wrote:


Why do you claim that 'More platters also means slower seeks
and generally slower performance.'?
	More platters -> more heads -> heavier head assembly -> slower seek 
time


I recall seeing many designs with more platters that have slower seek 
times in benchmarks, and I always presumed this was the reason.  That's 
the basis for that comment.  I'll disclaim that section a bit.


Actually, now that 8.3 can sync to disk every second instead of at every 
commit, I wonder, did someone do some enlightening benchmarks ?


I've seen some really heavy workloads where using async commit helped 
group commits in a larger batches usefully, but I personally haven't found 
it to be all that useful if you're already got a caching controller to 
accelerate writes on the kinds of hardware most people have.  It's a great 
solution for situations without a usable write cache though.


Also, there is a thing called write barriers, which supposedly could be 
used to implement fsync-like behaviour without the penalty, if the disk, 
the OS, the controller, and the filesystem support it (that's a lot of 
ifs)...


The database can't use fsync-like behavior for the things it calls fsync 
for; it needs the full semantics.  You're either doing the full operation, 
or you're cheating and it doesn't do what it's supposed to.  Write 
barriers aren't any improvement over a good direct I/O sync write setup 
for the WAL.  There may be some limited value to that approach for the 
database writes at checkpoint time, but again there's a real fsync coming 
at the end of that and it's not satisfied until everything is on disk (or 
in a good disk controller cache).


Gigabyte should revamp their i-RAM to use ECC RAM of a larger 
capacity... and longer lasting battery backup...


I saw a rumor somewhere that they were close to having a new version of 
that using DDR2 ready, which would make it pretty easy to have 8GB on 
there.


I wonder, how many write cycles those Flash drives can take before 
reliability becomes a problem...


The newer SSD drives with good write leveling should last at least as long 
as you'd expect a mechanical drive to, even in a WAL application.  Lesser 
grades of flash used as disk could be a problem though.


--
* Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Planning a new server - help needed

2008-03-29 Thread Peter Eisentraut
Laszlo Nagy wrote:
> Question 1. We are going to use PostgreSQL 3.1 with FreeBSD. The pg docs
> say that it is better to use FreeBSD because it can alter the I/O
> priority of processes dynamically.

Where does it say that?

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Planning a new server - help needed

2008-03-29 Thread James Mansion

PFC wrote:



Why do you claim that 'More platters also means slower seeks
and generally slower performance.'?


More platters -> more heads -> heavier head assembly -> slower 
seek time

Note sure I've sen a lot of evidence of that in drive specifications!

Gigabyte should revamp their i-RAM to use ECC RAM of a larger 
capacity... and longer lasting battery backup...
You would think a decent capacitor or rechargable button battery would 
be enough to dump it to a flash memory.

No problem with flash wear then.
I wonder, how many write cycles those Flash drives can take before 
reliability becomes a problem...
Hard to get data isn't it? I believe its hundreds of thousands to 
millions now. Now each record in most OLTP
tables is rewritten a few times unless its stuff that can go into temp 
tables etc, which should be elsewhere.

Index pages clearly get rewritten often.

I suspect a mix of storage technologies will be handy for some time yet 
- WAL on disk, and temp tables on

disk with no synchronous fsync requirement.

I think life is about to get interesting in DBMS storage.  All good for 
us users.


James


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Planning a new server - help needed

2008-03-29 Thread PFC



Why do you claim that 'More platters also means slower seeks
and generally slower performance.'?


More platters -> more heads -> heavier head assembly -> slower seek time
But..
	More platters -> higher density -> less seek distance (in mm of head  
movement) -> faster seek time


	As usual, no clear-cut case, a real-life test would tell more interesting  
things.



I'm not entirely sure why the extra platters should really count
as more moving parts since I think the platter assembly and
head assembly are both single parts in effect, albeit they will
be more massive with more platters. I'm not sure how much
extra bearing friction that will mean, but it is reasonable that
some extra energy is going to be needed.


	Since the bearings are only on one side of the axle (not both), a heavier  
platter assembly would put more stress on the bearing if the disk is  
subject to vibrations (like, all those RAID disks seeking together) which  
would perhaps shorten its life. Everything with conditionals of course ;)
	I remember reading a paper on vibration from many RAID disks somewhere a  
year or so ago, vibration from other disks seeking at the exact same time  
and in the same direction would cause resonances in the housing chassis  
and disturb the heads of disks, slightly worsening seek times and  
reliability. But, on the other hand, the 7 disks raided in my home storage  
server never complained, even though the $30 computer case vibrates all  
over the place when they seek. Perhaps if they were subject to 24/7 heavy  
torture, a heavier/better damped chassis would be a good investment.



It may be worth considering an alternative approach. I suspect
that a god RAID1 or RAID1+0 is worthwhile for WAL, but


	Actually, now that 8.3 can sync to disk every second instead of at every  
commit, I wonder, did someone do some enlightening benchmarks ? I remember  
benchmarking 8.2 on a forum style load and using a separate disk for WAL  
(SATA, write cache off) made a huge difference (as expected) versus one  
disk for everything (SATA, and write cache off). Postgres beat the crap  
out of MyISAM, lol.
	Seems like Postgres is one of the rare apps which gets faster and meaner  
with every release, instead of getting slower and more bloated like  
everyone else.


	Also, there is a thing called write barriers, which supposedly could be  
used to implement fsync-like behaviour without the penalty, if the disk,  
the OS, the controller, and the filesystem support it (that's a lot of  
ifs)...



I haven't done this, so YMMV.  But the prices are getting
interesting for OLTP where most disks are massively
oversized. The latest Samsung and SanDisk are expensive
in the UK but the Transcend 16GB TS16GSSD25S-S  SATA
is about $300 equiv - it can do 'only' 'up to' 28MB/s write and


	Gigabyte should revamp their i-RAM to use ECC RAM of a larger capacity...  
and longer lasting battery backup...
	I wonder, how many write cycles those Flash drives can take before  
reliability becomes a problem...





--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Planning a new server - help needed

2008-03-29 Thread James Mansion

Greg Smith wrote:
As for SCSI vs. SATA, I collected up the usual arguments on both sides 
at http://www.postgresqldocs.org/index.php/SCSI_vs._IDE/SATA_Disks



Why do you claim that 'More platters also means slower seeks
and generally slower performance.'?

On the face of it, it should mean that the number of track step
operations is reduced, even if the drive doesn't buffer a slice
of tracks aross all platters (which would help if it did).

I'm not entirely sure why the extra platters should really count
as more moving parts since I think the platter assembly and
head assembly are both single parts in effect, albeit they will
be more massive with more platters. I'm not sure how much
extra bearing friction that will mean, but it is reasonable that
some extra energy is going to be needed.

Recent figures I've seen suggest that the increased storage
density per platter, plus the extra number of platters, means
that the streaming speed of good 7200rpm SATA drives is
very close to that of good 15000rpm SAS drives - and you
can choose which bit of the disk to use to reduce seek time and
maximise transfer rate with the oversize drives. You can
get about 100MB/s out of both technologies, streaming.

It may be worth considering an alternative approach. I suspect
that a god RAID1 or RAID1+0 is worthwhile for WAL, but
you might consider a RAID1 of a pair of SSDs for data. They
will use a lot of your budget, but the seek time is negligible so the
effective usable performance is higher than you get with
spinning disks - so you might trade a fancy controller with
battery-backed cache for straight SSD.

I haven't done this, so YMMV.  But the prices are getting
interesting for OLTP where most disks are massively
oversized. The latest Samsung and SanDisk are expensive
in the UK but the Transcend 16GB TS16GSSD25S-S  SATA
is about $300 equiv - it can do 'only' 'up to' 28MB/s write and
you wouldn't want to put WAL on one, but sustaining
15-20MB/s through random access to a real disk isn't
trivial. If average access is 10ms, and you write 100MB/s
streaming, then you have to ask yourself if you going to do
80 or more seeks a second.

James

James





--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance