Re[2]: wikipedia down, slashdot covering, mysql mentioned

[DebugasRu]

> At least this mailing list has progressed beyond "Why didn't they have
> a UPS?", I suppose. :-)
Yes that was my primary intend :)
I'd really like people to care more about their data safety and more
awareness about the problems involved.

>> That didn't get rid of the race condition, it just moved it. So now you
>> write the transaction, and you read it off, and then you write that it
>> was OK -- but what if that write gets interrupted?
I simply will not delete the old original data (data state of my DB before
transaction was started) that i keep just in case i will need to roll back.

>> 
>> So you write the transaction, and you read it off, and then you write
>> that it was OK, and then you read that off, and then you write that the
>> OK was OK. No, that could get interrupted to...

I don't see why i need to care to write that "Ok was Ok". I just go
and check if "it was Ok with original data". I don't see why i would
need to have additional log file on data log file.


>> Meanwhile, it turns out that you've been reading back from cache instead
>> of platters all along. Or perhaps it turns out that the battery-backed
>> cache in the disk controller has a dead battery, or simply doesn't work.
That's completely different problem - it is a problem of how the
interface between hard drive and OS software is defined. Is there a
function in this interface that will return physically written data
and not cashed one. At this point no DBMS will save you indeed.

>> I think you've confused an RDBMS with a system administrator. Next
>> thing you know you'll be complaining that MySQL isn't ACID-compliant
>> because it can't survive a fire.
if DB is resident across several servers there should be protection
from one server permanently going off line (kind of fire isn't it ?)

GW> Exactly.  No ACID database can ensure integerity in such a situation.
GW> Postgres, Oracle, or any other transactional DB would have suffered the 
GW> same fate in these two cases (LiveJournal, Wiki).
I can't talk about how Oracle deals with such problems because i have no access 
to its source code
that's where mysql has a huge advantage! :)




-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Greg Whalin]

Rich Lafferty wrote:
On Thu, Feb 24, 2005 at 09:10:32AM -0500, Greg Whalin <[EMAIL PROTECTED]> wrote:
Exactly.  No ACID database can ensure integerity in such a situation. 
Postgres, Oracle, or any other transactional DB would have suffered the 
same fate in these two cases (LiveJournal, Wiki).

FWIW, my understanding of LiveJournal's integrity problem after the
power outage involved tables they weren't yet able to migrate off of
MyISAM, and getting replication content in all directions. The
stuff migrated to InnoDB already came up fine. Their replication
setup is a bit complex -- see
http://www.danga.com/words/2004_mysqlcon/mysql-slides.pdf for an
overview.
(My kit at the same facility was hit too, and recovered correctly, for
what it's worth. :-)
At least this mailing list has progressed beyond "Why didn't they have
a UPS?", I suppose. :-)
  -Rich
They had most of their dbs transitioned to innodb, but even some of 
those came up corrupted due to write caching being enabled on individual 
drives.  Their myisam tables simply needed indexes rebuilt (which is a 
pain in the butt and takes forever).  Their complete story: 
http://www.livejournal.com/community/lj_dev/670215.html

Needless to say, between LJ and Wiki, I am fairly paranoid about db 
corruption now (though our writes are considerably less than either of 
those two places as we are 99% reads ... our reads are considerably less 
than either of those sites as well :) )

Greg
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Rich Lafferty]

On Thu, Feb 24, 2005 at 09:10:32AM -0500, Greg Whalin <[EMAIL PROTECTED]> wrote:
> 
> Exactly.  No ACID database can ensure integerity in such a situation. 
> Postgres, Oracle, or any other transactional DB would have suffered the 
> same fate in these two cases (LiveJournal, Wiki).

FWIW, my understanding of LiveJournal's integrity problem after the
power outage involved tables they weren't yet able to migrate off of
MyISAM, and getting replication content in all directions. The
stuff migrated to InnoDB already came up fine. Their replication
setup is a bit complex -- see
http://www.danga.com/words/2004_mysqlcon/mysql-slides.pdf for an
overview.

(My kit at the same facility was hit too, and recovered correctly, for
what it's worth. :-)

At least this mailing list has progressed beyond "Why didn't they have
a UPS?", I suppose. :-)

  -Rich

-- 
Rich Lafferty --+---
 Ottawa, Ontario, Canada|  Save the Pacific Northwest Tree Octopus!
 http://www.lafferty.ca/|http://zapatopi.net/treeoctopus.html
[EMAIL PROTECTED] ---+---

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Greg Whalin]

Rich Lafferty wrote:
On Thu, Feb 24, 2005 at 02:38:46PM +0200, DebugasRu <[EMAIL PROTECTED]> wrote:
Probably fsync() had failed to flush some part of a 16 kB page to
disk.
so what ? one of trade-offs would be to re-read the data from the disk
and compare it with what it should be (another copy on the disk) and
only after that fix the transaction,

That didn't get rid of the race condition, it just moved it. So now you
write the transaction, and you read it off, and then you write that it
was OK -- but what if that write gets interrupted?
So you write the transaction, and you read it off, and then you write
that it was OK, and then you read that off, and then you write that the
OK was OK. No, that could get interrupted to...
Meanwhile, it turns out that you've been reading back from cache instead
of platters all along. Or perhaps it turns out that the battery-backed
cache in the disk controller has a dead battery, or simply doesn't work.
If the disk promises data is written but it is not then there is nothing
MySQL can do to *tell* that something is amiss. 

I think you've confused an RDBMS with a system administrator. Next
thing you know you'll be complaining that MySQL isn't ACID-compliant
because it can't survive a fire.
  -Rich
Exactly.  No ACID database can ensure integerity in such a situation. 
Postgres, Oracle, or any other transactional DB would have suffered the 
same fate in these two cases (LiveJournal, Wiki).

Greg
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Rich Lafferty]

On Thu, Feb 24, 2005 at 02:38:46PM +0200, DebugasRu <[EMAIL PROTECTED]> wrote:
> 
> > Probably fsync() had failed to flush some part of a 16 kB page to
> > disk.
> so what ? one of trade-offs would be to re-read the data from the disk
> and compare it with what it should be (another copy on the disk) and
> only after that fix the transaction,

That didn't get rid of the race condition, it just moved it. So now you
write the transaction, and you read it off, and then you write that it
was OK -- but what if that write gets interrupted?

So you write the transaction, and you read it off, and then you write
that it was OK, and then you read that off, and then you write that the
OK was OK. No, that could get interrupted to...

Meanwhile, it turns out that you've been reading back from cache instead
of platters all along. Or perhaps it turns out that the battery-backed
cache in the disk controller has a dead battery, or simply doesn't work.

If the disk promises data is written but it is not then there is nothing
MySQL can do to *tell* that something is amiss. 

I think you've confused an RDBMS with a system administrator. Next
thing you know you'll be complaining that MySQL isn't ACID-compliant
because it can't survive a fire.

  -Rich

-- 
Rich Lafferty --+---
 Ottawa, Ontario, Canada|  Save the Pacific Northwest Tree Octopus!
 http://www.lafferty.ca/|http://zapatopi.net/treeoctopus.html
[EMAIL PROTECTED] ---+---

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[DebugasRu]

>> Many data centers do not allow customers to install their own UPS
>> inside the rack.

The questions is not whether they have (wish to have, can afford etc ) UPS or 
not,
or if the OS is buggy ( fsync() function or others ),
the problem is the database management system failed to ensure
data integrity under stressed conditions. Of cause ensuring persistance of 
transactions when
underlying hardware/software is unreliable is a very difficult task requiring
multiple trade-offs and that's why i find it impossible to compare
performance of two given DBMS where one of them ensures data integrity
and the other one cannot. What would you prefer - the system that
fails in 1/billion's occasions but runs 10 times slower or the system
that fails 1/100 occasions even though it runs 10 times faster? Or is
it better to have a system that never fails but runs 100 times slower?

> Probably fsync() had failed to flush some part of a 16 kB page to disk.
so what ? one of trade-offs would be to re-read the data from the disk
and compare it with what it should be (another copy on the disk) and only after 
that fix the
transaction, otherwise roll it all back.
When you have multiple users keeping db copy for each of them
(versioning) becomes a nightmare...
Solutions do exist but they require expensive trade-offs and more
complex algorithms.




-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Heikki Tuuri]

Jochem,
- Original Message - 
From: "Jochem van Dieten" <[EMAIL PROTECTED]>
Newsgroups: mailing.database.myodbc
Sent: Wednesday, February 23, 2005 1:01 AM
Subject: Re: wikipedia down, slashdot covering, mysql mentioned


On Wed, 23 Feb 2005 00:22:55 +0200, Heikki Tuuri wrote:
a buggy fsync() in Linux is one of  the possible reasons here. If an 
InnoDB
tablespace gets corrupt in a power outage, it is most probably caused by 
a
bad fsync() implementation or configuration in the operating system or
hardware. An fsync() call should write the data physically to disk or to 
a
battery-backed, non-volatile disk cache. If it fails to do so, then any
database, PostgreSQL or anything, can get corrupt.
Can they? Even if the absolute ordering of writes is maintained, only
full pages are written and each page has a checksum? It seems to me
that then you might lose transactions that have not yet committed to
disk, but your database would not get corrupt.
you are right, if the disk subsystem writes database pages to the disk or a 
non-volatile cache in the same order that fsync() was called on the pages, 
then the database cannot get corrupt. Corruption could only result from a 
partial page write, and against that InnoDB uses the 'doublewrite' method:

http://dev.mysql.com/doc/mysql/en/innodb-disk-i-slash-o.html
In the Wikipedia case, the disk subsystem apparently had written partial 
pages for whom the good copy of the page was not present in the doublewrite 
buffer. That means that the writes could not physically happen in the same 
order that fsync() was called on them. Thus, fsync() was not a 'barrier' in 
disk writes.

Jochem
Regards,
Heikki
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Greg Whalin]

Daniel Kasak wrote:
Greg Whalin wrote:
Many data centers do not allow customers to install their own UPS 
inside the rack.  I am not sure if this is the case with Wikipedia, 
but it is definitely the case at the data center we are hosted in.  I 
would love to shove one in after reading the horror stories at 
Livejournal and now Wikipedia.

I see.
Do the data centers offer their own UPS ( as opposed to the customer 
setting up one for themselves ), or do they simple ban the use of UPSs 
outright?

Either way, if you *need* a UPS, you can't hide behind the excuse "oh 
yeah but XYZ wouldn't let me". You find a way to make it happen ... or 
if you can't you at least don't start blaming your software for what is 
obviously a hardware issue.

Our data center spans two power grids, has a massive battery backup, and 
is backed by two massive generators, making power loss pretty difficult.

However, we once lost power to our entire cage by hooking into too many 
servers into a single circuit and tripping a breaker.  We now have two 
power feeds coming into each rack, and redundant power supplies on each 
db server split between the two circuits.  We also carefully watch our 
power utilization to make sure we don't come close to hitting this limit.

As I understand it, both LJ and Wiki suffered from someone hitting EPO 
switch (emergancy power off used to kill all power for when fire 
fighters have to come through ... a safety to make sure the firemen are 
not electrocuted when putting out fires).  This EPO switch is precisly 
why UPSs are sometimes not allowed in each individual rack.  For safety 
reasons, they need to ensure that all power is off in event of 
fire/rescue crews having to come through.

I am not sure that LJ or Wiki blamed mysql.  In fact, the only mysql 
complaint I saw from LJ was that they had to spend time rebuilding index 
files for their few remaining myisam tables.  This I can easily believe 
as I see myisam indexes get corrupted when I sneeze in their general 
direction.  However, LJ mentioned that all of their innodb tables 
recovered with no problems.  In fact, both Wiki and LJ mentioned drive 
write caching to be the major cause of their problems.

Greg
--
[EMAIL PROTECTED]
Meetup.com
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Heikki Tuuri]

Hi!
If I understood correctly, Wikipedia did have an UPS at their data center. 
But the problem cut also that off.

Regards,
Heikki
- Original Message - 
From: "Daniel Kasak" <[EMAIL PROTECTED]>
Newsgroups: mailing.database.myodbc
Sent: Wednesday, February 23, 2005 12:56 AM
Subject: Re: wikipedia down, slashdot covering, mysql mentioned


Greg Whalin wrote:
Many data centers do not allow customers to install their own UPS
inside the rack.  I am not sure if this is the case with Wikipedia,
but it is definitely the case at the data center we are hosted in.  I
would love to shove one in after reading the horror stories at
Livejournal and now Wikipedia.
I see.
Do the data centers offer their own UPS ( as opposed to the customer
setting up one for themselves ), or do they simple ban the use of UPSs
outright?
Either way, if you *need* a UPS, you can't hide behind the excuse "oh
yeah but XYZ wouldn't let me". You find a way to make it happen ... or
if you can't you at least don't start blaming your software for what is
obviously a hardware issue.
--
Daniel Kasak
IT Developer
NUS Consulting Group
Level 5, 77 Pacific Highway
North Sydney, NSW, Australia 2060
T: (+61) 2 9922-7676 / F: (+61) 2 9922 7989
email: [EMAIL PROTECTED]
website: http://www.nusconsulting.com.au
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: 
http://lists.mysql.com/[EMAIL PROTECTED]


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Heikki Tuuri]

Greg,
looks like Jens Axboe and others are still working to get fsync() safe in 
Linux-2.6.xx:

http://groups-beta.google.com/group/linux.kernel/browse_frm/thread/bbe45994b0277f7a/cc6d86c50514da81?q=fsync+linux+kernel+axboe&_done=%2Fgroups%3Fas_q%3Dfsync+linux+kernel+axboe%26safe%3Dimages%26as_scoring%3Dd%26lr%3D%26hl%3Den%26&_doneTitle=Back+to+Search&&d#cc6d86c50514da81
I was not aware that fsync() would have had more problems in Linux-2.4 than 
in 2.6.

I am afraid they can never make fsync() to work correctly on all hardware. 
But there clearly is room for improvement.

Regards,
Heikki
- Original Message - 
From: "Greg Whalin" <[EMAIL PROTECTED]>
Newsgroups: mailing.database.myodbc
Sent: Wednesday, February 23, 2005 12:49 AM
Subject: Re: wikipedia down, slashdot covering, mysql mentioned


I was under the impression that fsync() was only buggy in Linux in the
2.4 kernels.  Is it still problematic in 2.6 series?
Greg
--
[EMAIL PROTECTED]
Meetup.com
Heikki Tuuri wrote:
Peter,
a buggy fsync() in Linux is one of  the possible reasons here. If an
InnoDB tablespace gets corrupt in a power outage, it is most probably
caused by a bad fsync() implementation or configuration in the operating
system or hardware. An fsync() call should write the data physically to
disk or to a battery-backed, non-volatile disk cache. If it fails to do
so, then any database, PostgreSQL or anything, can get corrupt.
In the Wikipedia case, James Day posted an error message that InnoDB
printed. It showed that the page checksum was wrong. Probably fsync()
had failed to flush some part of a 16 kB page to disk.
Unfortunately, fsync() is badly implemented in many, or most,
OS/hardware configurations. Some ways to improve the situation:
- use hdparm, or proprietary disk configuration tools of the disk vendor
to switch off write caching in the SCSI controller and the disk itself;
- use a battery-backed disk cache in the SCSI disk controller;
- use certified hardware from one vendor: for example, Solaris/Sparc
might work more reliably than an arbitrary Linux/x86 configuration;
- do not use disk striping, software RAID, NFS, or anything exotic;
- use the MySQL replication to keep a backup server.
None of the above is guaranteed to remove corruption problems, and
switching off write caching can seriously lower performance. In the
Wikipedia case, battery-backed SCSI controllers did not help at all.
Thus, using an Uninterrupted Power System is probably the best way to
prepare against power outages.
It is possible to implement fsync() correctly and efficiently in the
operating system and hardware. There are no physical limitations that
would make it impossible. But, unfortunately, in practice, many
implementations are seriously flawed.
Best regards,
Heikki Tuuri
Innobase Oy
Foreign keys, transactions, and row level locking for MySQL
InnoDB Hot Backup - a hot backup tool for InnoDB which also backs up
MyISAM tables
http://www.innodb.com/order.php
Order MySQL Network from http://www.mysql.com/network/
- Original Message - From: "Peter Wilm" <[EMAIL PROTECTED]>
Newsgroups: mailing.database.myodbc
Sent: Tuesday, February 22, 2005 12:40 PM
Subject: Re: wikipedia down, slashdot covering, mysql mentioned

Could this possibly be a problem with a bad fsync implementation in
linux (< 2.6.5)?
See: http://www.ussg.iu.edu/hypermail/linux/kernel/0403.2/0527.html
Scott Haneda schrieb:
There is a thread over at /. about WikiMedia being out due to a power
outage
<http://slashdot.org/articles/05/02/22/0151213.shtml?tid=95>
MySql is getting bashed pretty hard in some cases as apparently, in
power
failures, you get database corruption.  (this is all from the posts,
not my
opinion at all, I am asking only to learn more) Also, it is mentioned
the
MySql devs just do not have data corruption from power failure a high
priority.
Many are leaning on Postgres.  For me, MySql has never let me down
(Knock on
PSU) but I have to ask, for more data on this matter.  I run MySql on
OS X,
I sometimes have to reboot, it comes back up fine, I do however, do
nothing
special to shut down mysql, should I be?

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:
http://lists.mysql.com/[EMAIL PROTECTED]


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: 
http://lists.mysql.com/[EMAIL PROTECTED]


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Jochem van Dieten]

On Wed, 23 Feb 2005 00:22:55 +0200, Heikki Tuuri wrote:
> 
> a buggy fsync() in Linux is one of  the possible reasons here. If an InnoDB
> tablespace gets corrupt in a power outage, it is most probably caused by a
> bad fsync() implementation or configuration in the operating system or
> hardware. An fsync() call should write the data physically to disk or to a
> battery-backed, non-volatile disk cache. If it fails to do so, then any
> database, PostgreSQL or anything, can get corrupt.

Can they? Even if the absolute ordering of writes is maintained, only
full pages are written and each page has a checksum? It seems to me
that then you might lose transactions that have not yet committed to
disk, but your database would not get corrupt.

Jochem

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Daniel Kasak]

Greg Whalin wrote:
Many data centers do not allow customers to install their own UPS 
inside the rack.  I am not sure if this is the case with Wikipedia, 
but it is definitely the case at the data center we are hosted in.  I 
would love to shove one in after reading the horror stories at 
Livejournal and now Wikipedia.
I see.
Do the data centers offer their own UPS ( as opposed to the customer 
setting up one for themselves ), or do they simple ban the use of UPSs 
outright?

Either way, if you *need* a UPS, you can't hide behind the excuse "oh 
yeah but XYZ wouldn't let me". You find a way to make it happen ... or 
if you can't you at least don't start blaming your software for what is 
obviously a hardware issue.

--
Daniel Kasak
IT Developer
NUS Consulting Group
Level 5, 77 Pacific Highway
North Sydney, NSW, Australia 2060
T: (+61) 2 9922-7676 / F: (+61) 2 9922 7989
email: [EMAIL PROTECTED]
website: http://www.nusconsulting.com.au
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Greg Whalin]

I was under the impression that fsync() was only buggy in Linux in the 
2.4 kernels.  Is it still problematic in 2.6 series?

Greg
--
[EMAIL PROTECTED]
Meetup.com
Heikki Tuuri wrote:
Peter,
a buggy fsync() in Linux is one of  the possible reasons here. If an 
InnoDB tablespace gets corrupt in a power outage, it is most probably 
caused by a bad fsync() implementation or configuration in the operating 
system or hardware. An fsync() call should write the data physically to 
disk or to a battery-backed, non-volatile disk cache. If it fails to do 
so, then any database, PostgreSQL or anything, can get corrupt.

In the Wikipedia case, James Day posted an error message that InnoDB 
printed. It showed that the page checksum was wrong. Probably fsync() 
had failed to flush some part of a 16 kB page to disk.

Unfortunately, fsync() is badly implemented in many, or most, 
OS/hardware configurations. Some ways to improve the situation:

- use hdparm, or proprietary disk configuration tools of the disk vendor 
to switch off write caching in the SCSI controller and the disk itself;
- use a battery-backed disk cache in the SCSI disk controller;
- use certified hardware from one vendor: for example, Solaris/Sparc 
might work more reliably than an arbitrary Linux/x86 configuration;
- do not use disk striping, software RAID, NFS, or anything exotic;
- use the MySQL replication to keep a backup server.

None of the above is guaranteed to remove corruption problems, and 
switching off write caching can seriously lower performance. In the 
Wikipedia case, battery-backed SCSI controllers did not help at all. 
Thus, using an Uninterrupted Power System is probably the best way to 
prepare against power outages.

It is possible to implement fsync() correctly and efficiently in the 
operating system and hardware. There are no physical limitations that 
would make it impossible. But, unfortunately, in practice, many 
implementations are seriously flawed.

Best regards,
Heikki Tuuri
Innobase Oy
Foreign keys, transactions, and row level locking for MySQL
InnoDB Hot Backup - a hot backup tool for InnoDB which also backs up 
MyISAM tables
http://www.innodb.com/order.php

Order MySQL Network from http://www.mysql.com/network/
- Original Message - From: "Peter Wilm" <[EMAIL PROTECTED]>
Newsgroups: mailing.database.myodbc
Sent: Tuesday, February 22, 2005 12:40 PM
Subject: Re: wikipedia down, slashdot covering, mysql mentioned

Could this possibly be a problem with a bad fsync implementation in
linux (< 2.6.5)?
See: http://www.ussg.iu.edu/hypermail/linux/kernel/0403.2/0527.html
Scott Haneda schrieb:
There is a thread over at /. about WikiMedia being out due to a power 
outage
<http://slashdot.org/articles/05/02/22/0151213.shtml?tid=95>

MySql is getting bashed pretty hard in some cases as apparently, in 
power
failures, you get database corruption.  (this is all from the posts, 
not my
opinion at all, I am asking only to learn more) Also, it is mentioned 
the
MySql devs just do not have data corruption from power failure a high
priority.

Many are leaning on Postgres.  For me, MySql has never let me down 
(Knock on
PSU) but I have to ask, for more data on this matter.  I run MySql on 
OS X,
I sometimes have to reboot, it comes back up fine, I do however, do 
nothing
special to shut down mysql, should I be?

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: 
http://lists.mysql.com/[EMAIL PROTECTED]



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Greg Whalin]

Many data centers do not allow customers to install their own UPS inside 
the rack.  I am not sure if this is the case with Wikipedia, but it is 
definitely the case at the data center we are hosted in.  I would love 
to shove one in after reading the horror stories at Livejournal and now 
Wikipedia.

Greg
[EMAIL PROTECTED]
Meetup.com
Daniel Kasak wrote:
Scott Haneda wrote:
There is a thread over at /. about WikiMedia being out due to a power 
outage


MySql is getting bashed pretty hard in some cases as apparently, in power
failures, you get database corruption.  (this is all from the posts, 
not my
opinion at all, I am asking only to learn more) Also, it is mentioned the
MySql devs just do not have data corruption from power failure a high
priority.

Many are leaning on Postgres.  For me, MySql has never let me down 
(Knock on
PSU) but I have to ask, for more data on this matter.  I run MySql on 
OS X,
I sometimes have to reboot, it comes back up fine, I do however, do 
nothing
special to shut down mysql, should I be?
 

There are certainly some people who have nothing better to do than 
preach the benefits of Postgres over MySQL on Slashdot.
The simple fact is that no matter how fancy your software is, if your 
computer experiences a power outage, you can't *insist* that one 
particular application somehow protect your data from the fate of 
everything else on your computer.

We have a UPS connected to our 2 servers. The Linux server is running 
apcupsd and controlling the Win2k server ( also running apcupsd in slave 
mode ). Both servers are connected via a crossover cable to each other ( 
so we don't have to put the network switches on the UPS ).

If WikiMedia can't implement something at least as good as the above, 
then how can they or anyone else point the finger at MySQL for data 
corruption? The mind boggles over what they expect.

Having said that, our MySQL server has experienced a power 'issue' ( my 
boss tripped over the power cord coming from the UPS ). We both turned 
white :) But MySQL started up OK, rolled back a transaction or 2, and 
continued on it's way.

There's nothing like a UPS though. Or better yet, a UPS, daily backups, 
and a transaction log.


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Daniel Kasak]

Scott Haneda wrote:
There is a thread over at /. about WikiMedia being out due to a power outage

MySql is getting bashed pretty hard in some cases as apparently, in power
failures, you get database corruption.  (this is all from the posts, not my
opinion at all, I am asking only to learn more) Also, it is mentioned the
MySql devs just do not have data corruption from power failure a high
priority.
Many are leaning on Postgres.  For me, MySql has never let me down (Knock on
PSU) but I have to ask, for more data on this matter.  I run MySql on OS X,
I sometimes have to reboot, it comes back up fine, I do however, do nothing
special to shut down mysql, should I be?
 

There are certainly some people who have nothing better to do than 
preach the benefits of Postgres over MySQL on Slashdot.
The simple fact is that no matter how fancy your software is, if your 
computer experiences a power outage, you can't *insist* that one 
particular application somehow protect your data from the fate of 
everything else on your computer.

We have a UPS connected to our 2 servers. The Linux server is running 
apcupsd and controlling the Win2k server ( also running apcupsd in slave 
mode ). Both servers are connected via a crossover cable to each other ( 
so we don't have to put the network switches on the UPS ).

If WikiMedia can't implement something at least as good as the above, 
then how can they or anyone else point the finger at MySQL for data 
corruption? The mind boggles over what they expect.

Having said that, our MySQL server has experienced a power 'issue' ( my 
boss tripped over the power cord coming from the UPS ). We both turned 
white :) But MySQL started up OK, rolled back a transaction or 2, and 
continued on it's way.

There's nothing like a UPS though. Or better yet, a UPS, daily backups, 
and a transaction log.

--
Daniel Kasak
IT Developer
NUS Consulting Group
Level 5, 77 Pacific Highway
North Sydney, NSW, Australia 2060
T: (+61) 2 9922-7676 / F: (+61) 2 9922 7989
email: [EMAIL PROTECTED]
website: http://www.nusconsulting.com.au
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Heikki Tuuri]

Peter,
a buggy fsync() in Linux is one of  the possible reasons here. If an InnoDB 
tablespace gets corrupt in a power outage, it is most probably caused by a 
bad fsync() implementation or configuration in the operating system or 
hardware. An fsync() call should write the data physically to disk or to a 
battery-backed, non-volatile disk cache. If it fails to do so, then any 
database, PostgreSQL or anything, can get corrupt.

In the Wikipedia case, James Day posted an error message that InnoDB 
printed. It showed that the page checksum was wrong. Probably fsync() had 
failed to flush some part of a 16 kB page to disk.

Unfortunately, fsync() is badly implemented in many, or most, OS/hardware 
configurations. Some ways to improve the situation:

- use hdparm, or proprietary disk configuration tools of the disk vendor to 
switch off write caching in the SCSI controller and the disk itself;
- use a battery-backed disk cache in the SCSI disk controller;
- use certified hardware from one vendor: for example, Solaris/Sparc might 
work more reliably than an arbitrary Linux/x86 configuration;
- do not use disk striping, software RAID, NFS, or anything exotic;
- use the MySQL replication to keep a backup server.

None of the above is guaranteed to remove corruption problems, and switching 
off write caching can seriously lower performance. In the Wikipedia case, 
battery-backed SCSI controllers did not help at all. Thus, using an 
Uninterrupted Power System is probably the best way to prepare against power 
outages.

It is possible to implement fsync() correctly and efficiently in the 
operating system and hardware. There are no physical limitations that would 
make it impossible. But, unfortunately, in practice, many implementations 
are seriously flawed.

Best regards,
Heikki Tuuri
Innobase Oy
Foreign keys, transactions, and row level locking for MySQL
InnoDB Hot Backup - a hot backup tool for InnoDB which also backs up MyISAM 
tables
http://www.innodb.com/order.php

Order MySQL Network from http://www.mysql.com/network/
- Original Message - 
From: "Peter Wilm" <[EMAIL PROTECTED]>
Newsgroups: mailing.database.myodbc
Sent: Tuesday, February 22, 2005 12:40 PM
Subject: Re: wikipedia down, slashdot covering, mysql mentioned


Could this possibly be a problem with a bad fsync implementation in
linux (< 2.6.5)?
See: http://www.ussg.iu.edu/hypermail/linux/kernel/0403.2/0527.html
Scott Haneda schrieb:
There is a thread over at /. about WikiMedia being out due to a power 
outage
<http://slashdot.org/articles/05/02/22/0151213.shtml?tid=95>

MySql is getting bashed pretty hard in some cases as apparently, in power
failures, you get database corruption.  (this is all from the posts, not 
my
opinion at all, I am asking only to learn more) Also, it is mentioned the
MySql devs just do not have data corruption from power failure a high
priority.

Many are leaning on Postgres.  For me, MySql has never let me down (Knock 
on
PSU) but I have to ask, for more data on this matter.  I run MySql on OS 
X,
I sometimes have to reboot, it comes back up fine, I do however, do 
nothing
special to shut down mysql, should I be?

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: 
http://lists.mysql.com/[EMAIL PROTECTED]


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[Peter Wilm]

Could this possibly be a problem with a bad fsync implementation in 
linux (< 2.6.5)?
See: http://www.ussg.iu.edu/hypermail/linux/kernel/0403.2/0527.html

Scott Haneda schrieb:
There is a thread over at /. about WikiMedia being out due to a power outage

MySql is getting bashed pretty hard in some cases as apparently, in power
failures, you get database corruption.  (this is all from the posts, not my
opinion at all, I am asking only to learn more) Also, it is mentioned the
MySql devs just do not have data corruption from power failure a high
priority.
Many are leaning on Postgres.  For me, MySql has never let me down (Knock on
PSU) but I have to ask, for more data on this matter.  I run MySql on OS X,
I sometimes have to reboot, it comes back up fine, I do however, do nothing
special to shut down mysql, should I be?

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

Re: wikipedia down, slashdot covering, mysql mentioned

[DebugasRu]

SH> MySql is getting bashed pretty hard in some cases as apparently, in power
SH> failures, you get database corruption.

One can do a simple test
start lengthy transaction and plug off the computer while the
transaction is still in progress.
Will your DBMS recover the errors after switching your computer on
again ? If not then that DBMS is a joke.




-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]

wikipedia down, slashdot covering, mysql mentioned

[Scott Haneda]

There is a thread over at /. about WikiMedia being out due to a power outage


MySql is getting bashed pretty hard in some cases as apparently, in power
failures, you get database corruption.  (this is all from the posts, not my
opinion at all, I am asking only to learn more) Also, it is mentioned the
MySql devs just do not have data corruption from power failure a high
priority.

Many are leaning on Postgres.  For me, MySql has never let me down (Knock on
PSU) but I have to ask, for more data on this matter.  I run MySql on OS X,
I sometimes have to reboot, it comes back up fine, I do however, do nothing
special to shut down mysql, should I be?
-- 
-
Scott HanedaTel: 415.898.2602
 Fax: 313.557.5052
<[EMAIL PROTECTED]>  Novato, CA U.S.A.



-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]