Re: [h2] Database corrupt after computer power failure.

[Mikael Nordenberg]

We were mainly looking at the code quality / neatness.
But if I remember correctly we identified places where things would not
clean up correctly if an exception were thrown. It was a few years back now
though, so I don't remember exactly what we discovered.

We did however rescue almost all our data. It turned out it was only one or
two records (of millions) which were broken when the driver crashed and
refused to read the database.
The recovery tool was also useless for us, since it also crashed (or
refused to read the file).
If I remember correctly we selected all records "surrounding" the faulty
ones, and recovered the data that way.

/Mikael

2018-05-02 11:08 GMT+02:00 Noel Grandin :

>
>
> On 2018/05/02 10:48 AM, Mikael Nordenberg wrote:
>
>> No offence Thomas, but after reading the source code for H2 when we
>> investigated our problems, we decided to use a different database. To us,
>> it seemed an overwhelming task to get that code in order to make it make it
>> stable enough for our needs.
>> We switched to Postgres, and all our problems went away.
>>
>>
> Did you identifier any problems in particular?
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Re: [h2] Database corrupt after computer power failure.

[Noel Grandin]

On 2018/05/02 10:48 AM, Mikael Nordenberg wrote:
No offence Thomas, but after reading the source code for H2 when we investigated our problems, we decided to use a 
different database. To us, it seemed an overwhelming task to get that code in order to make it make it stable enough for 
our needs.

We switched to Postgres, and all our problems went away.



Did you identifier any problems in particular?

--
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Re: [h2] Database corrupt after computer power failure.

[Mikael Nordenberg]

No offence Thomas, but after reading the source code for H2 when we
investigated our problems, we decided to use a different database. To us,
it seemed an overwhelming task to get that code in order to make it make it
stable enough for our needs.
We switched to Postgres, and all our problems went away.

/Mikael

2018-05-02 9:05 GMT+02:00 Gautham Sampath :

> Hi,
>
> We are currently using h2 version: 1.3.174 in our customer environment. We
> are facing the same problem as Mikael Nordenberg. Is this problem
> resolved in the latest release version of H2 DB? Will H2 DB upgrade from
> PageStore to MVstore will solve this problem?
> Even in our case below exception was thrown due the power shutdown:
>
> 2018-04-27 06:03:38,595 [ERROR] [] UpdateThread 
> hibernate.util.JDBCExceptionReporter
> - File corrupted while reading record: "[16227] stream data key:16326
> pos:11 remaining:0". Possible solution: use the recovery tool; SQL
> statement:
> .
> .
> 2018-04-27 06:03:38,595 [ERROR] [] UpdateThread dao.jpa.Impl -
> Error in Impl:getAll()
> javax.persistence.PersistenceException: 
> org.hibernate.exception.GenericJDBCException:
> could not execute query
> at org.hibernate.ejb.AbstractEntityManagerImpl.convert(
> AbstractEntityManagerImpl.java:1214)
> at org.hibernate.ejb.AbstractEntityManagerImpl.convert(
> AbstractEntityManagerImpl.java:1147)
> at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:255)
> .
> .
>
> We connect with the db through this URL: jdbc:h2:db/Axxx;AUTO_SERVER=TRUE
>
> Please let me your views on the same. Thanks in advance.
>
> Regards,
> Gautham
>
> On Saturday, May 9, 2015 at 1:45:45 PM UTC+5:30, Thomas Mueller Graf wrote:
>>
>> Hi,
>>
>> Until this problem is fixed, it might make sense to use the PageStore
>> storage engine, by appending ";mv_store=false" to the database URL.
>>
>> Regards,
>> Thomas
>>
>>
>> On Friday, May 8, 2015, Mikael Nordenberg  wrote:
>>
>>> Thank you for your response and time to investigate this.
>>>
>>> That is very interesting!
>>> I attach another case when this happened, in case it can be used to
>>> verify your assumption.
>>> I have not been able to reproduce this on a test machine yet.
>>> But it has happened twice in just a matter of days on one client
>>> machine, so it may be reproducible on that PC.
>>> (The affected PC runs Windows 7, and uses a Samsung 850 EVO SSD drive if
>>> that matters.)
>>>
>>> /Mikael
>>>
>>>
>>> Den fredag 8 maj 2015 kl. 18:12:07 UTC+2 skrev Thomas Mueller:

 Hi,

 Thanks a lot! I have analyzed the database file, and I think I know
 what the problem is. It looks like the disk (or operating system)
 re-ordered write operations, so that changes later in time (and later in
 the file) were written, but one earlier change (both in time and in the
 file) was not written.

 (Detail for later reference: various entries (in chunks 'aad4' to
 'aadb', from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later
 than 16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2
 actually contains the earlier chunk 'aa7b' from 16:04:24.742).

 The MVStore should detect this and automatically discard chunks that
 were written later than 16:05:10.378. It does not currently do that, this
 is a problem. I will implement this for the next release.

 I think what also can happen is write re-ordering causes a file to be
 truncated too early. This would explain a different corruption problem. The
 fix for that would need to be different however (truncation would need to
 wait for 45 seconds).

 Regards,
 Thomas




 On Thursday, May 7, 2015, Mikael Nordenberg  wrote:

> Same corruption happened again today on a clients computer when power
> disappeared.
> Will try to reproduce tomorrow in our lab.
>
> /Mikael
>
> Den torsdag 7 maj 2015 kl. 15:42:48 UTC+2 skrev Mikael Nordenberg:
>>
>> Hi,
>>
>> We are using H2 version 1.4.185 with mv-store.
>> After an unclean shutdown the database file was corrupt. The database
>> is attached.
>> Opening the database results in http://pastebin.com/PXceifPx
>>
>> The database is used from a single thread.
>> Hope this gives anything...
>>
>> /Mikael
>>
> --
> You received this message because you are subscribed to the Google
> Groups "H2 Database" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to h2-database+unsubscr...@googlegroups.com.
> To post to this group, send email to h2-database@googlegroups.com.
> Visit this group at http://groups.google.com/group/h2-database.
> For more options, visit https://groups.google.com/d/optout.
>
 --
>>> You received this message because you are subscribed to the Google
>>> Groups 

Re: [h2] Database corrupt after computer power failure.

[Gautham Sampath]

Hi,

We are currently using h2 version: 1.3.174 in our customer environment. We 
are facing the same problem as Mikael Nordenberg. Is this problem resolved 
in the latest release version of H2 DB? Will H2 DB upgrade from PageStore 
to MVstore will solve this problem? 
Even in our case below exception was thrown due the power shutdown:

2018-04-27 06:03:38,595 [ERROR] [] UpdateThread 
hibernate.util.JDBCExceptionReporter - File corrupted while reading record: 
"[16227] stream data key:16326 pos:11 remaining:0". Possible solution: use 
the recovery tool; SQL statement:
.
.
2018-04-27 06:03:38,595 [ERROR] [] UpdateThread dao.jpa.Impl - 
Error in Impl:getAll() 
javax.persistence.PersistenceException: 
org.hibernate.exception.GenericJDBCException: could not execute query
at 
org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1214)
at 
org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1147)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:255)
.
.

We connect with the db through this URL: jdbc:h2:db/Axxx;AUTO_SERVER=TRUE

Please let me your views on the same. Thanks in advance. 

Regards,
Gautham

On Saturday, May 9, 2015 at 1:45:45 PM UTC+5:30, Thomas Mueller Graf wrote:
>
> Hi,
>
> Until this problem is fixed, it might make sense to use the PageStore 
> storage engine, by appending ";mv_store=false" to the database URL.
>
> Regards,
> Thomas
>
>
> On Friday, May 8, 2015, Mikael Nordenberg  
> wrote:
>
>> Thank you for your response and time to investigate this.
>>
>> That is very interesting!
>> I attach another case when this happened, in case it can be used to 
>> verify your assumption.
>> I have not been able to reproduce this on a test machine yet.
>> But it has happened twice in just a matter of days on one client machine, 
>> so it may be reproducible on that PC.
>> (The affected PC runs Windows 7, and uses a Samsung 850 EVO SSD drive if 
>> that matters.)
>>
>> /Mikael
>>
>>
>> Den fredag 8 maj 2015 kl. 18:12:07 UTC+2 skrev Thomas Mueller:
>>>
>>> Hi,
>>>
>>> Thanks a lot! I have analyzed the database file, and I think I know what 
>>> the problem is. It looks like the disk (or operating system) re-ordered 
>>> write operations, so that changes later in time (and later in the file) 
>>> were written, but one earlier change (both in time and in the file) was not 
>>> written.
>>>
>>> (Detail for later reference: various entries (in chunks 'aad4' to 
>>> 'aadb', from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later 
>>> than 16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2 
>>> actually contains the earlier chunk 'aa7b' from 16:04:24.742).
>>>
>>> The MVStore should detect this and automatically discard chunks that 
>>> were written later than 16:05:10.378. It does not currently do that, this 
>>> is a problem. I will implement this for the next release.
>>>
>>> I think what also can happen is write re-ordering causes a file to be 
>>> truncated too early. This would explain a different corruption problem. The 
>>> fix for that would need to be different however (truncation would need to 
>>> wait for 45 seconds).
>>>
>>> Regards,
>>> Thomas
>>>
>>>
>>>
>>>
>>> On Thursday, May 7, 2015, Mikael Nordenberg  wrote:
>>>
 Same corruption happened again today on a clients computer when power 
 disappeared.
 Will try to reproduce tomorrow in our lab.

 /Mikael

 Den torsdag 7 maj 2015 kl. 15:42:48 UTC+2 skrev Mikael Nordenberg:
>
> Hi,
>
> We are using H2 version 1.4.185 with mv-store.
> After an unclean shutdown the database file was corrupt. The database 
> is attached.
> Opening the database results in http://pastebin.com/PXceifPx
>
> The database is used from a single thread.
> Hope this gives anything...
>
> /Mikael
>
 -- 
 You received this message because you are subscribed to the Google 
 Groups "H2 Database" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to h2-database+unsubscr...@googlegroups.com.
 To post to this group, send email to h2-database@googlegroups.com.
 Visit this group at http://groups.google.com/group/h2-database.
 For more options, visit https://groups.google.com/d/optout.

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "H2 Database" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to h2-database+unsubscr...@googlegroups.com.
>> To post to this group, send email to h2-database@googlegroups.com.
>> Visit this group at http://groups.google.com/group/h2-database.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop 

[h2] Database corrupt after computer power failure.

[Thomas Mueller]

Hi,

I forgot to write: this should be fixed now (actually since quite a long
time).

Regards,
Thomas

On Sat, May 9, 2015 at 10:15 AM, Thomas Mueller <
thomas.tom.muel...@gmail.com
> wrote:

> Hi,
>
> Until this problem is fixed, it might make sense to use the PageStore
> storage engine, by appending ";mv_store=false" to the database URL.
>
> Regards,
> Thomas
>
>
> On Friday, May 8, 2015, Mikael Nordenberg  > wrote:
>
>> Thank you for your response and time to investigate this.
>>
>> That is very interesting!
>> I attach another case when this happened, in case it can be used to
>> verify your assumption.
>> I have not been able to reproduce this on a test machine yet.
>> But it has happened twice in just a matter of days on one client machine,
>> so it may be reproducible on that PC.
>> (The affected PC runs Windows 7, and uses a Samsung 850 EVO SSD drive if
>> that matters.)
>>
>> /Mikael
>>
>>
>> Den fredag 8 maj 2015 kl. 18:12:07 UTC+2 skrev Thomas Mueller:
>>>
>>> Hi,
>>>
>>> Thanks a lot! I have analyzed the database file, and I think I know what
>>> the problem is. It looks like the disk (or operating system) re-ordered
>>> write operations, so that changes later in time (and later in the file)
>>> were written, but one earlier change (both in time and in the file) was not
>>> written.
>>>
>>> (Detail for later reference: various entries (in chunks 'aad4' to
>>> 'aadb', from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later
>>> than 16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2
>>> actually contains the earlier chunk 'aa7b' from 16:04:24.742).
>>>
>>> The MVStore should detect this and automatically discard chunks that
>>> were written later than 16:05:10.378. It does not currently do that, this
>>> is a problem. I will implement this for the next release.
>>>
>>> I think what also can happen is write re-ordering causes a file to be
>>> truncated too early. This would explain a different corruption problem. The
>>> fix for that would need to be different however (truncation would need to
>>> wait for 45 seconds).
>>>
>>> Regards,
>>> Thomas
>>>
>>>
>>>
>>>
>>> On Thursday, May 7, 2015, Mikael Nordenberg  wrote:
>>>
 Same corruption happened again today on a clients computer when power
 disappeared.
 Will try to reproduce tomorrow in our lab.

 /Mikael

 Den torsdag 7 maj 2015 kl. 15:42:48 UTC+2 skrev Mikael Nordenberg:
>
> Hi,
>
> We are using H2 version 1.4.185 with mv-store.
> After an unclean shutdown the database file was corrupt. The database
> is attached.
> Opening the database results in http://pastebin.com/PXceifPx
>
> The database is used from a single thread.
> Hope this gives anything...
>
> /Mikael
>
 --
 You received this message because you are subscribed to the Google
 Groups "H2 Database" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to h2-database+unsubscr...@googlegroups.com.
 To post to this group, send email to h2-database@googlegroups.com.
 Visit this group at http://groups.google.com/group/h2-database.
 For more options, visit https://groups.google.com/d/optout.

>>> --
>> You received this message because you are subscribed to the Google Groups
>> "H2 Database" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to h2-database+unsubscr...@googlegroups.com.
>> To post to this group, send email to h2-database@googlegroups.com.
>> Visit this group at http://groups.google.com/group/h2-database.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Re: [h2] Database corrupt after computer power failure.

[Thomas Mueller]

Hi,

Until this problem is fixed, it might make sense to use the PageStore
storage engine, by appending ;mv_store=false to the database URL.

Regards,
Thomas


On Friday, May 8, 2015, Mikael Nordenberg mik...@ikanos.se wrote:

 Thank you for your response and time to investigate this.

 That is very interesting!
 I attach another case when this happened, in case it can be used to verify
 your assumption.
 I have not been able to reproduce this on a test machine yet.
 But it has happened twice in just a matter of days on one client machine,
 so it may be reproducible on that PC.
 (The affected PC runs Windows 7, and uses a Samsung 850 EVO SSD drive if
 that matters.)

 /Mikael


 Den fredag 8 maj 2015 kl. 18:12:07 UTC+2 skrev Thomas Mueller:

 Hi,

 Thanks a lot! I have analyzed the database file, and I think I know what
 the problem is. It looks like the disk (or operating system) re-ordered
 write operations, so that changes later in time (and later in the file)
 were written, but one earlier change (both in time and in the file) was not
 written.

 (Detail for later reference: various entries (in chunks 'aad4' to 'aadb',
 from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later than
 16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2
 actually contains the earlier chunk 'aa7b' from 16:04:24.742).

 The MVStore should detect this and automatically discard chunks that were
 written later than 16:05:10.378. It does not currently do that, this is a
 problem. I will implement this for the next release.

 I think what also can happen is write re-ordering causes a file to be
 truncated too early. This would explain a different corruption problem. The
 fix for that would need to be different however (truncation would need to
 wait for 45 seconds).

 Regards,
 Thomas




 On Thursday, May 7, 2015, Mikael Nordenberg mik...@ikanos.se wrote:

 Same corruption happened again today on a clients computer when power
 disappeared.
 Will try to reproduce tomorrow in our lab.

 /Mikael

 Den torsdag 7 maj 2015 kl. 15:42:48 UTC+2 skrev Mikael Nordenberg:

 Hi,

 We are using H2 version 1.4.185 with mv-store.
 After an unclean shutdown the database file was corrupt. The database
 is attached.
 Opening the database results in http://pastebin.com/PXceifPx

 The database is used from a single thread.
 Hope this gives anything...

 /Mikael

  --
 You received this message because you are subscribed to the Google
 Groups H2 Database group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to h2-database+unsubscr...@googlegroups.com.
 To post to this group, send email to h2-database@googlegroups.com.
 Visit this group at http://groups.google.com/group/h2-database.
 For more options, visit https://groups.google.com/d/optout.

  --
 You received this message because you are subscribed to the Google Groups
 H2 Database group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to h2-database+unsubscr...@googlegroups.com
 javascript:_e(%7B%7D,'cvml','h2-database%2bunsubscr...@googlegroups.com');
 .
 To post to this group, send email to h2-database@googlegroups.com
 javascript:_e(%7B%7D,'cvml','h2-database@googlegroups.com');.
 Visit this group at http://groups.google.com/group/h2-database.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups H2 
Database group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Database corrupt after computer power failure.

[Thomas Mueller]

Hi,

Thanks a lot! I have analyzed the database file, and I think I know what
the problem is. It looks like the disk (or operating system) re-ordered
write operations, so that changes later in time (and later in the file)
were written, but one earlier change (both in time and in the file) was not
written.

(Detail for later reference: various entries (in chunks 'aad4' to 'aadb',
from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later than
16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2
actually contains the earlier chunk 'aa7b' from 16:04:24.742).

The MVStore should detect this and automatically discard chunks that were
written later than 16:05:10.378. It does not currently do that, this is a
problem. I will implement this for the next release.

I think what also can happen is write re-ordering causes a file to be
truncated too early. This would explain a different corruption problem. The
fix for that would need to be different however (truncation would need to
wait for 45 seconds).

Regards,
Thomas




On Thursday, May 7, 2015, Mikael Nordenberg mik...@ikanos.se wrote:

 Same corruption happened again today on a clients computer when power
 disappeared.
 Will try to reproduce tomorrow in our lab.

 /Mikael

 Den torsdag 7 maj 2015 kl. 15:42:48 UTC+2 skrev Mikael Nordenberg:

 Hi,

 We are using H2 version 1.4.185 with mv-store.
 After an unclean shutdown the database file was corrupt. The database is
 attached.
 Opening the database results in http://pastebin.com/PXceifPx

 The database is used from a single thread.
 Hope this gives anything...

 /Mikael

  --
 You received this message because you are subscribed to the Google Groups
 H2 Database group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to h2-database+unsubscr...@googlegroups.com.
 To post to this group, send email to h2-database@googlegroups.com.
 Visit this group at http://groups.google.com/group/h2-database.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups H2 
Database group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.