Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-04-13 Thread GOULD, Nicholas
All,

Does anyone have experience of how long OWLIM should take to go through it's 
transaction log and update all indicies?

We have loaded a large volume of data (~7.5 GB of ttl files) into a repository 
and have a backup file of 8.7GB. We stopped Tomcat and on restart it is now 
going through transactions, presumably synching any differences between the 
disk and memory indicies when shutdown.

Many thanks,

Nic


From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] on behalf of Barry Bishop 
[barry.bis...@ontotext.com]
Sent: 01 April 2012 09:12
To: Barry Norton
Cc: owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hello all,

This message is displayed (and OWLIM refuses to continue) only after the
following sequence of events:

1. OWLIM is stopped and not shutdown properly
2. OWLIM is restarted and the recover process starts
3. OWLIM is stopped again without allowing the recovery process to finish
4. OWLIM is restarted

At this point, the recovery code is aware of the backed up indices, some
half built replacement indices and the transaction log. Without knowing
which set of indices is correct, it informs the user and stops.

(This is not an ideal situation and it is one that can not occur in OWLIM 5)

The important point is that you should always allow OWLIM time to
shutdown properly and should not kill tomcat, even if it seems to take a
long time. If you have got in to this situation, the probable means to
get started again is to shut down tomcat and delete any *.old files in
the owlim storage folder. Then restart and allow time for OWLIM to
replay its transaction log.

The cause of this is that OWLIM 4 (and before) uses a lazy writing
strategy for dirty pages. These are only swapped to disk when memory has
been exhausted and more pages are required. In other words, the state of
the indices on disk and in memory can diverge quite considerably and in
the event of a sudden stop, the correct indices must be rebuilt using
the transaction log (a file called 'backup' in the OWLIM storage folder).

OWLIM 5 still has this mode of operation available (it is called 'fast'
mode), but the default mode flushes updates to disk at the point of each
commit. It does this in such a way, that a failure at any time leaves
the disk image consistent with the state of the database at the end of
the last successful commit operation. For sustained loading operations,
this can have lower performance, but the advantages are:
1. instant shutdown time
2. instant recovery time (even if the last instance was stopped abruptly)
3. improved concurrency of reads and write operations

This last point means that export the database programmatically is now a
viable means to backup the database.

To summarise:

OWLIM 4 - this error occurs during initialisation when the OWLIM process
has been killed, restarted and not allowed time to recover properly,
i.e. killed again. It can be avoided by giving owlim enough time to
shutdown properly (which can be a long time after loading large amounts
of data on a machine with large memory). If it does happen, shutdown,
delete the *.old files, restart and wait.

OWLIM 5 - using the default transaction mode, this problem will not
occur. The fast transaction mode is available when administrators need
maximum performance and reduced query/update concurrency, but will have
no guarantees about data security if OWLIM is killed before shutting
properly - switching between transaction modes requires a restart.

I hope this helps,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
www.ontotext.com


On 01/04/12 01:29, Barry Norton wrote:

 Guys, how are you shutting down your database that it needs to be
 recovered so often?

 And are you interrupting the recovery process?

 Barry



 On 31/03/2012 20:44, Blasetti, Luciano (CIOK) wrote:
 Hi All,
 I also experienced quite frequently this issue on a tomcat instance
 running only openrdf-sesame (OWLIM 4.3 SE) and openrdf-workbench. No
 way to restore the database.
 Moreover, since there's no out-of-the-box functionalities to perform
 a hot backup on a non clustered configuration (and the programmatic
 export can't work on large triplestore due to timeout issues), this
 strange behavior leads to data loss.

 I hope that the forthcoming OWLIM 5.0 will be more robust and reliable.

 Regards,
 Luciano
 
 From: owlim-discussion-boun...@ontotext.com
 [owlim-discussion-boun...@ontotext.com] On Behalf Of Marek Šurek
 [marek_su...@yahoo.co.uk]
 Sent: Saturday, March 31, 2012 11:18 AM
 To: Monika Solanki; owlim-discussion@ontotext.com
 Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database
 restorer  tool has already failed once before to perform image
 restore.

 Hi Monika

Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-04-13 Thread GOULD, Nicholas
Thanks for the quick response Barry. We figured that we would have to wait for 
it to complete. I have a project manager breathing down my neck for an ETA on 
this so if you are able to give us an idea of how long 'a long time' could be 
that would be helpful.

The transaction file is up to around 9000 after 1 hour 40 mins and as stated 
before the backup is 8.7 GB. Is it possible to estimate the number of 
transactions in the backup file?

Thanks again for your help.

Nic


From: Barry Bishop [barry.bis...@ontotext.com]
Sent: 13 April 2012 12:33
To: GOULD, Nicholas
Cc: Barry Norton; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hello Nicholas,

This can take a long time, I'm afraid. If the indices are damaged then
OWLIM will replay it's entire transaction log in order to recreate them.

My suggestion is just to wait until it completes. Terminating the
recovery process means that the next restart will fail to initialise and
cause the the message in the subject of this email to be displayed.

The good news is that the transaction mechanisms in OWLIM 5.0 have been
completely refactored in order to fix these kinds of inconveniences. We
will release this version next week.

Please see:

http://owlim.ontotext.com/display/OWLIMv50/OWLIM-SE+Indexing+Specifics#OWLIM-SEIndexingSpecifics-TransactionMode

I hope this helps,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
www.ontotext.com


On 13/04/12 13:24, GOULD, Nicholas wrote:
 All,

 Does anyone have experience of how long OWLIM should take to go through it's 
 transaction log and update all indicies?

 We have loaded a large volume of data (~7.5 GB of ttl files) into a 
 repository and have a backup file of 8.7GB. We stopped Tomcat and on restart 
 it is now going through transactions, presumably synching any differences 
 between the disk and memory indicies when shutdown.

 Many thanks,

 Nic

 
 From: owlim-discussion-boun...@ontotext.com 
 [owlim-discussion-boun...@ontotext.com] on behalf of Barry Bishop 
 [barry.bis...@ontotext.com]
 Sent: 01 April 2012 09:12
 To: Barry Norton
 Cc: owlim-discussion@ontotext.com
 Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer 
 tool has already failed once before to perform image restore.

 Hello all,

 This message is displayed (and OWLIM refuses to continue) only after the
 following sequence of events:

 1. OWLIM is stopped and not shutdown properly
 2. OWLIM is restarted and the recover process starts
 3. OWLIM is stopped again without allowing the recovery process to finish
 4. OWLIM is restarted

 At this point, the recovery code is aware of the backed up indices, some
 half built replacement indices and the transaction log. Without knowing
 which set of indices is correct, it informs the user and stops.

 (This is not an ideal situation and it is one that can not occur in OWLIM 5)

 The important point is that you should always allow OWLIM time to
 shutdown properly and should not kill tomcat, even if it seems to take a
 long time. If you have got in to this situation, the probable means to
 get started again is to shut down tomcat and delete any *.old files in
 the owlim storage folder. Then restart and allow time for OWLIM to
 replay its transaction log.

 The cause of this is that OWLIM 4 (and before) uses a lazy writing
 strategy for dirty pages. These are only swapped to disk when memory has
 been exhausted and more pages are required. In other words, the state of
 the indices on disk and in memory can diverge quite considerably and in
 the event of a sudden stop, the correct indices must be rebuilt using
 the transaction log (a file called 'backup' in the OWLIM storage folder).

 OWLIM 5 still has this mode of operation available (it is called 'fast'
 mode), but the default mode flushes updates to disk at the point of each
 commit. It does this in such a way, that a failure at any time leaves
 the disk image consistent with the state of the database at the end of
 the last successful commit operation. For sustained loading operations,
 this can have lower performance, but the advantages are:
 1. instant shutdown time
 2. instant recovery time (even if the last instance was stopped abruptly)
 3. improved concurrency of reads and write operations

 This last point means that export the database programmatically is now a
 viable means to backup the database.

 To summarise:

 OWLIM 4 - this error occurs during initialisation when the OWLIM process
 has been killed, restarted and not allowed time to recover properly,
 i.e. killed again. It can be avoided by giving owlim enough time to
 shutdown properly (which can be a long time after loading large amounts
 of data on a machine with large memory). If it does happen, shutdown

Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-04-13 Thread Barry Bishop

Hi Nicholas,

Yikes, sorry to hear that you are in such a tight spot.

Do you know roughly how big your transactions are? This would provide a 
way to make a guess about how many are in the backup file.


I just tried an experiment:

LUBM(1) is made up of 15 files of approximately 10,000 statements each. 
The size of the backup file is 4MB.


If you (by some fluke) have the same average transaction size then I 
would guess there are this many transactions:


(8000MB/4MB)*15 = 30,000 transactions.

This is very 'back of a fag packet stuff' and depends completely on your 
average transaction size. If it was 5000 statements then expect 60,000 
transactions, if it was 30,000 statements per transaction then expect 
about 10,000 transactions.


Sorry, but this is the best I can do at this stage.

Best of luck,
barry

On 13/04/12 13:47, GOULD, Nicholas wrote:

Thanks for the quick response Barry. We figured that we would have to wait for 
it to complete. I have a project manager breathing down my neck for an ETA on 
this so if you are able to give us an idea of how long 'a long time' could be 
that would be helpful.

The transaction file is up to around 9000 after 1 hour 40 mins and as stated 
before the backup is 8.7 GB. Is it possible to estimate the number of 
transactions in the backup file?

Thanks again for your help.

Nic


From: Barry Bishop [barry.bis...@ontotext.com]
Sent: 13 April 2012 12:33
To: GOULD, Nicholas
Cc: Barry Norton; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hello Nicholas,

This can take a long time, I'm afraid. If the indices are damaged then
OWLIM will replay it's entire transaction log in order to recreate them.

My suggestion is just to wait until it completes. Terminating the
recovery process means that the next restart will fail to initialise and
cause the the message in the subject of this email to be displayed.

The good news is that the transaction mechanisms in OWLIM 5.0 have been
completely refactored in order to fix these kinds of inconveniences. We
will release this version next week.

Please see:

http://owlim.ontotext.com/display/OWLIMv50/OWLIM-SE+Indexing+Specifics#OWLIM-SEIndexingSpecifics-TransactionMode

I hope this helps,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
www.ontotext.com


On 13/04/12 13:24, GOULD, Nicholas wrote:

All,

Does anyone have experience of how long OWLIM should take to go through it's 
transaction log and update all indicies?

We have loaded a large volume of data (~7.5 GB of ttl files) into a repository 
and have a backup file of 8.7GB. We stopped Tomcat and on restart it is now 
going through transactions, presumably synching any differences between the 
disk and memory indicies when shutdown.

Many thanks,

Nic


From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] on behalf of Barry Bishop 
[barry.bis...@ontotext.com]
Sent: 01 April 2012 09:12
To: Barry Norton
Cc: owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hello all,

This message is displayed (and OWLIM refuses to continue) only after the
following sequence of events:

1. OWLIM is stopped and not shutdown properly
2. OWLIM is restarted and the recover process starts
3. OWLIM is stopped again without allowing the recovery process to finish
4. OWLIM is restarted

At this point, the recovery code is aware of the backed up indices, some
half built replacement indices and the transaction log. Without knowing
which set of indices is correct, it informs the user and stops.

(This is not an ideal situation and it is one that can not occur in OWLIM 5)

The important point is that you should always allow OWLIM time to
shutdown properly and should not kill tomcat, even if it seems to take a
long time. If you have got in to this situation, the probable means to
get started again is to shut down tomcat and delete any *.old files in
the owlim storage folder. Then restart and allow time for OWLIM to
replay its transaction log.

The cause of this is that OWLIM 4 (and before) uses a lazy writing
strategy for dirty pages. These are only swapped to disk when memory has
been exhausted and more pages are required. In other words, the state of
the indices on disk and in memory can diverge quite considerably and in
the event of a sudden stop, the correct indices must be rebuilt using
the transaction log (a file called 'backup' in the OWLIM storage folder).

OWLIM 5 still has this mode of operation available (it is called 'fast'
mode), but the default mode flushes updates to disk at the point of each
commit. It does this in such a way, that a failure at any time leaves
the disk image

Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-04-13 Thread Barry Bishop

Hi Nicholas,

Potentially yes, but if that is the case I am surprised that it is 
running so slowly.


Did you have reasonable performance during loading and how long did that 
take?


barry

On 13/04/12 14:32, GOULD, Nicholas wrote:

Thanks Barry,

Our ttl files are small, but lots of them, only around 30 triples per file.

I think, extrapolating from what you said below, that this means we are looking 
at a very high number of transactions?

Nic


From: Barry Bishop [barry.bis...@ontotext.com]
Sent: 13 April 2012 13:15
To: GOULD, Nicholas
Cc: Barry Norton; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hi Nicholas,

Yikes, sorry to hear that you are in such a tight spot.

Do you know roughly how big your transactions are? This would provide a
way to make a guess about how many are in the backup file.

I just tried an experiment:

LUBM(1) is made up of 15 files of approximately 10,000 statements each.
The size of the backup file is 4MB.

If you (by some fluke) have the same average transaction size then I
would guess there are this many transactions:

(8000MB/4MB)*15 = 30,000 transactions.

This is very 'back of a fag packet stuff' and depends completely on your
average transaction size. If it was 5000 statements then expect 60,000
transactions, if it was 30,000 statements per transaction then expect
about 10,000 transactions.

Sorry, but this is the best I can do at this stage.

Best of luck,
barry

On 13/04/12 13:47, GOULD, Nicholas wrote:

Thanks for the quick response Barry. We figured that we would have to wait for 
it to complete. I have a project manager breathing down my neck for an ETA on 
this so if you are able to give us an idea of how long 'a long time' could be 
that would be helpful.

The transaction file is up to around 9000 after 1 hour 40 mins and as stated 
before the backup is 8.7 GB. Is it possible to estimate the number of 
transactions in the backup file?

Thanks again for your help.

Nic


From: Barry Bishop [barry.bis...@ontotext.com]
Sent: 13 April 2012 12:33
To: GOULD, Nicholas
Cc: Barry Norton; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hello Nicholas,

This can take a long time, I'm afraid. If the indices are damaged then
OWLIM will replay it's entire transaction log in order to recreate them.

My suggestion is just to wait until it completes. Terminating the
recovery process means that the next restart will fail to initialise and
cause the the message in the subject of this email to be displayed.

The good news is that the transaction mechanisms in OWLIM 5.0 have been
completely refactored in order to fix these kinds of inconveniences. We
will release this version next week.

Please see:

http://owlim.ontotext.com/display/OWLIMv50/OWLIM-SE+Indexing+Specifics#OWLIM-SEIndexingSpecifics-TransactionMode

I hope this helps,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
www.ontotext.com


On 13/04/12 13:24, GOULD, Nicholas wrote:

All,

Does anyone have experience of how long OWLIM should take to go through it's 
transaction log and update all indicies?

We have loaded a large volume of data (~7.5 GB of ttl files) into a repository 
and have a backup file of 8.7GB. We stopped Tomcat and on restart it is now 
going through transactions, presumably synching any differences between the 
disk and memory indicies when shutdown.

Many thanks,

Nic


From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] on behalf of Barry Bishop 
[barry.bis...@ontotext.com]
Sent: 01 April 2012 09:12
To: Barry Norton
Cc: owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hello all,

This message is displayed (and OWLIM refuses to continue) only after the
following sequence of events:

1. OWLIM is stopped and not shutdown properly
2. OWLIM is restarted and the recover process starts
3. OWLIM is stopped again without allowing the recovery process to finish
4. OWLIM is restarted

At this point, the recovery code is aware of the backed up indices, some
half built replacement indices and the transaction log. Without knowing
which set of indices is correct, it informs the user and stops.

(This is not an ideal situation and it is one that can not occur in OWLIM 5)

The important point is that you should always allow OWLIM time to
shutdown properly and should not kill tomcat, even if it seems to take a
long time. If you have got in to this situation, the probable means to
get started again is to shut down tomcat and delete any *.old files

Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-04-13 Thread GOULD, Nicholas
Load took around 4 hours.


From: Barry Bishop [barry.bis...@ontotext.com]
Sent: 13 April 2012 13:52
To: GOULD, Nicholas
Cc: Barry Norton; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool 
has already failed once before to perform image restore.

Hi Nicholas,

Potentially yes, but if that is the case I am surprised that it is
running so slowly.

Did you have reasonable performance during loading and how long did that
take?

barry

On 13/04/12 14:32, GOULD, Nicholas wrote:
 Thanks Barry,

 Our ttl files are small, but lots of them, only around 30 triples per file.

 I think, extrapolating from what you said below, that this means we are 
 looking at a very high number of transactions?

 Nic

 
 From: Barry Bishop [barry.bis...@ontotext.com]
 Sent: 13 April 2012 13:15
 To: GOULD, Nicholas
 Cc: Barry Norton; owlim-discussion@ontotext.com
 Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer 
 tool has already failed once before to perform image restore.

 Hi Nicholas,

 Yikes, sorry to hear that you are in such a tight spot.

 Do you know roughly how big your transactions are? This would provide a
 way to make a guess about how many are in the backup file.

 I just tried an experiment:

 LUBM(1) is made up of 15 files of approximately 10,000 statements each.
 The size of the backup file is 4MB.

 If you (by some fluke) have the same average transaction size then I
 would guess there are this many transactions:

 (8000MB/4MB)*15 = 30,000 transactions.

 This is very 'back of a fag packet stuff' and depends completely on your
 average transaction size. If it was 5000 statements then expect 60,000
 transactions, if it was 30,000 statements per transaction then expect
 about 10,000 transactions.

 Sorry, but this is the best I can do at this stage.

 Best of luck,
 barry

 On 13/04/12 13:47, GOULD, Nicholas wrote:
 Thanks for the quick response Barry. We figured that we would have to wait 
 for it to complete. I have a project manager breathing down my neck for an 
 ETA on this so if you are able to give us an idea of how long 'a long time' 
 could be that would be helpful.

 The transaction file is up to around 9000 after 1 hour 40 mins and as stated 
 before the backup is 8.7 GB. Is it possible to estimate the number of 
 transactions in the backup file?

 Thanks again for your help.

 Nic

 
 From: Barry Bishop [barry.bis...@ontotext.com]
 Sent: 13 April 2012 12:33
 To: GOULD, Nicholas
 Cc: Barry Norton; owlim-discussion@ontotext.com
 Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer 
 tool has already failed once before to perform image restore.

 Hello Nicholas,

 This can take a long time, I'm afraid. If the indices are damaged then
 OWLIM will replay it's entire transaction log in order to recreate them.

 My suggestion is just to wait until it completes. Terminating the
 recovery process means that the next restart will fail to initialise and
 cause the the message in the subject of this email to be displayed.

 The good news is that the transaction mechanisms in OWLIM 5.0 have been
 completely refactored in order to fix these kinds of inconveniences. We
 will release this version next week.

 Please see:

 http://owlim.ontotext.com/display/OWLIMv50/OWLIM-SE+Indexing+Specifics#OWLIM-SEIndexingSpecifics-TransactionMode

 I hope this helps,
 barry

 Barry Bishop
 OWLIM Product Manager
 Ontotext AD
 Tel: +43 650 2000 237
 email: barry.bis...@ontotext.com
 www.ontotext.com


 On 13/04/12 13:24, GOULD, Nicholas wrote:
 All,

 Does anyone have experience of how long OWLIM should take to go through 
 it's transaction log and update all indicies?

 We have loaded a large volume of data (~7.5 GB of ttl files) into a 
 repository and have a backup file of 8.7GB. We stopped Tomcat and on 
 restart it is now going through transactions, presumably synching any 
 differences between the disk and memory indicies when shutdown.

 Many thanks,

 Nic

 
 From: owlim-discussion-boun...@ontotext.com 
 [owlim-discussion-boun...@ontotext.com] on behalf of Barry Bishop 
 [barry.bis...@ontotext.com]
 Sent: 01 April 2012 09:12
 To: Barry Norton
 Cc: owlim-discussion@ontotext.com
 Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer 
 tool has already failed once before to perform image restore.

 Hello all,

 This message is displayed (and OWLIM refuses to continue) only after the
 following sequence of events:

 1. OWLIM is stopped and not shutdown properly
 2. OWLIM is restarted and the recover process starts
 3. OWLIM is stopped again without allowing the recovery process to finish
 4. OWLIM is restarted

 At this point, the recovery code is aware of the backed up indices, some
 half built replacement indices and the transaction log. Without

Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-04-01 Thread Barry Bishop

Hello all,

This message is displayed (and OWLIM refuses to continue) only after the 
following sequence of events:


1. OWLIM is stopped and not shutdown properly
2. OWLIM is restarted and the recover process starts
3. OWLIM is stopped again without allowing the recovery process to finish
4. OWLIM is restarted

At this point, the recovery code is aware of the backed up indices, some 
half built replacement indices and the transaction log. Without knowing 
which set of indices is correct, it informs the user and stops.


(This is not an ideal situation and it is one that can not occur in OWLIM 5)

The important point is that you should always allow OWLIM time to 
shutdown properly and should not kill tomcat, even if it seems to take a 
long time. If you have got in to this situation, the probable means to 
get started again is to shut down tomcat and delete any *.old files in 
the owlim storage folder. Then restart and allow time for OWLIM to 
replay its transaction log.


The cause of this is that OWLIM 4 (and before) uses a lazy writing 
strategy for dirty pages. These are only swapped to disk when memory has 
been exhausted and more pages are required. In other words, the state of 
the indices on disk and in memory can diverge quite considerably and in 
the event of a sudden stop, the correct indices must be rebuilt using 
the transaction log (a file called 'backup' in the OWLIM storage folder).


OWLIM 5 still has this mode of operation available (it is called 'fast' 
mode), but the default mode flushes updates to disk at the point of each 
commit. It does this in such a way, that a failure at any time leaves 
the disk image consistent with the state of the database at the end of 
the last successful commit operation. For sustained loading operations, 
this can have lower performance, but the advantages are:

1. instant shutdown time
2. instant recovery time (even if the last instance was stopped abruptly)
3. improved concurrency of reads and write operations

This last point means that export the database programmatically is now a 
viable means to backup the database.


To summarise:

OWLIM 4 - this error occurs during initialisation when the OWLIM process 
has been killed, restarted and not allowed time to recover properly, 
i.e. killed again. It can be avoided by giving owlim enough time to 
shutdown properly (which can be a long time after loading large amounts 
of data on a machine with large memory). If it does happen, shutdown, 
delete the *.old files, restart and wait.


OWLIM 5 - using the default transaction mode, this problem will not 
occur. The fast transaction mode is available when administrators need 
maximum performance and reduced query/update concurrency, but will have 
no guarantees about data security if OWLIM is killed before shutting 
properly - switching between transaction modes requires a restart.


I hope this helps,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
www.ontotext.com


On 01/04/12 01:29, Barry Norton wrote:


Guys, how are you shutting down your database that it needs to be 
recovered so often?


And are you interrupting the recovery process?

Barry



On 31/03/2012 20:44, Blasetti, Luciano (CIOK) wrote:

Hi All,
I also experienced quite frequently this issue on a tomcat instance 
running only openrdf-sesame (OWLIM 4.3 SE) and openrdf-workbench. No 
way to restore the database.
Moreover, since there's no out-of-the-box functionalities to perform 
a hot backup on a non clustered configuration (and the programmatic 
export can't work on large triplestore due to timeout issues), this 
strange behavior leads to data loss.


I hope that the forthcoming OWLIM 5.0 will be more robust and reliable.

Regards,
Luciano

From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] On Behalf Of Marek Šurek 
[marek_su...@yahoo.co.uk]

Sent: Saturday, March 31, 2012 11:18 AM
To: Monika Solanki; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database 
restorer  tool has already failed once before to perform image 
restore.


Hi Monika,
I also experienced this exception number of times on OWLIM 4.3. The  
problem appears only when I ran openrdf-sesame and client application 
at the same Tomcat server and due to developing client application I 
had to restart Tomcat very offen to see changes in my client 
application. My solution was to divide those two applications into 
different tomcat instances so now they run independently(my client 
application runs on different Tomcat as openrdf-sesame) and I don't 
restart Tomcat instance with openrdf-sesame.war almost never.

I hope this would help,
Marek


From: Monika Solankimsolanki.maili...@gmail.com
To: owlim-discussion@ontotext.com
Sent: Saturday, 31 March 2012, 10:59
Subject: [Owlim-discussion] CRITICAL FAILURE !!! The database

[Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-03-31 Thread Monika Solanki
Hello Again,

A couple of times I have had this error thrown to me and I have had to
populate the repository again.

CRITICAL FAILURE !!! The database restorer tool has already failed once
before to perform image restore.

I would appreciate any help on how I can possibly avoid this error.

Thanks,

Monika
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-03-31 Thread Marek Šurek
Hi Monika, 

I also experienced this exception number of times on OWLIM 4.3. The  problem 
appears only when I ran openrdf-sesame and client application at the same 
Tomcat server and due to developing client application I had to restart Tomcat 
very offen to see changes in my client application. My solution was to divide 
those two applications into different tomcat instances so now they run 
independently(my client application runs on different Tomcat as openrdf-sesame) 
and I don't restart Tomcat instance with openrdf-sesame.war almost never.
I hope this would help,
Marek




 From: Monika Solanki msolanki.maili...@gmail.com
To: owlim-discussion@ontotext.com 
Sent: Saturday, 31 March 2012, 10:59
Subject: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has 
already failed once before to perform image restore.
 

Hello Again,

A couple of times I have had this error thrown to me and I have had to populate 
the repository again. 

CRITICAL FAILURE !!! The database restorer tool has already failed once before 
to perform image restore.

I would appreciate any help on how I can possibly avoid this error. 

Thanks,

Monika


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-03-31 Thread Blasetti, Luciano (CIOK)
Hi All,
I also experienced quite frequently this issue on a tomcat instance running 
only openrdf-sesame (OWLIM 4.3 SE) and openrdf-workbench. No way to restore the 
database.
Moreover, since there's no out-of-the-box functionalities to perform a hot 
backup on a non clustered configuration (and the programmatic export can't work 
on large triplestore due to timeout issues), this strange behavior leads to 
data loss.

I hope that the forthcoming OWLIM 5.0 will be more robust and reliable.

Regards,
Luciano

From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] On Behalf Of Marek Šurek 
[marek_su...@yahoo.co.uk]
Sent: Saturday, March 31, 2012 11:18 AM
To: Monika Solanki; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer  
tool has already failed once before to perform image restore.

Hi Monika,
I also experienced this exception number of times on OWLIM 4.3. The  problem 
appears only when I ran openrdf-sesame and client application at the same 
Tomcat server and due to developing client application I had to restart Tomcat 
very offen to see changes in my client application. My solution was to divide 
those two applications into different tomcat instances so now they run 
independently(my client application runs on different Tomcat as openrdf-sesame) 
and I don't restart Tomcat instance with openrdf-sesame.war almost never.
I hope this would help,
Marek


From: Monika Solanki msolanki.maili...@gmail.com
To: owlim-discussion@ontotext.com
Sent: Saturday, 31 March 2012, 10:59
Subject: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has 
already failed once before to perform image restore.

Hello Again,

A couple of times I have had this error thrown to me and I have had to populate 
the repository again.

CRITICAL FAILURE !!! The database restorer tool has already failed once before 
to perform image restore.

I would appreciate any help on how I can possibly avoid this error.

Thanks,

Monika


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.commailto:Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has already failed once before to perform image restore.

2012-03-31 Thread Barry Norton


Guys, how are you shutting down your database that it needs to be 
recovered so often?


And are you interrupting the recovery process?

Barry



On 31/03/2012 20:44, Blasetti, Luciano (CIOK) wrote:

Hi All,
I also experienced quite frequently this issue on a tomcat instance running 
only openrdf-sesame (OWLIM 4.3 SE) and openrdf-workbench. No way to restore the 
database.
Moreover, since there's no out-of-the-box functionalities to perform a hot 
backup on a non clustered configuration (and the programmatic export can't work 
on large triplestore due to timeout issues), this strange behavior leads to 
data loss.

I hope that the forthcoming OWLIM 5.0 will be more robust and reliable.

Regards,
Luciano

From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] On Behalf Of Marek Šurek 
[marek_su...@yahoo.co.uk]
Sent: Saturday, March 31, 2012 11:18 AM
To: Monika Solanki; owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer  
tool has already failed once before to perform image restore.

Hi Monika,
I also experienced this exception number of times on OWLIM 4.3. The  problem 
appears only when I ran openrdf-sesame and client application at the same 
Tomcat server and due to developing client application I had to restart Tomcat 
very offen to see changes in my client application. My solution was to divide 
those two applications into different tomcat instances so now they run 
independently(my client application runs on different Tomcat as openrdf-sesame) 
and I don't restart Tomcat instance with openrdf-sesame.war almost never.
I hope this would help,
Marek


From: Monika Solankimsolanki.maili...@gmail.com
To: owlim-discussion@ontotext.com
Sent: Saturday, 31 March 2012, 10:59
Subject: [Owlim-discussion] CRITICAL FAILURE !!! The database restorer tool has 
already failed once before to perform image restore.

Hello Again,

A couple of times I have had this error thrown to me and I have had to populate 
the repository again.

CRITICAL FAILURE !!! The database restorer tool has already failed once before 
to perform image restore.

I would appreciate any help on how I can possibly avoid this error.

Thanks,

Monika


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.commailto:Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion