[h2] License Statement for PR 2637

[Sven Uhlig]

Hello,

here is my license statement for
https://github.com/h2database/h2database/pull/2637

I wrote the code, it's mine, and I'm contributing it to H2 for
distribution multiple-licensed under the MPL 2.0, and the EPL 1.0
(https://h2database.com/html/license.html).

Best Regards
Sven Uhlig.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/4b7af0cb-40fe-0d18-b890-701ce63239ef%40resident-uhlig.de.

Re: [h2] corrupt data base

['Sven Eric Panitz' via H2 Database]

Thanks for trying to help.

Reading the second block would be done somewhere around here?

// read the first two blocks
ByteBuffer fileHeaderBlocks = fileStore.readFully(0, 2 * 
BLOCK_SIZE);
byte[] buff = new byte[BLOCK_SIZE];
for (int i = 1; i <= BLOCK_SIZE; i += BLOCK_SIZE) {
fileHeaderBlocks.get(buff);
// the following can fail for various reasons


like this?
ByteBuffer fileHeaderBlocks = fileStore.readFully(BLOCK_SIZE,  
BLOCK_SIZE);


Whatever I try, I get the same error message. At least a bit catching 
errors leads to some small output files, but not to my large data base I am 
trying to restore.


Am Montag, 9. Juli 2018 12:33:51 UTC+2 schrieb Noel Grandin:
>
>
>
> On 2018/07/09 12:15 PM, 'Sven Eric Panitz' via H2 Database wrote: 
> > Okay I cloned the repository and I am willing to tweak the source code 
> to ignore the errors. Anyone out there who can 
> > guide me a bit. Reading chunks gives the error. 
> > 
>
> try making the loop at the top of readStoreHeader use the second block, 
> that might pointer to slightly older, but still 
> consistent data. 
>
> And just comment out that throws code, and see how far you get. 
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Re: [h2] corrupt data base

['Sven Eric Panitz' via H2 Database]

Okay I cloned the repository and I am willing to tweak the source code to 
ignore the errors. Anyone out there who can guide me a bit. Reading chunks 
gives the error.

java.lang.IllegalStateException: Chunk 60877 is invalid [1.4.197/6]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:881)
at org.h2.mvstore.MVStore.loadChunkMeta(MVStore.java:772)
at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:730)
at org.h2.mvstore.MVStore.(MVStore.java:380)
at org.h2.mvstore.MVStore$Builder.open(MVStore.java:3210)
at org.h2.tools.Recover.dumpMVStoreFile(Recover.java:606)
at org.h2.tools.Recover.process(Recover.java:348)
at org.h2.tools.Recover.runTool(Recover.java:196)
at org.h2.tools.Recover.main(Recover.java:159)

What would be a good attemp to ignore this invalid value?

Am Sonntag, 8. Juli 2018 16:51:05 UTC+2 schrieb Sven Eric Panitz:
>
> In other words. No simple way to rescue my data. So I hope to find some 
> recent back up files
>
> Am Sonntag, 8. Juli 2018 15:06:53 UTC+2 schrieb Noel Grandin:
>>
>> If you are willing to build from source, you can sometimes tweak the 
>> source code to ignore certain errors and insert good values when it seems 
>> bad ones, in order to run the Recover tool.
>> ​
>>
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Re: [h2] corrupt data base

['Sven Eric Panitz' via H2 Database]

In other words. No simple way to rescue my data. So I hope to find some 
recent back up files

Am Sonntag, 8. Juli 2018 15:06:53 UTC+2 schrieb Noel Grandin:
>
> If you are willing to build from source, you can sometimes tweak the 
> source code to ignore certain errors and insert good values when it seems 
> bad ones, in order to run the Recover tool.
> ​
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] corrupt data base

['Sven Eric Panitz' via H2 Database]

I am using H2 within a scala/lift Web application. Due to a 
OutOfMemoryError my data base is corrupted.

The recovery tool however seems to have problems with the data-base:

java -cp lib/h2*.jar org.h2.tools.Recover
Exception in thread "main" java.lang.IllegalStateException: Chunk 60877 is 
invalid [1.4.187/6]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:773)
at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:695)
at org.h2.mvstore.MVStore.(MVStore.java:348)
at org.h2.mvstore.MVStore$Builder.open(MVStore.java:2782)
at org.h2.mvstore.MVStoreTool.info(MVStoreTool.java:315)
at org.h2.tools.Recover.process(Recover.java:341)
at org.h2.tools.Recover.runTool(Recover.java:195)
at org.h2.tools.Recover.main(Recover.java:158)


Any chance, that I can rescue my data base?

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Re: Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

Update (just that this ) won't happen:

The issue could be resolved, the pull request 
https://github.com/h2database/h2database/pull/391/ 

 has 
been merged into master.

Adding/removing null spatial keys into MVSpatialIndex 

 is 
pointless, as they cannot be found based on a bounding box. Therefore, they 
are ignored. Thanks for watching.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Re: Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

I have created a pull request on this 
issue: https://github.com/h2database/h2database/pull/391/
I would appreciate some feedback on this. 

Basically, the whole isNull() funtionality in *SpatialDataType *is not 
working at all. Maybe Null Geometries can be added (and retrieved with 
hacks) to an R-Tree this way, but nodes can never be split in a defined 
manner. So in a large tree with many null and not-null geometries, it is 
best to give at least some bogus bounding box in which entries can then be 
arranged.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Re: Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

Any idea anyone? The example script I provided makes it quite easy to 
reproduce the error...

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Re: Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

..forgot to mention, the *get* function takes place in MVRTreeMap.java.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Re: Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

Hi,

Nicolas, thanks for the remark, and sorry for not providing an example. I 
do have the latest version so the handling of null geometries is up to 
date. One test function that has been added in the pull request you 
mentioned only checks for a small table (size one I believe). However, in 
that case, the spatial key is not used for lookup because the MVTree is 
small enough to contain only leaves.


Here is a small example that yields the error I mean. Note that the test 
script *passes for small table sizes.*

public void testIndexUpdateNullGeometry2() throws SQLException {
deleteDb("spatial");
Connection conn = getConnection(URL);
Statement stat = conn.createStatement();
stat.execute("drop table if exists DUMMY_11;");
stat.execute(
"CREATE TABLE PUBLIC.DUMMY_11 (fid serial,  GEOM GEOMETRY, Name 
varchar(255));");
stat.execute("CREATE SPATIAL INDEX PUBLIC_DUMMY_11_SPATIAL_INDEX on"
+ " PUBLIC.DUMMY_11(GEOM);");
for (int i = 0; i < 100; i++) {
stat.execute("insert into PUBLIC.DUMMY_11(geom) values(null);");
}
stat.execute("update PUBLIC.DUMMY_11 set Name='test' where fid = 5");
conn.close();
deleteDb("spatial");
}



The row fails to be found here, because with larger tables, the root Page 
is not a leaf any more, relying on the above mentioned contains relation...

protected Object get(Page p, Object key) {
if (!p.isLeaf()) {
List l = new ArrayList();
for (int i = 0; i < p.getKeyCount(); i++) {
Object o = get(p.getChildPage(i), key);
if (o != null) {
l.add(o);
}
}
for (int i = 0; i < p.getKeyCount(); i++) {
if (contains(p, i, key)) {
Object o = get(p.getChildPage(i), key);
if (o != null) {
return o;
}
}
}
} else
{
for (int i = 0; i < p.getKeyCount(); i++) {
if (keyType.equals(p.getKey(i), key)) {
return p.getValue(i);
}
}
}
return null;
}


kind regards,

Sven

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Re: Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

I forgot another way to find such keys, maybe this is even safer. Couldn't 
the *Page *be passed down to the *contains *function and perform a binary 
search on the key in question?

public boolean contains(Page p, int index, Object key) {
SpatialKey a = (SpatialKey) p.getKey(index);
SpatialKey b = (SpatialKey) key;

if (a.isNull() || b.isNull()) {
return p.binarySearch(a) >= 0; // <
}
for (int i = 0; i < dimensions; i++) {
if (a.min(i) > b.min(i) || a.max(i) < b.max(i)) {
return false;
}
}
return true;
}



-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[h2] Entries in a table containing Geometries cannot be found when the Geometry is empty ( Row not found when trying to delete from index )

[Sven]

Hello,

suppose I have a table with Geometries. This kind of datatype is handled by 
the ValueGeometry class, and an entry appears to be handled in a *Page, *that 
organizes its entries in an R-Tree, using the *SpatialKey *class as keys. 
Getting an entry can then be effectively performed by a simple 
contains-relation that is well-defined on the bounding box of 
each SpatialKey - unless the Geometry is null.

The *contains *Function in *SpatialDataType.java *always returns false when 
a key to be found is null. In that case, one can never access a row with a 
null-geometry, and I end up with an exception like this

Caused by: org.h2.jdbc.JdbcSQLException: Row not found when trying to 
delete from index "PUBLIC.GEOMETRIES: 105" [90112-192]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.mvstore.db.MVSpatialIndex.remove(MVSpatialIndex.java:177)
at org.h2.mvstore.db.MVTable.removeRow(MVTable.java:677)
at org.h2.table.Table.updateRows(Table.java:489)
at org.h2.command.dml.Update.update(Update.java:145)
at org.h2.command.CommandContainer.update(CommandContainer.java:98)
at org.h2.command.Command.executeUpdate(Command.java:258)
... 10 more


This is caused by the following function in SpatialDataType.java:

public boolean contains(Object objA, Object objB) {
SpatialKey a = (SpatialKey) objA;
SpatialKey b = (SpatialKey) objB;
if (a.isNull() || b.isNull()) {
return false;
}
for (int i = 0; i < dimensions; i++) {
if (a.min(i) > b.min(i) || a.max(i) < b.max(i)) {
return false;
}
}
return true;
}


Now in order for the key to be found, a at least some contains-relation HAS 
to be present for empty SpatialKeys, for which we only have the ID.

What *seems *to work is a simple relation that works on the IDs abusing the 
<= relation:

public boolean contains(Object objA, Object objB) {
SpatialKey a = (SpatialKey) objA;
SpatialKey b = (SpatialKey) objB;
if (a.isNull() || b.isNull()) {
return a.getId() <= b.getId(); // <---
}
for (int i = 0; i < dimensions; i++) {
if (a.min(i) > b.min(i) || a.max(i) < b.max(i)) {
return false;
}
}
return true;
}


Even simpler seems to be to always return true:

  public boolean contains(Object objA, Object objB) {
SpatialKey a = (SpatialKey) objA;
SpatialKey b = (SpatialKey) objB;
if (a.isNull() || b.isNull()) {
return true; // <
}
for (int i = 0; i < dimensions; i++) {
if (a.min(i) > b.min(i) || a.max(i) < b.max(i)) {
return false;
}
}
return true;
}

However, I have no insight as to how the MVTree is then built up, and how 
that may affect performance. 

In any case, this change will *never work on existing databases, and any 
changes to this relation will render all empty Geometries unreachable.*


Does anyone have a deeper understanding of this kind of data retrieval, or 
do you have a remark on the fixes I suggest? I would greatly appreciate any 
remark...

Kind regards,

Sven

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Re: H2 page size and caching

[Sven Mathijssen]

Vince,

Thanks for your answer.

 I don't know about Maven, but I checkout the H2 trunk using subclipse
 and build using the Ant build.xml script that's in the project root
 directory. I've also built successfully using the built.bat
 command-line script.

Yes, I understand, only when I checkout the program using subclipse,
the root of the project is trunk/h2/src/installer. I expected it to be
at trunk/ or something, because now I am unable to access the src
directory from the path mentioned. No problem in building, but I'm
interested in the source. I once managed to check out correctly. Going
to try again now!

 - How does H2 behave when using a cache size of 0? I connect to the
 database using an URL like: jdbc:h2:~/experiments/
 db;DB_CLOSE_ON_EXIT=FALSE;CACHE_SIZE=0.

Besides, what I mean by this is does H2 reserve at least SOME space
for buffering data at all? Cache size of 0 is of course not
preferable, but I want to have the effect of swapping buffer contents
to disk as soon as possible. Is there some lower limit on the cache
size? The amount of page transfers is very, very high. I expected it
to be high, but my own storage engine prototype (very simple one) uses
 16K for buffering data from disk and generates a number of page
transfers orders of magnitude lower than H2 does, which surprises me a
little. All this still *assumes* the use of buffers each having a size
equal to the page size used and also *assumes* that the paging model
from database theory is used, which I am not able to figure out yet.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups H2 
Database group.
To post to this group, send email to h2-database@googlegroups.com
To unsubscribe from this group, send email to 
h2-database+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/h2-database?hl=en
-~--~~~~--~~--~--~---

Re: H2 page size and caching

[Sven Mathijssen]

Found some interesting things in the source (DiskFile.java):

 * There are 'blocks' of 128 bytes (DiskFile.BLOCK_SIZE). Each objects
own one
 * or more pages; each page size is 64 blocks
(DiskFile.BLOCKS_PER_PAGE). That
 * is 8 KB page size. However pages are not read or written as one
unit; only
 * individual objects (multiple blocks at a time) are read or written.

As I expected, 8 KB page size (which is, I think, kind of usual for a
database system). Maybe the high values for number of I/O operations
are due to the last comment: Only individual objects are read or
written.

This information is extracted by issuing

SELECT * FROM INFORMATION_SCHEMA.SETTINGS WHERE NAME =
'info.FILE_DISK_READ' ORDER BY NAME ASC;

Both before and after any statement is sent to the database. In this
way, I was hoping to count page transfers, but as I can see now, what
I'm probably doing is counting I/O *block* operations.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups H2 
Database group.
To post to this group, send email to h2-database@googlegroups.com
To unsubscribe from this group, send email to 
h2-database+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/h2-database?hl=en
-~--~~~~--~~--~--~---

H2 page size and caching

[Sven Mathijssen]

Thomas,

I was wondering about the following things:

- Does H2 use paging by default? If not, is it possible to use this?
- What is the default page size used (assuming paging is used to store
data on disk).
- Are there any guidelines somewhere about checking out the source
code in an IDE (I use Eclipse). It should be possible to checkout from
the SVN repository and build using Maven, but all I get when checking
out the project using subclipse is
- How does H2 behave when using a cache size of 0? I connect to the
database using an URL like: jdbc:h2:~/experiments/
db;DB_CLOSE_ON_EXIT=FALSE;CACHE_SIZE=0. Background: it's for running
some experiments for my master's project, and I'm trying to force the
system into swapping from and to disk, counting the number of I/O
operations in the process).

Thanks!

Sven
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups H2 
Database group.
To post to this group, send email to h2-database@googlegroups.com
To unsubscribe from this group, send email to 
h2-database+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/h2-database?hl=en
-~--~~~~--~~--~--~---