I'm seeing something interesting. If I comment out the call to compact(),
1. the DB file size increases only on the first run of the program but
not on subsequent runs, which is exactly what should happen.
2. the "File corrupted while reading record " exception also is no
more.
class Database {
...
private synchronized void closeOpenFilesAndUnlock(boolean flush) {
stopWriter();
if (pageStore != null) {
if (flush) {
try {
pageStore.checkpoint();
if (!readOnly) {
lockMeta(pageStore.getPageStoreSession());
// pageStore.compact(compactMode); <------
commented out, it works
Is there any internal, idempotent verification-function in H2 that I could
call in between the statements of the compact() method, to be able to hit
the exact point at which corruption begins to happens?
On Saturday, May 10, 2014 4:14:35 PM UTC+5:30, hs wrote:
>
> Hi,
>
> I'm reporting back on my unit-testing results.
>
> class TestFileSystem {
> ...
> public void test() throws Exception {
> testFileSystem(getBaseDir() + "/fs"); // Passes
> testAbsoluteRelative(); // Passes
> testDirectories(getBaseDir()); // Passes
> testMoveTo(getBaseDir()); // Passes
>
>
>
> // ALL subsequent ones fail!
>
> // testUnsupportedFeatures(getBaseDir());
> // ...
>
>
> The first 4 testXXX() calls now pass. I did manage to discover and fix a
> few bugs in my FileChannel and RandomAccessFile classes using these tests,
> so thanks!
> The testUnsupportedFeatures() call fails because my file-system does
> indeed support some of those features instead of throwing
> UnsupportedOperationException.
> All of the rest of the testXXX() calls that test more exotic H2 features
> like zip:, memfs:, etc fail because I don't support them. I hope H2 does
> not rely on those features if my connection URL is simply "
> jdbc:h2:foofs:/abc;CIPHER=AES".
>
> Can I now conclude that my file-system code meets the requirements needed
> to run H2? Obviously not(!)... because the original problems -- namely,
> DB file size increasing upon each run, and the exception during
> connection-close -- still remain! The PageStore.compact() method is not
> doing its compacting job for some reason and due to complete lack of H2
> internals knowledge I don't know how to debug this further.
>
> Any suggestions on what to do next?
>
> Regards,
> /PN
>
>
> PS: Btw, I found a minor difference between H2's Channel and that of the
> JDK spec. The last asserEquals in the following snippet fails...
>
> class TestFileSystem {
> ...
> private void testRandomAccess(String fsBase, int seed) throws
> Exception {
> StringBuilder buff = new StringBuilder();
> String s = FileUtils.createTempFile(fsBase + "/tmp", ".tmp",
> false, false);
> File file = new File(TestBase.BASE_TEST_DIR + "/tmp");
> file.getParentFile().mkdirs();
> file.delete();
> RandomAccessFile ra = new RandomAccessFile(file, "rw");
> FileUtils.delete(s);
> FileChannel f = FileUtils.open(s, "rw");
> assertEquals(s, f.toString());
>
>
> ... because in the JDK spec there's no requirement that a file channel's
> toString() must return the name of the file it is backing. As a quick fix,
> I've added toString() to my own Channel class but this is just an fyi.
>
> On Wednesday, April 30, 2014 9:59:10 PM UTC+5:30, Thomas Mueller wrote:
>>
>> Hi,
>>
>> Sure, H2 we can change, but I guess you want to support other
>> applications as well, and some are not as easy to change. If you can show
>> there is a bug in H2 or the unit test of H2: patches are welcome!
>>
>> > But if I could keep it optional, I would have preferred to /not/ use
>> it at all!
>>
>> What I mean is:
>>
>> Legacy: For some libraries and applications that use java.io.* directly,
>> you can use the bootclasspath part (changed java.io. classes). That way
>> the application and library doesn't have to be changed, but starting the
>> JVM is more complicated.
>>
>> Clean: For other libraries and applications, for example H2, that do have
>> a file system abstraction or use Java 7 style NIO2. That simplifies
>> starting those applications, as no bootclasspath hacks are needed.
>>
>> There might be mixed cases (both Legacy and Clean) within the same JVM,
>> that would result in Legacy mode. But there are also Clean use cases, for
>> example running the H2 server.
>>
>> Regards,
>> Thomas
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Apr 30, 2014 at 9:19 AM, PN <pingn...@gmail.com> wrote:
>>
>>> Hi Thomas.
>>>
>>>
>>> > I'm overriding 4 classes of the java.io package
>>>>
>>>> In theory, what you do should work, but making it 100% compatible with
>>>> java.io.* is very hard, if not impossible: some applications may rely on
>>>> implementation details. You could argue this is a bug in the application,
>>>> but it's still problematic to have to change the bootclasspath to use your
>>>> tool.
>>>>
>>>
>>> By "application", I believe you mean H2 here. I would greatly appreciate
>>> if you could share some insights on why achieving 100% compatibility with
>>> java.io.* would be hard or impossible, especially when H2, like all
>>> Java clients, would, in my guess,
>>> a) be relying only on the public, documented class interface of
>>> java.io.*
>>> AND
>>> b) H2's test suite (such as TestFileSystem) further asserting on
>>> this compatibility or lack thereof.
>>> AND
>>> c) the fact of H2's availability on multiple OSes and JVMs.
>>>
>>> In fact, wouldn't this the current situation be a good usecase for
>>> re-examining (a) or (b)?
>>>
>>> Just fyi, all that I'm doing in my tweaks to the 4 java.io.* classes is
>>> scheme-routing. Meaning, every method body in my version now has a giant
>>> if-else inside: If the scheme of the filename is "myfs:" then I route it to
>>> my code; else I use the original code. That's all!
>>>
>>> So, if TestFileSystem is unable to pass some test, it may very well be
>>> due to some bug in my code. Or, it could be because the H2 test suite is
>>> too 'tight' in some of its assumptions (bootclasspath use or no-use being
>>> one example, but there could be others too). Or, both.
>>>
>>> Since H2 has been around for many years, and would be much more stable
>>> and robust by now in every way especially compared to my code which is just
>>> work-in-progress, I would love to use H2 to test my code. But only if I
>>> know at the outset that things like bootclasspath are allowed and no
>>> implementation-specific java.io.* features are in use.
>>>
>>>
>>> What I would do is keep that part (the java.io.* part) optional, so your
>>>> tool can be used without having to change the bootclasspath. Then of
>>>> course
>>>> can only be used with H2 and Java 7+, but it's much cleaner and easier to
>>>> use.
>>>>
>>>
>>> But if I could keep it optional, I would have preferred to /not/ use it
>>> at all! My constraints is: The older Java apps, that have no counterpart
>>> for NIO2 of Java 7 or FilePath of H2, have to run alongside H2 in the same
>>> JVM!
>>>
>>> Regards,
>>> /PN
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "H2 Database" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to h2-database...@googlegroups.com.
>>> To post to this group, send email to h2-da...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/h2-database.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
--
You received this message because you are subscribed to the Google Groups "H2
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to h2-database+unsubscr...@googlegroups.com.
To post to this group, send email to h2-database@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.
