Andy, sorry, I'll attach the patch again (as TestTransSystem4.patch in https://issues.apache.org/jira/browse/JENA-115). Were you able to see the problems with TestTransSystem on your Windows box?
Simon From: Andy Seaborne <[email protected]> To: [email protected] Date: 09/15/2011 09:10 AM Subject: Re: [jira] [Commented] (JENA-91) extremely large buffer is being created in ObjectFileStorage (nothing attached) I can recreate on windows/64, not linux (only tried 64 bit) I have not tried (windows|linux)/32. Some a few tests of the overall datasets change by architecture but most do not - the current failures are neutral. I do have to use real hardware, not a VM, which slows investigation time down. (This is unrelated - I have never got a VirtualBox particular stable with a 64bit OS; I don't usually even get to the tests.) Some of the errors are spurious - failing because windows handling of mmaped files and delete. These can be coded around and I'll check in some mods for that. TestBlockAccessMapped fileaccess_03, fileaccess_04 (these were reusing a mapped file) and maybe some, but not all, of the others. Simon - the patch file TestTransSystem4.patch is not marked as licensed for Apache works. Andy On 14/09/11 22:25, Simon Helsen wrote: > perhaps it is considered costly and not everyone needs it. Anyhow, here > it is: > > > > > From: Andy Seaborne <[email protected]> > To: [email protected] > Date: 09/14/2011 05:20 PM > Subject: Re: [jira] [Commented] (JENA-91) extremely large buffer is > being created in ObjectFileStorage > > > ------------------------------------------------------------------------ > > > > Simon, > > Thanks, I can produce them from those file (not sure why the normal > build does not do it, it's > > mvn surefire-report:report-only > ) > Andy > > On 14/09/11 22:11, Simon Helsen wrote: > > Andy, > > > > keep in mind that my patched version of TestTransSystem also fails in > > direct mode so please try that as well in case the regular tests only > > fail because of known limitations. We (IBM) only use direct mode and my > > internal tests fail equally in direct mode. > > Here is the report (not sure about html, it only contains .txt and .xml) > > > > > > > > Simon > > > > > > > > From: Andy Seaborne <[email protected]> > > To: [email protected] > > Date: 09/14/2011 05:02 PM > > Subject: Re: [jira] [Commented] (JENA-91) extremely large buffer is > > being created in ObjectFileStorage > > > > > > ------------------------------------------------------------------------ > > > > > > > > One note: some of the tests are known to fail on Windows 64 bit because > > windows does not release memory mapped files in the same way Linux does > > (Linux does it mid-run). That might cover some of the test failures but > > not all of them. > > > > I'll find a windows/64 machine. > > > > Simon - could you send me the surefire HTML report please? > > > > Andy > > > > > > > > On 14/09/11 19:04, Simon Helsen wrote: > > > actually, I just executed exactly what you (Paolo) did, but on > > Windows. My > > > results are not quite like what Dave saw, but they fail > nonetheless. I'll > > > add this to Jira > > > > > > Here is the output: > > > > > > D:\jfsDev\TxTDB>svn up > > > At revision 1170749. > > > > > > D:\jfsDev\TxTDB>mvn clean test > > > [INFO] Scanning for projects... > > > [INFO] > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building TxTDB 0.9.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ tx-tdb --- > > > [INFO] Deleting D:\jfsDev\TxTDB\target > > > [INFO] > > > [INFO] --- maven-antrun-plugin:1.3:run (ant-timestamp) @ tx-tdb --- > > > [INFO] Executing tasks > > > [mkdir] Created dir: D:\jfsDev\TxTDB\target > > > [echo] version=0.9.0-SNAPSHOT > > > [echo] build.time=2011-09-14 14:00 EDT > > > [INFO] Executed tasks > > > [INFO] > > > [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ > > > tx-tdb --- > > > > > > [debug] execute contextualize > > > [INFO] Using 'UTF-8' encoding to copy filtered resources. > > > [INFO] Copying 1 resource to com/hp/hpl/jena/tdb/ > > > [INFO] > > > [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ > tx-tdb > > > --- > > > [INFO] Compiling 268 source files to D:\jfsDev\TxTDB\target\classes > > > [INFO] > > > [INFO] --- maven-resources-plugin:2.5:testResources > > > (default-testResources) @ tx > > > -tdb --- > > > [debug] execute contextualize > > > [INFO] Using 'UTF-8' encoding to copy filtered resources. > > > [INFO] skip non existing resourceDirectory > > > D:\jfsDev\TxTDB\src\test\resources > > > [INFO] > > > [INFO] --- maven-compiler-plugin:2.3.2:testCompile > > (default-testCompile) @ > > > tx-td > > > b --- > > > [INFO] Compiling 115 source files to > D:\jfsDev\TxTDB\target\test-classes > > > [INFO] > > > [INFO] --- maven-surefire-plugin:2.9:test (default-test) @ tx-tdb --- > > > [INFO] Surefire report directory: > D:\jfsDev\TxTDB\target\surefire-reports > > > > > > ------------------------------------------------------- > > > T E S T S > > > ------------------------------------------------------- > > > Running com.hp.hpl.jena.tdb.assembler.TS_TDBAssembler > > > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 2.087 sec > > > Running com.hp.hpl.jena.tdb.base.block.TS_Block > > > Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.803 sec > > > Running com.hp.hpl.jena.tdb.base.file.TS_File > > > Tests run: 39, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: > > 0.054 sec > > > <<< F > > > AILURE! > > > Running com.hp.hpl.jena.tdb.base.objectfile.TS_ObjectFile > > > Tests run: 48, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.091 sec > > > Running com.hp.hpl.jena.tdb.base.record.TS_Record > > > Tests run: 54, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.047 sec > > > Running com.hp.hpl.jena.tdb.base.recordfile.TS_RecordFile > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 0.002 sec > > > Running com.hp.hpl.jena.tdb.graph.TS_Graph > > > Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.674 sec > > > Running com.hp.hpl.jena.tdb.index.TS_Index > > > Tests run: 97, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.242 sec > > > Running com.hp.hpl.jena.tdb.lib.TS_LibTDB > > > Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 0.009 sec > > > Running com.hp.hpl.jena.tdb.migrate.TS_Migrate > > > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 0.01 sec > > > Running com.hp.hpl.jena.tdb.nodetable.TS_NodeTable > > > Tests run: 37, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 0.05 sec > > > Running com.hp.hpl.jena.tdb.solver.TS_SolverTDB > > > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 0.077 sec > > > Running com.hp.hpl.jena.tdb.store.TS_Store > > > Tests run: 181, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 2.094 > > > sec<<< > > > FAILURE! > > > Running com.hp.hpl.jena.tdb.sys.TS_Sys > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > 0.013 sec > > > Running com.hp.hpl.jena.tdb.transaction.TS_Transaction > > > Tests run: 77, Failures: 13, Errors: 0, Skipped: 0, Time elapsed: 3.569 > > > sec<<< > > > FAILURE! > > > Running com.hp.hpl.jena.tdb.TS_Factory > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec > > > Running com.hp.hpl.jena.tdb.TS_Jena > > > Tests run: 73, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: > > 0.096 sec > > > > > > Results : > > > > > > Failed tests: > > > fileaccess_03(com.hp.hpl.jena.tdb.base.file.TestBlockAccessMappe > > > d) > > > fileaccess_04(com.hp.hpl.jena.tdb.base.file.TestBlockAccessMapped) > > > nodetrans_03(com.hp.hpl.jena.tdb.transaction.TestNodeTableTransDisk): > > > expected > > > :<[DoesNotExist]> but was:<[0000000000000000]> > > > trans_04(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > trans_06(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_01(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_02(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_03(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_04(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_05(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_06(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_07(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_08(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_09(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > > > > trans_readBlock_11(com.hp.hpl.jena.tdb.transaction.TestTransSequentialDisk) > > > > > > Tests in error: > > > dataset2(com.hp.hpl.jena.tdb.store.TestDatasetTDBPersist) > > > dataset3(com.hp.hpl.jena.tdb.store.TestDatasetTDBPersist) > > > > > > Tests run: 697, Failures: 15, Errors: 2, Skipped: 0 > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD FAILURE > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 25.664s > > > [INFO] Finished at: Wed Sep 14 14:00:52 EDT 2011 > > > [INFO] Final Memory: 30M/83M > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [ERROR] Failed to execute goal > > > org.apache.maven.plugins:maven-surefire-plugin:2. > > > 9:test (default-test) on project tx-tdb: There are test failures. > > > [ERROR] > > > [ERROR] Please refer to D:\jfsDev\TxTDB\target\surefire-reports for the > > > individu > > > al test results. > > > [ERROR] -> [Help 1] > > > [ERROR] > > > [ERROR] To see the full stack trace of the errors, re-run Maven > with the > > > -e swit > > > ch. > > > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > > > [ERROR] > > > [ERROR] For more information about the errors and possible solutions, > > > please rea > > > d the following articles: > > > [ERROR] [Help 1] > > > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExc > > > eption > > > D:\jfsDev\TxTDB> > > > > > > > > > > > > From: > > > Paolo Castagna<[email protected]> > > > To: > > > [email protected] > > > Date: > > > 09/14/2011 01:45 PM > > > Subject: > > > Re: [jira] [Commented] (JENA-91) extremely large buffer is being > created > > > in ObjectFileStorage > > > > > > > > > > > > Hi Dave, > > > thanks (it helps). > > > > > > If there are tests failing on Windows 7, it's bad (and good at > > > the same time that the test suite is signaling the problem). > > > > > > Just to double check there are no changes in TxTDB trunk that > > > I do not have on my local copy, I've just did: > > > > > > cd /tmp/txtdb/ > > > svn up > > > At revision 1170728. > > > > > > mvn clean test > > > > > > [INFO] Scanning for projects... > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building TxTDB > > > [INFO] task-segment: [clean, test] > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [...] > > > > > > Tests run: 697, Failures: 0, Errors: 0, Skipped: 0 > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD SUCCESSFUL > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 1 minute 13 seconds > > > [INFO] Finished at: Wed Sep 14 18:36:44 BST 2011 > > > [INFO] Final Memory: 42M/296M > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > There is obviously something bad going when we run TxTDB on > > > Windows 7. I am afraid I cannot be more helpful on this. :-( > > > > > > @Andy, do you have a Windows 7 machine to investigate the > > > problem that Simon and Dave are experiencing? > > > > > > Paolo > > > > > > Dave Reynolds wrote: > > >> On Wed, 2011-09-14 at 16:38 +0100, Paolo Castagna wrote: > > >>> Hi Simon, > > >>> indeed, very strange. > > >>> > > >>> I do not have a Windows 7 machine to try, but if you have a Linux > > > machine > > >>> you could try there, it would be really interesting to know the > result. > > >>> > > >>> Could it be something related to your Maven cache and somehow you > still > > > have > > >>> an old ARQ SNAPSHOT? You could try to manually delete the ARQ > artifacts > > > from > > >>> your Maven cache (usually in ~/.m2/repository/com/hp/hpl/jena/arq) > > >>> > > >>> This is how you or others could run the test on different > environments: > > >>> > > >>> cd /tmp/ > > >>> svn co > > > > http://svn.apache.org/repos/asf/incubator/jena/Experimental/TxTDB/trunk/ > > >>> txtdb > > >>> cd /tmp/txtdb/ > > >>> wget > > >>> > > > > > > https://issues.apache.org/jira/secure/attachment/12494332/TestTransSystem4.patch > > > > > >>> patch -p0< TestTransSystem4.patch > > >>> mvn package > > >>> mvn exec:java -Dexec.classpathScope="test" > > >>> -Dexec.mainClass="com.hp.hpl.jena.tdb.transaction.TestTransSystem" > > >>> > > >>> (you need to adapt things to Windows) > > >> > > >> FWIW I get errors trying that sequence on Windows. > > >> > > >> Environment: > > >> Windows 7 64 bit > > >> Java 1.6.0_26-b03 > > >> > > >> START (disk, 100 iterations) > > >> 000: ..java.lang.UnsupportedOperationException: Quad: predicate cannot > > >> be null > > >> at com.hp.hpl.jena.sparql.core.Quad.<init>(Quad.java:49) > > >> at com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:127) > > >> at com.hp.hpl.jena.tdb.lib.TupleLib.quad(TupleLib.java:118) > > >> at com.hp.hpl.jena.tdb.lib.TupleLib.access$100(TupleLib.java:32) > > >> at com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:76) > > >> at com.hp.hpl.jena.tdb.lib.TupleLib$4.convert(TupleLib.java:72) > > >> at org.openjena.atlas.iterator.Iter$4.next(Iter.java:268) > > >> at com.hp.hpl.jena.tdb.store.GraphTDBBase > > >> $ProjectQuadsToTriples.next(GraphTDBBase.java:183) > > >> at com.hp.hpl.jena.tdb.store.GraphTDBBase > > >> $ProjectQuadsToTriples.next(GraphTDBBase.java:171) > > >> at > > >> > > > > > > com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:68) > > >> at > > >> com.hp.hpl.jena.util.iterator.Map1Iterator.next(Map1Iterator.java:35) > > >> at > > >> > > > > > > com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:68) > > >> at > > >> > > > > > > com.hp.hpl.jena.rdf.model.impl.StmtIteratorImpl.next(StmtIteratorImpl.java:33) > > >> at > > >> > > > > > > com.hp.hpl.jena.rdf.model.impl.StmtIteratorImpl.next(StmtIteratorImpl.java:21) > > >> at > > >> > > > > > > com.hp.hpl.jena.tdb.transaction.TestTransSystem.readModel(TestTransSystem.java:446) > > >> at com.hp.hpl.jena.tdb.transaction.TestTransSystem.access > > >> $100(TestTransSystem.java:59) > > >> at com.hp.hpl.jena.tdb.transaction.TestTransSystem > > >> $Reader.call(TestTransSystem.java:163) > > >> at java.util.concurrent.FutureTask > > >> $Sync.innerRun(FutureTask.java:303) > > >> at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > >> at java.util.concurrent.ThreadPoolExecutor > > >> $Worker.runTask(ThreadPoolExecutor.java:886) > > >> at java.util.concurrent.ThreadPoolExecutor > > >> $Worker.run(ThreadPoolExecutor.java:908) > > >> at java.lang.Thread.run(Thread.java:619) > > >> > > >> > > >> However, I get some test errors during the build and had to force > Maven > > >> to ignore those in order to get a build, so this is not a clean run. > > >> > > >> Surefire logs for the test-during-build errors are attached. > > >> > > >> This is not my normal dev environment so there is little chance of > stale > > >> maven cache. > > >> > > >> Dave > > >> > > >> > > >> > > > > > > > > > > > > > > > > > >
