Hi Murray, Thanks for taking a look! I was able to make an additional test, setting jspwiki's lucene version to 7.5 (the same brought by default by tika) and had the same outcome.
I did look at other tests retrieving attachments and PageRenamerTest grabs them the same way. I still have to take a better look at it, though.. best regards, juan pablo El jue., 9 may. 2019 13:22, Murray Altheim <murra...@altheim.com> escribió: > Hi Juan Pablo, > > I can duplicate the error on Ubuntu 18.10, Java version: > > java version "1.8.0_211" > Java(TM) SE Runtime Environment (build 1.8.0_211-b12) > Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode) > > with: > > 2019-05-09 23:11:26,236 [main] DEBUG > org.apache.wiki.search.LuceneSearchProvider - Scheduling page > test-tika/aaa-diagram.pdf for index update > 2019-05-09 23:11:26,236 [main] INFO > org.apache.wiki.providers.BasicAttachmentProvider - Uploading attachment > favicon.png to page test-tika > 2019-05-09 23:11:26,236 [main] INFO > org.apache.wiki.providers.BasicAttachmentProvider - Saving attachment > contents to > > /home/altheim/workspace/jspwiki/jspwiki-tika-searchprovider/target/test-classes/testrepository1557400283189/test-tika-att/favicon.png-dir/1.png > 2019-05-09 23:11:26,280 [main] DEBUG > org.apache.wiki.search.LuceneSearchProvider - Scheduling page > test-tika/favicon.png for index update > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > elapsed: 3.367 s <<< FAILURE! - in > org.apache.wiki.search.tika.TikaSearchProviderTest > [ERROR] testGetAttachmentContent Time elapsed: 3.358 s <<< ERROR! > java.lang.NullPointerException > at > org.apache.wiki.search.tika.TikaSearchProviderTest.testGetAttachmentContent(TikaSearchProviderTest.java:58) > > [INFO] > [INFO] Results: > [INFO] > [ERROR] Errors: > [ERROR] TikaSearchProviderTest.testGetAttachmentContent:58 » > NullPointer > [INFO] > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 > > It's late here so I'll hopefully get to this again tomorrow. I still > suspect > instantiation somewhere inside of Lucene, where I've seen strange > behaviours > when multiple versions are present, including NPEs. > > Cheers, > > Murray > > On 9/05/19 9:46 PM, Juan Pablo Santos Rodríguez wrote: > > Hi, > > > > Made some updates, as I was able to grab an Ubuntu box. The build seems > to > > fail, consistently, on ubuntu. Regarding the NPE, the SearchManager (tsp > on > > the test) is *not* null, but the attachment is. Oddly this only happens > on > > ubuntu. > > > > I'll try to debug some more, but I've limited access to this box, so > extra > > eyes are more than welcome. > > > > br, > > juan pablo > > > > El mar., 7 may. 2019 0:24, Juan Pablo Santos Rodríguez < > > juanpablo.san...@gmail.com> escribió: > > > >> My daily update on this: > >> > >> definitely, seems something is weird on the ubuntu nodes; I've been > trying > >> several combinations of JDK 8, maven installations, increasing the -Xmx > >> through MAVEN_OPTS, and all failed, on ubuntu nodes. I've modified the > >> build to run on windows nodes, and suddenly there aren't any problems. > I've > >> also tried on my local pc (windows based) with several JDK 8 and memory > >> settings and haven't been able to reproduce the test failure. > >> > >> I'll try tomorrow on an Ubuntu box at work to see if I can reproduce the > >> test failure. Meanwhile, has anybody been able to reproduce it? > >> What do we do with the build process? Comment out the test, move it to > >> Windows nodes,..? > >> > >> > >> best regards, > >> juan pablo > >> > >> On Sun, May 5, 2019 at 11:44 PM Juan Pablo Santos Rodríguez < > >> juanpablo.san...@gmail.com> wrote: > >> > >>> Hi, > >>> > >>> hmm, didn't realize that Tika and JSPWiki pull different versions > (7.5.0 > >>> and 8.0.0). However, JSPWiki "forces" it's Lucene version to > >>> Tika via dependencyManagement section in pom, so when pulling the > >>> dependencies only Lucene 8 appears in classpath for both > >>> JSPWiki and Tika. We could have binary incompatibilities down the line, > >>> but I think we are not using Tika enough for them to appear. > >>> Also, there are occasions of tests failing where Tika isn't even loaded > >>> in memory (https://builds.apache.org/job/jspwiki-test/18/console). > >>> Tests which have been running for ages, and with NPEs on objects that > are > >>> obviously not null at that point.. Tika related code runs ok, > >>> the error comes on an NPE afterwards. > >>> > >>> Weirder, the offending test fails with: > >>> > >>> [ERROR] testGetAttachmentContent Time elapsed: 3.892 s <<< ERROR! > >>> java.lang.NullPointerException > >>> at > org.apache.wiki.search.tika.TikaSearchProviderTest.testGetAttachmentContent(TikaSearchProviderTest.java:58) > >>> > >>> being the offending line: > >>> String pdfIndexed = tsp.getAttachmentContent( attPdf ); > >>> > >>> and tsp being loaded as: > >>> TikaSearchProvider tsp = ( TikaSearchProvider > >>> )engine.getSearchManager().getSearchEngine(); > >>> > >>> (so, only tsp could yield the NPE) > >>> > >>> I modified the Jenkinsfile to perform a sed on that test so the above > >>> line gets replaced by: > >>> TikaSearchProvider tsp = new TikaSearchProvider();tsp.initialize( > engine, > >>> props ); > >>> > >>> tsp *cannot* be null beyond that point.. But it is: > >>> https://builds.apache.org/job/jspwiki-test/19/ shows the same NPE on > the > >>> same > >>> line yielding it as before. > >>> > >>> Perhaps is that the build is running with a very adjusted memory > setting > >>> (i.e.: the default maven setting is not enough for the jspwiki > >>> build?). Will keep digging tomorrow.. > >>> > >>> > >>> best regards, > >>> juan pablo > >>> > >>> On Sun, May 5, 2019 at 10:50 PM Murray Altheim <murra...@altheim.com> > >>> wrote: > >>> > >>>> Hi Juan Pablo, > >>>> > >>>> My suspicion is still related to there being two Lucene engines in the > >>>> mix. > >>>> > >>>> If JSPWiki and Tika use *different* versions of Lucene there will be > two > >>>> Lucenes the classpath, and I've personally had all sorts of issues > with > >>>> Lucene instantiating when two components both use it (I seem to > remember > >>>> the "Version" can't be resolved, or something like that). Put it this > >>>> way: > >>>> I've noted that two Lucene engines in the same project might not like > >>>> each > >>>> other too much. I think I might have even had problems if they use the > >>>> same version number there can be contention. So when I heard > something in > >>>> Tika related to Lucene was popping an NPE it set off my Lucene > Alarm... > >>>> > >>>> But I've got some time this week so I'll be trying out the latest > >>>> release, > >>>> hopefully in the next few days. > >>>> > >>>> Cheers, > >>>> > >>>> Murray > >>>> > >>>> On 6/05/19 8:42 AM, Juan Pablo Santos Rodríguez wrote: > >>>>> Hi Murray, > >>>>> > >>>>> thanks for looking into it :-) > >>>>> > >>>>> My findings so far: Lucene index related test are already in the main > >>>>> build, so that part should be covered, and regarding the Tika-related > >>>> ones, > >>>>> they don't interact with Lucene, they only parse a couple of files, > and > >>>>> check the extracted information (which is afterwards sent to Lucene, > >>>> but > >>>>> that's outside the scope of the test). I was suspecting of something > >>>> weird > >>>>> on the node in which the build kept running and made a new Jenkins > job > >>>> to > >>>>> do some tests (and avoid spamming dev@j.a.o) at > >>>>> https://builds.apache.org/job/jspwiki-test/ > >>>>> [...] > >>>> > >>>> > ........................................................................... > >>>> Murray Altheim <murray18 at altheim dot com> = = > >>>> === > >>>> http://www.altheim.com/murray/ > === > >>>> === > >>>> > = = > >>>> === > >>>> In the evening > >>>> The rice leaves in the garden > >>>> Rustle in the autumn wind > >>>> That blows through my reed hut. > >>>> -- Minamoto no Tsunenobu > >>>> > >>>> > > > > -- > > ........................................................................... > Murray Altheim <murray18 at altheim dot com> = = === > http://www.altheim.com/murray/ === > === > = = > === > In the evening > The rice leaves in the garden > Rustle in the autumn wind > That blows through my reed hut. > -- Minamoto no Tsunenobu > >
