https://issues.apache.org/jira/browse/SOLR-10335 is tracking the upgrade in Solr to Tika 1.15. Please chime in on that issue.
You should be able to swap in POI 3.16 (final) wherever you had earlier versions, make sure to include: poi, poi-scratchpad, poi-ooxml, poi-ooxml-schemas. And make sure to include tika-parsers (1.15), tika-core, tika-java7, tika-xmp. Also, include commons-collections4 (which is new in POI w Tika 1.14). (I assume you have already added curvesapi?) -----Original Message----- From: Gytis Mikuciunas [mailto:gyt...@gmail.com] Sent: Saturday, June 3, 2017 5:39 AM To: solr-user@lucene.apache.org Subject: RE: Solr 6.4. Can't index MS Visio vsdx files Great Tim. What do I need to do to integrate it on my current installation? On May 31, 2017 16:24, "Allison, Timothy B." <talli...@mitre.org> wrote: Apache Tika 1.15 is now available. -----Original Message----- From: Allison, Timothy B. [mailto:talli...@mitre.org] Sent: Tuesday, May 9, 2017 7:45 AM To: solr-user@lucene.apache.org Subject: RE: Solr 6.4. Can't index MS Visio vsdx files Probably better to ask on the Tika list. We'll push the release asap after PDFBox 2.0.6 is out. Andreas plans to cut the release candidate for PDFBox this Friday. Tika will probably have an RC by Monday 5/15, with the release happening later in the week...That's if there are no surprises...[2] You can get a recent build if you'd like to test [1]. Best, Tim [1] https://builds.apache.org/view/Tika/job/Tika-trunk/ [2] If you are curious, for the comparison reports btwn PDFBox 2.0.5 and 2.0.6-SNAPSHOT on ~500k pdfs, see: http://162.242.228.174/ reports/reports_pdfbox_2_0_6.tar.gz -----Original Message----- From: Gytis Mikuciunas [mailto:gyt...@gmail.com] Sent: Tuesday, May 9, 2017 7:17 AM To: solr-user@lucene.apache.org Subject: Re: Solr 6.4. Can't index MS Visio vsdx files Are there any news regarding Tika 1.15? Maybe it's already ready for download somewhere G. On Wed, Apr 12, 2017 at 6:57 PM, Allison, Timothy B. <talli...@mitre.org> wrote: > The release candidate for POI was just cut...unfortunately, I think > after Nick Burch fixed the 'PolylineTo' issue...thank you, btw, for opening that! > > That'll be done within a week unless there are surprises. Once that's > out, I have to update a few things, but I'd think we'd have a > candidate for Tika a week later, then a week for release. > > You can get nightly builds here: https://builds.apache.org/ > > Please ask on the POI or Tika users lists for how to get the > latest/latest running, and thank you, again, for opening the issue on POI's Bugzilla. > > Best, > > Tim > > -----Original Message----- > From: Gytis Mikuciunas [mailto:gyt...@gmail.com] > Sent: Wednesday, April 12, 2017 1:00 AM > To: solr-user@lucene.apache.org > Subject: Re: Solr 6.4. Can't index MS Visio vsdx files > > when 1.15 will be released? maybe you have some beta version and I > could test it :) > > SAX sounds interesting, and from info that I found in google it could > solve my issues. > > On Tue, Apr 11, 2017 at 10:48 PM, Allison, Timothy B. > <talli...@mitre.org> > wrote: > > > It depends. We've been trying to make parsers more, erm, flexible, > > but there are some problems from which we cannot recover. > > > > Tl;dr there isn't a short answer. :( > > > > My sense is that DIH/ExtractingDocumentHandler is intended to get > > people up and running with Solr easily but it is not really a great > > idea for production. See Erick's gem: https://lucidworks.com/2012/ > > 02/14/indexing-with-solrj/ > > > > As for the Tika portion... at the very least, Tika _shouldn't_ cause > > the ingesting process to crash. At most, it should fail at the file > > level and not cause greater havoc. In practice, if you're > > processing millions of files from the wild, you'll run into bad > > behavior and need to defend against permanent hangs, oom, memory leaks. > > > > Also, at the least, if there's an exception with an embedded file, > > Tika should catch it and keep going with the rest of the file. If > > this doesn't happen let us know! We are aware that some types of > > embedded file stream problems were causing parse failures on the > > entire file, and we now catch those in Tika 1.15-SNAPSHOT and don't > > let them percolate up through the parent file (they're reported in > > the > metadata though). > > > > Specifically for your stack traces: > > > > For your initial problem with the missing class exceptions -- I > > thought we used to catch those in docx and log them. I haven't been > > able to track this down, though. I can look more if you have a need. > > > > For "Caused by: org.apache.poi.POIXMLException: Invalid 'Row_Type' > > name 'PolylineTo' ", this problem might go away if we implemented a > > pure SAX parser for vsdx. We just did this for docx and pptx > > (coming in 1.15) and these are more robust to variation because they > > aren't requiring a match with the ooxml schema. I haven't looked > > much at vsdx, but that _might_ help. > > > > For "TODO Support v5 Pointers", this isn't supported and would > > require contributions. However, I agree that POI shouldn't throw a > > Runtime exception. Perhaps open an issue in POI, or maybe we should > > catch this special example at the Tika level? > > > > For "Caused by: java.lang.ArrayIndexOutOfBoundsException:", the POI > > team _might_ be able to modify the parser to ignore a stream if > > there's an exception, but that's often a sign that something needs > > to be fixed with the parser. In short, the solution will come from POI. > > > > Best, > > > > Tim > > > > -----Original Message----- > > From: Gytis Mikuciunas [mailto:gyt...@gmail.com] > > Sent: Tuesday, April 11, 2017 1:56 PM > > To: solr-user@lucene.apache.org > > Subject: RE: Solr 6.4. Can't index MS Visio vsdx files > > > > Thanks for your responses. > > Are there any posibilities to ignore parsing errors and continue > indexing? > > because now solr/tika stops parsing whole document if it finds any > > exception > > > > On Apr 11, 2017 19:51, "Allison, Timothy B." <talli...@mitre.org> wrote: > > > > > You might want to drop a note to the dev or user's list on Apache POI. > > > > > > I'm not extremely familiar with the vsd(x) portion of our code base. > > > > > > The first item ("PolylineTo") may be caused by a mismatch btwn > > > your doc and the ooxml spec. > > > > > > The second item appears to be an unsupported feature. > > > > > > The third item may be an area for improvement within our > > > codebase...I can't tell just from the stacktrace. > > > > > > You'll probably get more helpful answers over on POI. Sorry, I > > > can't help with this... > > > > > > Best, > > > > > > Tim > > > > > > P.S. > > > > 3.1. ooxml-schemas-1.3.jar instead of > > > > poi-ooxml-schemas-3.15.jar > > > > > > You shouldn't need both. Ooxml-schemas-1.3.jar should be a super > > > set of poi-ooxml-schemas-3.15.jar > > > > > > > > > > > >
