All, As Brian pointed out, optimaize is no longer maintained, and it has some dependencies that have aged out. Should we replace our baseline langdetect in tika-app and tika-server in 3.x? I'd say that we should go with our OpenNLP based language detection, but that, too, is effectively EOL'd because OpenNLP >= 2.3.0 requires Java 17. Thoughts?
Best, Tim ---------- Forwarded message --------- From: Brian Laskey <blas...@us.ibm.com> Date: Fri, Mar 8, 2024 at 2:38 PM Subject: RE: Replacing full tika-app.jar to directly using tiki-core / and parsers To: u...@tika.apache.org <u...@tika.apache.org> Hi Tim Thanks this is helpful. For tika-app we found the dependency on org.apache.tika » tika-langdetect-optimaize brings in some older 3rd party jars, and unfortunately it appears that the com.optimaize.languagedetector » language-detector 0.6 is unmaintained so it’s dependencies on vulnerable versions of guava (18.0) cause us problems with security scans. I could be wrong but I don’t believe we need this component for our usage of just detect and parse? We have a sort of microservice process (java based) which is ingesting files parsed from tika. It was nice that we could separate the tika process in it’s own heap space as a separate java process rather than adding it to our app, but I suppose we could work around that Thank you Brian Laskey From: Tim Allison <talli...@apache.org> Reply-To: "u...@tika.apache.org" <u...@tika.apache.org> Date: Friday, March 8, 2024 at 9:44 AM To: "u...@tika.apache.org" <u...@tika.apache.org> Subject: [EXTERNAL] Re: Replacing full tika-app.jar to directly using tiki-core / and parsers Hi Brian, A few thoughts: 1) tika-app is basically tika-core + tika-parsers-standard-package. Which components are you trying to avoid? tika-serialization and jackson? boilerpipecontenthandler and some of its dependencies? I ask, because we Hi Brian, A few thoughts: 1) tika-app is basically tika-core + tika-parsers-standard-package. Which components are you trying to avoid? tika-serialization and jackson? boilerpipecontenthandler and some of its dependencies? I ask, because we could factor out a tika-app-core with no parsers in Tika 3.x, which is what we do now with tika-server-core and tika-server-standard. 2) Unrelated, there are probably more efficient ways of running Tika than calling it per file on the commandline. That is a robust option, at least! If all you want is detect and text extraction, and you want to run it from the commandline, write two classes, whose main()s call: System.out.println(Tika.detect(File f)); or System.out.println(Tika.parseToString(File f)) On Thu, Mar 7, 2024 at 5:04 PM Brian Laskey <blas...@us.ibm.com> wrote: Hello Tika community, Our team is migrating away from usage of tika-app.jar (2.6 currently) to something with more minimal third party dependencies which we can control. Is there any good documentation or pathway to describe how a team could map the tika-app functionality we use to the same behavior using just tika-core and tika-parsers-standard-package (I assume)? The tika-app functions we use today are: Mime-type detection java -jar tika-app.jar -d <file> and Text extraction attempts java -jar tika-app.jar -t <file> Is there a subset of tika parser jars we would need to include to have equivalent functionality if we wrote our own wrapper main class? Thank you, Brian Laskey