[ https://issues.apache.org/jira/browse/PDFBOX-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388439#comment-15388439 ]
Luis Filipe Nassif commented on PDFBOX-3429: -------------------------------------------- Hi Tilman, I was going to write a test code with only PDFBox, without Tika, thank you very much. I've got same results with your code: ~100% cpu usage with v1.8.10 and ~80% with 2.0.2, with 12 threads (actually my system has 6 physical cores x 2 hiperthreading). I will attach the graphs. I modified a little bit your code to used only one specific file (will attach it too), parsed 10000, but it was chosen ramdonly, so I think there is nothing special with the file. Modified code below: {code} public class BenchmarkPDFBox3429 { public static final int JOB_COUNT = 10000; public static final int THREAD_COUNT = 12; public static void main(String[] args) throws IOException, InterruptedException { java.util.logging.Logger.getLogger("org.apache").setLevel(java.util.logging.Level.OFF); System.out.println("Benchmark is running"); System.out.printf("%s pdf strips with %s thread(s).", JOB_COUNT, THREAD_COUNT); long startTime = System.currentTimeMillis(); ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT); File file = new File("f:/teste/pdf2/000000000000B265.pdf"); for (int i = 0; i < JOB_COUNT; i++) { executor.execute(new Job(file)); } executor.shutdown(); while (!executor.isTerminated()) { Thread.sleep(1000); } long duration = System.currentTimeMillis() - startTime; System.out.println(String.format("\nFinished all jobs. %s ms", duration)); } static class Job implements Runnable { private final File file; Job(File file) { this.file = file; } @Override public void run() { try { extractFromPdf(file); } catch (IOException e) { e.printStackTrace(); } finally { } } public static String extractFromPdf(File file) throws IOException { PDDocument doc = null; try { doc = PDDocument.load(file); PDFTextStripper stripper = new PDFTextStripper(); return stripper.getText(doc); }finally{ if(doc != null) doc.close(); } } } } {code} Also, I repeated the test with only 4 threads (1/3 of 12) and cpu usage was around 33% (1/3 of 100%) as expected, so maybe the problem arises only in high concurrent scenarios. Finally, I repeated the test with 48 threads and got same results: cpu usage only around 80%. > Improve ExtractText Concurrency > ------------------------------- > > Key: PDFBOX-3429 > URL: https://issues.apache.org/jira/browse/PDFBOX-3429 > Project: PDFBox > Issue Type: Improvement > Components: Text extraction > Affects Versions: 2.0.1 > Environment: Win7, jdk1.8.0_60 x64 > Reporter: Luis Filipe Nassif > Priority: Minor > Labels: optimization > Attachments: cpu-pdfbox-2.0.1.png, cpu-pdfbox1.8.10.png > > > While testing Tika 1.13, which uses PDFBox 2.0.1, from a multithreaded text > extraction application, I noted cpu usage aroung 80% in my 6 core computer > when processing a dataset of ~75 thousands of pdfs (18GB). It took 5min25sec > to complete the text extraction. With Tika 1.10, which uses PDFBox 1.8.10, > cpu usage stays aroung 100%. It took 4min37sec to complete. The dataset is > read from a ramdrive, so there is no i/o bottleneck. I suspect there is some > new synchronization code that blocks the threads for a non trivial amount of > time, resulting in less cpu usage than before. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pdfbox.apache.org For additional commands, e-mail: dev-h...@pdfbox.apache.org