[
https://issues.apache.org/jira/browse/PDFBOX-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387997#comment-15387997
]
Tilman Hausherr commented on PDFBOX-3429:
-----------------------------------------
It is at 100% with 4 threads on a 4 core system. But I don't have the same test
as you do (I didn't use TIKA, only PDFBox alone), so this doesn't mean much.
Here's some code:
{code}
public class BenchmarkPDFBox3429
{
public static final int JOB_COUNT = 500;
public static final int THREAD_COUNT = 4;
public static void main(String[] args) throws IOException,
InterruptedException
{
System.out.println("Benchmark is running");
System.out.printf("%s pdf strips with %s thread(s).", JOB_COUNT,
THREAD_COUNT);
long startTime = System.currentTimeMillis();
ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
File[] list = new File("XXXXXXXXX").listFiles(new FilenameFilter()
{
@Override
public boolean accept(File dir, String name)
{
return name.toLowerCase().endsWith(".pdf");
}
});
for (int index = 0; index < JOB_COUNT; ++index)
{
executor.execute(new Job(list[index]));
}
executor.shutdown();
while (!executor.isTerminated())
{
Thread.sleep(1000);
}
long duration = System.currentTimeMillis() - startTime;
System.out.println(String.format("\nFinished all jobs. %s ms",
duration));
}
static class Job implements Runnable
{
private final File file;
Job(File file)
{
this.file = file;
}
@Override
public void run()
{
try
{
extractFromPdf(file);
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
}
}
public static String extractFromPdf(File file) throws IOException
{
try (PDDocument doc = PDDocument.load(file))
{
PDFTextStripper stripper = new PDFTextStripper();
return stripper.getText(doc);
}
}
}
}
{code}
> Improve ExtractText Concurrency
> -------------------------------
>
> Key: PDFBOX-3429
> URL: https://issues.apache.org/jira/browse/PDFBOX-3429
> Project: PDFBox
> Issue Type: Improvement
> Components: Text extraction
> Affects Versions: 2.0.1
> Environment: Win7, jdk1.8.0_60 x64
> Reporter: Luis Filipe Nassif
> Priority: Minor
> Labels: optimization
> Attachments: cpu-pdfbox-2.0.1.png, cpu-pdfbox1.8.10.png
>
>
> While testing Tika 1.13, which uses PDFBox 2.0.1, from a multithreaded text
> extraction application, I noted cpu usage aroung 80% in my 6 core computer
> when processing a dataset of ~75 thousands of pdfs (18GB). It took 5min25sec
> to complete the text extraction. With Tika 1.10, which uses PDFBox 1.8.10,
> cpu usage stays aroung 100%. It took 4min37sec to complete. The dataset is
> read from a ramdrive, so there is no i/o bottleneck. I suspect there is some
> new synchronization code that blocks the threads for a non trivial amount of
> time, resulting in less cpu usage than before.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
