[
https://issues.apache.org/jira/browse/PDFBOX-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Allison updated PDFBOX-2252:
--------------------------------
Attachment: pdfs_directionality.xlsx
Table with counts of ltr and rtl tokens per pdf in a large portion of our
combined Common Crawl (subset) and govdocs1. Text was extracted with Tika
1.10=pdfbox 1.8.10. Can't vouch for licensing on any of these.
Good candidates seem to have > 100 tokens and a balance of > 20%...e.g. :
govdocs1/302/302975.pdf
> PDFTextStripper has problem with documents with mixed language directions
> -------------------------------------------------------------------------
>
> Key: PDFBOX-2252
> URL: https://issues.apache.org/jira/browse/PDFBOX-2252
> Project: PDFBox
> Issue Type: Bug
> Components: Text extraction
> Affects Versions: 1.8.6, 2.0.0
> Reporter: Amir
> Assignee: Maruan Sahyoun
> Priority: Critical
> Fix For: 2.1.0
>
> Attachments: BidiMirroring.txt, IsMirroredDeviations.txt,
> PDFTextStripper-201709271718.patch, PDFTextStripper-201709272018.patch,
> PDFTextStripper.java.patch, PDFTextStripper.java.patch, atest.pdf,
> bugzilla867751.pdf, overlap.jpg, pdfs_directionality.xlsx, test.pdf,
> wikipedia_dl_lyric_test.pdf
>
>
> When the input document of PDFTextStripper is a combination of right-to-left
> and left-to-right languages, the output characters of one language is
> reversed.
> A sample bilingual pdf document is attached.
> PDFTextStripper has a variable "isRtlDominant" in "writePage" function, which
> is defined as follows: boolean isRtlDominant = rtlCount > ltrCount;
> This class clearly count the number of rtl characters and decide if the whole
> content should be revered or not. It's not true, it must operate on each
> word, not the whole document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
