[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17348688#comment-17348688 ] Hudson commented on TIKA-3270: -- UNSTABLE: Integrated in Jenkins build Tika » tika-main-jdk8 #244 (See [https://ci-builds.apache.org/job/Tika/job/tika-main-jdk8/244/]) TIKA-3270 -- when rendering a page for OCR, do not include electronic text (as default) (tallison: [https://github.com/apache/tika/commit/8ec5d4483c953f6024cc470e780037e01530d7dd]) * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java * (edit) CHANGES.txt * (add) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/NoTextPDFRenderer.java * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-package/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParserConfig.java * (edit) tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParser.java > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Assignee: Tim Allison >Priority: Major > Fix For: 2.0.0 > > Attachments: test-no-text.png, test.png, tiger-no-text.png, tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17348657#comment-17348657 ] Tim Allison commented on TIKA-3270: --- This is a breaking change. The default is now to render the page without text when running OCR. We can revisit this if this causes havoc. > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Assignee: Tim Allison >Priority: Major > Fix For: 2.0.0 > > Attachments: test-no-text.png, test.png, tiger-no-text.png, tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17348620#comment-17348620 ] Tim Allison commented on TIKA-3270: --- [~tilman] it really is that easy! :D > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Priority: Major > Attachments: test-no-text.png, test.png, tiger-no-text.png, tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264228#comment-17264228 ] Luís Filipe Nassif commented on TIKA-3270: -- Is checking for missing ToUnicode mapping easy? I have never seen this part of the code. And if we also could detect garbage text with lang stats as suggested by [~tilman] , I think this would be a great improvement to OCR of PDFs! Thanks for opening this, [~tallison]! > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Priority: Major > Attachments: tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263644#comment-17263644 ] Tim Allison commented on TIKA-3270: --- Hahaha...the former...do we have one within our small unit tests or can you generate one. :D Thank you! > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Priority: Major > Attachments: tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263638#comment-17263638 ] Tilman Hausherr commented on TIKA-3270: --- If your question was whether there is a method to find such files in a large group - I can't think of one. > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Priority: Major > Attachments: tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263634#comment-17263634 ] Tilman Hausherr commented on TIKA-3270: --- Don't know, but I just created one by printing from PDFBox into PDF. > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Priority: Major > Attachments: tiger.pdf > > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[ https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263623#comment-17263623 ] Tim Allison commented on TIKA-3270: --- Thank you [~tilman] ! Are examples of vector graphics easy enough to find in PDFBox's or Tika's unit test files? > Render non-text in PDFs for OCR > --- > > Key: TIKA-3270 > URL: https://issues.apache.org/jira/browse/TIKA-3270 > Project: Tika > Issue Type: Improvement >Reporter: Tim Allison >Priority: Major > > When we render a PDF page for OCR, we are relying on PDFBox to render all of > the contents of the page, including text that may be available via regular > extraction methods. > The result of this is that if a user selects ocr_and_text, there can be > duplicate text -- text as stored in PDFs and the text generated via OCR. In > the xhtml output, we do mark a separate "div" for OCR so that users can > distinguish, but still, it might be useful not to have to run OCR on text > that was reliably extracted. > One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a > technical/implementation recommendation by [~tilman] to subclass PDFRenderer > and PageDrawer to render only the image components of a page. > This would be a new, non-breaking feature. This is not a blocker on 2.0.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-3270) Render non-text in PDFs for OCR
[
https://issues.apache.org/jira/browse/TIKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263618#comment-17263618
]
Tilman Hausherr commented on TIKA-3270:
---
{quote}
to render only the image components of a page
{quote}
I wrote about not rendering the text. We still need to render vector graphics.
These may be texts.
> Render non-text in PDFs for OCR
> ---
>
> Key: TIKA-3270
> URL: https://issues.apache.org/jira/browse/TIKA-3270
> Project: Tika
> Issue Type: Improvement
>Reporter: Tim Allison
>Priority: Major
>
> When we render a PDF page for OCR, we are relying on PDFBox to render all of
> the contents of the page, including text that may be available via regular
> extraction methods.
> The result of this is that if a user selects ocr_and_text, there can be
> duplicate text -- text as stored in PDFs and the text generated via OCR. In
> the xhtml output, we do mark a separate "div" for OCR so that users can
> distinguish, but still, it might be useful not to have to run OCR on text
> that was reliably extracted.
> One solution to this was proposed by [~lfcnassif] on TIKA-3258, with a
> technical/implementation recommendation by [~tilman] to subclass PDFRenderer
> and PageDrawer to render only the image components of a page.
> This would be a new, non-breaking feature. This is not a blocker on 2.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
