[jira] [Comment Edited] (TIKA-1857) Enhance PDFParser to extract text from XFA forms
[
https://issues.apache.org/jira/browse/TIKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149014#comment-15149014
]
Tim Allison edited comment on TIKA-1857 at 2/16/16 7:49 PM:
from TIKA-1607's
[comment|https://issues.apache.org/jira/browse/TIKA-1607?focusedCommentId=15148914&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15148914]
bq. In the case of XFA forms, the form IS the content.
Got it. Doh. Thank you.
As I look at a few of these docs from govdocs1 w/ XFA data, it looks like the
form also contains the PDF's standard metadata...(author etc.) which is not
necessarily stored in the older mechanism: COSDictionary. govdocs1's
{{517660.pdf}} shows this -- the author and title can be extracted from the
XFA, but that info is not extracted with our current methods.
bq. I'll support whichever way you pick, but I personally can't see use cases
where extracting that workaround message is the intent when using Tika. I do
see value in keeping the entire DOM though. Maybe you can do as you suggest,
but "in addition" to returning the XFA text as the content?
Y, that would be in addition. Thank you, again.
was (Author: talli...@mitre.org):
from TIKA-1607's
[comment|https://issues.apache.org/jira/browse/TIKA-1607?focusedCommentId=15148914&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15148914]
bq. In the case of XFA forms, the form IS the content.
Got it. Doh. Thank you.
As I look at a few of these docs from govdocs1 w/ XFA data, it looks like the
form also contains the PDF's standard metadata...(author etc.) which is not
necessarily stored in the older mechanism: COSDictionary.
bq. I'll support whichever way you pick, but I personally can't see use cases
where extracting that workaround message is the intent when using Tika. I do
see value in keeping the entire DOM though. Maybe you can do as you suggest,
but "in addition" to returning the XFA text as the content?
Y, that would be in addition. Thank you, again.
> Enhance PDFParser to extract text from XFA forms
>
>
> Key: TIKA-1857
> URL: https://issues.apache.org/jira/browse/TIKA-1857
> Project: Tika
> Issue Type: Improvement
> Components: parser
>Reporter: Pascal Essiembre
>Priority: Trivial
> Labels: patch
> Fix For: 1.13
>
> Attachments: 041617_filled_out.pdf, xfa_in_govdocs1.txt
>
>
> Extract text from PDF Forms (XFA). Information about XFA:
> https://en.wikipedia.org/wiki/XFA
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (TIKA-1857) Enhance PDFParser to extract text from XFA forms
[
https://issues.apache.org/jira/browse/TIKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149171#comment-15149171
]
Tim Allison edited comment on TIKA-1857 at 2/16/16 8:09 PM:
I've only looked at a handful of files that contain xfa...this metadata is
entirely new to me. The files I've looked at come from govdocs1 and are fairly
old by now.
In the attached {{041617_filled_out.pdf}}, I've added content to the forms and
saved the document.
With the patch, I'm getting all of the boilerplate from the xfa extraction, but
I'm not getting any content from the form because it isn't in
{{<(speak|text|exData)>}} elements. However, with our old code, I am seeing
the entered data, e.g. {{my_exhibitor}}.
Is this PDF storing the contents of the form in both the xfa _and_ in the
traditional AcroForm?
I imagine that won't happen in all PDFs, and there will be an either/or?
To avoid duplication of content, do we want to skip processing of AcroForm data
if XFA exists? Will we miss anything?
The other major question: I like the narrow focus that the current regexes
yield, but why wouldn't we want to run our HtmlParser or our DcXMLParser
against the bytes and pull everything out? We'd have to skip inline/embedded
images or handle those properly at some point...but any other reasons?
[~tilman], have you worked with XFA? Any recommendations for pulling as much
info as we can without duplication?
We could make this configurable, of course. :)
was (Author: talli...@mitre.org):
I've only looked at a handful of files that contain xfa...this metadata is
entirely new to me. The files I've looked at come from govdocs1 and are fairly
old by now.
In the attached, I've added content to the forms and saved the document.
With the patch, I'm getting all of the boilerplate from the xfa extraction, but
I'm not getting any content from the form because it isn't in
{{<(speak|text|exData)>}} elements. However, with our old code, I am seeing
the entered data, e.g. {{my_exhibitor}}.
Is this PDF storing the contents of the form in both the xfa _and_ in the
traditional AcroForm?
I imagine that won't happen in all PDFs, and there will be an either/or?
To avoid duplication of content, do we want to skip processing of AcroForm data
if XFA exists? Will we miss anything?
[~tilman], have you worked with XFA? Any recommendations for pulling as much
info as we can without duplication?
We could make this configurable, of course. :)
> Enhance PDFParser to extract text from XFA forms
>
>
> Key: TIKA-1857
> URL: https://issues.apache.org/jira/browse/TIKA-1857
> Project: Tika
> Issue Type: Improvement
> Components: parser
>Reporter: Pascal Essiembre
>Priority: Trivial
> Labels: patch
> Fix For: 1.13
>
> Attachments: 041617_filled_out.pdf, xfa_in_govdocs1.txt
>
>
> Extract text from PDF Forms (XFA). Information about XFA:
> https://en.wikipedia.org/wiki/XFA
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (TIKA-1857) Enhance PDFParser to extract text from XFA forms
[ https://issues.apache.org/jira/browse/TIKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153851#comment-15153851 ] Maruan Sahyoun edited comment on TIKA-1857 at 2/19/16 7:33 AM: --- Sorry for my delay in answering your question. May I propose the following strategy: a) for static XFA if there is datasets.data use that content for the field values otherwise extract from the AcroForm. b) for dynamic XFA scrape/extract info from the XFA. Why a different proposal for a) from yours? Adobe Reader/Acrobat use the information from dataset.data for the field value over the possibly differing content in AcroForm (which might happen if the form has been filled out with an XFA aware processor and afterwards was amended with a non XFA aware processor) was (Author: msahyoun): Sorry for my delay in answering your question. May I propose the following strategy: a) for static XFA if there is datasets.data use that content for the filed values otherwise extract from the AcroForm. b) for dynamic XFA scrape/extract info from the XFA. Why a different proposal for a) from yours? Adobe Reader/Acrobat use the information from dataset.data for the field value over the possibly differing content in AcroForm (which might happen if the form has been filled out with an XFA aware processor and afterwards was amended with a non XFA aware processor) > Enhance PDFParser to extract text from XFA forms > > > Key: TIKA-1857 > URL: https://issues.apache.org/jira/browse/TIKA-1857 > Project: Tika > Issue Type: Improvement > Components: parser >Reporter: Pascal Essiembre >Priority: Trivial > Labels: patch > Fix For: 1.13 > > Attachments: 041617_filled_out.pdf, xfa_in_govdocs1.txt > > > Extract text from PDF Forms (XFA). Information about XFA: > https://en.wikipedia.org/wiki/XFA -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TIKA-1857) Enhance PDFParser to extract text from XFA forms
[ https://issues.apache.org/jira/browse/TIKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865598#comment-15865598 ] Tim Allison edited comment on TIKA-1857 at 2/14/17 12:12 PM: - Are you able to share mocked up xml, sanitized of patient data? was (Author: talli...@mitre.org): Are you able to submit the triggering document? If not, are you able to share it personally? > Enhance PDFParser to extract text from XFA forms > > > Key: TIKA-1857 > URL: https://issues.apache.org/jira/browse/TIKA-1857 > Project: Tika > Issue Type: Improvement > Components: parser >Reporter: Pascal Essiembre > Labels: patch > Fix For: 1.13 > > Attachments: 041617_filled_out.pdf, govdocs1_xfas.zip, > xfa_in_govdocs1.txt > > > Extract text from PDF Forms (XFA). Information about XFA: > https://en.wikipedia.org/wiki/XFA -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (TIKA-1857) Enhance PDFParser to extract text from XFA forms
[ https://issues.apache.org/jira/browse/TIKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865806#comment-15865806 ] Kenneth Lui edited comment on TIKA-1857 at 2/14/17 2:04 PM: I cannot copy the file out of the secured environment. But this is a file I found on the Internet to have the same issue and I used this to test my pdfbox script as well. Edit: the comment seems to be not obvious that I attached doc8.pdf. That is the file I am referring to. was (Author: hkkenneth): I cannot copy the file out of the secured environment. But this is a file I found on the Internet to have the same issue and I used this to test my pdfbox script as well. > Enhance PDFParser to extract text from XFA forms > > > Key: TIKA-1857 > URL: https://issues.apache.org/jira/browse/TIKA-1857 > Project: Tika > Issue Type: Improvement > Components: parser >Reporter: Pascal Essiembre > Labels: patch > Fix For: 1.13 > > Attachments: 041617_filled_out.pdf, doc8.pdf, govdocs1_xfas.zip, > xfa_in_govdocs1.txt > > > Extract text from PDF Forms (XFA). Information about XFA: > https://en.wikipedia.org/wiki/XFA -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (TIKA-1857) Enhance PDFParser to extract text from XFA forms
[
https://issues.apache.org/jira/browse/TIKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887106#comment-15887106
]
Tim Allison edited comment on TIKA-1857 at 2/28/17 3:10 PM:
{noformat}
IT IS EASY
JUST TRY
DUDE
DO YOUR OWN JOB
DON'T EXPECT ME TO DO IT!
IT'S XML!
READ THE DOCUMENTATION
DUDE
LEARN BEFORE YOU CODE
{noformat}
Is now extracted as:
{noformat}
Nazwa pełna: IT IS EASY
Nazwisko: DUDE
ImiePierwsze: JUST TRY
Województwo: DO YOUR OWN JOB
Powiat: DON'T EXPECT ME TO DO IT!
Gmina: IT'S XML!
Miejscowość: READ THE DOCUMENTATION
Kod pocztowy: DUDE
Poczta: LEARN BEFORE YOU CODE
{noformat}
Once our git is back up and running, I'll push the fix. Thank you for raising
this issue and sharing a triggering document.
was (Author: talli...@mitre.org):
{noformat}
IT IS EASY
0123456789
JUST TRY
DUDE
2015-02-19
PL
DO YOUR OWN JOB
DON'T EXPECT ME TO DO IT!
IT'S XML!
012345678
READ THE DOCUMENTATION
DUDE
LEARN BEFORE YOU CODE
{noformat}
Is now extracted as:
{noformat}
Nazwa pełna:
IT IS EASY
REGON:
REGON:
REGON:
Nazwisko:
DUDE
ImiePierwsze:
JUST TRY
DataUrodzenia:
2015-02-19
PESEL:
Numer Identyfikacji Podatkowej:
Numer PESEL:
Kraj:
KodKraju:
PL
Województwo:
DO YOUR OWN JOB
Powiat:
DON'T EXPECT ME TO DO IT!
Gmina:
IT'S XML!
Ulica:
Nr domu:
012345678
Nr lokalu:
Miejscowość:
READ THE DOCUMENTATION
Kod pocztowy:
DUDE
Poczta:
LEARN BEFORE YOU CODE
{noformat}
Once our git is back up and running, I'll push the fix. Thank you for raising
this issue and sharing a triggering document.
> Enhance PDFParser to extract text from XFA forms
>
>
> Key: TIKA-1857
> URL: https://issues.apache.org/jira/browse/TIKA-1857
> Project: Tika
> Issue Type: Improvement
> Components: parser
>Reporter: Pascal Essiembre
> Labels: patch
> Fix For: 1.13
>
> Attachments: 041617_filled_out.pdf, doc8.pdf, govdocs1_xfas.zip,
> xfa_in_govdocs1.txt
>
>
> Extract text from PDF Forms (XFA). Information about XFA:
> https://en.wikipedia.org/wiki/XFA
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
