[jira] [Commented] (TIKA-1308) Support in memory parse mode(don't create temp file): to support run Tika in GAE
[
https://issues.apache.org/jira/browse/TIKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372060#comment-14372060
]
Tyler Palsulich commented on TIKA-1308:
---
This would be a great feature! The default would need to be disabled, though,
since some files are larger than memory. And, as mentioned above, some parsers
require writing the output to a file in order to use the external parsing
library.
Support in memory parse mode(don't create temp file): to support run Tika in
GAE
Key: TIKA-1308
URL: https://issues.apache.org/jira/browse/TIKA-1308
Project: Tika
Issue Type: Improvement
Components: parser
Affects Versions: 1.5
Reporter: jefferyyuan
Labels: gae
Fix For: 1.8
I am trying to use Tika in GAE and write a simple servlet to extract meta
data info from jpeg:
{code}
String urlStr = req.getParameter(imageUrl);
byte[] oldImageData = IOUtils.toByteArray(new URL(urlStr));
ByteArrayInputStream bais = new ByteArrayInputStream(oldImageData);
Metadata metadata = new Metadata();
BodyContentHandler ch = new BodyContentHandler();
AutoDetectParser parser = new AutoDetectParser();
parser.parse(bais, ch, metadata, new ParseContext());
bais.close();
{code}
This fails with exception:
{code}
Caused by: java.lang.SecurityException: Unable to create temporary file
at java.io.File.createTempFile(File.java:1986)
at
org.apache.tika.io.TemporaryResources.createTemporaryFile(TemporaryResources.java:66)
at org.apache.tika.io.TikaInputStream.getFile(TikaInputStream.java:533)
at org.apache.tika.parser.jpeg.JpegParser.parse(JpegParser.java:56)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242
{code}
Checked the code, in
org.apache.tika.parser.jpeg.JpegParser.parse(InputStream, ContentHandler,
Metadata, ParseContext), it creates a temp file from the input stream.
I can understand why tika create temp file from the stream: so tika can parse
it multiple times.
But as GAE and other cloud servers are getting more popular, is it possible
to avoid create temp file: instead we can copy the origin stream to a
byteArray stream, so tika can also parse it multiple times.
-- This will have a limit on the file size, as tika keeps the whole file in
memory, but this can make tika work in GAE and maybe other cloud server.
We can add a parameter in parser.parse to indicate whether do in memory parse
only.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TIKA-1308) Support in memory parse mode(don't create temp file): to support run Tika in GAE
[ https://issues.apache.org/jira/browse/TIKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008704#comment-14008704 ] Hong-Thai Nguyen commented on TIKA-1308: A virtual FileSystem may be a solution, If you're on Java 7. The NIO APIs with FileSytemProvider [1] allows you define or inject a Virtual FileSystem (eg. Common VFS [2]). [1] http://docs.oracle.com/javase/7/docs/api/java/nio/file/spi/FileSystemProvider.html [2] http://commons.apache.org/proper/commons-vfs/filesystems.html Support in memory parse mode(don't create temp file): to support run Tika in GAE Key: TIKA-1308 URL: https://issues.apache.org/jira/browse/TIKA-1308 Project: Tika Issue Type: Improvement Components: parser Affects Versions: 1.5 Reporter: yuanyun.cn Labels: gae Fix For: 1.6 I am trying to use Tika in GAE and write a simple servlet to extract meta data info from jpeg: String urlStr = req.getParameter(imageUrl); byte[] oldImageData = IOUtils.toByteArray(new URL(urlStr)); ByteArrayInputStream bais = new ByteArrayInputStream(oldImageData); Metadata metadata = new Metadata(); BodyContentHandler ch = new BodyContentHandler(); AutoDetectParser parser = new AutoDetectParser(); parser.parse(bais, ch, metadata, new ParseContext()); bais.close(); This fails with exception: Caused by: java.lang.SecurityException: Unable to create temporary file at java.io.File.createTempFile(File.java:1986) at org.apache.tika.io.TemporaryResources.createTemporaryFile(TemporaryResources.java:66) at org.apache.tika.io.TikaInputStream.getFile(TikaInputStream.java:533) at org.apache.tika.parser.jpeg.JpegParser.parse(JpegParser.java:56) at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242 Checked the code, in org.apache.tika.parser.jpeg.JpegParser.parse(InputStream, ContentHandler, Metadata, ParseContext), it creates a temp file from the input stream. I can understand why tika create temp file from the stream: so tika can parse it multiple times. But as GAE and other cloud servers are getting more popular, is it possible to avoid create temp file: instead we can copy the origin stream to a byteArray stream, so tika can also parse it multiple times. -- This will have a limit on the file size, as tika keeps the whole file in memory, but this can make tika work in GAE and maybe other cloud server. We can add a parameter in parser.parse to indicate whether do in memory parse only. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TIKA-1308) Support in memory parse mode(don't create temp file): to support run Tika in GAE
[ https://issues.apache.org/jira/browse/TIKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007292#comment-14007292 ] Chris A. Mattmann commented on TIKA-1308: - Sounds like a great issue - looking forward to seeing a patch. Thanks! Support in memory parse mode(don't create temp file): to support run Tika in GAE Key: TIKA-1308 URL: https://issues.apache.org/jira/browse/TIKA-1308 Project: Tika Issue Type: Improvement Components: parser Affects Versions: 1.5 Reporter: yuanyun.cn Labels: gae Fix For: 1.6 I am trying to use Tika in GAE and write a simple servlet to extract meta data info from jpeg: String urlStr = req.getParameter(imageUrl); byte[] oldImageData = IOUtils.toByteArray(new URL(urlStr)); ByteArrayInputStream bais = new ByteArrayInputStream(oldImageData); Metadata metadata = new Metadata(); BodyContentHandler ch = new BodyContentHandler(); AutoDetectParser parser = new AutoDetectParser(); parser.parse(bais, ch, metadata, new ParseContext()); bais.close(); This fails with exception: Caused by: java.lang.SecurityException: Unable to create temporary file at java.io.File.createTempFile(File.java:1986) at org.apache.tika.io.TemporaryResources.createTemporaryFile(TemporaryResources.java:66) at org.apache.tika.io.TikaInputStream.getFile(TikaInputStream.java:533) at org.apache.tika.parser.jpeg.JpegParser.parse(JpegParser.java:56) at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242 Checked the code, in org.apache.tika.parser.jpeg.JpegParser.parse(InputStream, ContentHandler, Metadata, ParseContext), it creates a temp file from the input stream. I can understand why tika create temp file from the stream: so tika can parse it multiple times. But as GAE and other cloud servers are getting more popular, is it possible to avoid create temp file: instead we can copy the origin stream to a byteArray stream, so tika can also parse it multiple times. -- This will have a limit on the file size, as tika keeps the whole file in memory, but this can make tika work in GAE and maybe other cloud server. We can add a parameter in parser.parse to indicate whether do in memory parse only. -- This message was sent by Atlassian JIRA (v6.2#6252)
