Hi Brian,
A trace would be great.
It's good to know that the Workbench upload works. That focuses the
investigation on the incoming XML.
- Florian
Hi Florian,
This will be rather difficult to reproduce. The "Limit exceeded!"
server exception occurs when the file is being uploaded from a
particular Mobile App going through WiFi to my server. However, the
exception does not occur if the same file is uploaded from the CMIS
Workbench to my server. I hope this isn't a timing-sensitive
problem.
I am going to look at a packet trace to see if I can see any
differences between how the Mobile App is transferring the file vs.
the CMIS Workbench.
... Brian ...
-----Original Message-----
From: Florian Müller [mailto:[email protected]]
Sent: Monday, March 31, 2014 12:31 PM
To: Inouye, Brian
Cc: [email protected]
Subject: Re: create service with a large content stream causes a
"Limit exceeded!" exception
Hi Brian,
I tried a lot but I cannot reproduce it.
Is it possible to reproduce it with a smaller content and send me the
request? I would like to debug this.
- Florian
Hi Florian,
No, we did not change anything in the OpenCMIS code.
AtomEntryParser uses XMLUtils to create the parser:
XMLStreamReader parser = XMLUtils.createParser(cappedStream);
XMLUtils explicitly sets isCoalescing to FALSE.
XML_INPUT_FACTORY.setProperty(XMLInputFactory.IS_COALESCING,
Boolean.FALSE);
... Brian ...
-----Original Message-----
From: Florian Müller [mailto:[email protected]]
Sent: Friday, March 28, 2014 1:38 PM
To: Inouye, Brian
Cc: [email protected]
Subject: Re: create service with a large content stream causes a
"Limit exceeded!" exception
Hi Brian,
I've revalidated that uploading huge documents works as expected.
What you are seeing is only possible if the property
"javax.xml.stream.isCoalescing" at the XMLInputFactory has been set to
TRUE somehow. Did you change anything in the OpenCMIS code?
- Florian
Hi Brian,
The parser should process the the content of base64 tag in small
chunks.
For some reasons it tries to read much more than that in your
setup.
I'll look into it.
- Florian
Hi Florian,
Here is the exception stack. It looks like it's using the
Woodstox
parser (I assume that's what com.ctc.wstx is). The exception is
happening when AtomEntryParser is calling readBase64, so I think
it
doesn't like the amount of the base64 encoded content stream. The
rest of the XML, excluding the content stream, is very small.
We don't have control over the Client so our options are limited.
... Brian ...
org.apache.chemistry.opencmis.commons.exceptions.CmisInvalidArgumentException:
Limit exceeded!
at
org.apache.chemistry.opencmis.server.shared.CappedInputStream.checkL
e
ngth(CappedInputStream.java:71)
at
org.apache.chemistry.opencmis.server.shared.CappedInputStream.read(C
a
ppedInputStream.java:107)
at com.ctc.wstx.io.BaseReader.readBytes(BaseReader.java:155)
at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:368)
at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:111)
at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:87)
at
com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.
java:57)
at
com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:991)
at
com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReade
r
.java:4647)
at
com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:
3721)
at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.
j
ava:3675)
at
com.ctc.wstx.sr.BasicStreamReader.getTextLength(BasicStreamReader.ja
v
a:907)
at
org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.re
a
dBase64(AtomEntryParser.java:461)
at
org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
r
seCmisContent(AtomEntryParser.java:375)
at
org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
r
seEntry(AtomEntryParser.java:251)
at
org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
r
se(AtomEntryParser.java:214)
at
org.apache.chemistry.opencmis.server.impl.atompub.ObjectService$Crea
t
e.serve(ObjectService.java:97)
at
org.apache.chemistry.opencmis.server.shared.Dispatcher.dispatch(Disp
a
tcher.java:88)
at
org.apache.chemistry.opencmis.server.impl.atompub.CmisAtomPubServlet.
dispatch(CmisAtomPubServlet.java:292)
at
org.apache.chemistry.opencmis.server.impl.atompub.CmisAtomPubServlet.
service(CmisAtomPubServlet.java:228)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(App
l
icationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(Application
F
ilterChain.java:206)
at
com.xerox.docushare.amber.util.UTF8Filter.doFilter(UTF8Filter.java:28)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(App
l
icationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(Application
F
ilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapper
V
alve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContext
V
alve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.
j
ava:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.
j
ava:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVa
l
ve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.ja
v
a:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.jav
a
:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.proc
e
ss(Http11Protocol.java:602)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
-----Original Message-----
From: Florian Müller [mailto:[email protected]]
Sent: Thursday, March 27, 2014 4:46 PM
To: Inouye, Brian
Cc: [email protected]
Subject: Re: create service with a large content stream causes a
"Limit exceeded!" exception
One additional remark:
To avoid the Base64 encoding create an empty document and then set
the content. It's much more efficient when you have to handle
documents of that size.
Or use the Browser binding.
- Florian
Hi Florian,
The Client is sending the content stream embedded in the
cmisra:content element, so the resulting XML is huge.
<?xml version="1.0" encoding="UTF-8" ?> <entry
xmlns="http://www.w3.org/2005/Atom";
xmlns:app="http://www.w3.org/2007/app";
xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/";>
<cmisra:content>
<cmisra:mediatype>video/quicktime</cmisra:mediatype>
<cmisra:base64>[base64 encoding of
file]</cmisra:base64></cmisra:content>
<cmisra:object
xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/";>
<cmis:properties>...</cmis:properties>
</cmisra:object>
<title>Demo.mov</title>
</entry>
... Brian ...
Brian Inouye, Xerox Corporation
-----Original Message-----
From: Florian Müller [mailto:[email protected]]
Sent: Thursday, March 27, 2014 3:03 PM
To: Inouye, Brian
Cc: [email protected]
Subject: Re: create service with a large content stream causes a
"Limit exceeded!" exception
Hi Brian,
The CappedInputStream does not limit the content stream, but the
"envelope" around it. In case of the AtomPub binding it is
limiting
the size of the XML, but doesn't count the embedded document
content.
Could you check how the request is created and if it contains a
big
XML portion?
- Florian
Hi,
I've run into a limitation in my CMIS Provider which uses
OpenCMIS
0.10.0. When a Client sends a create request to my CMIS
Provider
and the accompanying content stream is large, say 1.4 GB,
CappedInputStream raises an exception
CmisInvalidArgumentException("Limit exceeded!").
AtomEntryParser.java creates the CappedInputStream object,
passing
in a constant MAX_STREAM_LENGTH which is set to 10 * 1024 *
1024.
public class CappedInputStream extends InputStream {
...
private void checkLength() throws IOException {
if (counter > max) {
throw new CmisInvalidArgumentException("Limit
exceeded!");
}
}
...
}
public class AtomEntryParser {
...
private static final long MAX_STREAM_LENGTH = 10 * 1024 *
1024;
...
public void parse(InputStream stream) throws
XMLStreamException, IOException {
...
cappedStream = new CappedInputStream(stream,
MAX_STREAM_LENGTH);
...
}
}
What can I do to prevent this exception from occurring? I tried
doubling the value, but the exception still occurs.
... Brian ...
Brian Inouye, Xerox Corporation
