[appengine-java] "Content is not allowed in prolog" when parsing perfectly valid XML on GAE
Hey guys, I've been beating my head against this absolutely infuriating bug for the last 48 hours, so I thought I'd finally throw in the towel and try asking here before I throw my laptop out the window. I'm trying to parse the response XML from a call I made to AWS SimpleDB. The response is coming back on the wire just fine; for example, it may look like: http://sdb.amazonaws.com/doc/ 2009-04-15/">AudioCourseDocumentContentsLectureSetMetaDataProfessorsTag42330b4a- e134-6aec-e62a-5869ac2b45750.071759 I pass in this XML to a parser with XMLEventReader eventReader = xmlInputFactory.createXMLEventReader(response.getContent()); and call eventReader.nextEvent(); a bunch of times to get the data I want. Here's the bizarre part -- it works great inside the local server. The response comes in, I parse it, everyone's happy. The problem is that when I deploy the code to Google App Engine, the outgoing request still works, and the response XML seems 100% identical and correct to me, but the response fails to parse with the following exception: com.amazonaws.http.HttpClient handleResponse: Unable to unmarshall response (ParseError at [row,col]:[1,1] Message: Content is not allowed in prolog.): http://sdb.amazonaws.com/doc/ 2009-04-15/">AudioCourseDocumentContentsLectureSetMetaDataProfessorsTag42330b4a- e134-6aec-e62a-5869ac2b45750.071759 javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1] Message: Content is not allowed in prolog. at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(Unknown Source) at com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(Unknown Source) at com.amazonaws.transform.StaxUnmarshallerContext.nextEvent(StaxUnmarshallerContext.java: 153) ... (rest of lines omitted) I have double, triple, quadruple checked this XML for 'invisible characters' or non-UTF8 encoded characters, etc. I looked at it byte- by-byte in an array for byte-order-marks or something of that nature. Nothing; it passes every validation test I could throw at it. Even stranger, it happens if I use a Saxon-based parser as well -- but ONLY on GAE, it always works fine in my local environment. It makes it very hard to trace the code for problems when I can only run the debugger on an environment that works perfectly (I haven't found any good way to remotely debug on GAE). Nevertheless, using the primitive means I have, I've tried a million approaches including: * XML with and without the prolog * With and without newlines * With and without the "encoding=" attribute in the prolog * Both newline styles * With and without the chunking information present in the HTTP stream And I've tried most of these in multiple combinations where it made sense they would interact -- nothing! I'm at my wit's end. Has anyone seen an issue like this before that can hopefully shed some light on it? Thanks! -Adrian -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Uploading to blobstore gives OutOfMemoryError
I'm having the exact same issue too, down to the line numbers in the stack trace. Has anyone figured out what is causing this? I find it hard to believe we've all made the exact same identical mistake, all within the last month. Is something wrong on Google's end? Cheers, Adrian On Jun 2, 7:14 pm, Jean Hsu wrote: > Hi all, > > I am trying to set up a basic file upload to blobstore, but I get > this OutOfMemoryError: > > WARNING: Error for /_ah/upload/ > aghvbWdkcmVzc3IcCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGMACDA > java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2786) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71) > at > javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java: > 316) > at javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:186) > at javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:109) > at > com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(UploadBlobServlet.java: > 135) > at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access > $000(UploadBlobServlet.java:72) > at com.google.appengine.api.blobstore.dev.UploadBlobServlet > $1.run(UploadBlobServlet.java:100) > at java.security.AccessController.doPrivileged(Native Method) > at > com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobServlet.java: > 98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > 511); > > I used the Memory Analyzer on Eclipse and it said that the memory leak > suspect is QueuedThreadPool. I found this information about a memory > leak bug: > > http://jira.codehaus.org/browse/JETTY-1188 > > How can I figure out what release of jetty is running locally? It > looks like they fixed this in version 6.1.23: > > http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11160&vers... > > Has anyone else had this issue? > > Thanks, > Jean -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Uploading to blobstore gives OutOfMemoryError
I've also posted about the issue on Stack Overflow here: http://stackoverflow.com/questions/3177092/uploading-to-blobstore-gives-a-java-heap-outofmemoryerror Hopefully getting some more eyeballs on it will help. On Jul 4, 10:47 pm, Adrian Petrescu wrote: > I'm having the exact same issue too, down to the line numbers in the > stack trace. > > Has anyone figured out what is causing this? I find it hard to believe > we've all made the exact same identical mistake, all within the last > month. Is something wrong on Google's end? > > Cheers, > Adrian > > On Jun 2, 7:14 pm, Jean Hsu wrote: > > > > > Hi all, > > > I am trying to set up a basic file upload to blobstore, but I get > > this OutOfMemoryError: > > > WARNING: Error for /_ah/upload/ > > aghvbWdkcmVzc3IcCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGMACDA > > java.lang.OutOfMemoryError: Java heap space > > at java.util.Arrays.copyOf(Arrays.java:2786) > > at > > java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71) > > at > > javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java: > > 316) > > at javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:186) > > at > > javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:109) > > at > > com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(Uploa > > dBlobServlet.java: > > 135) > > at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access > > $000(UploadBlobServlet.java:72) > > at com.google.appengine.api.blobstore.dev.UploadBlobServlet > > $1.run(UploadBlobServlet.java:100) > > at java.security.AccessController.doPrivileged(Native Method) > > at > > com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobS > > ervlet.java: > > 98) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > > at > > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > > 511); > > > I used the Memory Analyzer on Eclipse and it said that the memory leak > > suspect is QueuedThreadPool. I found this information about a memory > > leak bug: > > >http://jira.codehaus.org/browse/JETTY-1188 > > > How can I figure out what release of jetty is running locally? It > > looks like they fixed this in version 6.1.23: > > >http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11160&vers... > > > Has anyone else had this issue? > > > Thanks, > > Jean -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Uploading to blobstore gives OutOfMemoryError
Hi Thomas, Thanks for your reply :) But I'm not sure what you mean by "a plain normal HttpServlet#doPost to handle the upload"; in what sense am I not doing that? Do you mean the fact that I'm uploading to Blobstore? Why would that be a problem, and how else would I get them in there? Thanks, Adrian On Mon, Jul 5, 2010 at 10:03 PM, Thomas wrote: > Hi: > > I read the post on Stack Overflow. My opinion is that you had better > use a plain normal HttpServlet#doPost to handle the upload. I also > encountered 500 Internal Server Error, but it worked well after I > replace original handler with a plain servlet. > > Hope the above helps! > > On 7月6日, 上午8時25分, Adrian Petrescu wrote: > > I've also posted about the issue on Stack Overflow here: > http://stackoverflow.com/questions/3177092/uploading-to-blobstore-giv... > > > > Hopefully getting some more eyeballs on it will help. > > > > On Jul 4, 10:47 pm, Adrian Petrescu wrote: > > > > > I'm having the exact same issue too, down to the line numbers in the > > > stack trace. > > > > > Has anyone figured out what is causing this? I find it hard to believe > > > we've all made the exact same identical mistake, all within the last > > > month. Is something wrong on Google's end? > > > > > Cheers, > > > Adrian > > > > > On Jun 2, 7:14 pm, Jean Hsu wrote: > > > > > > Hi all, > > > > > > I am trying to set up a basic file upload to blobstore, but I get > > > > this OutOfMemoryError: > > > > > > WARNING: Error for /_ah/upload/ > > > > aghvbWdkcmVzc3IcCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGMACDA > > > > java.lang.OutOfMemoryError: Java heap space > > > > at java.util.Arrays.copyOf(Arrays.java:2786) > > > > at > java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71) > > > > at > > > > > javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java: > > > > 316) > > > > at > javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:186) > > > > at > javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:109) > > > > at > > > > > com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(Uploa > dBlobServlet.java: > > > > 135) > > > > at > com.google.appengine.api.blobstore.dev.UploadBlobServlet.access > > > > $000(UploadBlobServlet.java:72) > > > > at com.google.appengine.api.blobstore.dev.UploadBlobServlet > > > > $1.run(UploadBlobServlet.java:100) > > > > at java.security.AccessController.doPrivileged(Native Method) > > > > at > > > > > com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobS > ervlet.java: > > > > 98) > > > > at > javax.servlet.http.HttpServlet.service(HttpServlet.java:713) > > > > at > javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > > > > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > > > > 511); > > > > > > I used the Memory Analyzer on Eclipse and it said that the memory > leak > > > > suspect is QueuedThreadPool. I found this information about a memory > > > > leak bug: > > > > > >http://jira.codehaus.org/browse/JETTY-1188 > > > > > > How can I figure out what release of jetty is running locally? It > > > > looks like they fixed this in version 6.1.23: > > > > > > > http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11160&vers... > > > > > > Has anyone else had this issue? > > > > > > Thanks, > > > > Jean > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Uploading to blobstore gives OutOfMemoryError
Okay guys, I think I've figured out the source of the problem.
If you fail to add a "setName()" attribute on your FileInput element (or,
presumably, any other in the form), the resulting HTTP request is
unparseable which causes the MimeMultipart parser to explode in the way all
of you have encountered. I fixed this just by adding a
"fileInput.setName("file");" and now it uploads to the Blobstore upload url
without any problems.
Even though the problem turned out to be mine, and was easily fixable, I
still consider this a bug in AppEngine because there's no way such a simple
mistake should cause an OutOfMemoryException and crash; it could even be the
source of DOS vulnerabilities in the future just by crafting malicious HTTP
POST requests. I'll file a bug report with Google.
Cheers,
Adrian
On Mon, Jul 5, 2010 at 10:40 PM, Thomas wrote:
> Hi Adrian:
>
> By plain normal servlet I meaned the url path (of the call to
> blobstoreService.createUploadUrl(url) ) is mapping to a simple plain
> HttpServlet implementation. And it is better to have the servlet
> bypass any filter mapping in development phase. It's how I finally
> succeeded to upload lots of blobs without any problems.
>
> On 7月6日, 上午10時07分, Adrian Petrescu wrote:
> > Hi Thomas,
> >
> > Thanks for your reply :) But I'm not sure what you mean by "a plain
> normal
> > HttpServlet#doPost to handle the upload"; in what sense am I not doing
> that?
> > Do you mean the fact that I'm uploading to Blobstore? Why would that be a
> > problem, and how else would I get them in there?
> >
> > Thanks,
> > Adrian
> >
> > On Mon, Jul 5, 2010 at 10:03 PM, Thomas wrote:
> > > Hi:
> >
> > > I read the post on Stack Overflow. My opinion is that you had better
> > > use a plain normal HttpServlet#doPost to handle the upload. I also
> > > encountered 500 Internal Server Error, but it worked well after I
> > > replace original handler with a plain servlet.
> >
> > > Hope the above helps!
> >
> > > On 7月6日, 上午8時25分, Adrian Petrescu wrote:
> > > > I've also posted about the issue on Stack Overflow here:
> > >http://stackoverflow.com/questions/3177092/uploading-to-blobstore-giv.
> ..
> >
> > > > Hopefully getting some more eyeballs on it will help.
> >
> > > > On Jul 4, 10:47 pm, Adrian Petrescu wrote:
> >
> > > > > I'm having the exact same issue too, down to the line numbers in
> the
> > > > > stack trace.
> >
> > > > > Has anyone figured out what is causing this? I find it hard to
> believe
> > > > > we've all made the exact same identical mistake, all within the
> last
> > > > > month. Is something wrong on Google's end?
> >
> > > > > Cheers,
> > > > > Adrian
> >
> > > > > On Jun 2, 7:14 pm, Jean Hsu wrote:
> >
> > > > > > Hi all,
> >
> > > > > > I am trying to set up a basic file upload to blobstore, but I
> get
> > > > > > this OutOfMemoryError:
> >
> > > > > > WARNING: Error for /_ah/upload/
> > > > > > aghvbWdkcmVzc3IcCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGMACDA
> > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > > at java.util.Arrays.copyOf(Arrays.java:2786)
> > > > > > at
> > > java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71)
> > > > > > at
> >
> > >
> javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java:
> > > > > > 316)
> > > > > > at
> > > javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:186)
> > > > > > at
> > > javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:109)
> > > > > > at
> >
> > >
> com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(Uploa
> > > dBlobServlet.java:
> > > > > > 135)
> > > > > > at
> > > com.google.appengine.api.blobstore.dev.UploadBlobServlet.access
> > > > > > $000(UploadBlobServlet.java:72)
> > > > > > at
> com.google.appengine.api.blobstore.dev.UploadBlobServlet
> > > > > > $1.run(UploadBlobServlet.java:100)
> > > > > > at java.security.AccessController.doPrivileged(Native
> Method)
> > > > > > at
> >
> > &
Re: [appengine-java] Re: Uploading to blobstore gives OutOfMemoryError
No, we had the enctype; that's what the "uploadForm.setEncoding(FormPanel. ENCODING_MULTIPART);" was about. As several of us have discovered, it's the name we were missing. The requirement to have encoding set is well-documented, the name is not so much. And the point remains that even if it is documented, running out of memory NOT the right way to respond to it, because any malicious user can send a POST request with no name. On Mon, Oct 25, 2010 at 2:42 AM, zen wrote: > Despite all the documentations and examples you have missed adding the > enctype! > > add this to the form enctype="multipart/form-data" you should have no > problems. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Exact JVM version
Hi App Engine team,
Would you be able to provide information about the exact JVM version
that App Engine is running? I ask because I believe I might be the
victim of an XML-parsing bug that only affects JVM versions: 1.6.0_06,
1.6.0_13, and 1.6.0_17.
However, on App Engine, whenever I print out
System.getProperty("java.vm.version"), it only gives me the very non-
specific "1.6.0" instead of the usual format: "16.0-b13" (on a desktop
JVM).
I've tried searching around but I couldn't find a definitive answer.
So I thought I'd ask here: what is (or how can I find out) the exact
JVM version you guys are running?
If it's a custom version, can you at least tell me if it's forked from
one of the three versions I listed above?
Thanks!
-Adrian
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
