[appengine-java] Re: Uploading Files to Blobstore from external app
You said the blob ends up in the blobstore but your webhook is not called? What's the appid? On Friday, 9 March 2012 21:32:58 UTC+11, Killian wrote: Hi Stuart, Yes apologies if I wasn't clear. On the app engine side of things I'm calling createUploadURL and placing the resulting url in the task payload as follows: BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); String uploadURL = blobstoreService.createUploadUrl(/+url); task.payload(uploadURL); Alternatively, would anyone know of sample code in java trying to programmatically upload blobs from an external app (posting a response to a pull task queue) might make things easier. I'm surprise I couldn't find anything like that on the web. In a nutshell all I'm trying to do is reproduce the architecture of the first pull task system presented in the Google I/O talk http://youtu.be/AM0ZPO7-lcE in java, to place a response to the pull task queue for the app engine app to consume. Thanks for your help :-) On Thursday, 8 March 2012 22:13:45 UTC, Stuart Langley wrote: Is that the actual code? You need to call createUploadURL for each blob you want to upload. On Friday, 9 March 2012 00:03:43 UTC+11, Killian wrote: Hi Lads, This issue has been bugging me for a while now and just wasted the last 2 days with no solution found. I'm getting desperate at this stage ! My aim was to use the Task Queue Pull service to run some NLP processing outside of app engine on an external VM. That's working fine however all the trouble started when I discovered that the output produced by the external VM could be fed back into the Task Queue. Following the example in the Google I/O video http://youtu.be/AM0ZPO7-lcE , I then decided to provide the response of the task queue as a Blob upload to the app engine. So I have the working example http://code.google.com/appengine/docs/java/blobstore/overview.htmlimplement and working perfectly. (ie: I can manually upload a file using the jsp form and store it). The problem is that for obvious reasons I need to send this post request from the VM programmatically. And that's were things start getting messy. Since the video only provides an example in Python and not in java I've been struggling find a solution to this. Looking around it seems like I need the httpcomponents-client-4.1.3 apache library to create post requests. So this is the workflow I have: I generate the upload url from the app engine side and place it in the task queue payload which is then retrieved by the VM correctly. on the VM side this is what I do: String uploadURL = http://host/_ah/upload/agtsaW5ndWFib3gxMHIbCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGAEM ; DefaultHttpClient httpclient = new DefaultHttpClient(); HttpPost method = new HttpPost(uploadURL); MultipartEntity entity = new MultipartEntity(); File f = new File(dummyFileToUpload.txt); FileBody fileBody = new FileBody(f); entity.addPart(myFile, fileBody); method.setEntity(entity); HttpResponse response = httpclient.execute(method); System.out.println(response.getStatusLine()); PROBLEM: When I do this both in development mode or production mode, I receive a http 404 not found response. and it seems like the upload servlet isn't run at all (if I'm in deug mode for instance). However when I look in the blob store, the blob was indeed uploaded correctly which is good news of course however it means I can't get a proper response from the app engine side and more importantly can't run anything in the upload servlet. I'm guessing my post request mustn't be created properly which would explain why the app engine side is reacting weird? Is there anything obviously wrong in my approach? Would really appreciate any advice on this. Thanks in advance :-) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/V_aklEwhcQQJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Uploading Files to Blobstore from external app
The appid is linguabox10 In the mean time I deleted the blobs which I had uploaded and since yesterday when I click on the blob viewer in the app engine dashboard I get a message Oops! We couldn't retrieve your list of Kinds. I'm not sure if these issues are related? Thanks for your help Killian On Friday, 9 March 2012 11:04:30 UTC, Stuart Langley wrote: You said the blob ends up in the blobstore but your webhook is not called? What's the appid? On Friday, 9 March 2012 21:32:58 UTC+11, Killian wrote: Hi Stuart, Yes apologies if I wasn't clear. On the app engine side of things I'm calling createUploadURL and placing the resulting url in the task payload as follows: BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); String uploadURL = blobstoreService.createUploadUrl(/+url); task.payload(uploadURL); Alternatively, would anyone know of sample code in java trying to programmatically upload blobs from an external app (posting a response to a pull task queue) might make things easier. I'm surprise I couldn't find anything like that on the web. In a nutshell all I'm trying to do is reproduce the architecture of the first pull task system presented in the Google I/O talk http://youtu.be/AM0ZPO7-lcE in java, to place a response to the pull task queue for the app engine app to consume. Thanks for your help :-) On Thursday, 8 March 2012 22:13:45 UTC, Stuart Langley wrote: Is that the actual code? You need to call createUploadURL for each blob you want to upload. On Friday, 9 March 2012 00:03:43 UTC+11, Killian wrote: Hi Lads, This issue has been bugging me for a while now and just wasted the last 2 days with no solution found. I'm getting desperate at this stage ! My aim was to use the Task Queue Pull service to run some NLP processing outside of app engine on an external VM. That's working fine however all the trouble started when I discovered that the output produced by the external VM could be fed back into the Task Queue. Following the example in the Google I/O video http://youtu.be/AM0ZPO7-lcE , I then decided to provide the response of the task queue as a Blob upload to the app engine. So I have the working example http://code.google.com/appengine/docs/java/blobstore/overview.htmlimplement and working perfectly. (ie: I can manually upload a file using the jsp form and store it). The problem is that for obvious reasons I need to send this post request from the VM programmatically. And that's were things start getting messy. Since the video only provides an example in Python and not in java I've been struggling find a solution to this. Looking around it seems like I need the httpcomponents-client-4.1.3 apache library to create post requests. So this is the workflow I have: I generate the upload url from the app engine side and place it in the task queue payload which is then retrieved by the VM correctly. on the VM side this is what I do: String uploadURL = http://host/_ah/upload/agtsaW5ndWFib3gxMHIbCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGAEM ; DefaultHttpClient httpclient = new DefaultHttpClient(); HttpPost method = new HttpPost(uploadURL); MultipartEntity entity = new MultipartEntity(); File f = new File(dummyFileToUpload.txt); FileBody fileBody = new FileBody(f); entity.addPart(myFile, fileBody); method.setEntity(entity); HttpResponse response = httpclient.execute(method); System.out.println(response.getStatusLine()); PROBLEM: When I do this both in development mode or production mode, I receive a http 404 not found response. and it seems like the upload servlet isn't run at all (if I'm in deug mode for instance). However when I look in the blob store, the blob was indeed uploaded correctly which is good news of course however it means I can't get a proper response from the app engine side and more importantly can't run anything in the upload servlet. I'm guessing my post request mustn't be created properly which would explain why the app engine side is reacting weird? Is there anything obviously wrong in my approach? Would really appreciate any advice on this. Thanks in advance :-) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/Gy_ICjSG-8AJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Uploading Files to Blobstore from external app
It looks like the URL that you specified for the callback in createUploadURL does not exist - there's a 404 when we try and call it. On Friday, 9 March 2012 22:39:58 UTC+11, Killian wrote: The appid is linguabox10 In the mean time I deleted the blobs which I had uploaded and since yesterday when I click on the blob viewer in the app engine dashboard I get a message Oops! We couldn't retrieve your list of Kinds. I'm not sure if these issues are related? Thanks for your help Killian On Friday, 9 March 2012 11:04:30 UTC, Stuart Langley wrote: You said the blob ends up in the blobstore but your webhook is not called? What's the appid? On Friday, 9 March 2012 21:32:58 UTC+11, Killian wrote: Hi Stuart, Yes apologies if I wasn't clear. On the app engine side of things I'm calling createUploadURL and placing the resulting url in the task payload as follows: BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); String uploadURL = blobstoreService.createUploadUrl(/+url); task.payload(uploadURL); Alternatively, would anyone know of sample code in java trying to programmatically upload blobs from an external app (posting a response to a pull task queue) might make things easier. I'm surprise I couldn't find anything like that on the web. In a nutshell all I'm trying to do is reproduce the architecture of the first pull task system presented in the Google I/O talk http://youtu.be/AM0ZPO7-lcE in java, to place a response to the pull task queue for the app engine app to consume. Thanks for your help :-) On Thursday, 8 March 2012 22:13:45 UTC, Stuart Langley wrote: Is that the actual code? You need to call createUploadURL for each blob you want to upload. On Friday, 9 March 2012 00:03:43 UTC+11, Killian wrote: Hi Lads, This issue has been bugging me for a while now and just wasted the last 2 days with no solution found. I'm getting desperate at this stage ! My aim was to use the Task Queue Pull service to run some NLP processing outside of app engine on an external VM. That's working fine however all the trouble started when I discovered that the output produced by the external VM could be fed back into the Task Queue. Following the example in the Google I/O video http://youtu.be/AM0ZPO7-lcE , I then decided to provide the response of the task queue as a Blob upload to the app engine. So I have the working example http://code.google.com/appengine/docs/java/blobstore/overview.htmlimplement and working perfectly. (ie: I can manually upload a file using the jsp form and store it). The problem is that for obvious reasons I need to send this post request from the VM programmatically. And that's were things start getting messy. Since the video only provides an example in Python and not in java I've been struggling find a solution to this. Looking around it seems like I need the httpcomponents-client-4.1.3 apache library to create post requests. So this is the workflow I have: I generate the upload url from the app engine side and place it in the task queue payload which is then retrieved by the VM correctly. on the VM side this is what I do: String uploadURL = http://host/_ah/upload/agtsaW5ndWFib3gxMHIbCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGAEM ; DefaultHttpClient httpclient = new DefaultHttpClient(); HttpPost method = new HttpPost(uploadURL); MultipartEntity entity = new MultipartEntity(); File f = new File(dummyFileToUpload.txt); FileBody fileBody = new FileBody(f); entity.addPart(myFile, fileBody); method.setEntity(entity); HttpResponse response = httpclient.execute(method); System.out.println(response.getStatusLine()); PROBLEM: When I do this both in development mode or production mode, I receive a http 404 not found response. and it seems like the upload servlet isn't run at all (if I'm in deug mode for instance). However when I look in the blob store, the blob was indeed uploaded correctly which is good news of course however it means I can't get a proper response from the app engine side and more importantly can't run anything in the upload servlet. I'm guessing my post request mustn't be created properly which would explain why the app engine side is reacting weird? Is there anything obviously wrong in my approach? Would really appreciate any advice on this. Thanks in advance :-) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/9r3EKX4CVjoJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Uploading Files to Blobstore from external app
haha ! My god I can't believe it was so easy ! I think I was probably over thinking it. Yes so the only reason why the app engine side was returning this 404 code is because I was created the upload url on random names instead of giving it the servlet name upload. Thank you so much :-) On Friday, 9 March 2012 12:03:18 UTC, Stuart Langley wrote: It looks like the URL that you specified for the callback in createUploadURL does not exist - there's a 404 when we try and call it. On Friday, 9 March 2012 22:39:58 UTC+11, Killian wrote: The appid is linguabox10 In the mean time I deleted the blobs which I had uploaded and since yesterday when I click on the blob viewer in the app engine dashboard I get a message Oops! We couldn't retrieve your list of Kinds. I'm not sure if these issues are related? Thanks for your help Killian On Friday, 9 March 2012 11:04:30 UTC, Stuart Langley wrote: You said the blob ends up in the blobstore but your webhook is not called? What's the appid? On Friday, 9 March 2012 21:32:58 UTC+11, Killian wrote: Hi Stuart, Yes apologies if I wasn't clear. On the app engine side of things I'm calling createUploadURL and placing the resulting url in the task payload as follows: BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); String uploadURL = blobstoreService.createUploadUrl(/+url); task.payload(uploadURL); Alternatively, would anyone know of sample code in java trying to programmatically upload blobs from an external app (posting a response to a pull task queue) might make things easier. I'm surprise I couldn't find anything like that on the web. In a nutshell all I'm trying to do is reproduce the architecture of the first pull task system presented in the Google I/O talk http://youtu.be/AM0ZPO7-lcE in java, to place a response to the pull task queue for the app engine app to consume. Thanks for your help :-) On Thursday, 8 March 2012 22:13:45 UTC, Stuart Langley wrote: Is that the actual code? You need to call createUploadURL for each blob you want to upload. On Friday, 9 March 2012 00:03:43 UTC+11, Killian wrote: Hi Lads, This issue has been bugging me for a while now and just wasted the last 2 days with no solution found. I'm getting desperate at this stage ! My aim was to use the Task Queue Pull service to run some NLP processing outside of app engine on an external VM. That's working fine however all the trouble started when I discovered that the output produced by the external VM could be fed back into the Task Queue. Following the example in the Google I/O video http://youtu.be/AM0ZPO7-lcE , I then decided to provide the response of the task queue as a Blob upload to the app engine. So I have the working example http://code.google.com/appengine/docs/java/blobstore/overview.htmlimplement and working perfectly. (ie: I can manually upload a file using the jsp form and store it). The problem is that for obvious reasons I need to send this post request from the VM programmatically. And that's were things start getting messy. Since the video only provides an example in Python and not in java I've been struggling find a solution to this. Looking around it seems like I need the httpcomponents-client-4.1.3 apache library to create post requests. So this is the workflow I have: I generate the upload url from the app engine side and place it in the task queue payload which is then retrieved by the VM correctly. on the VM side this is what I do: String uploadURL = http://host/_ah/upload/agtsaW5ndWFib3gxMHIbCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGAEM ; DefaultHttpClient httpclient = new DefaultHttpClient(); HttpPost method = new HttpPost(uploadURL); MultipartEntity entity = new MultipartEntity(); File f = new File(dummyFileToUpload.txt); FileBody fileBody = new FileBody(f); entity.addPart(myFile, fileBody); method.setEntity(entity); HttpResponse response = httpclient.execute(method); System.out.println(response.getStatusLine()); PROBLEM: When I do this both in development mode or production mode, I receive a http 404 not found response. and it seems like the upload servlet isn't run at all (if I'm in deug mode for instance). However when I look in the blob store, the blob was indeed uploaded correctly which is good news of course however it means I can't get a proper response from the app engine side and more importantly can't run anything in the upload servlet. I'm guessing my post request mustn't be created properly which would explain why the app engine side is reacting weird? Is there anything obviously wrong in my approach? Would really appreciate any advice on this. Thanks in advance :-) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit
Re: [appengine-java] Re: Slow cold starts
That totally sucks. I can understand that it takes up more resources to have an app always loaded in an instance but when minimum instances is set, then it should mean that the app is kept in memory. If what you write is true (and I guess it is since nobody from Google has objected to it) then Google is not fulfilling what they themselves claim: Because App Engine keeps idle instances in reserve, it is unlikely that requests will enter the pending queue except in exceptionally high load spikes. http://code.google.com/appengine/docs/adminconsole/performancesettings.html On Thursday, March 8, 2012 10:10:03 PM UTC+1, Mark Rathwell wrote: Just wanted to follow up on this: Idle instances set to 1: 7 of 10 requests were loading requests (20-30 seconds) Idle instances set to 2: 3 of 9 requests were loading requests (and cost was about 4 times as much) This is a test app, with no other traffic, no new versions uploaded, etc., and these requests were about once a day for the last few weeks. I'm giving up on finding a setting that will mimic always on for low traffic apps, and just setting up a cron job to ping the apps regularly. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/ly0xBCd9rVkJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Unable to load several versions of AppEngine Project and Seeing HardDeadLineExceeded Exceptions
I'm also seeing those problems since the latest maitainance on MS. My app is with a high error rate because of this, and since multithreaded is enabled, the error is multiplied. Please, if your issue is similar to thoes ones, star this production issue: http://code.google.com/p/googleappengine/issues/detail?id=7108 Em quinta-feira, 8 de março de 2012 09h42min25s UTC-3, Luke escreveu: i also have similar error. anyone figured out why this happen? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/zgjyGf_R3C0J. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Slow cold starts
Even we are finding these slow cold starts, very similar to what Jeff described earlier. Can someone from Google please respond? Thanks, Arun From: Anders Testson anders...@gmail.com To: google-appengine-java@googlegroups.com Sent: Friday, 9 March 2012 7:24 PM Subject: Re: [appengine-java] Re: Slow cold starts That totally sucks. I can understand that it takes up more resources to have an app always loaded in an instance but when minimum instances is set, then it should mean that the app is kept in memory. If what you write is true (and I guess it is since nobody from Google has objected to it) then Google is not fulfilling what they themselves claim: Because App Engine keeps idle instances in reserve, it is unlikely that requests will enter the pending queue except in exceptionally high load spikes. http://code.google.com/appengine/docs/adminconsole/performancesettings.html On Thursday, March 8, 2012 10:10:03 PM UTC+1, Mark Rathwell wrote: Just wanted to follow up on this: Idle instances set to 1: 7 of 10 requests were loading requests (20-30 seconds) Idle instances set to 2: 3 of 9 requests were loading requests (and cost was about 4 times as much) This is a test app, with no other traffic, no new versions uploaded, etc., and these requests were about once a day for the last few weeks. I'm giving up on finding a setting that will mimic always on for low traffic apps, and just setting up a cron job to ping the apps regularly. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/ly0xBCd9rVkJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
