[google-appengine] Datastore Viewer Not Working
I've been trying to access our Datastore viewer in the AppEngine admin interface and it has been throwing an error for the better part of a week. Are there any known issues going on? We're not on the HRD and do have 250 million records if that makes a difference, appid is noticeorange. Thanks, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/q-FruHifaVwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[appengine-java] Re: Conversion API - Adding Assets Example Wrongly Documented
Thanks for the code Raj, saved me.
If it's useful to anyone else, based on your sample I wrote a general
purpose URL-PDF function that will search the HTML for any dependent
images and insert them, only external dependency is commons.io
-Casey
String content = IOUtils.toString(new
URL(url).openStream());
Asset asset = new Asset(text/html, content.getBytes(),
in.html);
Matcher m = Pattern.compile(img[^]*src=\([^\]*),
Pattern.CASE_INSENSITIVE).matcher(content);
ListAsset assetsList = new ArrayListAsset();
assetsList.add(asset);
while (m.find()) {
String relative = m.group(1);
URL absolute = new URL(new URL(url), relative);
byte[] image_data =
IOUtils.toByteArray(absolute.openStream());
String guessedContentType = null;
if (relative.toLowerCase().endsWith(.png)) {
guessedContentType = image/png;
} else if (relative.toLowerCase().endsWith(.jpg)) {
guessedContentType = image/jpeg;
} else if (relative.toLowerCase().endsWith(.gif)) {
guessedContentType = image/gif;
}
Asset subAsset = new Asset(guessedContentType == null
|| response.getContentType() != null ? response.getContentType() :
guessedContentType, image_data, relative);
assetsList.add(subAsset);
}
Document document = new Document(assetsList);
Conversion conversion = new Conversion(document,
application/pdf);
ConversionService service =
ConversionServiceFactory.getConversionService();
ConversionResult result = service.convert(conversion);
if (result.success()) {
response.getOutputStream().write(result.getOutputDoc().getAssets().get(0).getData());
} else {
logger().severe(result.getErrorCode().name());
response.getOutputStream().write(result.getErrorCode().name().getBytes());
}
On Jan 5, 10:23 am, RAJ er.rajes...@gmail.com wrote:
I have solved this issue now :) (Java Documentation must be updated
with right example)
code
Asset asset = new Asset(
text/html, bsome data/bimg src=static/icon.gif/
.getBytes(), testfile.html);
byte[] image_data = // Read yourimageinto bytes[] array;
Asset subAsset = new Asset(
image/gif, image_data, static/icon.gif);
ListAsset assetsList = new ArrayListAsset();
assetsList.add(asset);
assetsList.add(subAsset);
Document document = new Document(assetsList);
/code
Cheers,
Raj
On Jan 4, 5:31 pm, RAJ er.rajes...@gmail.com wrote:
Hi All,
I am trying to producepdfreports out of my html String as mentioned
in below tutorial:
http://code.google.com/appengine/docs/java/conversion/overview.html
code
// Create a conversion request from HTML to PNG.
Asset asset = new Asset(
text/html, bsome data/b.getBytes(), testfile.html);
Document document = new Document(asset);
/code
My HTML String has few pictures to be embedded. Thus, I am trying
Additional Functions (Adding Assets) example to add images as sub
asset.
But, example mentioned in tutorial is wrong. I am not able to add
imageas sub-asset in my main html string. There is no java api
mentioned for the same!
Has someone done this earlier using Java? (Example in python is
correctly described)
Cheers,
Raj
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [google-appengine] Datastore Small Operations and Nov 7th
App ID has been sent via private reply, but for anyone else looking around it's onlyrounder-sc. Thanks for taking a look. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Dd6QvVMoJ4sJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Datastore Small Operations and Nov 7th
It appears that on the same day billing was enabled (Nov 7th), what counted as a Datastore Small Operation also changed. Is that true? In all of my billing previews before Nov 7th, the count was at an acceptable level, but started to climb on Nov 7th, topping out at around 100x what it was previously. Causing an accumulation of over $1,000 in bills that should have been less than a tenth of that. I left for vacation expecting the billing preview level to stay the same after billing was enabled, but appears that count() operations were not included prior to the Nov 7th update? Code wasn't changed since mid-October, and here are the counts for small operations (in millions): Nov 6th: 14 (Days before the 6th were at or under this level) Nov 7th: 10 Nov 8th: 41 Nov 9th: 118 Nov 10th: 192 Nov 11th: 462 Nov 12th: 632 And then it levels out here, using up all quota each day, which was my fault for leaving so high as I had imported a lot of data prior to leaving. Code didn't change, traffic patterns didn't change (there may have been a bit of a fight with spammers hammering the site a bit, but that doesn't appear to be a totally new thing - still trying to post-mortem that but very difficult weeks after the fact). Only my bill went crazy. All posts that talk about these operations changing reference these two URLs: http://code.google.com/appengine/docs/quotas.html#Resources http://code.google.com/appengine/docs/kb/billing.html#operations_charged_for One of which can't be found and the other doesn't seem to have any information related to this. Can someone clarify what changed and when? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Scheduler Oddities
It looks like we are being charged at the higher rate of 30-35 instance hours / hour even though only ~10 are active at a time and max-idle is set to 5. Although it's still hard to tell as we're behind ~5 days in billing history summaries. Is the current scheduler going to be changed before new-billing is implemented? Seems like a must. On Sep 1, 4:48 pm, Millisecond millisec...@gmail.com wrote: The table at the top of the dashboard is showing 23 active instances, QPS of 2.6 and a latency of 481ms. Which makes some sense as I'm hovering between 40 and 80 QPS overall. I've set the scheduler to have max 5 idle instances and the min latency to 60ms, but when I pull down the graph to the Instances display, it has me hovering between ~5 and ~12 active and 30-35 total. Is the graph just not accurate and we're billed for what's in the text area? Are we charged for active instances or total instances or instances in the text area? Why isn't it more-or-less active + 5 as max = total? With a 3-4 day delay on my billing history reporting, this is going to be very hard to tweak if I can't see correct numbers on the dashboard. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Scheduler Oddities
I agree that we shouldn't be, it just seems that we are. I'll just have to wait for more days to show up in Billing History where I have screenshots of the dashboard available to compare against. Frustrating to have to wait 5 days to see what something is going to be billed at. We have basically 2 cycles to make changes and see the effects before billing goes into effect and the two things biting us are instance-hours and datastore writes. One is totally opaque and the other appears to be inconsistently reported in the live graphs (right now I have 20 Total in text, 35 total / 12 active in the graph). On Sep 5, 3:40 pm, Johan Euphrosine pro...@google.com wrote: You should never be charged for more than: Active Instance + Max Idle instances, even if the scheduler keeps more than Max Idle instances around. Setting Min-Pending-Latency to 60ms instruct the scheduler to wait at least 60ms if all the instances are busy before deciding to spawn a new instance for handling an incoming request. You can maximize existing instance usage over new instance creation by increasing that value, but this could come at the expense of increasing request latency. Hope that helps. On Mon, Sep 5, 2011 at 3:12 PM, Millisecond millisec...@gmail.com wrote: It looks like we are being charged at the higher rate of 30-35 instance hours / hour even though only ~10 are active at a time and max-idle is set to 5. Although it's still hard to tell as we're behind ~5 days in billing history summaries. Is the current scheduler going to be changed before new-billing is implemented? Seems like a must. On Sep 1, 4:48 pm, Millisecond millisec...@gmail.com wrote: The table at the top of the dashboard is showing 23 active instances, QPS of 2.6 and a latency of 481ms. Which makes some sense as I'm hovering between 40 and 80 QPS overall. I've set the scheduler to have max 5 idle instances and the min latency to 60ms, but when I pull down the graph to the Instances display, it has me hovering between ~5 and ~12 active and 30-35 total. Is the graph just not accurate and we're billed for what's in the text area? Are we charged for active instances or total instances or instances in the text area? Why isn't it more-or-less active + 5 as max = total? With a 3-4 day delay on my billing history reporting, this is going to be very hard to tweak if I can't see correct numbers on the dashboard. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: 回覆：Re: [google-appengine] Data Transfer to AWS
Update on our progress: -Currently rather busy optimizing what we can on GAE as we can't take a $7,000 / mo bill until the transfer is complete and it's going to take a non-trivial amount of time to move. -Very app-specific, but our data is really broken out into two categories, configuration and data. We're going to move the configuration over, start the servers on the new side and then have a process pull the data, taking however many days are required. User's will just see data appear over time. -Planning on using the remote API from a command-line utility (we'll probably have to write from scratch as it's so specific) to pull into SDB newer entries first. We'll run it on the AWS side to do the transfer. http://code.google.com/appengine/docs/java/tools/remoteapi.html -Biggest outstanding problem is id matching and allocation. We'll need to start new data records with an ID higher than what's allocated in GAE so the subsequent pull doesn't clobber those allocated in the meantime. Haven't tackled that yet, but assuming the solution won't be super hard. Also spending some time wishing that AppScale would run on Beanstalk / SDB, so our code changes would be super minimal. Digging through their code now to see how hard it would be to add Beanstalk / SDB support to their current setup. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Scheduler Oddities
Jon, Thank you for the detailed response and taking the time to look at my specific application. I'll take a look at the new billing statement in the morning (I'm central european time) and see what I can tell. On Sep 5, 10:49 pm, Jon McAlister jon...@google.com wrote: Hi millisecond, I can illuminate some of the issues here. As Johan said, the bill will be based on active-instances [the orange line] plus max-idle-instances. That is correct. I can see though how your 09-01 billing report is confusing though, I can explain that. Also, I'm very sorry that the billing reports are delayed so long, we're working on that. The billing reports are computed on the usage for the day, based on Pacific Standard Time. In your case, the setting of max-idle-instances=5 was made at 2011/09/01-04:28:41. So, it applied to the remaining 19.5 hours of the day, but not to the initial 4.5 hours for the day. This is why it didn't drop as low as expected (it dropped from ~3000 instance-hours to 764 instance hours, but obviously it should be going lower than that). But, we can figure out the average billed instance rate for the latter 19.5 hours of the day by solving for: 3000 / 24 * 4.5 + N * 19.5 = 764 Which is N=10.3. As such, I would expect the 09-02 billing report to have ~250 instance hours. We'll see in a few hours if this is correct or not. The number 10.3 seems correct though. Your active-instances rate graph is a sawtooth graph with a trough of 3 and a peak of 13. So, the average of that graph, plus 5, coming out to 10.3, looks correct to me. I'll check back in on you once the 09-02 billing report comes out in a few hours for you. On Mon, Sep 5, 2011 at 7:12 AM, Johan Euphrosine pro...@google.com wrote: The Max Idle instances slider help text states: *You will not be charged for instances over the specified maximum* So if you are, it is a billing bug and you should ask a refund. Feel free to a production issue with your application id if you want us to track the instance inconsistencies for your application: http://code.google.com/p/googleappengine/issues/entry?template=Produc... I would also suggest to fill a feature request for faster billing preview/reporting: http://code.google.com/p/googleappengine/issues/entry?template=Featur... Hope that helps. On Mon, Sep 5, 2011 at 4:00 PM, Millisecond millisec...@gmail.com wrote: I agree that we shouldn't be, it just seems that we are. I'll just have to wait for more days to show up in Billing History where I have screenshots of the dashboard available to compare against. Frustrating to have to wait 5 days to see what something is going to be billed at. We have basically 2 cycles to make changes and see the effects before billing goes into effect and the two things biting us are instance-hours and datastore writes. One is totally opaque and the other appears to be inconsistently reported in the live graphs (right now I have 20 Total in text, 35 total / 12 active in the graph). On Sep 5, 3:40 pm, Johan Euphrosine pro...@google.com wrote: You should never be charged for more than: Active Instance + Max Idle instances, even if the scheduler keeps more than Max Idle instances around. Setting Min-Pending-Latency to 60ms instruct the scheduler to wait at least 60ms if all the instances are busy before deciding to spawn a new instance for handling an incoming request. You can maximize existing instance usage over new instance creation by increasing that value, but this could come at the expense of increasing request latency. Hope that helps. On Mon, Sep 5, 2011 at 3:12 PM, Millisecond millisec...@gmail.com wrote: It looks like we are being charged at the higher rate of 30-35 instance hours / hour even though only ~10 are active at a time and max-idle is set to 5. Although it's still hard to tell as we're behind ~5 days in billing history summaries. Is the current scheduler going to be changed before new-billing is implemented? Seems like a must. On Sep 1, 4:48 pm, Millisecond millisec...@gmail.com wrote: The table at the top of the dashboard is showing 23 active instances, QPS of 2.6 and a latency of 481ms. Which makes some sense as I'm hovering between 40 and 80 QPS overall. I've set the scheduler to have max 5 idle instances and the min latency to 60ms, but when I pull down the graph to the Instances display, it has me hovering between ~5 and ~12 active and 30-35 total. Is the graph just not accurate and we're billed for what's in the text area? Are we charged for active instances or total instances or instances in the text area? Why isn't it more-or-less active + 5 as max = total? With a 3-4 day delay on my billing history reporting, this is going to be very hard to tweak if I can't see correct numbers on the dashboard. -- You received this message because you are subscribed
[google-appengine] Data Transfer to AWS
Won't rehash the pricing discussions, but because of the pricing changes and the way it's been handled we're 90% sure we're going to move to AWS. Anyways... now I'm tasked with figuring out how to get our almost TB of data over there either into SimpleDB or into an RDS instance, not sure yet. Have other people done the move, how did it go? Did you pull from AWS / push from GAE? Take an intermediate backup and then load from some other mechanism (thinking S3) into SDB / RDS? Our app is adding over 1 GB / day with 20MM reads and 20MM writes, and we'd rather not take it offline for too long... Thinking of some crazy scheme based on descending keys to move data over from key A downwards, shut off app, move over everything from key A upwards as we're mostly only writing new data. Maybe special-case a few object classes and use timestamps to detect deltas. Realize it's pretty app-specific, but wondering if other people have tackled that same problem and what their experiences were. Thanks for any and all help, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Keep it short: Who is forced to leave GAE?
We're using Java and expect a 7x increase. With threading, and some optimizations we should bring that down to 3 or 4x - I guess we're a bit lucky, way less than a lot of people. But, when you're over $1k/mo already and a small business, it's really hard to take. And more than the cost, the way this whole thing has been handled is very odd. Will explore the options to go to another cloud provider, but what a pain in the arse to move a TB of data and a handful of apps with a collective ~50k LoC and 30 million reqs/day. Hopefully we can find a reasonable path forward. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Scheduler Oddities
The table at the top of the dashboard is showing 23 active instances, QPS of 2.6 and a latency of 481ms. Which makes some sense as I'm hovering between 40 and 80 QPS overall. I've set the scheduler to have max 5 idle instances and the min latency to 60ms, but when I pull down the graph to the Instances display, it has me hovering between ~5 and ~12 active and 30-35 total. Is the graph just not accurate and we're billed for what's in the text area? Are we charged for active instances or total instances or instances in the text area? Why isn't it more-or-less active + 5 as max = total? With a 3-4 day delay on my billing history reporting, this is going to be very hard to tweak if I can't see correct numbers on the dashboard. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Data Transfer to AWS
App Rocket looks awesome. Assuming I can start it up in a non-default Python instance (I'm normally Java) the same way I've done for some of the Google-supplied python tools that'll really help. Thanks! -Casey On Sep 1, 7:23 pm, Waleed Abdulla wal...@ninua.com wrote: Another tool I used in the past to move my data into GAE (it works both ways) is App Rocket. It uses timestamps to do real-time replication between GAE and mysql. You might need to tweak it a little bit to fit your need exactly, but it gets you most of the way there. https://github.com/k7d/approcket/wiki/ On Thu, Sep 1, 2011 at 10:15 AM, Daniel danielshaneup...@gmail.com wrote: We will be facing the same problem as we leave app engine. I believe I'm going to attempt to use the bulk downloader to get the data to a local computer. I am slightly worried about the cost of transferring 11TB. I believe that the process make take a couple of days, so I plan on moving inbound records to the new database before I start the d/l, that way I can have reduced functionality over the data transfer period but no data will be lost. -Daniel On Sep 1, 8:22 am, Millisecond millisec...@gmail.com wrote: Won't rehash the pricing discussions, but because of the pricing changes and the way it's been handled we're 90% sure we're going to move to AWS. Anyways... now I'm tasked with figuring out how to get our almost TB of data over there either into SimpleDB or into an RDS instance, not sure yet. Have other people done the move, how did it go? Did you pull from AWS / push from GAE? Take an intermediate backup and then load from some other mechanism (thinking S3) into SDB / RDS? Our app is adding over 1 GB / day with 20MM reads and 20MM writes, and we'd rather not take it offline for too long... Thinking of some crazy scheme based on descending keys to move data over from key A downwards, shut off app, move over everything from key A upwards as we're mostly only writing new data. Maybe special-case a few object classes and use timestamps to detect deltas. Realize it's pretty app-specific, but wondering if other people have tackled that same problem and what their experiences were. Thanks for any and all help, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] C2DM Rate Limiting
Hey all, We're running into an odd issue, and not sure if it's us, the AppEngine group, or the C2DM Android group at Google so I thought I'd post here to start and see where we can get. We have implemented a C2DM server on AppEngine servicing several hundred of our clients' Android applications, sending a lot of messages. It was working quite well for a while, but now we're seeing a lot of Error=QuotaExceeded returns from the C2DM server. Seems like one of three causes: -Our back-off algorithms aren't correct. Hard to tell though as the google docs are very light on what's an acceptable rate: http://code.google.com/android/c2dm/index.html I have turned off all retry logic, only sending original messages and it still triggers, so guessing it's not a back-off problem. -We're just plain hitting a C2DM limit, but would be great to be able to talk to someone at Google-Android about raising it. -Google is limited based on source IP and lumping all GAE requests together in a DDoS prevention bucket. Which would mean that over time our ability to send messages from GAE would only become worse. At this point I'm just looking to eliminate possibilities and the third one there seems to be the easiest to rule out. Does anyone on the AppEngine team have insight into whether that might be what's happening? Thanks, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[appengine-java] Blobstore Write
Hey all, Am I reading it right that there really no way to write data into a Blobstore object programmatically? I swear I had read something about it, but can only see the InputStream additions now and maybe I'd misread it as an OutputStream. I have several million images I've collected from RSS feed polling, and several million more coming that I would love to use with the new fast image serving functionality, but I can't get them into the Blobstore in the first place as no user is ever involved with the creation of them. Toyed with the idea of using an HTTP POST from inside an AppEngine task, but that seems fragile and wasteful. Is this something that will ever be supported with Blobstore? Thanks, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Transaction exception when not using a transaction ?
Hmmm.
How is your object model structured? Do entities have other entities
as direct references?
If you have class A { B b } and setting b to an instance of B that
already existed might throw the error you're seeing.
I changed my structure to be class A { String bKey } for a variety
of other reasons, but guess it also may have gotten around this
problem.
Alternatively, post as much code as you can and I'll see if anything
else rings a bell, early on I spent a frustrated day or two working
around this exception in a variety of places so know it can be
frustrating.
-C
On Jun 3, 6:33 am, JD liva...@gmail.com wrote:
I added a call to flush() after every change on the PM-managed object,
but that did not help.
Also doing a pm.flush() does not throw this exception, but doing a
pm.close() does - which seems to be contrary to your reasoning (I
would expect flush to throw the same exception if multiple entity
groups were at cause).
On Jun 3, 1:33 am, Millisecond millisec...@gmail.com wrote:
Even though you're not using transactions, I think it's trying to make
the .close() call atomic (maybe with an internal implicit
transaction), failing or succeeding as a whole. And as I understand
it, entities not in the same group can be stored on separate machines
so can't be operated on atomically.
I've worked around this by calling a flush() after most any change on
a PM-managed object until close(). If this has a huge number of reqs/
s, you may want to re-architect to get the objects in the same entity
group as multiple flushes won't be very efficient.
-C
On Jun 2, 8:26 pm, JD liva...@gmail.com wrote:
I have a PersistenceManager
PersistenceManager pm = PMF.get().getPersistenceManager();
which I use to do a bunch of operations on different objects, but
WITHOUT transaction (run queries, store entities and lookup entities).
I then close the manager with pm.close() and get this obscure error
that complains about multiple entity groups inside a transaction even
though I am not using a transaction (you will notice that the error is
not the usual one where it prints the different entity groups).
I am not 100% confident but have the impression that this error
started happening with the 1.3.4 release.
100% reproducible use case. Would appreciate input from App Engine
developers.
com.myapp.servlet.task.PopulateUserPages doAction: Illegal argument
javax.jdo.JDOFatalUserException: Illegal argument
at
org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(Nuc
leusJDOHelper.java:
344)
at
org.datanucleus.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:
281)
at
com.myapp.dao.jdo.DatastoreService.release(DatastoreService.java:
631)
at
com.myapp.servlet.AbstractBaseServlet.releaseService(AbstractBaseServlet.ja
va:
243)
at
com.myapp.servlet.task.PopulateUserPages.doAction(PopulateUserPages.java:
132)
at com.myapp.servlet.task.Dispatcher.doAction(Dispatcher.java:35)
at
com.myapp.servlet.task.TaskServlet.doAction(TaskServlet.java:36)
at
com.myapp.servlet.AbstractBaseServlet.doGenericAction(AbstractBaseServlet.j
ava:
194)
at
com.myapp.servlet.AbstractBaseServlet.doPost(AbstractBaseServlet.java:
84)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
511)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1166)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlo
bUploadFilter.java:
97)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionF
ilter.java:
35)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans
actionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
418)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionH
[appengine-java] Re: Transaction exception when not using a transaction ?
Even though you're not using transactions, I think it's trying to make the .close() call atomic (maybe with an internal implicit transaction), failing or succeeding as a whole. And as I understand it, entities not in the same group can be stored on separate machines so can't be operated on atomically. I've worked around this by calling a flush() after most any change on a PM-managed object until close(). If this has a huge number of reqs/ s, you may want to re-architect to get the objects in the same entity group as multiple flushes won't be very efficient. -C On Jun 2, 8:26 pm, JD liva...@gmail.com wrote: I have a PersistenceManager PersistenceManager pm = PMF.get().getPersistenceManager(); which I use to do a bunch of operations on different objects, but WITHOUT transaction (run queries, store entities and lookup entities). I then close the manager with pm.close() and get this obscure error that complains about multiple entity groups inside a transaction even though I am not using a transaction (you will notice that the error is not the usual one where it prints the different entity groups). I am not 100% confident but have the impression that this error started happening with the 1.3.4 release. 100% reproducible use case. Would appreciate input from App Engine developers. com.myapp.servlet.task.PopulateUserPages doAction: Illegal argument javax.jdo.JDOFatalUserException: Illegal argument at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(Nuc leusJDOHelper.java: 344) at org.datanucleus.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java: 281) at com.myapp.dao.jdo.DatastoreService.release(DatastoreService.java: 631) at com.myapp.servlet.AbstractBaseServlet.releaseService(AbstractBaseServlet.ja va: 243) at com.myapp.servlet.task.PopulateUserPages.doAction(PopulateUserPages.java: 132) at com.myapp.servlet.task.Dispatcher.doAction(Dispatcher.java:35) at com.myapp.servlet.task.TaskServlet.doAction(TaskServlet.java:36) at com.myapp.servlet.AbstractBaseServlet.doGenericAction(AbstractBaseServlet.j ava: 194) at com.myapp.servlet.AbstractBaseServlet.doPost(AbstractBaseServlet.java: 84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 511) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1166) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlo bUploadFilter.java: 97) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionF ilter.java: 35) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans actionCleanupFilter.java: 43) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionH andlerMap.java: 238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 542) at org.mortbay.jetty.HttpConnection $RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequ estParser.java: 76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceReques t(JettyServletEngineAdapter.java: 135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java: 250) at com.google.apphosting.base.RuntimePb$EvaluationRuntime $6.handleBlockingRequest(RuntimePb.java:5838) at com.google.apphosting.base.RuntimePb$EvaluationRuntime $6.handleBlockingRequest(RuntimePb.java:5836) at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingAp plicationHandler.java: 24) at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java: 398)
[google-appengine] Re: Unscheduled downtime/problems lately
Thanks Ikai, I look forward to reading the post-mortem. The last one was indeed very good. Will the post-mortem address the dramatically increased latency since the 25th? It's ongoing and has already caused tens of thousands of errors on our site alone. As far as I can tell, there's been no official communication regarding that. And it's still continuing with 1500ms+ for a lot of today: http://code.google.com/status/appengine/detail/datastore/2010/06/01#ae-trust-detail-datastore-query-latency -C On Jun 1, 3:16 pm, Ikai L (Google) ika...@google.com wrote: I just wanted to let you guys know to hang in there. We're working on a post-mortem describing what went wrong and steps we are taking to prevent this from happening again. I don't have an ETA for this yet, but we've done this before (http://groups.google.com/group/google-appengine/browse_thread/thread/...) and are committed to as much transparency as we can with regards to these issues with you guys. On Tue, Jun 1, 2010 at 2:59 PM, Mark mar...@gmail.com wrote: +1 as worried, I'm investing time in app engine and this is quite discouraging to say the least! How often does this happen? (I don't have a live app yet), but should I abandon app engine now before getting myself in trouble? On May 27, 1:32 pm, Jody Belka j...@jj79.org wrote: And what do you know, no-one from Google has made any response whatsoever to this thread. And yet we know they are around, they occasionally do write messages after all. Very disappointing. On 25 May 2010 21:51, Waleed Abdulla wal...@ninua.com wrote: It's definitely worrisome. My app is down at the moment, my task queue backlog is at 4 days, and I'm very disappointed. On Tue, May 25, 2010 at 12:46 PM, Flips p...@script-network.com wrote: These latency spikes are really annoying. Will the business app engine edition use the same bigtable cluster as we do? On May 25, 9:03 pm, James jamesk...@gmail.com wrote: +1 On May 25, 10:13 am, Jody Belka j...@jj79.org wrote: From reading this list, there appear to have been quite few incidents lately where multiple sites start reporting errors, with the status page showing ... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine Blog:http://googleappengine.blogspot.com Twitter:http://twitter.com/app_engine Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Unscheduled downtime/problems lately
Yes, app id noticeorange. 3500+ latency related errors today in 2 major and ~10 minor spikes. Most days since the 25th have had 5k+ errors, with several 20k+. It's usually related to ms/req spikes coinciding with the high latency spikes on the query latency graph. -C On Jun 1, 3:34 pm, Ikai L (Google) ika...@google.com wrote: The post-mortem should address the outage as well as the latency leading up to the outage. We're keeping an eye on latency now. Are you experience latency in your application? On Tue, Jun 1, 2010 at 3:31 PM, Millisecond millisec...@gmail.com wrote: Thanks Ikai, I look forward to reading the post-mortem. The last one was indeed very good. Will the post-mortem address the dramatically increased latency since the 25th? It's ongoing and has already caused tens of thousands of errors on our site alone. As far as I can tell, there's been no official communication regarding that. And it's still continuing with 1500ms+ for a lot of today: http://code.google.com/status/appengine/detail/datastore/2010/06/01#a... -C On Jun 1, 3:16 pm, Ikai L (Google) ika...@google.com wrote: I just wanted to let you guys know to hang in there. We're working on a post-mortem describing what went wrong and steps we are taking to prevent this from happening again. I don't have an ETA for this yet, but we've done this before ( http://groups.google.com/group/google-appengine/browse_thread/thread/...) and are committed to as much transparency as we can with regards to these issues with you guys. On Tue, Jun 1, 2010 at 2:59 PM, Mark mar...@gmail.com wrote: +1 as worried, I'm investing time in app engine and this is quite discouraging to say the least! How often does this happen? (I don't have a live app yet), but should I abandon app engine now before getting myself in trouble? On May 27, 1:32 pm, Jody Belka j...@jj79.org wrote: And what do you know, no-one from Google has made any response whatsoever to this thread. And yet we know they are around, they occasionally do write messages after all. Very disappointing. On 25 May 2010 21:51, Waleed Abdulla wal...@ninua.com wrote: It's definitely worrisome. My app is down at the moment, my task queue backlog is at 4 days, and I'm very disappointed. On Tue, May 25, 2010 at 12:46 PM, Flips p...@script-network.com wrote: These latency spikes are really annoying. Will the business app engine edition use the same bigtable cluster as we do? On May 25, 9:03 pm, James jamesk...@gmail.com wrote: +1 On May 25, 10:13 am, Jody Belka j...@jj79.org wrote: From reading this list, there appear to have been quite few incidents lately where multiple sites start reporting errors, with the status page showing ... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine Blog:http://googleappengine.blogspot.com Twitter:http://twitter.com/app_engine Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group
[google-appengine] Re: Unscheduled downtime/problems lately
Since I posted that my dashboard has been wonky and the errors counting _backwards_. So, there are only ~2k errors at the moment, but there definitely were ~3500 when I first posted. Really don't think I'm going crazy but I do feel like it with the dashboard at times... -C On Jun 1, 3:41 pm, Millisecond millisec...@gmail.com wrote: Yes, app id noticeorange. 3500+ latency related errors today in 2 major and ~10 minor spikes. Most days since the 25th have had 5k+ errors, with several 20k+. It's usually related to ms/req spikes coinciding with the high latency spikes on the query latency graph. -C On Jun 1, 3:34 pm, Ikai L (Google) ika...@google.com wrote: The post-mortem should address the outage as well as the latency leading up to the outage. We're keeping an eye on latency now. Are you experience latency in your application? On Tue, Jun 1, 2010 at 3:31 PM, Millisecond millisec...@gmail.com wrote: Thanks Ikai, I look forward to reading the post-mortem. The last one was indeed very good. Will the post-mortem address the dramatically increased latency since the 25th? It's ongoing and has already caused tens of thousands of errors on our site alone. As far as I can tell, there's been no official communication regarding that. And it's still continuing with 1500ms+ for a lot of today: http://code.google.com/status/appengine/detail/datastore/2010/06/01#a... -C On Jun 1, 3:16 pm, Ikai L (Google) ika...@google.com wrote: I just wanted to let you guys know to hang in there. We're working on a post-mortem describing what went wrong and steps we are taking to prevent this from happening again. I don't have an ETA for this yet, but we've done this before ( http://groups.google.com/group/google-appengine/browse_thread/thread/...) and are committed to as much transparency as we can with regards to these issues with you guys. On Tue, Jun 1, 2010 at 2:59 PM, Mark mar...@gmail.com wrote: +1 as worried, I'm investing time in app engine and this is quite discouraging to say the least! How often does this happen? (I don't have a live app yet), but should I abandon app engine now before getting myself in trouble? On May 27, 1:32 pm, Jody Belka j...@jj79.org wrote: And what do you know, no-one from Google has made any response whatsoever to this thread. And yet we know they are around, they occasionally do write messages after all. Very disappointing. On 25 May 2010 21:51, Waleed Abdulla wal...@ninua.com wrote: It's definitely worrisome. My app is down at the moment, my task queue backlog is at 4 days, and I'm very disappointed. On Tue, May 25, 2010 at 12:46 PM, Flips p...@script-network.com wrote: These latency spikes are really annoying. Will the business app engine edition use the same bigtable cluster as we do? On May 25, 9:03 pm, James jamesk...@gmail.com wrote: +1 On May 25, 10:13 am, Jody Belka j...@jj79.org wrote: From reading this list, there appear to have been quite few incidents lately where multiple sites start reporting errors, with the status page showing ... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http
[google-appengine] Query Latency, etc
The query latency has been red on and off all morning, requests are returning 500's and not counting as errors (saw another thread I was going to reference about other people seeing this as well, but can't find it now), but no alerts/status updates: http://code.google.com/status/appengine/detail/datastore/2010/05/28#ae-trust-detail-datastore-query-latency The only day marked as 'service disruption' was May 25th, but we've thrown 75,000+ errors in the past week due to latency, not including May 25th. Billing is missing and no word from Google on whether it's intentional: http://groups.google.com/group/google-appengine/browse_thread/thread/beaba487d51f6339/6f48263f24ca553d?lnk=gstq=billing#6f48263f24ca553d Would be great to hear about how this is all just a temporary thing, what the problem is, and what Google's plan is to fix it. But, at the moment, this feels like normal operation of AppEngine and nobody at Google is concerned which really has me concerned. -C -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Query Latency, etc
We're going the wrong direction, with regard to available information, query latency graph is now broken again today. Going back in time, the only day that has a functioning query latency graph in the last 10 (!) is May 26th. http://code.google.com/status/appengine/detail/datastore/2010/05/26#ae-trust-detail-datastore-query-latency -C On May 28, 9:39 am, Bay troels...@gmail.com wrote: There just needs to be more information. This is a service people pay money for. Where are the explanations of todays errors? Adding an issue at the tracker almost always goes unnoticed - writing in these (three different) forum groups almost always goes unnoticed. Can we expect these 500 error that have been coming and going for months to be fixed one day, or are we supposed to pull our apps and rewrite them into something that is less scalable but -actually- working? This is not supposed to be hard. Everybody are confined to java and python - nothing more, errors can be contained. No filesystem, everything in a database which prohibit any operation that look like join and has indexes for everything that can be fetched except if one actually knows the unique key. With this in mind, things are simple. Its not supposed to be that hard - if you can't do what companies that allow SQL and multiple languages can, then please buy a hosting company and get some best practices from there. You've got the money - you've got our money, as we pay for the service. If you need more money to make it work upp the prices a little, because -random- 500 errors are unacceptable for everybody. A few answers and a few pieces of information would be nice. Thank you. On May 28, 6:04 pm, Millisecond millisec...@gmail.com wrote: The query latency has been red on and off all morning, requests are returning 500's and not counting as errors (saw another thread I was going to reference about other people seeing this as well, but can't find it now), but no alerts/status updates:http://code.google.com/status/appengine/detail/datastore/2010/05/28#a... The only day marked as 'service disruption' was May 25th, but we've thrown 75,000+ errors in the past week due to latency, not including May 25th. Billing is missing and no word from Google on whether it's intentional:http://groups.google.com/group/google-appengine/browse_thread/thread/... Would be great to hear about how this is all just a temporary thing, what the problem is, and what Google's plan is to fix it. But, at the moment, this feels like normal operation of AppEngine and nobody at Google is concerned which really has me concerned. -C -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Billing missing?
We're seeing (and wondering) the same things On May 26, 3:26 pm, jay kyburz@gmail.com wrote: Hey all, I just noticed that the billing info for my app is missing and it's billing status is Free. Is this just some compensation for the downtime or is something wrong with my app? Jay. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Request was aborted after waiting too long
Odd, our app had gotten to a point where it was working pretty well until this. Since maintenance yesterday it's been all over the place, not even sure how many errors it's throwing, the counters keep resetting. Sometimes on their own, and once on deploying a new version with 1.3.3.1. But about 500 errors in the last 15 minutes. App id: noticeorange -C On May 6, 12:03 pm, bsb b...@pearcomp.com wrote: And for me, it's magically working flawlessly again today. Google, did you do something? That would be good. If you didn't, it means the service is extremely unreliable: That'd be really bad... Ben On May 6, 8:02 pm, Joshua Smith joshuaesm...@charter.net wrote: +1 Here is my error: 05-06 11:00AM 46.656 / 500 10544ms 0cpu_ms 0kb Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7,gzip(gfe),gzip(gfe) 216.230.77.82 - - [06/May/2010:11:00:57 -0700] GET / HTTP/1.1 500 0 - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7,gzip(gfe),gzip(gfe) fix3dpdf.v-central.com W05-06 11:00AM 57.200 Request was aborted after waiting too long to attempt to service your request. This may happen sporadically when the App Engine serving cluster is under unexpectedly high or uneven load. If you see this message frequently, please contact the App Engine team. On May 6, 2010, at 12:53 PM, bFlood wrote: my task queue is filled with these today. each task had been paired back to 20 small entity puts. On May 6, 11:42 am, mkmanning michaell...@gmail.com wrote: One of my apps started getting this yesterday and the problem continued all day, and is still there this a.m. This is happening any time you visit the app. The error log for / shows: Request was aborted after waiting too long to attempt to service your request. This may happen sporadically when the App Engine serving cluster is under unexpectedly high or uneven load. If you see this message frequently, please contact the App Engine team. This started after my app had been up and running for months, with no changes; worked the day before, then dead. Nothing in the system status shows what the cause may be or what (if anything) is being done about this. On May 5, 12:56 pm, bsb b...@pearcomp.com wrote: I also get the same error with cron jobs in another app of mine. This has only started happening recently. Can someone @ Google please take a look at this? On May 5, 10:18 am, Benjamin Schuster-Böckler b...@pearcomp.com wrote: I'm getting loads of these. It's ok with the deferred tasks, as they are rescheduled, but the cron jobs (to be run every 10min) just don't run if this happens, causing unpredictable behaviour of my app. Effectually, the cron only runs eery few hours, because most of the time, this error occurs. What can I do to fix this? Do I really need to have a dummy task to constantly hammer instances so the app stays loaded? Thanks in advance, Ben -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit
[appengine-java] Wildcard root url-pattern and 1.3.3
Trying to deploy my app (noticeorange) with the 1.3.3 runtime is giving me some hiccups with my main servlet mapping. I map all requests into this one servlet and then figure stuff out from there. Has worked fine until just now when I tried to upgrade my dev environment to 1.3.3. Only / is mapping to the servlet with either of these two entries in web.xml: servlet-mapping servlet-namenotice/servlet-name url-pattern//url-pattern /servlet-mapping OR: servlet-mapping servlet-namenotice/servlet-name url-pattern/*/url-pattern /servlet-mapping Is there some other way to do this now? Is it failing for other people or just me? Thanks, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Read-only mode exceptions
I'm trying to make my app robust to survive through these read-only modes that we seem to be in fairly often lately. All calls to the cache were surrounded with a try-catch for MemcacheServiceException as spelled out here (even though I don't use a StrictErrorHandler, during the last maintenance it was still failing): http://code.google.com/appengine/docs/java/howto/maintenance.html But now during this maintenance, I'm receiving a GCacheException on all cache.put() calls causing my app to still be down. Is that expected? Are there other exceptions I may need to catch? If it's expected, an update to that page would be nice. Thanks, -C -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Image API Transparency cut
Hey all, I'm trying to take a PNG that has all pixels filled with nothing transparent, and cut out part of that image along a predefined mask. So I would end up with an image that is part transparent and part filled. Is this possible with the Image API? The only way I'm able to punch out pixels of an originating image appears to be with a full color value not a transparent pixel. Thanks, -Casey -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Request was aborted after waiting too long to attempt to service your request.
I'm getting this log message a lot in my application: Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please see http://code.google.com/appengine/docs/quotas.html for more details. I do have excessive latency in that I run non-trivial tasks in the TaskQueue handlers interacting with external systems averaging about 10 seconds per handler with a couple per cycle pushing the 30 second limit, but on the Dashboard my requests/second never seem to exceed ~1.5. I have billing enabled and my understanding is that gives me 500 simultaneous requests. Which if all of my TaskQueues were executing at once I don't think I'd reach, although it might get close. But wouldn't that show on the dashboard? Is there anything else that could be causing this? I see the DataStore has been in Anomoly for the past few days, but nobody seems worried about it and assume roughly this latency will be normal. Thanks, -C -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Request was aborted after waiting too long to attempt to service your request.
Ah, you're right, only this bit stuck in my head quotas with billing enabled can accommodate around 500 requests per second from http://code.google.com/appengine/docs/quotas.html Thanks for the pointer to the math working that number back from the 30 request / s limit. Seems a little optimistic though that a request could be handled in 75ms (or faster with the -implied- math on the quota page) with the average query latency these days being way over that itself. Here's hoping those come down soon. Also thanks for the pointer to upping the limit. -C On Jan 22, 9:30 am, Don Schwarz schwa...@google.com wrote: On Fri, Jan 22, 2010 at 10:55 AM, Millisecond millisec...@gmail.com wrote: I'm getting this log message a lot in my application: Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please seehttp://code.google.com/appengine/docs/quotas.htmlformore details. I do have excessive latency in that I run non-trivial tasks in the TaskQueue handlers interacting with external systems averaging about 10 seconds per handler with a couple per cycle pushing the 30 second limit, but on the Dashboard my requests/second never seem to exceed ~1.5. I have billing enabled and my understanding is that gives me 500 simultaneous requests. That's not accurate. Where did you hear this? Here's the documentation we have on this: http://code.google.com/appengine/docs/java/runtime.htmlsays: *** An application can process around 30 active dynamic requests simultaneously. This means that an application whose average server-side request processing time is 75 milliseconds can serve up to (1000 ms/second / 75 ms/request) * 30 = 400 requests/second without incurring any additional latency. Applications that are heavily CPU-bound may incur some additional latency in long-running requests in order to make room for other apps sharing the same servers. Requests for static files are not affected by this limit. *If your application is making efficient use of resources and traffic is about to exceed your expected maximum queries per second, you can request that the simultaneous dynamic request limit be raisedhttp://code.google.com/appengine/kb/billing.html#cpu. App Engine can scale far beyond 30 simultaneous requests; this default limit is in place to prevent a poorly performing or malicious app from hoarding resources.* I suggest that you calculate the maximum number of simultaneous requests that you need to serve and fill out the above form. Thanks, Don -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] TaskQueue and null host name
Hello, I'm just testing out super basic TaskQueue stuff in the dev environment and running into an exception I just can't seem to shake. Code: Queue queue = QueueFactory.getQueue(mail-queue); queue.add(url(/api/RunMonitor)); Exception: java.lang.IllegalArgumentException: Host name may not be null at org.apache.commons.httpclient.HttpHost.init(Unknown Source) at org.apache.commons.httpclient.HttpHost.init(Unknown Source) at org.apache.commons.httpclient.HttpMethodBase.setURI(Unknown Source) at org.apache.commons.httpclient.HttpMethodBase.init(Unknown Source) at org.apache.commons.httpclient.methods.ExpectContinueMethod.init (Unknown Source) at org.apache.commons.httpclient.methods.EntityEnclosingMethod.init (Unknown Source) at org.apache.commons.httpclient.methods.PostMethod.init(Unknown Source) at com.google.appengine.api.urlfetch.dev.LocalURLFetchService $RedirectablePostMethod.init(LocalURLFetchService.java:348) at com.google.appengine.api.urlfetch.dev.LocalURLFetchService $4.buildMethod(LocalURLFetchService.java:95) I've tried this about half a dozen ways, starting with different -- address parameters thinking something wasn't picking up right on the default 0.0.0.0 argument and on 1.2.8 and 1.3 versions. The same code works fine if I publish to to production and run it there. My environment: -OSX -IntelliJ w/plugin -JDK1.6 Thanks! -Millisecond -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: TaskQueue and null host name
It's in a normal doGet method of a servlet. Re-enabled it to double-check that I was failing on the version in doGet (originally I had it in an init() method) and ran into something that may be the cause. If I request the servlet from http://localhost:8080/ it fails, the same servlet also responds to http://localhost:8080/Index and there it works fine. I can't see anything in my code making it fail from the bare URL, but there is a fair amount going on in that servlet. I was just doing it there for expediency while testing, can't see any good reason to enqueue something from / anyway so should be fine. Thanks, -C On Dec 15, 6:26 pm, Max Ross (Google) maxr+appeng...@google.com wrote: Where is your code that adds the task to the queue? Is it being run during server initializaiton, perhaps as part of a static block or a LoadOnStartup servlet? Thanks, Max On Tue, Dec 15, 2009 at 8:27 AM, Millisecond millisec...@gmail.com wrote: Hello, I'm just testing out super basic TaskQueue stuff in the dev environment and running into an exception I just can't seem to shake. Code: Queue queue = QueueFactory.getQueue(mail-queue); queue.add(url(/api/RunMonitor)); Exception: java.lang.IllegalArgumentException: Host name may not be null at org.apache.commons.httpclient.HttpHost.init(Unknown Source) at org.apache.commons.httpclient.HttpHost.init(Unknown Source) at org.apache.commons.httpclient.HttpMethodBase.setURI(Unknown Source) at org.apache.commons.httpclient.HttpMethodBase.init(Unknown Source) at org.apache.commons.httpclient.methods.ExpectContinueMethod.init (Unknown Source) at org.apache.commons.httpclient.methods.EntityEnclosingMethod.init (Unknown Source) at org.apache.commons.httpclient.methods.PostMethod.init(Unknown Source) at com.google.appengine.api.urlfetch.dev.LocalURLFetchService $RedirectablePostMethod.init(LocalURLFetchService.java:348) at com.google.appengine.api.urlfetch.dev.LocalURLFetchService $4.buildMethod(LocalURLFetchService.java:95) I've tried this about half a dozen ways, starting with different -- address parameters thinking something wasn't picking up right on the default 0.0.0.0 argument and on 1.2.8 and 1.3 versions. The same code works fine if I publish to to production and run it there. My environment: -OSX -IntelliJ w/plugin -JDK1.6 Thanks! -Millisecond -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
