[google-appengine] Entities not being returning in query result -- index bug?
I was alerted by one of my users that some entries they are associated with were not appearing to them. Indeed, when I query the datastore for these entities (with a single property filter), they are not returned. After writing them to the datastore, the index is updated and they are returned in the query. Perhaps technically my query is not guaranteed to return the entities as it is weakly consistent, but none of the entities were changed recently and usually any inconsistent results are resolved quite quickly. (it has been several hours now) So it seems like the index entries for this property on these entities were lost or damaged somehow. What to do? Wait and hope the index will be regenerated? I can write entities for this user to the datastore to regenerate the index...but doing it for all my users is not really an option. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/8c53a518-2e5b-456e-98d5-79d38993e973n%40googlegroups.com.
[google-appengine] Tasks failing silently with error 'Instance Unavailable'
Tasks have been failing silently in my taskqueue all day. No PermanentTaskFailures or other exceptions are being thrown -- nothing in my logs. The tasks are just retrying a number of times, with the reason for retries being 'Instance Unavailable'. Then they disappear. I received an email from AppEngine Support about a month ago about a bug they were planning to fix: > Currently, we have a bug in the system where tasks enqueued from a custom > domain (e.g. your_domain.com) are always routed to the default module > even if a non-default module is specified in the app's dispatch.yaml/xml > file. This bug will be fixed by February 22, 2016. However, this email did not mention the app I'm having issues with and the bug didn't seem relevant to my app. Were changes made to the TaskQueues today? Is anyone else experiencing similar problems? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f4af3c51-e9fb-436d-8c6d-f56a82669dfb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Experiencing latency issues. Can anyone confirm?
We've been experiencing latency issues over the last day. As of about 16 hours ago, latency started rising to 24s for the average request and it has now returned to about 4s, which is way too high. Earlier in the day we had many datastore timeouts, and I can see that this was an AppEngine datastore latency issue here : https://code.google.com/status/appengine/detail/datastore/2014/09/25#ae-trust-detail-datastore-get-latency But although the above page shows that datastore latency has returned to normal and we are no longer getting 500 error from timeouts, our request latency is very high. Here is our latency graph: http://i.imgur.com/obUyWaO.png That version was deployed 2 days ago. You can see that our usual latency is far below 4 seconds. Can anyone give insight or corraborate on this? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Appengine SDK 1.9.4 is slow to query the datastore
I upgraded from 1.9.2 to 1.9.4 and have noticed that datastore queries are taking substantially longer -- here's a comparison for loading the same page (not using NDB): before: instance.py:385] Request handled by Instancea05df4abeccece9cdfeec56ee573ea1627a0 : 1/8, total: 3 in 0.9100s after: instance.py:385] Request handled by Instanceb2f99bd20bb11df4a1e2fcf0a0585b5da31a : 1/8, total: 16 in 67.9088s The generated query was made more complex in 1.9.3, which is probably partially the reason: https://code.google.com/p/googleappengine/source/diff?spec=svn427old=410r=427format=unidiffpath=%2Ftrunk%2Fpython%2Fgoogle%2Fappengine%2Fdatastore%2Fdatastore_sqlite_stub.py generated query before: SELECT Entities.__path__, Entities.entity FROM dev~application!!EntitiesAS Entities INNER JOIN dev~application!!EntitiesByProperty AS ebp_0 ON Entities.__path__ = ebp_0.__path__ INNER JOIN dev~application!!EntitiesByProperty AS ebp_1 ON Entities.__path__ = ebp_1.__path__ INNER JOIN dev~application!!EntitiesByProperty AS ebp_2 ON Entities.__path__ = ebp_2.__path__ WHERE ebp_0.kind = :1 AND ebp_0.name = :2 AND ebp_0.value = :3 AND ebp_1.kind = :4 AND ebp_1.name = :5 AND ebp_1.value = :6 AND ebp_2.kind = :7 AND ebp_2.name = :8 ORDER BY ebp_2.value DESC, Entities.__path__ ASC generated query after: SELECT Entities.__path__, Entities.entity FROM dev~application!!EntitiesAS Entities INNER JOIN (SELECT kind, name, value, __path__ FROM dev~application!!EntitiesByProperty ORDER BY kind, name, value, __path__)AS ebp_0 ON Entities.__path__ = ebp_0.__path__ INNER JOIN (SELECT kind, name, value,__path__ FROM dev~application!!EntitiesByProperty ORDER BY kind, name, value, __path__)AS ebp_1 ON Entities.__path__ = ebp_1.__path__ INNER JOIN (SELECT kind, name, value,__path__ FROM dev~application!!EntitiesByProperty ORDER BY kind, name, value, __path__)AS ebp_2 ON Entities.__path__ = ebp_2.__path__ WHERE ebp_0.kind = :1 AND ebp_0.name = :2AND ebp_0 .value = :3 AND ebp_1.kind = :4 AND ebp_1.name = :5 AND ebp_1.value = :6AND ebp_2 .kind = :7 AND ebp_2.name = :8 ORDER BY ebp_2.value DESC, Entities.__path__ ASC; Was the change made in 1.9.3 really necessary? This has made developing locally unusably slow for me. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Appengine SDK 1.9.4 is slow to query the datastore
Guess I should have searched the issues more carefully before posting: https://code.google.com/p/googleappengine/issues/detail?id=10845q=stars%3A2sort=-idcolspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Safety of using threading.currentThread on GoogleAppEngine (Python)
Ok, that makes sense, and if you haven't seen any memory leaks with it, it sounds like threading.local is probably the right way to go. Thanks for the discussion, it's been very helpful! On Monday, June 17, 2013 3:56:12 PM UTC+1, timh wrote: Thread local is specific to a single request. I have not seen any evidence of memory leaks using it. (I have thread.local in heavy use on a number of projects). I think you are assuming there is a thread pool, and threads and therefore thread.local is re-used. I don't have any evidence to back up this claim but I don't think that is the case. I believe a new thread is being created for each request, hence anything thread local will be gc'd at some point. On Monday, June 17, 2013 6:43:06 PM UTC+8, bmurr wrote: You're right, looks like threading.local also fixes the concurrency problem. But surely if Python can't tell when an alien thread gets joined, then the variables used in threading.local will never go out of scope and get garbage collected? Which leads to a memory leak, like before? On Saturday, June 15, 2013 1:47:25 AM UTC+1, timh wrote: Why do you need to call currentThread() to create thread local storage? You can just create threadlocal storage. On Friday, June 14, 2013 8:15:25 PM UTC+8, bmurr wrote: After switching threadsafe to true in my app.yaml, I had some concurrency issues (detailed in this SO question, here : http://stackoverflow.com/questions/16966365/in-python-does-dummythread-uniquely-identify-the-current-thread ). Using threading.currentThread() with a thread-local storage approach got rid of my immediate problem, but raised an issue that I'm not sure about : The threads AppEngine creates are outside of threading.py's control, so Python creates these immortal DummyThread objects to represent them when you call threading.currentThread. My thinking on this was that even if DummyThread objects never die and thus create a memory leak, AppEngine probably doesn't create enough threads and our application is updated frequently enough (once per week, on average), that the number of DummyThreads will be unlikely to grow beyond a certain point and thus any memory leak resulting from DummyThreads will be insignificant. I'm really not sure whether I'm in for a world of hurt eventually here or whether I should be ok. I'd love to hear some opinions/comments on why this is a good/bad idea, or maybe some suggestions of an alternate approach. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Safety of using threading.currentThread on GoogleAppEngine (Python)
You're right, looks like threading.local also fixes the concurrency problem. But surely if Python can't tell when an alien thread gets joined, then the variables used in threading.local will never go out of scope and get garbage collected? Which leads to a memory leak, like before? On Saturday, June 15, 2013 1:47:25 AM UTC+1, timh wrote: Why do you need to call currentThread() to create thread local storage? You can just create threadlocal storage. On Friday, June 14, 2013 8:15:25 PM UTC+8, bmurr wrote: After switching threadsafe to true in my app.yaml, I had some concurrency issues (detailed in this SO question, here : http://stackoverflow.com/questions/16966365/in-python-does-dummythread-uniquely-identify-the-current-thread ). Using threading.currentThread() with a thread-local storage approach got rid of my immediate problem, but raised an issue that I'm not sure about : The threads AppEngine creates are outside of threading.py's control, so Python creates these immortal DummyThread objects to represent them when you call threading.currentThread. My thinking on this was that even if DummyThread objects never die and thus create a memory leak, AppEngine probably doesn't create enough threads and our application is updated frequently enough (once per week, on average), that the number of DummyThreads will be unlikely to grow beyond a certain point and thus any memory leak resulting from DummyThreads will be insignificant. I'm really not sure whether I'm in for a world of hurt eventually here or whether I should be ok. I'd love to hear some opinions/comments on why this is a good/bad idea, or maybe some suggestions of an alternate approach. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Safety of using threading.currentThread on GoogleAppEngine (Python)
After switching threadsafe to true in my app.yaml, I had some concurrency issues (detailed in this SO question, here :http://stackoverflow.com/questions/16966365/in-python-does-dummythread-uniquely-identify-the-current-thread). Using threading.currentThread() with a thread-local storage approach got rid of my immediate problem, but raised an issue that I'm not sure about : The threads AppEngine creates are outside of threading.py's control, so Python creates these immortal DummyThread objects to represent them when you call threading.currentThread. My thinking on this was that even if DummyThread objects never die and thus create a memory leak, AppEngine probably doesn't create enough threads and our application is updated frequently enough (once per week, on average), that the number of DummyThreads will be unlikely to grow beyond a certain point and thus any memory leak resulting from DummyThreads will be insignificant. I'm really not sure whether I'm in for a world of hurt eventually here or whether I should be ok. I'd love to hear some opinions/comments on why this is a good/bad idea, or maybe some suggestions of an alternate approach. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Blobstore filename created in MapReduce job too long to create BlobKey
Well, I think I have it solved.
I was using an older version of the mapreduce library, which uses
mapreduce.lib.files to interact with the blobstore.
The newer version of mapreduce uses google.appengine.api.files instead,
which doesn't cause this problem.
These two libraries seem pretty similar -- so I'm not sure what precisely
was causing the issue.
On Saturday, March 2, 2013 4:25:17 AM UTC, Jamie Niemasik wrote:
I've been receiving intermittent errors from MapReduce jobs. I'm running
Python 2.7.
The specific error is BadValueError: name must be under 500 bytes which
is raised when calling datastore.Key.from_path() within
blobstore.get_blob_key(); the filename being provided is way too long to
make a key from.
This all occurs within the code in the mapreduce package… nothing in my
code seems to affect it.
Some of the filenames are 288 bytes long, while some are 992. The M/R spec
name and id in each case is nearly the same and is very short; I don't see
where this variance comes from.
The sequence of events is this:
mapreduce.output_writers.init_job() creates a reasonable, short filename
and passes it to files.blobstore.create()
create() calls files.file._create('blobstore', …, filename)
_create() sets up an rpc with that filename and calls _make_call('Create',
...)
And that call sometimes returns a filename that's 288 bytes, sometimes
992. I have no idea why or how to work around this — any help would be
appreciated.
Thanks,
Jamie
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Blobstore filename created in MapReduce job too long to create BlobKey
I've also been having this problem.
Did you manage to find any workarounds?
On Saturday, March 2, 2013 4:25:17 AM UTC, Jamie Niemasik wrote:
I've been receiving intermittent errors from MapReduce jobs. I'm running
Python 2.7.
The specific error is BadValueError: name must be under 500 bytes which
is raised when calling datastore.Key.from_path() within
blobstore.get_blob_key(); the filename being provided is way too long to
make a key from.
This all occurs within the code in the mapreduce package… nothing in my
code seems to affect it.
Some of the filenames are 288 bytes long, while some are 992. The M/R spec
name and id in each case is nearly the same and is very short; I don't see
where this variance comes from.
The sequence of events is this:
mapreduce.output_writers.init_job() creates a reasonable, short filename
and passes it to files.blobstore.create()
create() calls files.file._create('blobstore', …, filename)
_create() sets up an rpc with that filename and calls _make_call('Create',
...)
And that call sometimes returns a filename that's 288 bytes, sometimes
992. I have no idea why or how to work around this — any help would be
appreciated.
Thanks,
Jamie
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
