[google-appengine] Re: Possible problem with Datastore
Hi Ankur,
The logic seems fine to me. Could you please have a counter for fetched
entries outside the try block to check if #fetched and #processed are same?
Thanks,
On Wed, Aug 12, 2009 at 10:46 PM, Ankur ankj...@gmail.com wrote:
Hello All,
I am facing some weird problem while doing some processing through
AppEngine's RemoteAPI. Here is the scenario:
* I have more 600k entries in a table in datastore. I am using a
sharded counter which gave me this count information.
* I am trying to iterate over ALL entries in my datastore using the
code snippet given below.
* What's happening is that this code is not reading ALL 600k entries
from the datastore. The loop finishes after 125k iterations! It should
have processed all 600k entries. But, the loop is finishing early. I
think the issue is with '__key__' field of AppEngine's Datastore.
Can any help me in debugging this problem?
Thanks,
Ankur
def download_data(my_datastore_table):
KIND = my_datastore_table
batchsize = 200
cnt = 0
num = 0
results = KIND.all().order('__key__').fetch(batchsize)
while results:
num_fetched = len(results)
last_key = results[-1].key()
try:
doSomethingWithResultsresults)
cnt = cnt + num_fetched
print %s records processed % cnt
except:
traceback.print_exc()
results = KIND.all().filter('__key__ ', last_key).order
('__key__').fetch(batchsize)
--
-Jyoti
www.videonym.com
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: how to add data content kept in two csv files and store them in a new entry
If you have unique identifiers in both the csv files then you can insert entities using the data from one csv and then update those entities using the data from other csv file. If you don't have any unique identifier but all rows in the csv files have one on one association and are in the same order then you could create a key using the line number while inserting the data from one csv file. Then update entities using another csv file. On Thu, May 14, 2009 at 1:43 AM, Nikhar nikhar...@gmail.com wrote: how to add data content kept in two csv files and store them in a new entry -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: RSS Language Translation
Nice app :) On Sun, Mar 29, 2009 at 12:30 PM, RSS Language Translation davide.rogn...@gmail.com wrote: Public domain (P) 2009 Davide Rognoni - xml2jsonp - Google AJAX Language API http://riagallery.appspot.com/rss-language-translation -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Django template
use ifequal
{% ifequal AccountData.AccountType Current %}
...{% endifequal %}
please rever to
http://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs for
more details
On Fri, Mar 20, 2009 at 1:12 PM, Ronn Ross ronn.r...@gmail.com wrote:
I'm trying to use a conditional statement in django template. Like so:{%
if AccountData.AccountType == Current %}
bAccount is current/b
{% endif %}
GAE is throwing this error:
TemplateSyntaxError: 'if' statement improperly formatted
{% if (AccountData.AccountType == Current) %}
Can someone help?
--
-Jyoti
www.videonym.com
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: hello
Hi Venky, I am assuming that your local server works fine without -a option. I can't think of any reason why adding -a option results in an error. You could try following things though: 1. Use --address 0.0.0.0 instead of -a0.0.0.0 2. Check if you have the latest version of the app engine sdk. On Mon, Mar 16, 2009 at 10:20 PM, venkat rambotla venkatrambo...@gmail.comwrote: Hi Jyothi, Thanks for ur suggestion... I am facing a problem when i am running ur suggested command from command prompt its saying 'dev_appserver.py is not a internal or external command,operable program or batch file.. please help Thanking You, Regards, Venky On Thu, Mar 12, 2009 at 10:13 PM, Jyoti Shete-Javadekar jyoti.javade...@gmail.com wrote: Hi Venky, Use -a0.0.0.0 option while starting your dev server. e.g. python dev_appserver.py -a0.0.0.0 your_app_name Once you start the server like this, you will be able to access your app using the IP address of the machine where dev server is running. http:// IP:8080 On Thu, Mar 12, 2009 at 2:27 AM, Venky venkatrambo...@gmail.com wrote: Hello I created a python application in ecilpse targeting app engine through pydev my application name is Helloworld its runs fine on my local browser http://localhost:8080 now how should i make my application access to other systems,What should be the url from other systems to access the application -- -Jyoti www.videonym.com -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: hello
Hi Venky, Use -a0.0.0.0 option while starting your dev server. e.g. python dev_appserver.py -a0.0.0.0 your_app_name Once you start the server like this, you will be able to access your app using the IP address of the machine where dev server is running. http:// IP:8080 On Thu, Mar 12, 2009 at 2:27 AM, Venky venkatrambo...@gmail.com wrote: Hello I created a python application in ecilpse targeting app engine through pydev my application name is Helloworld its runs fine on my local browser http://localhost:8080 now how should i make my application access to other systems,What should be the url from other systems to access the application -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: how to use google Analytics
Google Analytics works just as well with GAE apps as it does with any other websites. On Thu, Mar 12, 2009 at 11:15 AM, Wooble geoffsp...@gmail.com wrote: On Mar 12, 8:09 am, Coonay fla...@gmail.com wrote: google Analytics is easy to used in standard web server such as apache or lighttp,but that is a different case in GAE, ANYBODY got the trick to use GA in GAE? I just put the urchin script in my templates and it works fine. What exactly have you tried that's not working? -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Problem while uploading application on Google app engine
Hi Sunita, There could be issues with your app.yaml. Could you please check if your have configured it correctly? http://code.google.com/appengine/docs/python/tools/configuration.html On Tue, Feb 24, 2009 at 3:28 AM, sunita khilare khilare...@gmail.comwrote: I am getting one error while uploading application on Google app engine.I have developed application using new Google app engine Version 1.1.9 ValueError: dictionary update sequence element #0 has length 1; 2 is required Could u please guide me whts the problem is?? -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: What is Data Storage Limit in GAE DataStore
please refer to http://code.google.com/appengine/articles/quotas.html On Thu, Jan 15, 2009 at 11:34 AM, anand innu.an...@gmail.com wrote: How much data can I store in GAE datastore per application? I have a requirement to store a large volume of data (a lot of pics) in datastore. Is there any limit? Thanks for your help. Regards, Anand -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] URL Redirection + Pipe = Bad Request
Hi, I host a video mashup at videonym.appspot.com. I have redirected www.videonym.com to my app. I have noticed a strange behavior when url parameters contain a | (pipe) I get an Error 400 on www.videonym.com if url parameters have | e.g. http://www.videonym.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20|%20Lecture%201'criterion='title' However same url parameters wok fine on videonym.appspot.com e.g. http://videonym.appspot.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20|%20Lecture%201'criterion='title' Other than the above discrepancy both videonym.appspot.com and www.videonym.com work similarly. Is there any known issue with | and url redirection? -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: URL Redirection + Pipe = Bad Request
Please copy the example urls in address to reproduce the behavior I described. gmail seems to be url-encoding | hence directly clicking on the links won't reproduce the discrepancy. http://www.videonym.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20|%20Lecture%201'criterion='title'http://www.videonym.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20%7C%20Lecture%201'criterion='title' http://videonym.appspot.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20|%20Lecture%201'criterion='title'http://videonym.appspot.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20%7C%20Lecture%201'criterion='title' I am going to explicitly encode all url parameters to resolve this issue. However I am curious to know why videonym.appspot.com and www.videonym.combehave differently. On Wed, Jan 7, 2009 at 5:28 PM, Jyoti Shete-Javadekar jyoti.javade...@gmail.com wrote: Hi, I host a video mashup at videonym.appspot.com. I have redirected www.videonym.com to my app. I have noticed a strange behavior when url parameters contain a | (pipe) I get an Error 400 on www.videonym.com if url parameters have | e.g. http://www.videonym.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20|%20Lecture%201'criterion='title'http://www.videonym.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20%7C%20Lecture%201'criterion='title' However same url parameters wok fine on videonym.appspot.com e.g. http://videonym.appspot.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20|%20Lecture%201'criterion='title'http://videonym.appspot.com/?vertical=Technologyuser='stanforduniversity'video='Darwins%20Legacy%20%7C%20Lecture%201'criterion='title' Other than the above discrepancy both videonym.appspot.com and www.videonym.com work similarly. Is there any known issue with | and url redirection? -- -Jyoti www.videonym.com -- -Jyoti www.videonym.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Any updates on deleting all entities in the production datastore?
Hi, I want to delete all my data models in production environment and upload new set of entities. I have gone through previous threads regarding this requirement. So far the solution seems to be selecting bunch of entities and then deleting them through script or through data viewer. I tried both the ways but they are not efficient. I am able to delete maximum 100-200 rows at a time. Is there any update on any appengine utility to delete all models in the production datastore at once? thanks, Jyoti --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Is there any way to do prefix match for list type data model attributes
Hi, I tried using do prefix match for StringListProperty of a data model. But it doesn't work. Could someone please tell me if there is any other way to do so. Thanks in advance, Jyoti On Tue, Nov 18, 2008 at 10:34 AM, Jyoti Shete-Javadekar [EMAIL PROTECTED] wrote: As per the GQL documentation : ( http://code.google.com/appengine/docs/datastore/queriesandindexes.html#Defining_Indexes_With_index_yaml ) Query filters do not have an explicit way to match just part of a string value, but you can fake a prefix match using inequality filters: db.GqlQuery(SELECT * FROM MyModel WHERE prop = :1 AND prop :2, abc, uabc + u\ufffd) This matches every MyModel entity with a string property prop that begins with the characters abc. The unicode string u\ufffd represents the largest possible Unicode character. When the property values are sorted in an index, the values that fall in this range are all of the values that begin with the given prefix. Is this type of prefix match supported for StringListProperty as well? I tried the above inequality filters for an attribute having StringListProperty, however it did not give correct results for all queries. Is there any other way to apply prefix filter on StringListProperty attributes? Thanks, Jyoti --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] question regarding data model prefix match
As per the GQL documentation : ( http://code.google.com/appengine/docs/datastore/queriesandindexes.html#Defining_Indexes_With_index_yaml ) Query filters do not have an explicit way to match just part of a string value, but you can fake a prefix match using inequality filters: db.GqlQuery(SELECT * FROM MyModel WHERE prop = :1 AND prop :2, abc, uabc + u\ufffd) This matches every MyModel entity with a string property prop that begins with the characters abc. The unicode string u\ufffd represents the largest possible Unicode character. When the property values are sorted in an index, the values that fall in this range are all of the values that begin with the given prefix. Is this type of prefix match supported for StringListProperty as well? I tried the above inequality filters for an attribute having StringListProperty, however it did not give correct results for all queries. Is there any other way to apply prefix filter on StringListProperty attributes? Thanks, Jyoti --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
