[google-appengine] Rendering Query object values takes longer time
I have a query object which contains 3 entities each contains 51 attributes. While rendering the page with template render function, where the query object is being iterated, it takes more than 25 secs to render the page. If I remove the iteration of the query object, it takes less than a sec. Is this the known issue. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Local datastore import is too slow
I am also havin the same problem. Could you please clarify me if you got the answer On Nov 20, 5:35 am, "Jyoti Shete-Javadekar" <[EMAIL PROTECTED]> wrote: > Hi, > I am trying to load my development datastore using the bulk loader script. > However the import is very slow. I had to kill the import process since it > was not completed even after 12 hours. I have about 13K rows in the CSV > file. One data model entity is about 300 bytes. 10 entities are imported at > a time. The model has two unicode attributes, two unicode list attributes , > one url and one long attribute. I use unicode.split to populate list > attributes. I am running the bulk loader in a virtual machine having 512MB > memory. During the import about 91% memory is utilized. I have not specified > any custom index in index.yaml. > > Could someone please tell me why the import is so slow? What optimization > should I do to improve the performance? I have not yet loaded the data at > appspot. Above observations are for my local development server. Are there > any performance numbers available for the bulk loader script? > > Thanks in advance, > Jyoti --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: bulkload issues
On Fri, Nov 21, 2008 at 1:26 AM, [EMAIL PROTECTED] < [EMAIL PROTECTED]> wrote: > > local store is a dummy datastore and It could be slow... > > how are you managing the 10MBytes of datastore limitation with your > 500k entities ? This is a commercial proposal. Still it is in analysis stage. I have tried in local datastore as a prototype of the application for demo. Please clarify me, I have three possibilities for uploading data to the AppEngine datastore. These are: 1. For all 500K entities, I have the Instance list module. [ I created it from the cvs file using my own python script, so it doesn't raise any error or exceptions, already testing is completed] If I use this, will it be slow? 2. I have the csv file exported from MySql db. For this If I use bulkload utility, it checks for empty values, and raises import Failure error.Is any ways are available? 3. Using Fixture support, I can recreate the DataModels. Is it really necessary? Rescue me from the above three cylones! Can I use the same instance list modules for creating the datastore in AppEngine during application upload? Of else, Do I need to create the instance Model classes using Fixture kind of stuff? I created this instance list from > > On Nov 20, 9:11 am, Selva <[EMAIL PROTECTED]> wrote: > > I have a requirement of loading around 50 entities to a data > > model. Each entity contains max of 20 attributes. > > But the no.of attributes for each entity is varying(i.e, some records > > will not have all the field values). > > When I am trying to use bulkload utility to load the data. > > > > Since the absence of some attribute values, import failure error Iam > > getting. > > If I use empty field bw two commas, except string type data, converter > > method raises exception. > > > > Could any body help me to overcome this issue? Any other options to > > upload the huge amount of data values > > to the datastore in very short time. I used the straight forward > > instance creation mechanism, but after a period of 10 hr also it has > > not completed. > > > > FYI: The application is not yet been uploaded in App Engine. All the > > data store have been created in in the local > >machine. > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Working with a proxy for data storage and URL fetch
On Tue, Dec 2, 2008 at 2:52 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]>wrote: > > Have you checked out this http://code.google.com/p/approcket/ > > It allows you to setup replication from MySQL to AppEngine - I guess > that's what you are trying to achieve. > Any ways, It won't support for storing huge amount of data in the > datastore right? > > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Working with a proxy for data storage and URL fetch
On Wed, Dec 3, 2008 at 2:30 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]>wrote: > > It shoudn't be a problem, replication will just take longer. > > The replication is of course broken down into many small requests, in > order to avoid AppEngine request timeouts and running over high-CPU > quota. > > As far as I know, currently there isn't really any faster way to load > data into AppEngine datastore. > > Yes, I did a little analysis and gave a try with app rocket. You are > correct. > > > > > > Any ways, It won't support for storing huge amount of data in the > > > datastore right? > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: GAE will hurt Linux unless...
I don't think so, b'coz linux and its usage in app dev domain is a vast area and it cannot be beaten degraded by GAE. May be in games domain and the pkgs like photoshop could besome exceptions, but as a dev. platform point of view *nix operating systems will have its own image. I have worked in linux and now working in windows platform, Still Iam starving for the linux environment support for my regular activities. On Thu, Dec 4, 2008 at 3:45 AM, yejun <[EMAIL PROTECTED]> wrote: > > There's no such thing as directly using linux. > > On Dec 3, 5:09 pm, "Amir Michail" <[EMAIL PROTECTED]> wrote: > > On Wed, Dec 3, 2008 at 5:06 PM, yejun <[EMAIL PROTECTED]> wrote: > > > > > GAE runs on linux, so I couldn't see any reason linux will be hurt. > > > > GAE will cut down the number of people familiar with and directly > > using Linux even if it ends up increasing the number of Linux web > > servers. > > > > Amir > > > > > > > > > > > > > On Dec 3, 3:23 pm, Amir Michail <[EMAIL PROTECTED]> wrote: > > >> Hi, > > > > >> I suspect that the Google App Engine (and cloud computing more > > >> generally) will have the unintended effect of significantly reducing > > >> usage of Linux among web developers. > > > > >> The solution of course is for Google to release a user-friendly and > > >> slick Google OS built on top of Linux that makes the web the main > > >> source of apps. > > > > >> But even that's not enough since many people (even web devs) play > > >> games on Windows... and linux cannot compete at all in that regard. > > >> One could argue that the shift towards game consoles is making Windows > > >> less important for games, but that will take time... > > > > >> Amir > > > > --http://b4utweet.comhttp://chatbotgame.comhttp://numbrosia.comhttp:// > twitter.com/amichail > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
