hi nick agreed on parallel api cpu calls and how this relates to time but those values jumped dramatically at some point last week with little or no change in userland code. is there an answer to this 3X-5X jump in db write API time?
thanks brian On Sep 11, 2:17 pm, "Nick Johnson (Google)" <nick.john...@google.com> wrote: > Hi Peter, > > On Fri, Sep 11, 2009 at 2:18 AM, Peter Liu <tinyee...@gmail.com> wrote: > > > I see similar results. The total api cpu time is much higher then > > actual time spent on the actual call. I use a profiling api delegate > > that logs every api call duration, and the total api cpu reported on > > admin console is usually at least 2 times higher. > > This is because an API call may involve multiple machines. A batch put, for > example, can include operations on multiple tabletservers, which are all > executed simultaneously. > > -Nick Johnson > > > > > > > > > On Sep 9, 2:05 pm, Lec <lec...@gmail.com> wrote: > > > I cannot confirm what it was like previous to 1.2.5 release, but I > > > think I am seeing something similar. Although with task queues my > > > workers do data processing then store the data in datastore, i am NOT > > > seeingcpuproblems with workers. I only see problems with the > > > datastore in the main threads where i do not use workers. Did anyone > > > have this kind of experience? > > > > L > > > > On Sep 9, 3:54 am, herbie <4whi...@o2.co.uk> wrote: > > > > > It seems I'm not the only one. I was starting to think I was imagining > > > > it! Thanks for supporting this thread. > > > > Would anyone fro Google like to comment? > > > > > On Sep 8, 11:01 pm, Robert Kluin <robert.kl...@gmail.com> wrote: > > > > > > I have some code that is now using moreCPUas well. > > > > > > Previously: > > > > > 1530cpu_ms 616api_cpu_ms > > > > > 1404cpu_ms 995api_cpu_ms > > > > > 1104cpu_ms 695api_cpu_ms > > > > > > Now: > > > > > 4619cpu_ms 4133api_cpu_ms > > > > > 4619cpu_ms 4133api_cpu_ms (yes, it is exactly the same) > > > > > > That is unchanged code. Same exact data. > > > > > > Robert > > > > > > On Tue, Sep 8, 2009 at 3:23 PM, bFlood <bflood...@gmail.com> wrote: > > > > > > > I just ran a batch delete using the Task queue. It grabs the next > > 50 > > > > > > keys for a Kind and then calls db.delete(keys), so fairly simple > > > > > > stuff. here's some example results in the log: > > > > > > > 865ms 1032cpu_ms 952api_cpu_ms > > > > > > 1058ms 1040cpu_ms 952api_cpu_ms > > > > > > 947ms 49947cpu_ms 49869api_cpu_ms <--??? > > > > > > 1425ms 1035cpu_ms 952api_cpu_ms > > > > > > 1674ms 41181cpu_ms 41094api_cpu_ms > > > > > > > any thoughts? something seems wrong to me > > > > > > > cheers > > > > > > brian > > > > > > > On Sep 8, 8:56 am, bFlood <bflood...@gmail.com> wrote: > > > > > > > ok, I was able to check over my code again and even with rolling > > back > > > > > > > small changes, the largeCPUincreases are still there. at this > > point, > > > > > > > I have to agree with herbie's findings as well. It would be nice > > if > > > > > > > Google could weigh in on this troubling issue > > > > > > > > cheers > > > > > > > brian > > > > > > > > On Sep 8, 4:51 am, herbie <4whi...@o2.co.uk> wrote: > > > > > > > > > On Sep 8, 12:07 am, Stephen <sdea...@gmail.com> wrote: > > > > > > > > > > OK, but because api_cpu_ms is 96% of the total, then cpu_ms > > is also > > > > > > > > > almost 3x higher? The spike is showing up in the cpu_ms? > > > > > > > > > Yes in total the cpu_ms has gone up by nearly 3x too. > > > > > > > > > But as I understand it cpu_ms is the totalcpuusage for the > > request > > > > > > > > and api_cpu_ms is thecpuusage by GAE api calls. So the > > difference > > > > > > > > between the two is thecpuusage of my non api code. This > > difference > > > > > > > > hasn’t increased because the code hasn’t changed. > > > > > > > > > But yes, the newhighvalue for api_cpu_ms directly affects my > > quota > > > > > > > > because it makes the vast majority of cpu_ms. So we do pay for > > > > > > > > api_cpu_ms ! So for example if Google makes a change to > > db.put() > > > > > > > > (or any api call) so that it uses morecpu, we will be billed > > for > > > > > > > > morecpuusage even if our code hasn’t changed. > > > > > > > > > As my code/ indexes hasn’t changed and the api_cpu_ms has shot > > up the > > > > > > > > obvious conclusion is that an api/datastore change has caused > > it? > > > > > > > > > But there may be another ‘good’ reason for it, which I can’t > > think > > > > > > > > of, but as I’m going to have to pay for the increase in > > api_cpu_ms, > > > > > > > > I would really appreciate it if someone at Google could help. > > > > > > > > > On Sep 8, 12:07 am, Stephen <sdea...@gmail.com> wrote: > > > > > > > > > > On Sep 7, 8:57 pm, herbie <4whi...@o2.co.uk> wrote: > > > > > > > > > > > On Sep 7, 6:50 pm, Stephen <sdea...@gmail.com> wrote: > > > > > > > > > > > > What about cpu_ms, is that also higher for requests which > > write > > > > > > to the > > > > > > > > > > > data store? > > > > > > > > > > > No, not in relation to api_cpu_ms. For the request that > > does the > > > > > > most > > > > > > > > > > writing to the datastore api_cpu_ms accounts for 96% of the > > total > > > > > > > > > > cpu_ms value!. The so request handler does not much more > > than > > > > > > create > > > > > > > > > > new entities in the datastore. > > > > > > > > > > OK, but because api_cpu_ms is 96% of the total, then cpu_ms > > is also > > > > > > > > > almost 3x higher? The spike is showing up in the cpu_ms? > > > > > > > > > > cpu_ms is billed for, so if you have billing enabled you are > > being > > > > > > > > > overcharged. > > > > > > > > > > You could try asking for a refund here: > > >http://code.google.com/support/bin/request.py?contact_type=AppEngineB... > > -- > Nick Johnson, Developer Programs Engineer, App Engine --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
