Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
It's not my benchmark, it's Slim3's :) ...but you're right, it's bogus. I asked on the main appengine group too, and it turns out the low-level benchmark is doing lazy loading. With that fixed, their numbers come out like yours. I found this one too, which also gets results like yours: http://gaejava.appspot.com/ On Wed, Jun 8, 2011 at 4:44 AM, Erwin Streur wrote: > Indeed Dennis's measurements are very suspicious. First you should do > a couple of warming ups on each of the implementations to prevent > pollution like the JDO classpath scan for enhanced classes (which is > one of the reasons for the high initial run). Then do a couple of run > to determine a range of measurements to spot outlyers. your low-level > API 2millis is definately one. > > When I did the measurements I got the following results > low-level: 1150-1550 > Slim3: 1150-1600 > Objectify: 1950-2400 > JDO: 2100-2700 > > These measurements confirm that GAE designed implementations are > faster then the GAE implementation of a generic data access layer > (JDO), but not so extrem as initially posted. > > The initial response using JDO is a known issue and especially low > trafic website should not use it or use the always on feature (maybe > this will change in the new pricing model) > > Regards, > > Erwin > > On Jun 7, 11:00 am, Ian Marshall wrote: > > The low-level API does indeed look very fast. > > > > Just a comment on JDO: repeat runs roughly halve the JDO run time. I > > presume that this is because for repeat runs the JDO persistence > > manager factory has already been constructed. > > > > On Jun 6, 8:44 pm, DennisP wrote: > > > > > I'm looking at this online demo: > http://slim3demo.appspot.com/performance/ > > > > > Sample run: > > > The number of entities: 1 > > > low-level API:get: 2 millis > > > Slim3: 2490 millis > > > JDO: 6030 millis > > > > > Is the low-level API really that much faster? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-java@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Apologies, no offense meant. My impression was that if you wanted to, say,
display all that data, it's going to take around 1000 ms to get it, not 1
ms.
On Wed, Jun 8, 2011 at 10:55 AM, Yasuo Higa wrote:
> It is not bogus.
> LazyList#size() fetches all data as follows:
> public int size() {
>resolveAllData();
>return results.size();
> }
>
> Yasuo Higa
>
> On Wed, Jun 8, 2011 at 11:32 PM, Dennis Peterson
> wrote:
> > It's not my benchmark, it's Slim3's :) ...but you're right, it's bogus. I
> > asked on the main appengine group too, and it turns out the low-level
> > benchmark is doing lazy loading. With that fixed, their numbers come out
> > like yours.
> > I found this one too, which also gets results like yours:
> > http://gaejava.appspot.com/
> >
> > On Wed, Jun 8, 2011 at 4:44 AM, Erwin Streur
> wrote:
> >>
> >> Indeed Dennis's measurements are very suspicious. First you should do
> >> a couple of warming ups on each of the implementations to prevent
> >> pollution like the JDO classpath scan for enhanced classes (which is
> >> one of the reasons for the high initial run). Then do a couple of run
> >> to determine a range of measurements to spot outlyers. your low-level
> >> API 2millis is definately one.
> >>
> >> When I did the measurements I got the following results
> >> low-level: 1150-1550
> >> Slim3: 1150-1600
> >> Objectify: 1950-2400
> >> JDO: 2100-2700
> >>
> >> These measurements confirm that GAE designed implementations are
> >> faster then the GAE implementation of a generic data access layer
> >> (JDO), but not so extrem as initially posted.
> >>
> >> The initial response using JDO is a known issue and especially low
> >> trafic website should not use it or use the always on feature (maybe
> >> this will change in the new pricing model)
> >>
> >> Regards,
> >>
> >> Erwin
> >>
> >> On Jun 7, 11:00 am, Ian Marshall wrote:
> >> > The low-level API does indeed look very fast.
> >> >
> >> > Just a comment on JDO: repeat runs roughly halve the JDO run time. I
> >> > presume that this is because for repeat runs the JDO persistence
> >> > manager factory has already been constructed.
> >> >
> >> > On Jun 6, 8:44 pm, DennisP wrote:
> >> >
> >> > > I'm looking at this online
> >> > > demo:http://slim3demo.appspot.com/performance/
> >> >
> >> > > Sample run:
> >> > > The number of entities: 1
> >> > > low-level API:get: 2 millis
> >> > > Slim3: 2490 millis
> >> > > JDO: 6030 millis
> >> >
> >> > > Is the low-level API really that much faster?
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Google App Engine for Java" group.
> >> To post to this group, send email to
> >> google-appengine-java@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengine-java+unsubscr...@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/google-appengine-java?hl=en.
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> google-appengine-java@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/google-appengine-java?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Those multi-entity transactions are definitely interesting to me. There's some overhead but no getting around that. A while back I was playing around with some adhoc methods to do it in a specific case, but I suspect Slim3 is more solid and maybe faster than what I was doing. Definitely easier. When I get a chance I want to dig in and find out how it works. On Wed, Jun 8, 2011 at 10:26 AM, Mike Lawrence wrote: > I get... > > The number of entities: 1 > low level 1717 millis > slim3 1502 millis > objectify 2970 millis > jdo 3485 millis > > probably should modify this example > to do an average of several runs > > one important thing to note, is slim3 allows you > to update multiple entity types > in a single transaction. > not possible with the other 3 APIs > > > > On Jun 7, 4:00 am, Ian Marshall wrote: > > The low-level API does indeed look very fast. > > > > Just a comment on JDO: repeat runs roughly halve the JDO run time. I > > presume that this is because for repeat runs the JDO persistence > > manager factory has already been constructed. > > > > On Jun 6, 8:44 pm, DennisP wrote:> I'm > looking at this online demo:http://slim3demo.appspot.com/performance/ > > > > > Sample run: > > > The number of entities: 1 > > > low-level API:get: 2 millis > > > Slim3: 2490 millis > > > JDO: 6030 millis > > > > > Is the low-level API really that much faster? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-java@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
No I agree, and slim3 looks very interesting to me. It was just the very
fast low-level times I was wondering about, but it looks like normally slim3
and low-level will be about the same speed.
On Wed, Jun 8, 2011 at 11:42 AM, Yasuo Higa wrote:
> Hi Dennis,
>
> You can see all sources.
> http://slim3demo.appspot.com/performance/
>
> Java runtime reflections are very very slow.
> If you don't think so, please try it by you.
>
> Yasuo Higa
>
> On Thu, Jun 9, 2011 at 12:00 AM, Dennis Peterson
> wrote:
> > Apologies, no offense meant. My impression was that if you wanted to,
> say,
> > display all that data, it's going to take around 1000 ms to get it, not 1
> > ms.
> >
> > On Wed, Jun 8, 2011 at 10:55 AM, Yasuo Higa wrote:
> >>
> >> It is not bogus.
> >> LazyList#size() fetches all data as follows:
> >> public int size() {
> >>resolveAllData();
> >>return results.size();
> >> }
> >>
> >> Yasuo Higa
> >>
> >> On Wed, Jun 8, 2011 at 11:32 PM, Dennis Peterson
> >> wrote:
> >> > It's not my benchmark, it's Slim3's :) ...but you're right, it's
> bogus.
> >> > I
> >> > asked on the main appengine group too, and it turns out the low-level
> >> > benchmark is doing lazy loading. With that fixed, their numbers come
> out
> >> > like yours.
> >> > I found this one too, which also gets results like yours:
> >> > http://gaejava.appspot.com/
> >> >
> >> > On Wed, Jun 8, 2011 at 4:44 AM, Erwin Streur
> >> > wrote:
> >> >>
> >> >> Indeed Dennis's measurements are very suspicious. First you should do
> >> >> a couple of warming ups on each of the implementations to prevent
> >> >> pollution like the JDO classpath scan for enhanced classes (which is
> >> >> one of the reasons for the high initial run). Then do a couple of run
> >> >> to determine a range of measurements to spot outlyers. your low-level
> >> >> API 2millis is definately one.
> >> >>
> >> >> When I did the measurements I got the following results
> >> >> low-level: 1150-1550
> >> >> Slim3: 1150-1600
> >> >> Objectify: 1950-2400
> >> >> JDO: 2100-2700
> >> >>
> >> >> These measurements confirm that GAE designed implementations are
> >> >> faster then the GAE implementation of a generic data access layer
> >> >> (JDO), but not so extrem as initially posted.
> >> >>
> >> >> The initial response using JDO is a known issue and especially low
> >> >> trafic website should not use it or use the always on feature (maybe
> >> >> this will change in the new pricing model)
> >> >>
> >> >> Regards,
> >> >>
> >> >> Erwin
> >> >>
> >> >> On Jun 7, 11:00 am, Ian Marshall wrote:
> >> >> > The low-level API does indeed look very fast.
> >> >> >
> >> >> > Just a comment on JDO: repeat runs roughly halve the JDO run time.
> I
> >> >> > presume that this is because for repeat runs the JDO persistence
> >> >> > manager factory has already been constructed.
> >> >> >
> >> >> > On Jun 6, 8:44 pm, DennisP wrote:
> >> >> >
> >> >> > > I'm looking at this online
> >> >> > > demo:http://slim3demo.appspot.com/performance/
> >> >> >
> >> >> > > Sample run:
> >> >> > > The number of entities: 1
> >> >> > > low-level API:get: 2 millis
> >> >> > > Slim3: 2490 millis
> >> >> > > JDO: 6030 millis
> >> >> >
> >> >> > > Is the low-level API really that much faster?
> >> >>
> >> >> --
> >> >> You received this message because you are subscribed to the Google
> >> >> Groups
> >> >> "Google App Engine for Java" group.
> >> >> To post to this group, send email to
> >> >> google-appengine-java@googlegroups.com.
> >> >> To unsubscribe from this group, send email to
> >> >> google-appengine-java+unsubscr...@googlegroups.com.
> >> >> For more options, visit this group at
> >> >> http://groups.google.com/group/google-ap
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Thanks, I'd found that, I'm curious how it's implemented, since I made some efforts in that direction myself a while back. On Wed, Jun 8, 2011 at 11:49 AM, Yasuo Higa wrote: > Hi Dennis, > > The following document will help you about global transactions: > http://sites.google.com/site/slim3appengine/#gtx > > Yasuo Higa > > On Thu, Jun 9, 2011 at 12:33 AM, Dennis Peterson > wrote: > > Those multi-entity transactions are definitely interesting to me. There's > > some overhead but no getting around that. > > A while back I was playing around with some adhoc methods to do it in a > > specific case, but I suspect Slim3 is more solid and maybe faster than > what > > I was doing. Definitely easier. When I get a chance I want to dig in and > > find out how it works. > > > > On Wed, Jun 8, 2011 at 10:26 AM, Mike Lawrence > > wrote: > >> > >> I get... > >> > >> The number of entities: 1 > >> low level 1717 millis > >> slim3 1502 millis > >> objectify 2970 millis > >> jdo 3485 millis > >> > >> probably should modify this example > >> to do an average of several runs > >> > >> one important thing to note, is slim3 allows you > >> to update multiple entity types > >> in a single transaction. > >> not possible with the other 3 APIs > >> > >> > >> > >> On Jun 7, 4:00 am, Ian Marshall wrote: > >> > The low-level API does indeed look very fast. > >> > > >> > Just a comment on JDO: repeat runs roughly halve the JDO run time. I > >> > presume that this is because for repeat runs the JDO persistence > >> > manager factory has already been constructed. > >> > > >> > On Jun 6, 8:44 pm, DennisP wrote:> I'm > >> > looking at this online demo:http://slim3demo.appspot.com/performance/ > >> > > >> > > Sample run: > >> > > The number of entities: 1 > >> > > low-level API:get: 2 millis > >> > > Slim3: 2490 millis > >> > > JDO: 6030 millis > >> > > >> > > Is the low-level API really that much faster? > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine for Java" group. > >> To post to this group, send email to > >> google-appengine-java@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine-java+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine-java?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > google-appengine-java@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine-java?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-java@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer wrote: > On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber wrote: > > I am not comparing reflexion vs byte-code generation or anything like > that, > > apt generates code, is not a runtime technology. > > Like or not reflexion is known to be slower than actually writing the > code. > > This is entirely irrelevant. We've now established that the issue at > hand is a strange quirk in the Low-Level API and has nothing to do > with reflection. 100% (or close to it) of the performance "gain" of > Slim3 is because it calls .size() on a List before iterating it. > > > Slim3 generates the minimal possible code you need to talk with the low > > level api, if the low level bench is faster is just because its not > > converting the Entity to the Pojo. > > You missed the point - in the benchmark, Slim3 was *faster* than the > Low-Level API. This clearly indicated something was amiss, and now > we've uncovered the actual cause. It turns out to be something quite > interesting indeed. > > This is a classic case of what Feynman called Cargo Cult Science. You > all believed that Slim3 should be faster than other APIs because code > generation is faster than reflection, so when someone produced an > ill-conceived benchmark that seemed to confirm your preconceived > notion, you just accepted the entire narrative: "Slim3 is fast > because it doesn't use reflection!" This is sloppy thinking (see: > Confirmation Bias). > > If you haven't read this, it's a gem (I make a habit of re-reading it > at least once a year): > > http://www.lhup.edu/~DSIMANEK/cargocul.htm > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-java@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
