Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
I don't understand this code at all. You're querying the data and
then iterating over a separate call? And what does the extra call to
list.size() have anything to do with this?
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
Just like the 1ms low-level API query performance in the benchmark
that spawned this thread, even a casual observer should be able to
recognize the obvious flaw - the numbers don't show any expected
relationship between # of entity groups or the use of global
transactions. Interpreted literally, you would assume that local
transactions are much faster for 5 entity groups, but global
transactions are much faster for 4 entity groups.
It's pretty obvious that the benchmark author just ran one pass and
took the numbers as-is. If you actually run multiple passes, you'll
find that there is enormous variance in the timing. The only way you
can realistically measure results like this on appengine is to execute
the test 100 times and take a median.
FWIW, I deliberately haven't made any performance claims for Objectify
because I haven't put the necessary amount of due diligence into
creating a proper set of benchmarks. It is not easy to measure
performance, especially in a dynamic environment like appengine. I
only hope that the Slim3 authors have put more care and thought into
crafting their library than they have their benchmarks.
Jeff
On Wed, Jun 8, 2011 at 2:34 PM, Gal Dolber gal.dol...@gmail.com wrote:
Slim3 is not only fast, the api is completely awesome. It has been my
choice
for a year now for all gae projects.
It includes name safety and and amazing querying
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
It's *NO* bogus benchmark because the sample iterates the results.
http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java
Yasuo Higa
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
Just like the 1ms low-level API query performance in the benchmark
that spawned this thread, even a casual observer should be able to
recognize the obvious flaw - the numbers don't show any expected
relationship between # of entity groups or the use of global
transactions. Interpreted literally, you would assume that local
transactions are much faster for 5 entity groups, but global
transactions are much faster for 4 entity groups.
It's pretty obvious that the benchmark author just ran one pass and
took the numbers as-is. If you actually run multiple passes, you'll
find that there is enormous variance in the timing. The only way you
can realistically measure results like this on appengine is to execute
the test 100 times and take a median.
FWIW, I deliberately haven't made any performance claims for Objectify
because I haven't put the necessary amount of due diligence into
creating a proper set of benchmarks. It is not easy to measure
performance, especially in a dynamic environment like appengine. I
only hope that the Slim3 authors have put more care and thought into
crafting their library than they have their benchmarks.
Jeff
On Wed, Jun 8, 2011 at 2:34 PM, Gal Dolber gal.dol...@gmail.com wrote:
Slim3 is not only fast, the api is completely awesome. It has been my
choice
for a year now for all gae projects.
It includes name safety
[appengine-java] Re: Federated Login authentication Issues.
Fedarated Login authentication option is enabled. - The user logs in using the Google apps account. - The user will not be able to send emails. Email service is broken with FederatedLogin with Google Apps account. Why I need Federated Login: One of the requirement to list in the googlapps market place is to support OpenID authentication.Hence I have to choose the FederatedLogin authentication option. Otherwise, I am happy with 'GoogleAccounts API' authentication. Can Google please look into the following issues with FederatedLogin - bulkloader - client remote-api. - Emails service with Googleapps accounts. -Aswath www.AccountingGuru.in On Tue, May 31, 2011 at 4:00 PM, Aswath Satrasala aswath.satras...@gmail.com wrote: Hello, In the Application Setting of the Appengine console, set the Authentication Options to 'Federated Login'. Two features will not work - bulkloader - client remote-api. Any workarounds for the above? -Aswath www.AccountingGuru.in -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Sometimes arraylist stored in Datastore are not correctly retrieved
Hi all,
I've got an issue with a property with type ArrayListString.
Let's suppose I've got a class like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
@Inheritance(strategy=InheritanceStrategy.SUBCLASS_TABLE)
public class CommContact implements Serializable {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
protected Long idCont;
@Persistent(defaultFetchGroup=true)
protected ArrayListString tags;
// other properties
}
And another class DBContact that extends CommContact, like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
public class DBContact extends CommContact {
// some other properties...
}
Now, let's suppose that I do a query on datastore and I retrieve a
DBContact object. Sometimes (I cannot understand which is the pattern
for this issue...) the property tags is not correctly valued,
resulting in an empty array (but if I go to the DataViewer, I can
actually see that on the datastore some tags exist for this DBContact
object...).
What am I missing?
I really cannot understand why sometimes it works and sometimes not...
Thank you very much!
Bye
cghersi
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why does the info messages stopped appearing in the Logs?
Hi, A few trouble-shooting questions: · What's in your logging.properties file? · Where is this file? Has it moved recently? Have you more than one copy of this file? On Jun 8, 2:00 pm, Daniel vedm...@gmail.com wrote: Hi I always was writing the info messages to the log (like described in here : http://code.google.com/appengine/docs/java/runtime.html#Logging) I always used the .info method and the messages were perfectly shown in the Admin Log page, But recently i noticed the .info messages stopped to work, adn only .warning and higher are written into the Log... Why is that? How can I write .info messages into the log? import java.util.logging.Logger; private static final Logger log = Logger.getLogger(KeepSessionAliveServlet.class.getName()); log.info(KeepSessionAliveServlet); -- not working :( log.warning(KeepSessionAliveServlet); -- does work :/ Any ideas? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why does the info messages stopped appearing in the Logs?
Hi, A few trouble-shooting questions: · What's in your logging.properties file? · Where is this file? Has it moved recently? Have you more than one copy of this file? · Do you use java.util.logging.Logger or something else like Log4J? On Jun 8, 2:00 pm, Daniel vedm...@gmail.com wrote: Hi I always was writing the info messages to the log (like described in here : http://code.google.com/appengine/docs/java/runtime.html#Logging) I always used the .info method and the messages were perfectly shown in the Admin Log page, But recently i noticed the .info messages stopped to work, adn only .warning and higher are written into the Log... Why is that? How can I write .info messages into the log? import java.util.logging.Logger; private static final Logger log = Logger.getLogger(KeepSessionAliveServlet.class.getName()); log.info(KeepSessionAliveServlet); -- not working :( log.warning(KeepSessionAliveServlet); -- does work :/ Any ideas? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] getInputStream.read() returns -1 always
Hello all,
I am using the request.getInputStream().read() and the return value is
always -1. Here is my code
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws
IOException, ServletException
{
resp.getWriter().println(req.getInputStream().read());
resp.getWriter().println(req.getParameter(name));
}
I am getting -1 for the first print but I am able to get the post parameter
for the second.
Can someone help me in solving this Issue ?
Thanks,
Pradeep.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Memcache Problem
Hi, I am trying to cache a huge amount of data and getting this exception MemcacheServiceException: Memcache put: Error setting single item. Is there any possible way to cache data of huge size. How does google achieve it while displaying the search results. I bet there the data is far greater than my simple application. Thanks and Regards, Rittik Ray -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Google app engine plugin for Eclipse
Reading the howto on how to install the plugin and recommendations for Java versions on Google app engine. The recommendation is to use Java 1.6 as this is what the App Engine servers run. That's great. That's the JDK I'm using with Eclipse as well. Followed the plugin instructions in the Google howto to install: http://dl.google.com/eclipse/plugin/3.6 And bizarrely, it tells me it will only work with Java 1.5??? How do I get the Eclipse plugin to work with 1.6 or do I have to install an older version of the JDK to do Google app engine development despite the recommendation to use 1.6. Confused, Geoff. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How do I migrate old data (change schema) in a deployed Java app?
Hello, I have the same problem. But your solution is not working for me. Can you help? Thanks, SN -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/6MTAtbkeS_0J. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Dead link report
Page: https://code.google.com/appengine/docs/java/datastore/jdo/dataclasses.html Section: Class and Field Annotations Dead link: the DataNucleus documentationhttp://www.datanucleus.org/products/accessplatform_1_1/jdo/types.html -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/VVTqcDjgwGcJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Ok - so what you're saying is that the extra call to list.size()
before iterating through the list makes list iteration faster? Oddly
enough, this does seem to make a difference.
This looks like some sort of performance bug in the Low-Level API.
It's clearly not related to Slim3... except in as much as Slim3
inadvertently works around this bug. It's also very peculiar to
working with List results - you obviously can't call size() when
you're iterating through an unbounded result set.
The benchmark is bogus... which is to say, it does not show what you
think it shows.
Jeff
On Thu, Jun 9, 2011 at 12:09 AM, Yasuo Higa higaya...@gmail.com wrote:
It's *NO* bogus benchmark because the sample iterates the results.
http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java
Yasuo Higa
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global
Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
Just like the 1ms low-level API query performance in the benchmark
that spawned this thread, even a casual observer should be able to
recognize the obvious flaw - the numbers don't show any expected
relationship between # of entity groups or the use of global
transactions. Interpreted literally, you would assume that local
transactions are much faster for 5 entity groups, but global
transactions are much faster for 4 entity groups.
It's pretty obvious that the benchmark author just ran one pass and
took the numbers as-is. If you actually run multiple passes, you'll
find that there is enormous variance in the timing. The only way you
can realistically measure results like this on appengine is to execute
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Star this issue:
http://code.google.com/p/googleappengine/issues/detail?id=5167
I'm willing to bet I know exactly what's going on. When you call
size() first, the backing ArrayList is being created initially with
the proper size. If you don't call size() first, the backing
ArrayList is initialized small and reallocs organically up to the
10,000-element size. Thus the performance penalty.
The GAE internals obviously can obtain the size information ahead of
time; they just aren't initializing the ArrayList to the proper size
in this one case. I could probably find the exact line of code in the
SDK and submit a patch.
Jeff
On Thu, Jun 9, 2011 at 5:16 AM, Jeff Schnitzer j...@infohazard.org wrote:
Ok - so what you're saying is that the extra call to list.size()
before iterating through the list makes list iteration faster? Oddly
enough, this does seem to make a difference.
This looks like some sort of performance bug in the Low-Level API.
It's clearly not related to Slim3... except in as much as Slim3
inadvertently works around this bug. It's also very peculiar to
working with List results - you obviously can't call size() when
you're iterating through an unbounded result set.
The benchmark is bogus... which is to say, it does not show what you
think it shows.
Jeff
On Thu, Jun 9, 2011 at 12:09 AM, Yasuo Higa higaya...@gmail.com wrote:
It's *NO* bogus benchmark because the sample iterates the results.
http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java
Yasuo Higa
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global
Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
[appengine-java] Some very long logs are cut in the Admin Console
Hi all, I'm experiencing a problem checking the logs of my app in AdminConsole. I've got a very long task started as a cron job with a task queue. It produces a very long log, composed by hundreds of message. But suddenly the log is cut in the middle of an action. I'm totally sure that there has been no errors or exceptions in the meanwhile, and more-over the log prints flush as log status (instead of the usual 200). I'm sure that the servlet subject of this log correclty finished its work because I can actually see the results in the datastore. I'm using java.util.logging at the moment. So, my question is: is there any property that I need to place in my logging.properties file? My config file logging.properties looks like this: # Set the default logging level for all loggers to INFO .level = FINE # Limit the message that are printed on the console to INFO and above. java.util.logging.ConsoleHandler.level = FINE java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter Any hint about this? Thank you very much!! Bye cghersi -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.org wrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Ok, you actually got me to waste 1 hour on benchmarking and I hate to say that you are right. The use of reflexion seems not to be heavy enough to make the difference. Some extra features on slim3 make it slower when you don't make use them, I almost have a working path to avoid this problem, but even with the patch, both frameworks give almost the same performance On Thu, Jun 9, 2011 at 10:15 AM, Dennis Peterson dennisbpeter...@gmail.comwrote: Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.orgwrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Don't feel bad - I've now wasted over a day on this! Sorry if I've come across as grumpy in previous emails... but I've wasted over a day on this :-( Jeff On Thu, Jun 9, 2011 at 4:25 PM, Gal Dolber gal.dol...@gmail.com wrote: Ok, you actually got me to waste 1 hour on benchmarking and I hate to say that you are right. The use of reflexion seems not to be heavy enough to make the difference. Some extra features on slim3 make it slower when you don't make use them, I almost have a working path to avoid this problem, but even with the patch, both frameworks give almost the same performance On Thu, Jun 9, 2011 at 10:15 AM, Dennis Peterson dennisbpeter...@gmail.com wrote: Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.org wrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
When frameworks compete, everyone wins! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/3zCQizDuxPIJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
No problem... I say what I think, but have no shame to admit when I'm wrong On Thu, Jun 9, 2011 at 8:55 PM, Jay Young jayyoung9...@gmail.com wrote: When frameworks compete, everyone wins! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/3zCQizDuxPIJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: DLL how to handle data tables
I didn't hear from anyone on this topic so maybe I posted in the wrong forum. What I've decided is that I need a tool to handle this issue. I'm hoping there is not something better and already made(there probably is but I didn't find it, I didn't look too hard). Here's my plan: *OBJECTIVE* App Engine Data Entity Manager will handle modeling data visually and produce java source that can be directly inserted into an existing GWT project. Create objects that can be serialized, transferred to and from the client, and marked with the @PersistenceCapable flag to be stored. Expand and refine capabilities based on feedback and experience. *STAGE 1* Because of my lack of experience with GWT and Java I've already created this app in .NET Win Forms the first stage of this project is already created and functional. *STAGE 2* Port to App Engine and build server side functionality using the existing .NET project to assist in data modeling. *STAGE 3* Build primitive user interface to test server side functionality. *STAGE 4* Solicit feedback from other GWT developers and decide if further progress is necessary. *STAGE 5* Refine user interface. *STAGE 6* Plugin for Eclipse? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/6UBxyJgkj7cJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How do I migrate old data (change schema) in a deployed Java app?
What is this DataViewer? How do I use this? Eclipse? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/skuDsBkU2CUJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Sometimes arraylist stored in Datastore are not correctly retrieved
I have almost 0 experience I'm just reading to learn a bit. It seems rather ambitious to store an ArrayList? Maybe modifying your tags to be an array would be more successful. You could probably keep your g/setters and modify them slightly. Then you have to convert exiting datastores. I don't know. I'll be interested to see what you discover either way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/NwXCHDp2WZIJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Sometimes arraylist stored in Datastore are not correctly retrieved
Hi,
My declarations are a bit different but I have seen this
happening
GAE not loading the subitems by default in some cases, then if you
close the PersistenceManager, subitems List is null.
I have solved it by looping all items and subitems before closing the
PersistenceManager
On Jun 9, 4:05 am, cghersi cristiano.ghe...@gmail.com wrote:
Hi all,
I've got an issue with a property with type ArrayListString.
Let's suppose I've got a class like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
@Inheritance(strategy=InheritanceStrategy.SUBCLASS_TABLE)
public class CommContact implements Serializable {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
protected Long idCont;
@Persistent(defaultFetchGroup=true)
protected ArrayListString tags;
// other properties
}
And another class DBContact that extends CommContact, like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
public class DBContact extends CommContact {
// some other properties...
}
Now, let's suppose that I do a query on datastore and I retrieve a
DBContact object. Sometimes (I cannot understand which is the pattern
for this issue...) the property tags is not correctly valued,
resulting in an empty array (but if I go to the DataViewer, I can
actually see that on the datastore some tags exist for this DBContact
object...).
What am I missing?
I really cannot understand why sometimes it works and sometimes not...
Thank you very much!
Bye
cghersi
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
