Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
I don't understand this code at all. You're querying the data and
then iterating over a separate call? And what does the extra call to
list.size() have anything to do with this?
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
Just like the 1ms low-level API query performance in the benchmark
that spawned this thread, even a casual observer should be able to
recognize the obvious flaw - the numbers don't show any expected
relationship between # of entity groups or the use of global
transactions. Interpreted literally, you would assume that local
transactions are much faster for 5 entity groups, but global
transactions are much faster for 4 entity groups.
It's pretty obvious that the benchmark author just ran one pass and
took the numbers as-is. If you actually run multiple passes, you'll
find that there is enormous variance in the timing. The only way you
can realistically measure results like this on appengine is to execute
the test 100 times and take a median.
FWIW, I deliberately haven't made any performance claims for Objectify
because I haven't put the necessary amount of due diligence into
creating a proper set of benchmarks. It is not easy to measure
performance, especially in a dynamic environment like appengine. I
only hope that the Slim3 authors have put more care and thought into
crafting their library than they have their benchmarks.
Jeff
On Wed, Jun 8, 2011 at 2:34 PM, Gal Dolber gal.dol...@gmail.com wrote:
Slim3 is not only fast, the api is completely awesome. It has been my
choice
for a year now for all gae projects.
It includes name safety and and amazing querying
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
It's *NO* bogus benchmark because the sample iterates the results.
http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java
Yasuo Higa
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
Just like the 1ms low-level API query performance in the benchmark
that spawned this thread, even a casual observer should be able to
recognize the obvious flaw - the numbers don't show any expected
relationship between # of entity groups or the use of global
transactions. Interpreted literally, you would assume that local
transactions are much faster for 5 entity groups, but global
transactions are much faster for 4 entity groups.
It's pretty obvious that the benchmark author just ran one pass and
took the numbers as-is. If you actually run multiple passes, you'll
find that there is enormous variance in the timing. The only way you
can realistically measure results like this on appengine is to execute
the test 100 times and take a median.
FWIW, I deliberately haven't made any performance claims for Objectify
because I haven't put the necessary amount of due diligence into
creating a proper set of benchmarks. It is not easy to measure
performance, especially in a dynamic environment like appengine. I
only hope that the Slim3 authors have put more care and thought into
crafting their library than they have their benchmarks.
Jeff
On Wed, Jun 8, 2011 at 2:34 PM, Gal Dolber gal.dol...@gmail.com wrote:
Slim3 is not only fast, the api is completely awesome. It has been my
choice
for a year now for all gae projects.
It includes name safety
[appengine-java] Re: Federated Login authentication Issues.
Fedarated Login authentication option is enabled. - The user logs in using the Google apps account. - The user will not be able to send emails. Email service is broken with FederatedLogin with Google Apps account. Why I need Federated Login: One of the requirement to list in the googlapps market place is to support OpenID authentication.Hence I have to choose the FederatedLogin authentication option. Otherwise, I am happy with 'GoogleAccounts API' authentication. Can Google please look into the following issues with FederatedLogin - bulkloader - client remote-api. - Emails service with Googleapps accounts. -Aswath www.AccountingGuru.in On Tue, May 31, 2011 at 4:00 PM, Aswath Satrasala aswath.satras...@gmail.com wrote: Hello, In the Application Setting of the Appengine console, set the Authentication Options to 'Federated Login'. Two features will not work - bulkloader - client remote-api. Any workarounds for the above? -Aswath www.AccountingGuru.in -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Sometimes arraylist stored in Datastore are not correctly retrieved
Hi all,
I've got an issue with a property with type ArrayListString.
Let's suppose I've got a class like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
@Inheritance(strategy=InheritanceStrategy.SUBCLASS_TABLE)
public class CommContact implements Serializable {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
protected Long idCont;
@Persistent(defaultFetchGroup=true)
protected ArrayListString tags;
// other properties
}
And another class DBContact that extends CommContact, like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
public class DBContact extends CommContact {
// some other properties...
}
Now, let's suppose that I do a query on datastore and I retrieve a
DBContact object. Sometimes (I cannot understand which is the pattern
for this issue...) the property tags is not correctly valued,
resulting in an empty array (but if I go to the DataViewer, I can
actually see that on the datastore some tags exist for this DBContact
object...).
What am I missing?
I really cannot understand why sometimes it works and sometimes not...
Thank you very much!
Bye
cghersi
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why does the info messages stopped appearing in the Logs?
Hi, A few trouble-shooting questions: · What's in your logging.properties file? · Where is this file? Has it moved recently? Have you more than one copy of this file? On Jun 8, 2:00 pm, Daniel vedm...@gmail.com wrote: Hi I always was writing the info messages to the log (like described in here : http://code.google.com/appengine/docs/java/runtime.html#Logging) I always used the .info method and the messages were perfectly shown in the Admin Log page, But recently i noticed the .info messages stopped to work, adn only .warning and higher are written into the Log... Why is that? How can I write .info messages into the log? import java.util.logging.Logger; private static final Logger log = Logger.getLogger(KeepSessionAliveServlet.class.getName()); log.info(KeepSessionAliveServlet); -- not working :( log.warning(KeepSessionAliveServlet); -- does work :/ Any ideas? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why does the info messages stopped appearing in the Logs?
Hi, A few trouble-shooting questions: · What's in your logging.properties file? · Where is this file? Has it moved recently? Have you more than one copy of this file? · Do you use java.util.logging.Logger or something else like Log4J? On Jun 8, 2:00 pm, Daniel vedm...@gmail.com wrote: Hi I always was writing the info messages to the log (like described in here : http://code.google.com/appengine/docs/java/runtime.html#Logging) I always used the .info method and the messages were perfectly shown in the Admin Log page, But recently i noticed the .info messages stopped to work, adn only .warning and higher are written into the Log... Why is that? How can I write .info messages into the log? import java.util.logging.Logger; private static final Logger log = Logger.getLogger(KeepSessionAliveServlet.class.getName()); log.info(KeepSessionAliveServlet); -- not working :( log.warning(KeepSessionAliveServlet); -- does work :/ Any ideas? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] getInputStream.read() returns -1 always
Hello all,
I am using the request.getInputStream().read() and the return value is
always -1. Here is my code
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws
IOException, ServletException
{
resp.getWriter().println(req.getInputStream().read());
resp.getWriter().println(req.getParameter(name));
}
I am getting -1 for the first print but I am able to get the post parameter
for the second.
Can someone help me in solving this Issue ?
Thanks,
Pradeep.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Memcache Problem
Hi, I am trying to cache a huge amount of data and getting this exception MemcacheServiceException: Memcache put: Error setting single item. Is there any possible way to cache data of huge size. How does google achieve it while displaying the search results. I bet there the data is far greater than my simple application. Thanks and Regards, Rittik Ray -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Google app engine plugin for Eclipse
Reading the howto on how to install the plugin and recommendations for Java versions on Google app engine. The recommendation is to use Java 1.6 as this is what the App Engine servers run. That's great. That's the JDK I'm using with Eclipse as well. Followed the plugin instructions in the Google howto to install: http://dl.google.com/eclipse/plugin/3.6 And bizarrely, it tells me it will only work with Java 1.5??? How do I get the Eclipse plugin to work with 1.6 or do I have to install an older version of the JDK to do Google app engine development despite the recommendation to use 1.6. Confused, Geoff. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How do I migrate old data (change schema) in a deployed Java app?
Hello, I have the same problem. But your solution is not working for me. Can you help? Thanks, SN -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/6MTAtbkeS_0J. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Dead link report
Page: https://code.google.com/appengine/docs/java/datastore/jdo/dataclasses.html Section: Class and Field Annotations Dead link: the DataNucleus documentationhttp://www.datanucleus.org/products/accessplatform_1_1/jdo/types.html -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/VVTqcDjgwGcJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Ok - so what you're saying is that the extra call to list.size()
before iterating through the list makes list iteration faster? Oddly
enough, this does seem to make a difference.
This looks like some sort of performance bug in the Low-Level API.
It's clearly not related to Slim3... except in as much as Slim3
inadvertently works around this bug. It's also very peculiar to
working with List results - you obviously can't call size() when
you're iterating through an unbounded result set.
The benchmark is bogus... which is to say, it does not show what you
think it shows.
Jeff
On Thu, Jun 9, 2011 at 12:09 AM, Yasuo Higa higaya...@gmail.com wrote:
It's *NO* bogus benchmark because the sample iterates the results.
http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java
Yasuo Higa
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global
Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
Just like the 1ms low-level API query performance in the benchmark
that spawned this thread, even a casual observer should be able to
recognize the obvious flaw - the numbers don't show any expected
relationship between # of entity groups or the use of global
transactions. Interpreted literally, you would assume that local
transactions are much faster for 5 entity groups, but global
transactions are much faster for 4 entity groups.
It's pretty obvious that the benchmark author just ran one pass and
took the numbers as-is. If you actually run multiple passes, you'll
find that there is enormous variance in the timing. The only way you
can realistically measure results like this on appengine is to execute
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Star this issue:
http://code.google.com/p/googleappengine/issues/detail?id=5167
I'm willing to bet I know exactly what's going on. When you call
size() first, the backing ArrayList is being created initially with
the proper size. If you don't call size() first, the backing
ArrayList is initialized small and reallocs organically up to the
10,000-element size. Thus the performance penalty.
The GAE internals obviously can obtain the size information ahead of
time; they just aren't initializing the ArrayList to the proper size
in this one case. I could probably find the exact line of code in the
SDK and submit a patch.
Jeff
On Thu, Jun 9, 2011 at 5:16 AM, Jeff Schnitzer j...@infohazard.org wrote:
Ok - so what you're saying is that the extra call to list.size()
before iterating through the list makes list iteration faster? Oddly
enough, this does seem to make a difference.
This looks like some sort of performance bug in the Low-Level API.
It's clearly not related to Slim3... except in as much as Slim3
inadvertently works around this bug. It's also very peculiar to
working with List results - you obviously can't call size() when
you're iterating through an unbounded result set.
The benchmark is bogus... which is to say, it does not show what you
think it shows.
Jeff
On Thu, Jun 9, 2011 at 12:09 AM, Yasuo Higa higaya...@gmail.com wrote:
It's *NO* bogus benchmark because the sample iterates the results.
http://code.google.com/p/slim3/source/browse/trunk/slim3demo/src/slim3/demo/controller/performance/GetLLController.java
Yasuo Higa
The code linked from http://slim3demo.appspot.com/performance/ has
just gone back to calling list.size() without iterating the results,
which once again goes back to a bogus benchmark.
Jeff
On Wed, Jun 8, 2011 at 9:30 PM, Yasuo Higa higaya...@gmail.com wrote:
Slim3 uses LL API.
To resolve a strange issue that slim3 is faster than LL, I tried the
following samples:
One:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
Two:
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
ListEntity list =
pq.asList(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
// VERY IMPORTANT
list.size();
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
The second one is much faster than the first one.
I fixed the samples to call list.size().
http://slim3demo.appspot.com/performance/
As a result, LL is as fast as slim3 (^^;
Yasuo Higa
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Thank you for fixing the benchmark.
I am very curious. According to this new benchmark - it's hard to
tell without pushing the buttons a lot of times, but there seems to be
a trend - Slim3 is somewhat faster than the Low Level API.
Doesn't Slim3 use the Low Level API underneath? How can it possibly be
faster?
Jeff
On Wed, Jun 8, 2011 at 4:27 PM, Yasuo Higa higaya...@gmail.com wrote:
What I want to provide is a fair and casual benchmark.
As jeff advised, I modified samples as follows:
for (Entity e : service.getBarListUsingLL()) {
e.getKey();
e.getProperty(sortValue);
}
for (Bar bar : service.getBarListUsingSlim3()) {
bar.getKey();
bar.getSortValue();
}
for (BarObjectify bar : service.getBarListUsingObjectify()) {
bar.getKey();
bar.getSortValue();
}
for (BarJDO bar : service.getBarListUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
LL API is much slower than before.
http://slim3demo.appspot.com/performance/
Yasuo Higa
On Thu, Jun 9, 2011 at 7:45 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Slim3 may be a nice piece of software, but it has not been
demonstrated to be faster than anything (including JDO). It might or
might not be faster - I don't know - but based on the sloppy
benchmarking, I'm pretty certain that the people making this claim
don't know either.
There's another ill-conceived performance claim on the Slim3 website:
You may worry about the overhead of global transactions. Don't worry.
It is not very expensive. There are three problems with their little
performance test:
1) It only measures wall-clock time, not cost.
2) It does not measure what happens under contention.
3) The numbers are obviously wrong - they don't even pass a smoke test.
Look at these numbers (from the Slim3 home page):
Entity Groups Local Transaction(millis) Global
Transaction(millis)
1 67 70
2 450 415
3 213 539
4 1498 981
5 447 781
[appengine-java] Some very long logs are cut in the Admin Console
Hi all, I'm experiencing a problem checking the logs of my app in AdminConsole. I've got a very long task started as a cron job with a task queue. It produces a very long log, composed by hundreds of message. But suddenly the log is cut in the middle of an action. I'm totally sure that there has been no errors or exceptions in the meanwhile, and more-over the log prints flush as log status (instead of the usual 200). I'm sure that the servlet subject of this log correclty finished its work because I can actually see the results in the datastore. I'm using java.util.logging at the moment. So, my question is: is there any property that I need to place in my logging.properties file? My config file logging.properties looks like this: # Set the default logging level for all loggers to INFO .level = FINE # Limit the message that are printed on the console to INFO and above. java.util.logging.ConsoleHandler.level = FINE java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter Any hint about this? Thank you very much!! Bye cghersi -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.org wrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Ok, you actually got me to waste 1 hour on benchmarking and I hate to say that you are right. The use of reflexion seems not to be heavy enough to make the difference. Some extra features on slim3 make it slower when you don't make use them, I almost have a working path to avoid this problem, but even with the patch, both frameworks give almost the same performance On Thu, Jun 9, 2011 at 10:15 AM, Dennis Peterson dennisbpeter...@gmail.comwrote: Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.orgwrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
Don't feel bad - I've now wasted over a day on this! Sorry if I've come across as grumpy in previous emails... but I've wasted over a day on this :-( Jeff On Thu, Jun 9, 2011 at 4:25 PM, Gal Dolber gal.dol...@gmail.com wrote: Ok, you actually got me to waste 1 hour on benchmarking and I hate to say that you are right. The use of reflexion seems not to be heavy enough to make the difference. Some extra features on slim3 make it slower when you don't make use them, I almost have a working path to avoid this problem, but even with the patch, both frameworks give almost the same performance On Thu, Jun 9, 2011 at 10:15 AM, Dennis Peterson dennisbpeter...@gmail.com wrote: Haha, excellent. I studied cargo cults a bit in anthropology classes, long ago, and never suspected how relevant they would be. You would probably enjoy this: http://www.fanfiction.net/s/5782108/1/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.org wrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely irrelevant. We've now established that the issue at hand is a strange quirk in the Low-Level API and has nothing to do with reflection. 100% (or close to it) of the performance gain of Slim3 is because it calls .size() on a List before iterating it. Slim3 generates the minimal possible code you need to talk with the low level api, if the low level bench is faster is just because its not converting the Entity to the Pojo. You missed the point - in the benchmark, Slim3 was *faster* than the Low-Level API. This clearly indicated something was amiss, and now we've uncovered the actual cause. It turns out to be something quite interesting indeed. This is a classic case of what Feynman called Cargo Cult Science. You all believed that Slim3 should be faster than other APIs because code generation is faster than reflection, so when someone produced an ill-conceived benchmark that seemed to confirm your preconceived notion, you just accepted the entire narrative: Slim3 is fast because it doesn't use reflection! This is sloppy thinking (see: Confirmation Bias). If you haven't read this, it's a gem (I make a habit of re-reading it at least once a year): http://www.lhup.edu/~DSIMANEK/cargocul.htm Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
When frameworks compete, everyone wins! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/3zCQizDuxPIJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?
No problem... I say what I think, but have no shame to admit when I'm wrong On Thu, Jun 9, 2011 at 8:55 PM, Jay Young jayyoung9...@gmail.com wrote: When frameworks compete, everyone wins! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/3zCQizDuxPIJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: DLL how to handle data tables
I didn't hear from anyone on this topic so maybe I posted in the wrong forum. What I've decided is that I need a tool to handle this issue. I'm hoping there is not something better and already made(there probably is but I didn't find it, I didn't look too hard). Here's my plan: *OBJECTIVE* App Engine Data Entity Manager will handle modeling data visually and produce java source that can be directly inserted into an existing GWT project. Create objects that can be serialized, transferred to and from the client, and marked with the @PersistenceCapable flag to be stored. Expand and refine capabilities based on feedback and experience. *STAGE 1* Because of my lack of experience with GWT and Java I've already created this app in .NET Win Forms the first stage of this project is already created and functional. *STAGE 2* Port to App Engine and build server side functionality using the existing .NET project to assist in data modeling. *STAGE 3* Build primitive user interface to test server side functionality. *STAGE 4* Solicit feedback from other GWT developers and decide if further progress is necessary. *STAGE 5* Refine user interface. *STAGE 6* Plugin for Eclipse? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/6UBxyJgkj7cJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How do I migrate old data (change schema) in a deployed Java app?
What is this DataViewer? How do I use this? Eclipse? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/skuDsBkU2CUJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Sometimes arraylist stored in Datastore are not correctly retrieved
I have almost 0 experience I'm just reading to learn a bit. It seems rather ambitious to store an ArrayList? Maybe modifying your tags to be an array would be more successful. You could probably keep your g/setters and modify them slightly. Then you have to convert exiting datastores. I don't know. I'll be interested to see what you discover either way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/NwXCHDp2WZIJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Sometimes arraylist stored in Datastore are not correctly retrieved
Hi,
My declarations are a bit different but I have seen this
happening
GAE not loading the subitems by default in some cases, then if you
close the PersistenceManager, subitems List is null.
I have solved it by looping all items and subitems before closing the
PersistenceManager
On Jun 9, 4:05 am, cghersi cristiano.ghe...@gmail.com wrote:
Hi all,
I've got an issue with a property with type ArrayListString.
Let's suppose I've got a class like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
@Inheritance(strategy=InheritanceStrategy.SUBCLASS_TABLE)
public class CommContact implements Serializable {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
protected Long idCont;
@Persistent(defaultFetchGroup=true)
protected ArrayListString tags;
// other properties
}
And another class DBContact that extends CommContact, like this:
@PersistenceCapable(identityType = IdentityType.APPLICATION,
detachable = true)
public class DBContact extends CommContact {
// some other properties...
}
Now, let's suppose that I do a query on datastore and I retrieve a
DBContact object. Sometimes (I cannot understand which is the pattern
for this issue...) the property tags is not correctly valued,
resulting in an empty array (but if I go to the DataViewer, I can
actually see that on the datastore some tags exist for this DBContact
object...).
What am I missing?
I really cannot understand why sometimes it works and sometimes not...
Thank you very much!
Bye
cghersi
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [google-appengine] SSL/HTTPS Support on Google Apps domains
We are still actively working on it but unfortunately do not have any new news to report... Greg On Wed, Jun 8, 2011 at 5:22 AM, Ron ronsterc...@gmail.com wrote: Hello Any news on when SSL/HTTPS Support on Google Apps domains will be available? Thanks Ron -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Register Article on GAE
Talking about yourself, Robert? =) Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Jun 9, 2011 at 12:15 AM, Robert Kluin robert.kl...@gmail.comwrote: I've got to say, I really like Typhoon AE's approach for this purpose. Swap the 'fake' stubs out in the SDK for one that actually do stuff. Most of the GAE APIs are quite nice, simple but quite powerful. If your datastore access is that thin, I guess you just need something that implements some type of simple get and put? When I read that blog post it just made me think that some people probably prefer startup type culture. Robert On Wed, Jun 8, 2011 at 11:43, Brandon Wirtz drak...@digerat.com wrote: Actually it's 3 but who's counting. :-) No it's about 6, and I have a function that handles my most common datastore reads and writes so that when I port to a different data store I swap from GAEData.py to MSSQL.PY or AWSMYSQL.PY in the import and the code base doesn't have to change. I do the same thing with URLFetch and MemCache. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer Sent: Wednesday, June 08, 2011 12:51 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: Register Article on GAE If you can implement your data access with 6 lines of code, I'm jealous!! Jeff On Tue, Jun 7, 2011 at 10:54 PM, Brandon Wirtz drak...@digerat.com wrote: You are only locked in if you don't build your own functions to talk to datastore. If you have a Get Data function you can swap 6 lines of codes with the 6 lines you need for MSSQL when you move to Azure, or the 8 lines for LAMP, -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer Sent: Tuesday, June 07, 2011 10:39 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: Register Article on GAE I assume he found my name on this group, but I didn't ask. Presumably he's reading this thread :-) He seemed like a nice guy. I thought the article was pretty good, and casts GAE in a favorable light. Bringing up MongoDB and HBase makes sense; along with GAE these are the three horsemen of the SQLpocalypse. Most of the article dealt with the question of being locked in to Google and having to deal with high-sigma events like repricing of the whole business. When I talk to people who haven't drank the GAE koolaid, this is the #1 thing they always bring up... and I have to admit that it's a fair criticism, especially in light of recent events. I don't think it's compelling compared to the terrific benefit of being able to develop, deploy, and scale a app without hiring an ops team, but I don't expect this criticism to go away. Jeff On Tue, Jun 7, 2011 at 7:30 PM, Brandon Wirtz drak...@digerat.com wrote: Metz reached out to me after finding me on the group, but I don't think I said enough things that could be spun as bad so none of them made it in the article. Sent from my Samsung Epic™ 4G Ikai Lan (Google) ika...@google.com wrote: I think it's kind of neat how I recognize every single name in that article. Did the author find you guys via Google Groups? Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Jun 8, 2011 at 5:47 AM, Andrei gml...@gmail.com wrote: There is another article about Android http://www.computerworld.com/s/article/9217400/Major_damages_sought _ in_Oracle_Google_patent_dispute_ Does this make any difference for GAE? Probably not? On Jun 7, 9:59 am, Joshua Smith joshuaesm...@charter.net wrote: http://www.theregister.co.uk/2011/06/07/inside_google_app_engine/ Nice overview, and I didn't see any glaring errors. I'll be keeping a link to this to pass along whenever anyone asks me what GAE is (and isn't). An interesting observation was that many apps spill into non-GAE infrastructure when they need some functionality that GAE doesn't offer. That's true of several of our apps: one of my apps interfaces with S3 heavily because I wrote it before BlobStore existed; another uses GAE only as a massively scalable front-end lobby and scheduler, for access to non-GAE servers (in a private data center, and in EC2 when the private data center fills up). Also, it was interesting to see the coverage of App Scale. I was familiar with AETyphoon, but not App Scale. Nice to know there are options if google makes some really bad decisions about their pricing model in the coming months. -Joshua -- You
[google-appengine] SSL version
Hi, Does anybody know what version of SSL is being supported by Appspot (e.g SSL version 2.0, 3.0 or TLS version 1, ...) ? I tried to google but I could not find out any information about it ? Any response will be greatly appreciated ! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: FAQ for out of preview pricing changes
Suppose an app takes 10ms to produce a 50k response. The response might be for a mobile client, with slow connection, and take 1s to transfer. Is the instance tied up for the whole 1s, or is the instance tied only for 10ms, and then there is some independent buffering that takes care of delivering the response to the mobile browser of the client? Luca -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-_fdqp60vSYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: data structure size
Thanks for the reply Nick. There are various articles on the internet claiming Google had limitations on variable size early on, maybe it was just a misunderstanding On Jun 6, 12:52 am, Nick Johnson (Google) nick.john...@google.com wrote: Geoffrey is correct - there isn't and never was such a limit. In fact, it'd be difficult to implement, and I can't see any reason it would be at all useful. The only size limits are on the total memory consumed by your instance, and on the size of API calls. -Nick Johnson On Fri, May 27, 2011 at 9:32 AM, Geoffrey Spear geoffsp...@gmail.comwrote: On May 26, 10:11 am, JH ja...@tickettrackit.com wrote: I'm actually talking about python variables. Ints, Strings, Lists, Dicts, etc.. I don't believe there was ever such a limit. The 1MB limit is on the API calls, which makes it impossible to have a datastore entity larger than 1MB (and, thus, impossible to have individual properties larger than 1MB as well, although in practice you'll never fit a whole 1MB into a property because the entire serialized entity needs to be able to fit into the 1MB API call to write it). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: R: RE: [google-appengine] Google blocks traffic to appengine application
I apologize, I hit the wrong reply button. Here is my message in full: Actually, we use the industry standard X-Forwarded-For header to relay the original IP address. This is used by most reverse proxies to allow these issues to be solved in a practical and transparent manner. Chris Engineer, CloudFlare On Jun 8, 8:09 pm, Brandon Wirtz drak...@digerat.com wrote: Chris at cloudflare responded to me offlist that I am wrong. I am on my phone at hpdisover in vegas and can't forward his reply easily. If he wants to respond on list it would allow public discussion of why this issue cropped up. Chris may be correct that ip headers forward, but that wasn't my experience. I think their proxies aren't always configured the way they think they are and are often double bagged Sent from my Samsung Epic™ 4G Brandon Wirtz drak...@digerat.com wrote: The issue is that cloudflare uses headers that don identify the ip adress they are making the request on behalf of so gae sees them as a dos attack when their round robin requests don't round robin enough. this will happen a lot when bing bot or google bot comes though to index your site. Also if you have a site with more than 5000 pages you will find cloud flare often results in the cloudflare captcha being indexed. Sent from my Samsung Epic™ 4G Martino A. Sabia ezu...@gmail.com wrote: Problem disappeared without any change made either on my site or in CloudFlare, so I don't think it depends on their service. It would be great to know by Google what happened, since the message cames from them, but i suppose it will not happen :(. My only idea is that it has something to do with Ipv6 day, but i have no evidences to prove this. Brandon, I don't know if CloudFlare lied to me, I don't understand why they had too, but i have appreciated the fact they even responded to me even if i have only a free account on their service... enough said? The great trouble in all this is, in my point of view, that there is no way to have some kind of response from google even if i'm an early adopter of appengine and pay for their service. That scared me a lot. Martino. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visithttps://groups.google.com/d/msg/google-appengine/-/I0gte2ag83QJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] unable to establish loopback connection
Hi I have problems with eclipse and google appengine. I have windows 7 64 bit. I have already tryied to turn off firewalls and avast but it doesn't works. 9-giu-2011 9.14.43 com.google.apphosting.utils.jetty.JettyLogger info INFO: Logging to JettyLogger(null) via com.google.apphosting.utils.jetty.JettyLogger 9-giu-2011 9.14.52 com.google.apphosting.utils.config.AppEngineWebXmlReader readAppEngineWebXml INFO: Successfully processed C:\Users\Marty\Desktop\Progetto Web\df-mf- mpg-aip2011\war\WEB-INF/appengine-web.xml 9-giu-2011 9.14.52 com.google.apphosting.utils.config.AbstractConfigXmlReader readConfigXml INFO: Successfully processed C:\Users\Marty\Desktop\Progetto Web\df-mf- mpg-aip2011\war\WEB-INF/web.xml 9-giu-2011 9.14.52 com.google.apphosting.utils.jetty.JettyLogger info INFO: jetty-6.1.x Slim3 HOT reloading:true 9-giu-2011 9.14.57 com.google.apphosting.utils.jetty.JettyLogger warn AVVERTENZA: failed org.mortbay.jetty.nio.SelectChannelConnector $1@168e4805: java.io.IOException: Unable to establish loopback connection 9-giu-2011 9.14.57 com.google.apphosting.utils.jetty.JettyLogger warn AVVERTENZA: failed SelectChannelConnector@127.0.0.1:: java.io.IOException: Unable to establish loopback connection 9-giu-2011 9.14.57 com.google.apphosting.utils.jetty.JettyLogger warn AVVERTENZA: failed Server@5b32627: java.io.IOException: Unable to establish loopback connection java.io.IOException: Unable to establish loopback connection at sun.nio.ch.PipeImpl$Initializer.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.nio.ch.PipeImpl.init(Unknown Source) at sun.nio.ch.SelectorProviderImpl.openPipe(Unknown Source) at java.nio.channels.Pipe.open(Unknown Source) at sun.nio.ch.WindowsSelectorImpl.init(Unknown Source) at sun.nio.ch.WindowsSelectorProvider.openSelector(Unknown Source) at java.nio.channels.Selector.open(Unknown Source) at org.mortbay.io.nio.SelectorManager $SelectSet.init(SelectorManager.java:312) at org.mortbay.io.nio.SelectorManager.doStart(SelectorManager.java: 223) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java: 50) at org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelConnector.java: 314) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java: 50) at org.mortbay.jetty.Server.doStart(Server.java:235) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java: 50) at com.google.appengine.tools.development.JettyContainerService.startContainer(JettyContainerService.java: 186) at com.google.appengine.tools.development.AbstractContainerService.startup(AbstractContainerService.java: 157) at com.google.appengine.tools.development.DevAppServerImpl.start(DevAppServerImpl.java: 169) at com.google.appengine.tools.development.DevAppServerMain $StartAction.apply(DevAppServerMain.java:164) at com.google.appengine.tools.util.Parser $ParseResult.applyArgs(Parser.java:48) at com.google.appengine.tools.development.DevAppServerMain.init(DevAppServerMain.java: 113) at com.google.appengine.tools.development.DevAppServerMain.main(DevAppServerMain.java: 89) Caused by: java.net.ConnectException: Connection refused: connect at sun.nio.ch.Net.connect(Native Method) at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) at java.nio.channels.SocketChannel.open(Unknown Source) ... 22 more -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
R: Re: R: RE: [google-appengine] Google blocks traffic to appengine application
I'm sorry maybe I gonna say something totally wrong, but the real problem here, IMO, is that there are NO EVIDENCE at all in any place we can access, to say what was the real problem. So we're talking about something that can be, but even can be not. When Brandon said that Google see incoming traffic from CloudFlare as a DOS attack, for the header issue or whatever, I gone to appspot dashboard's Blacklist page to see if something was blocked. Well there was not, and there was neither an IP address from CloudFlare in the list of most active connections. I don't know if we can consider this an 'evidence' of something. Maybe Google have something in front of GAE that filters incoming traffic, but even for this, i have no evidence nor Google has declared this. Am I missing something here? For sure i can i say that i have other GAE apps without CloudFlare and, during the issue, i was able to access them without problems. So I have some legitimate suspect, but i can't consider them the real problem 'cause i can't tell. The other think i can say is that, even if traffic comes from geographically same origin but on different connections/networks there was different behaviors. We had some users blocked for long time (3-5 hours), some who doesn't had any problem, some that had blocked for a short period of time (30min-1hour). I have noticed that the blocking message was noticed especially by heavy user of the website (editors, owners, developers) and the more the user used the website in the past, the more was the blocking time. I hope that there will not be a similar issue in the future, anyway the first think that i will do is temporarily disable CloudFlare to see if it's the real problem :D. In the meantime it will be 'nice' if the only one who knows what REALLY happened, i.e. Google, will tell us... For Chris from CloudFlare: do you think CloudFlare will investigate or can you send some message to google (if you have some direct connection with them) on this issue, or simply for you there is no issue at all? For everybody: what is the best practice in this case? What to do now? Open a ticket with google hoping they will investigate... some clues? Martino -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pA-byZLu4UcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Custom user authentication...
Hi, I've decided to use protorpc, which is a very good project. I really like how to works. Once this part will be finished, I'll see if I use tipfy, webapp2 (can use protorpc), tornado,... for the web part. I didn't look at jsonrpc but protorpc is so good I don't want to look at any other projects. Regards -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/3XcH9neyeYkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Can't create new application
Hi Robert Thanks for responding. I had already tried logging in through the original account. Didn't work, as they all seem to map back to the 'new' account. Already filled out the SMS issues form, although this isn't really an SMS issue. I posted to the forum because I found that someone had had the exact same problem before me, and Google contacted them directly (not via the SMS form) Rob On Wed, Jun 8, 2011 at 1:04 PM, Robert Kluin robert.kl...@gmail.com wrote: Either log into the other account, or fill out the SMS issues form, https://appengine.google.com/waitlist/sms_issues as instructed in the FAQ: http://code.google.com/appengine/kb/sms.html#error On Wed, Jun 8, 2011 at 11:03, robm m.rob...@gmail.com wrote: Dear Marzia/Google I believe I have exactly this same problem. I created my first GAE test site a few months back, possibly using a different email address. Then I created a new gmail address. Whenever I log in, it is this gmail address that is used (m.rob...@gmail.com) Today I tried to create a second GAE application to host an app I've been working on. I try to publish from within Eclipse, and Eclipse lets me log in to GAE control panel. I click 'add application' (you have 10 remaining) and it takes me to a panel where I type in my carrier and phone number. (I remember doing this successfully months ago) I get an error saying that this account has already been validated (true) and can't be validated a second time. There is no way to get past this and add the second application. Help please! Rob m.rob...@gmail.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/bkQzc2pwenA2Sm9K. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: today is ipv6 day
why can't by appspot.com? On Jun 8, 10:37 pm, barrybhunter barrybhun...@gmail.com wrote: I beleive ghs.google.com has been ipv6 enabled for a while. So can access app engine via a Apps Domain over ipv6. On 8 June 2011 13:58, saintthor saintt...@gmail.com wrote: i can accesswww.google.comand blogspot.com via ipv6. but appspot.com is unavilable. when would it be avilable? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: today is ipv6 day
If you really need it then you should add the suggestion to the issue tracker - which will help see if anybody else needs it. What is your use case? Seems its a 'nice to have' but there is absolutly no requirement for ipv6. the vast majority of websites dont have any ipv6 support yet. On 9 June 2011 14:12, saintthor saintt...@gmail.com wrote: why can't by appspot.com? On Jun 8, 10:37 pm, barrybhunter barrybhun...@gmail.com wrote: I beleive ghs.google.com has been ipv6 enabled for a while. So can access app engine via a Apps Domain over ipv6. On 8 June 2011 13:58, saintthor saintt...@gmail.com wrote: i can accesswww.google.comand blogspot.com via ipv6. but appspot.com is unavilable. when would it be avilable? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: today is ipv6 day
It is my understanding and I can dig up the thread when it get home that ipv6 is supported by appengine if the visitors isp is on the google white list for ipv6. unrelated my happy ipv6 day video http://www.youtube.com/watch?v=3LYlz97SZvY -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Barry Hunter Sent: Thursday, June 09, 2011 6:27 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: today is ipv6 day If you really need it then you should add the suggestion to the issue tracker - which will help see if anybody else needs it. What is your use case? Seems its a 'nice to have' but there is absolutly no requirement for ipv6. the vast majority of websites dont have any ipv6 support yet. On 9 June 2011 14:12, saintthor saintt...@gmail.com wrote: why can't by appspot.com? On Jun 8, 10:37 pm, barrybhunter barrybhun...@gmail.com wrote: I beleive ghs.google.com has been ipv6 enabled for a while. So can access app engine via a Apps Domain over ipv6. On 8 June 2011 13:58, saintthor saintt...@gmail.com wrote: i can accesswww.google.comand blogspot.com via ipv6. but appspot.com is unavilable. when would it be avilable? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Help on Fantasm Fan In
@Mike_W, I've updated the docs to describe this new feature. http://code.google.com/p/fantasm/wiki/AdvancedConcepts#Fan-In j On Jun 8, 9:44 pm, Robert Kluin robert.kl...@gmail.com wrote: Hey Shawn, Awesome, I'll try to review it tomorrow! Hopefully Mike is still following this thread and will help us test it out. :) Robert On Wed, Jun 8, 2011 at 12:03, Shawn shawn.ru...@gmail.com wrote: Hi Robert, Done. http://code.google.com/p/fantasm/source/detail?r=147. Please give it a quick review if you are able. Thanks again. -- Shawn -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Y2FNTEJHRWdqczhK. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: Register Article on GAE
Off topic: Startup vs Established culture. I do have to say that startups from a managers perspective is easier.. You have a carrot and a stick built in. Carrot = Do great and we will all be rich and stick = Fail and we will run out of money and all be homeless on the street that gives you an interesting team dynamic, vs in a large established org. you have do well and you will get a bonus relative to how well you did vs your peers or stick fail and in 18 months after 3 bad reviews we'll consider letting you go From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Ikai Lan (Google) Sent: Thursday, June 09, 2011 2:40 AM To: Google App Engine Subject: Re: [google-appengine] Re: Register Article on GAE Talking about yourself, Robert? =) Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Jun 9, 2011 at 12:15 AM, Robert Kluin robert.kl...@gmail.com wrote: I've got to say, I really like Typhoon AE's approach for this purpose. Swap the 'fake' stubs out in the SDK for one that actually do stuff. Most of the GAE APIs are quite nice, simple but quite powerful. If your datastore access is that thin, I guess you just need something that implements some type of simple get and put? When I read that blog post it just made me think that some people probably prefer startup type culture. Robert On Wed, Jun 8, 2011 at 11:43, Brandon Wirtz drak...@digerat.com wrote: Actually it's 3 but who's counting. :-) No it's about 6, and I have a function that handles my most common datastore reads and writes so that when I port to a different data store I swap from GAEData.py to MSSQL.PY or AWSMYSQL.PY in the import and the code base doesn't have to change. I do the same thing with URLFetch and MemCache. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer Sent: Wednesday, June 08, 2011 12:51 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: Register Article on GAE If you can implement your data access with 6 lines of code, I'm jealous!! Jeff On Tue, Jun 7, 2011 at 10:54 PM, Brandon Wirtz drak...@digerat.com wrote: You are only locked in if you don't build your own functions to talk to datastore. If you have a Get Data function you can swap 6 lines of codes with the 6 lines you need for MSSQL when you move to Azure, or the 8 lines for LAMP, -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer Sent: Tuesday, June 07, 2011 10:39 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: Register Article on GAE I assume he found my name on this group, but I didn't ask. Presumably he's reading this thread :-) He seemed like a nice guy. I thought the article was pretty good, and casts GAE in a favorable light. Bringing up MongoDB and HBase makes sense; along with GAE these are the three horsemen of the SQLpocalypse. Most of the article dealt with the question of being locked in to Google and having to deal with high-sigma events like repricing of the whole business. When I talk to people who haven't drank the GAE koolaid, this is the #1 thing they always bring up... and I have to admit that it's a fair criticism, especially in light of recent events. I don't think it's compelling compared to the terrific benefit of being able to develop, deploy, and scale a app without hiring an ops team, but I don't expect this criticism to go away. Jeff On Tue, Jun 7, 2011 at 7:30 PM, Brandon Wirtz drak...@digerat.com wrote: Metz reached out to me after finding me on the group, but I don't think I said enough things that could be spun as bad so none of them made it in the article. Sent from my Samsung EpicT 4G Ikai Lan (Google) ika...@google.com wrote: I think it's kind of neat how I recognize every single name in that article. Did the author find you guys via Google Groups? Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Jun 8, 2011 at 5:47 AM, Andrei gml...@gmail.com wrote: There is another article about Android http://www.computerworld.com/s/article/9217400/Major_damages_sought _ in_Oracle_Google_patent_dispute_ Does this make any difference for GAE? Probably not? On Jun 7, 9:59 am, Joshua Smith joshuaesm...@charter.net wrote: http://www.theregister.co.uk/2011/06/07/inside_google_app_engine/ Nice overview, and I didn't see any glaring errors. I'll be keeping a link to this to pass along whenever anyone asks me what GAE is (and isn't). An interesting observation was that many apps spill into
RE: Re: R: RE: [google-appengine] Google blocks traffic to appengine application
Check you Logs and it will tell you a request was blocked. Sorry still on phone so I can't remember how to search exactly but you get a warning or error and on there you can see the ip that hit you. When I played with cloud flare most the time it would present an ip, but not always, and some times the ip would be a local rather than public ip. If you set your headers correctly on gae, they will do 75% of what cloud flare claims to do, with none of the risk of badness. Based on your use case I will likely be building a caching library for gae that would make this issue go away for python and java users, an integrated cache which would be better and make your gae costs lower. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Martino A. Sabia Sent: Thursday, June 09, 2011 3:35 AM To: google-appengine@googlegroups.com Subject: R: Re: R: RE: [google-appengine] Google blocks traffic to appengine application I'm sorry maybe I gonna say something totally wrong, but the real problem here, IMO, is that there are NO EVIDENCE at all in any place we can access, to say what was the real problem. So we're talking about something that can be, but even can be not. When Brandon said that Google see incoming traffic from CloudFlare as a DOS attack, for the header issue or whatever, I gone to appspot dashboard's Blacklist page to see if something was blocked. Well there was not, and there was neither an IP address from CloudFlare in the list of most active connections. I don't know if we can consider this an 'evidence' of something. Maybe Google have something in front of GAE that filters incoming traffic, but even for this, i have no evidence nor Google has declared this. Am I missing something here? For sure i can i say that i have other GAE apps without CloudFlare and, during the issue, i was able to access them without problems. So I have some legitimate suspect, but i can't consider them the real problem 'cause i can't tell. The other think i can say is that, even if traffic comes from geographically same origin but on different connections/networks there was different behaviors. We had some users blocked for long time (3-5 hours), some who doesn't had any problem, some that had blocked for a short period of time (30min-1hour). I have noticed that the blocking message was noticed especially by heavy user of the website (editors, owners, developers) and the more the user used the website in the past, the more was the blocking time. I hope that there will not be a similar issue in the future, anyway the first think that i will do is temporarily disable CloudFlare to see if it's the real problem :D. In the meantime it will be 'nice' if the only one who knows what REALLY happened, i.e. Google, will tell us... For Chris from CloudFlare: do you think CloudFlare will investigate or can you send some message to google (if you have some direct connection with them) on this issue, or simply for you there is no issue at all? For everybody: what is the best practice in this case? What to do now? Open a ticket with google hoping they will investigate... some clues? Martino -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pA-byZLu4UcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
It does uses a lazy list to do asynchronous prefetching: http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/datastore/LazyList.java On Tue, Jun 7, 2011 at 3:19 AM, Anders blabl...@gmail.com wrote: I doubt that the difference can be that large. The performance test code uses the low-level PreparedQuery#asList call. The question is if the list (ListEntity) contains entities loaded with data or if the list returned has a lazy loading implementation so that the actual data from the the datastore only gets loaded when entity properties are accessed. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/WVhBRXBqMWFMZ3dK. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: Re: R: RE: [google-appengine] Google blocks traffic to appengine application
Had a phone conversation with a friend who is not a gae employee, but an IT contractor at Google, who suggested that google infrastructure often blocks large proxies that aren't on their ISP whitelist, and that if this issue is to be resolved it isn't necessarily a GAE issue but a GOOGLE issue and Cloudflare will need to get on that white list. No clue how they do that, maybe a GAE employee can speak up, but this lends to CloudFlare needs to get some ducks in a row. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Martino A. Sabia Sent: Thursday, June 09, 2011 3:35 AM To: google-appengine@googlegroups.com Subject: R: Re: R: RE: [google-appengine] Google blocks traffic to appengine application I'm sorry maybe I gonna say something totally wrong, but the real problem here, IMO, is that there are NO EVIDENCE at all in any place we can access, to say what was the real problem. So we're talking about something that can be, but even can be not. When Brandon said that Google see incoming traffic from CloudFlare as a DOS attack, for the header issue or whatever, I gone to appspot dashboard's Blacklist page to see if something was blocked. Well there was not, and there was neither an IP address from CloudFlare in the list of most active connections. I don't know if we can consider this an 'evidence' of something. Maybe Google have something in front of GAE that filters incoming traffic, but even for this, i have no evidence nor Google has declared this. Am I missing something here? For sure i can i say that i have other GAE apps without CloudFlare and, during the issue, i was able to access them without problems. So I have some legitimate suspect, but i can't consider them the real problem 'cause i can't tell. The other think i can say is that, even if traffic comes from geographically same origin but on different connections/networks there was different behaviors. We had some users blocked for long time (3-5 hours), some who doesn't had any problem, some that had blocked for a short period of time (30min-1hour). I have noticed that the blocking message was noticed especially by heavy user of the website (editors, owners, developers) and the more the user used the website in the past, the more was the blocking time. I hope that there will not be a similar issue in the future, anyway the first think that i will do is temporarily disable CloudFlare to see if it's the real problem :D. In the meantime it will be 'nice' if the only one who knows what REALLY happened, i.e. Google, will tell us... For Chris from CloudFlare: do you think CloudFlare will investigate or can you send some message to google (if you have some direct connection with them) on this issue, or simply for you there is no issue at all? For everybody: what is the best practice in this case? What to do now? Open a ticket with google hoping they will investigate... some clues? Martino -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pA-byZLu4UcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Receiving mail at appspotmail.com. Message size limits, address filtering, spam protection?
I am sending an email to the *.appspot.mail address with a 7 meg attachment, and it's bouncing every time -- but not for other smaller files I've tested. Seems like there might be message size limits after all. I don't know why google wouldn't publish this information if there is a limit? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/6BbUVukbHOoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
If you are going to just iterate through a list with out doing any work, fetching everything up front is always going to be faster. However, we expect that you are going to be doing something with the entities you fetch. The lazy list tries to hid the cost of fetching the entities by doing it asynchronously while you are 'processing' the previous batch of entities. In my experiments (in Python) where work was actually being done (namely thread.sleep(X)) it gave a 10-15% speed up (depending on how much work you are doing, as the fetch time is a constant value). I would not have expected the difference to be this significant in Java though. We never know the # of results ahead of time so the difference couldn't be related to growing the List organically (as it always grows organically). One thing that should be noted, a wrapper that proactively converts entities (instead of doing the conversion on demand) will negate any benefit from async prefetching. Also a realistic benchmark should tak into account that some amount of work will be done on the entities fetched. On Thu, Jun 9, 2011 at 9:51 AM, Alfred Fuller arfuller+appeng...@google.com wrote: It does uses a lazy list to do asynchronous prefetching: http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/datastore/LazyList.java On Tue, Jun 7, 2011 at 3:19 AM, Anders blabl...@gmail.com wrote: I doubt that the difference can be that large. The performance test code uses the low-level PreparedQuery#asList call. The question is if the list (ListEntity) contains entities loaded with data or if the list returned has a lazy loading implementation so that the actual data from the the datastore only gets loaded when entity properties are accessed. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/WVhBRXBqMWFMZ3dK. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: data structure size
I remember reading one or more articles claiming this limit as well, but it was quite some time ago (maybe in 2008?). Probably they were confused by the 1mb entity limit. Damn, I guess *everything* I read on the internet isn't true? :/ Robert On Wed, Jun 8, 2011 at 23:15, JH ja...@mhztech.com wrote: Thanks for the reply Nick. There are various articles on the internet claiming Google had limitations on variable size early on, maybe it was just a misunderstanding On Jun 6, 12:52 am, Nick Johnson (Google) nick.john...@google.com wrote: Geoffrey is correct - there isn't and never was such a limit. In fact, it'd be difficult to implement, and I can't see any reason it would be at all useful. The only size limits are on the total memory consumed by your instance, and on the size of API calls. -Nick Johnson On Fri, May 27, 2011 at 9:32 AM, Geoffrey Spear geoffsp...@gmail.comwrote: On May 26, 10:11 am, JH ja...@tickettrackit.com wrote: I'm actually talking about python variables. Ints, Strings, Lists, Dicts, etc.. I don't believe there was ever such a limit. The 1MB limit is on the API calls, which makes it impossible to have a datastore entity larger than 1MB (and, thus, impossible to have individual properties larger than 1MB as well, although in practice you'll never fit a whole 1MB into a property because the entire serialized entity needs to be able to fit into the 1MB API call to write it). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Help on Fantasm Fan In
i'm also using it right now for some moderation flows i need to finish so i will test it too On 9 June 2011 10:14, Jason Collins jason.a.coll...@gmail.com wrote: @Mike_W, I've updated the docs to describe this new feature. http://code.google.com/p/fantasm/wiki/AdvancedConcepts#Fan-In j On Jun 8, 9:44 pm, Robert Kluin robert.kl...@gmail.com wrote: Hey Shawn, Awesome, I'll try to review it tomorrow! Hopefully Mike is still following this thread and will help us test it out. :) Robert On Wed, Jun 8, 2011 at 12:03, Shawn shawn.ru...@gmail.com wrote: Hi Robert, Done. http://code.google.com/p/fantasm/source/detail?r=147. Please give it a quick review if you are able. Thanks again. -- Shawn -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Y2FNTEJHRWdqczhK. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 500 Errors (DeadlineExceededError) on module imports and other random bits of code
Anyone else seeing a large amount of 500s (DeadlineExceededError) lately on module imports? Over the last few days parts of my application that have never generated errors (and have not changed recently) have been throwing 500 errors and timing out. Often it is the first import of a module that times out or it may time out in the middle of an SDK function. It is usually never the same line of code from error to error either. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 500 Errors (DeadlineExceededError) on module imports and other random bits of code
Hi Panda Yes, we are also facing the same issue on most of our servers hosted on app-engine. Same thing was happened about 16 hours ago as well. It is very strange that no google officials normally respond to such emails. regards, Atif -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/f0pqDrsyY9wJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] SLA credits and graceful degradation during service downtime
The prospective SLA [1] states that it does not apply to any performance issue related to apps that use and handle errors received from any API that is not an App Engine Datastore API. [1] http://code.google.com/appengine/sla.html I'd like my app to degrade gracefully when App Engine experiences problems. For instance, if the blobstore or the images service becomes unavailable, I'd rather catch any relevant exceptions and keep the site operational with limited functionality than spit 500 server errors for any page. Will that approach make my app ineligible for service credits according to the SLA if one of the App Engine services (such as the blobstore or the images service) experiences significant downtime? Will my app become eligible if it doesn't try to handle any GAE exceptions but just fails completely for every request until the broken service becomes available again? Thanks, -- Alexander -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 500 Errors (DeadlineExceededError) on module imports and other random bits of code
I've seen the same problems. It actually forced my to start paying for the Always-On feature. Otherwise my users would be waiting for 20-30 seconds for a request if it worked at all. It might be related to issue 1695: http://code.google.com/p/googleappengine/issues/detail?id=1695 Dan On Jun 9, 2:55 pm, Atif atif.gul...@gmail.com wrote: Hi Panda Yes, we are also facing the same issue on most of our servers hosted on app-engine. Same thing was happened about 16 hours ago as well. It is very strange that no google officials normally respond to such emails. regards, Atif -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Help on Fantasm Fan In
BTW - any intentions on plugging an observer pattern/plugin around it :) ? It would be awesome to have a list of observers watching for change on states (subject) On 9 June 2011 12:22, David Mora dla.m...@gmail.com wrote: i'm also using it right now for some moderation flows i need to finish so i will test it too On 9 June 2011 10:14, Jason Collins jason.a.coll...@gmail.com wrote: @Mike_W, I've updated the docs to describe this new feature. http://code.google.com/p/fantasm/wiki/AdvancedConcepts#Fan-In j On Jun 8, 9:44 pm, Robert Kluin robert.kl...@gmail.com wrote: Hey Shawn, Awesome, I'll try to review it tomorrow! Hopefully Mike is still following this thread and will help us test it out. :) Robert On Wed, Jun 8, 2011 at 12:03, Shawn shawn.ru...@gmail.com wrote: Hi Robert, Done. http://code.google.com/p/fantasm/source/detail?r=147. Please give it a quick review if you are able. Thanks again. -- Shawn -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Y2FNTEJHRWdqczhK. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://about.me/david.mora -- http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 500 Errors (DeadlineExceededError) on module imports and other random bits of code
We also see deadline exceptions and Memcache was throwing exceptions for some time. What's interesting is that in those cases capability API was returning DISABLED for all checks. On Jun 9, 9:57 pm, Dan dan.c.hoo...@gmail.com wrote: I've seen the same problems. It actually forced my to start paying for the Always-On feature. Otherwise my users would be waiting for 20-30 seconds for a request if it worked at all. It might be related to issue 1695:http://code.google.com/p/googleappengine/issues/detail?id=1695 Dan On Jun 9, 2:55 pm, Atif atif.gul...@gmail.com wrote: Hi Panda Yes, we are also facing the same issue on most of our servers hosted on app-engine. Same thing was happened about 16 hours ago as well. It is very strange that no google officials normally respond to such emails. regards, Atif -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 500 Errors (DeadlineExceededError) on module imports and other random bits of code
And make the mail.Send() calls too. We've been getting these consistently when we never did before: The API call mail.Send() took too long to respond and was cancelled. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/WQkKWuCBADYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 500 Errors (DeadlineExceededError) on module imports and other random bits of code
Came to the newsgroup for this exact issue as a matter of fact. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Sxc5g1z7hYIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] JSONBOT 0.7.1 released
Hello kids and parents !! I just want to announce the release of JSONBOT 0.7.1. This release consists of minor bug fixes and new xmpp auth code (SASL) which support DIGEST-MD5 and PLAIN authing. JSONBOT should run well again on systems with python2.5 installed. You can fetch it at http://jsonbot.googlecode.com Have fun playing with it! Bart About JSONBOT: JSONBOT is a remote event-driven framework for building bots that talk JSON to each other over XMPP. This distribution provides bots built on this framework for console, IRC, XMPP and Convore for the shell and WWW and XMPP for the Google Application engine. JSONBOT is all of the following: * a shell console bot * a shell IRC bot * a shell XMPP bot * a shell Convore bot * a Web bot running on Google Application Engine * a XMPP bot running on Google Application Engine * a Google Wave bot running op Google Application Engine * the XMPP bots are used to communicate between bots * plugin infrastructure to write your own functionality * event driven framework by the use of callbacks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Modelling Many to Many Relationship - Is the GAE Documentation's example appropriate for my model?
Hi all :) Let's say I have a app called Opposites Attract. And in it I have a *User * model, and each user has a list of *Likes* and a list of *Dislikes*. Each USER can have a maximum of: - 200 Likes. - 200 Dislikes. I want to query this information to find out for a given *Like*, what other users *Dislike* it or vice versa. I don't need to return EVERY person that likes or dislikes a particular thing. If I just returned the first 100 people found to like/dislike a particular thing, that would be acceptable. *How would it be best to model this?* Here (http://code.google.com/appengine/articles/modeling.html) it speaks of a potentially similar example under the *Many to Many* example, and if I were to implement it in this way, I would have a class like so: class User(db.Model): likes = db.ListProperty(db.Key) dislikes = db.ListProperty(db.Key) where the keys in both lists would refer to items from a *Thing* model. What I'm wondering is in the App Engine Documentation's example they mention: In the example above, the Contact side was chosen because a single person is not likely to belong to too many groups, whereas in a large contacts database, a group might contain hundreds of members. * With that quote in mind, does this method suit my example?* I want to design this app so that hundreds of thousands of people can like or dislike a thing. So I would feel better if the above quote read whereas in a large contacts database, a group might contain hundreds (OF THOUSANDS) of members. Also though I expect this doesn't change how I model this particular relationship I feel I should mention this in case it does: While my concern at the moment is searching ALL users that like or dislike a particular thing, my next step would be to search by *Gender* and *Location* For example: all GIRLS who likes Dogs everyone in Los Angeles who dislikes Cats -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/m3lX4RQ3FXoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Modelling Many to Many Relationship - Is the GAE Documentation's example appropriate for my model?
Depending on your entity group/transaction strategy here you might want to consider Referencing Users to a Dislike/Like Kind: http://code.google.com/appengine/docs/python/datastore/datamodeling.html#References What happens is (and premature i can say so) your like, dislikes properties are going to generate a lot of zig-zagging (due to merge join) when you query for this information (also depending on your index strategy) class User(db.Model): . class Like(db.Model): topic = db.StringProperty() #enumeration user = db.ReferenceProperty(User) class Dislike(db.Model): topic = db.StringProperty() #enumeration user = db.ReferenceProperty(User) Your key strategy can be (serial of topic + user.key().name()) And then you can use a pipeline to go thru your Like, Dislike structures and build a de-normalized Kind you can query safely on. - i think - On 9 June 2011 14:49, King sirhc...@gmail.com wrote: Hi all :) Let's say I have a app called Opposites Attract. And in it I have a * User* model, and each user has a list of *Likes* and a list of *Dislikes*. Each USER can have a maximum of: - 200 Likes. - 200 Dislikes. I want to query this information to find out for a given *Like*, what other users *Dislike* it or vice versa. I don't need to return EVERY person that likes or dislikes a particular thing. If I just returned the first 100 people found to like/dislike a particular thing, that would be acceptable. *How would it be best to model this?* Here (http://code.google.com/appengine/articles/modeling.html) it speaks of a potentially similar example under the *Many to Many* example, and if I were to implement it in this way, I would have a class like so: class User(db.Model): likes = db.ListProperty(db.Key) dislikes = db.ListProperty(db.Key) where the keys in both lists would refer to items from a *Thing* model. What I'm wondering is in the App Engine Documentation's example they mention: In the example above, the Contact side was chosen because a single person is not likely to belong to too many groups, whereas in a large contacts database, a group might contain hundreds of members. * With that quote in mind, does this method suit my example?* I want to design this app so that hundreds of thousands of people can like or dislike a thing. So I would feel better if the above quote read whereas in a large contacts database, a group might contain hundreds (OF THOUSANDS) of members. Also though I expect this doesn't change how I model this particular relationship I feel I should mention this in case it does: While my concern at the moment is searching ALL users that like or dislike a particular thing, my next step would be to search by *Gender* and * Location* For example: all GIRLS who likes Dogs everyone in Los Angeles who dislikes Cats -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/m3lX4RQ3FXoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
I modified the samples to use Iterator for a fair benchmark:
public IteratorEntity getBarsUsingLL() {
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
return pq.asIterator(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
}
public IteratorBar getBarsUsingSlim3() {
return Datastore.query(Bar.class).asIterator();
}
public IteratorBarObjectify getBarsUsingObjectify() {
Objectify ofy = ObjectifyService.begin();
return ofy.query(BarObjectify.class).iterator();
}
@SuppressWarnings(unchecked)
public ListBarJDO getBarsUsingJDO() {
ListBarJDO list = null;
PersistenceManager pm = PMF.get().getPersistenceManager();
try {
javax.jdo.Query q = pm.newQuery(BarJDO.class);
list = (ListBarJDO) q.execute();
list = (ListBarJDO) pm.detachCopyAll(list);
} finally {
pm.close();
}
return list;
}
private static final int COUNT = 5;
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (IteratorEntity ite = service.getBarsUsingLL();
ite.hasNext();) {
Entity e = ite.next();
e.getKey();
e.getProperty(sortValue);
}
}
sessionScope(getLL, (System.currentTimeMillis() - start) / COUNT);
return redirect(basePath);
}
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (IteratorBar ite = service.getBarsUsingSlim3();
ite.hasNext();) {
Bar bar = ite.next();
bar.getKey();
bar.getSortValue();
}
}
sessionScope(getSlim3, (System.currentTimeMillis() - start) / COUNT);
return redirect(basePath);
}
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (IteratorBarObjectify ite =
service.getBarsUsingObjectify(); ite
.hasNext();) {
BarObjectify bar = ite.next();
bar.getKey();
bar.getSortValue();
}
}
sessionScope(getObjectify, (System.currentTimeMillis() - start)
/ COUNT);
return redirect(basePath);
}
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (BarJDO bar : service.getBarsUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
}
sessionScope(getJDO, (System.currentTimeMillis() - start) / COUNT);
return redirect(basePath);
}
http://slim3demo.appspot.com/performance/
Slim3 is slightly faster than frameworks that use reflection,
but the difference is small.
Yasuo Higa
On Fri, Jun 10, 2011 at 2:29 AM, Alfred Fuller
arfuller+appeng...@google.com wrote:
If you are going to just iterate through a list with out doing any work,
fetching everything up front is always going to be faster. However, we
expect that you are going to be doing something with the entities you fetch.
The lazy list tries to hid the cost of fetching the entities by doing it
asynchronously while you are 'processing' the previous batch of entities. In
my experiments (in Python) where work was actually being done (namely
thread.sleep(X)) it gave a 10-15% speed up (depending on how much work you
are doing, as the fetch time is a constant value).
I would not have expected the difference to be this significant in Java
though. We never know the # of results ahead of time so the difference
couldn't be related to growing the List organically (as it always grows
organically).
One thing that should be noted, a wrapper that proactively converts entities
(instead of doing the conversion on demand) will negate any benefit from
async prefetching. Also a realistic benchmark should tak into account that
some amount of work will be done on the entities fetched.
On Thu, Jun 9, 2011 at 9:51 AM, Alfred Fuller
arfuller+appeng...@google.com wrote:
It does uses a lazy list to do asynchronous prefetching:
http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/datastore/LazyList.java
On Tue, Jun 7, 2011 at 3:19 AM, Anders blabl...@gmail.com wrote:
I doubt that the difference can be that large. The performance test code
uses the low-level PreparedQuery#asList call. The question is if the list
(ListEntity) contains entities loaded with data or if the list returned
has a lazy loading implementation so that the
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
It's still producing *wild* variance when I click those buttons. By a
factor of 2.
If you want an accurate benchmark:
1) You need more iterations. 5 is not enough. At a minimum I would
say 10. If this is too long for a single button click, you can halve
the # of entities fetched at once - it shouldn't matter for what we
are trying to measure.
2) You cannot use an arithmetic mean to measure average. GAE
performance is too erratic and outliers overwhelm the mean with noise.
Use a median value, this will be more stable and will more accurately
represent the true performance of the system.
Basically, you will know the measurement is proper when multiple
executions produce fairly consistent results. Right now you're just
measuring noise.
Jeff
On Thu, Jun 9, 2011 at 2:34 PM, Yasuo Higa higaya...@gmail.com wrote:
I modified the samples to use Iterator for a fair benchmark:
public IteratorEntity getBarsUsingLL() {
AsyncDatastoreService ds =
DatastoreServiceFactory.getAsyncDatastoreService();
Query q = new Query(Bar);
PreparedQuery pq = ds.prepare(q);
return pq.asIterator(FetchOptions.Builder.withDefaults().limit(
Integer.MAX_VALUE));
}
public IteratorBar getBarsUsingSlim3() {
return Datastore.query(Bar.class).asIterator();
}
public IteratorBarObjectify getBarsUsingObjectify() {
Objectify ofy = ObjectifyService.begin();
return ofy.query(BarObjectify.class).iterator();
}
@SuppressWarnings(unchecked)
public ListBarJDO getBarsUsingJDO() {
ListBarJDO list = null;
PersistenceManager pm = PMF.get().getPersistenceManager();
try {
javax.jdo.Query q = pm.newQuery(BarJDO.class);
list = (ListBarJDO) q.execute();
list = (ListBarJDO) pm.detachCopyAll(list);
} finally {
pm.close();
}
return list;
}
private static final int COUNT = 5;
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (IteratorEntity ite = service.getBarsUsingLL();
ite.hasNext();) {
Entity e = ite.next();
e.getKey();
e.getProperty(sortValue);
}
}
sessionScope(getLL, (System.currentTimeMillis() - start) / COUNT);
return redirect(basePath);
}
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (IteratorBar ite = service.getBarsUsingSlim3();
ite.hasNext();) {
Bar bar = ite.next();
bar.getKey();
bar.getSortValue();
}
}
sessionScope(getSlim3, (System.currentTimeMillis() - start) / COUNT);
return redirect(basePath);
}
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (IteratorBarObjectify ite =
service.getBarsUsingObjectify(); ite
.hasNext();) {
BarObjectify bar = ite.next();
bar.getKey();
bar.getSortValue();
}
}
sessionScope(getObjectify, (System.currentTimeMillis() - start)
/ COUNT);
return redirect(basePath);
}
@Override
public Navigation run() throws Exception {
long start = System.currentTimeMillis();
for (int i = 0; i COUNT; i++) {
for (BarJDO bar : service.getBarsUsingJDO()) {
bar.getKey();
bar.getSortValue();
}
}
sessionScope(getJDO, (System.currentTimeMillis() - start) / COUNT);
return redirect(basePath);
}
http://slim3demo.appspot.com/performance/
Slim3 is slightly faster than frameworks that use reflection,
but the difference is small.
Yasuo Higa
On Fri, Jun 10, 2011 at 2:29 AM, Alfred Fuller
arfuller+appeng...@google.com wrote:
If you are going to just iterate through a list with out doing any work,
fetching everything up front is always going to be faster. However, we
expect that you are going to be doing something with the entities you fetch.
The lazy list tries to hid the cost of fetching the entities by doing it
asynchronously while you are 'processing' the previous batch of entities. In
my experiments (in Python) where work was actually being done (namely
thread.sleep(X)) it gave a 10-15% speed up (depending on how much work you
are doing, as the fetch time is a constant value).
I would not have expected the difference to be this significant in Java
though. We never know the # of results ahead of time so the difference
couldn't be related to growing the List organically (as it always grows
organically).
One thing that should be noted, a wrapper that
[google-appengine] oauth vs federated login
hi, after checking out the federated login way to authenticate users to my app i decided that its not the way i want to go because it does not really look trustful if people want for example log in with twitter and the log in page is the same as the google one with a google url and a twitter logo. so now i want to try authorization via oauth. is there a generic oauth python package which supports twitter, google and facebook logins? any suggestion or best practice is appreciated. thank you -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
Interesting - in retrospect, of course it isn't realloc overhead. There would be at most 9 reallocs, and there's no way that could take hundreds of milliseconds. I'm not sure that this optimization of Python will be as effective in Javaland. The pattern of fetching is totally different in Python - they fetch N entities at a time into a list, so it makes sense to turn that list into an async iterator behind the scenes. Java already has iterators, so people probably only use Lists when they want to iterate the dataset multiple times. It'll be a less common case. I have mixed feelings about converting the Objectify Query.list() method to return a lazy List instead of an eagerly-converted List of POJOs. It would mean the List is no longer serializable. I've always thought that the Query methods that return Iterator are there for performance; the Query methods that return List are there for convenience. In this case, seems like convenience (ie serializability) should trump. But I'm not certain of that. Thanks for clearing this up! Jeff On Thu, Jun 9, 2011 at 10:29 AM, Alfred Fuller arfuller+appeng...@google.com wrote: If you are going to just iterate through a list with out doing any work, fetching everything up front is always going to be faster. However, we expect that you are going to be doing something with the entities you fetch. The lazy list tries to hid the cost of fetching the entities by doing it asynchronously while you are 'processing' the previous batch of entities. In my experiments (in Python) where work was actually being done (namely thread.sleep(X)) it gave a 10-15% speed up (depending on how much work you are doing, as the fetch time is a constant value). I would not have expected the difference to be this significant in Java though. We never know the # of results ahead of time so the difference couldn't be related to growing the List organically (as it always grows organically). One thing that should be noted, a wrapper that proactively converts entities (instead of doing the conversion on demand) will negate any benefit from async prefetching. Also a realistic benchmark should tak into account that some amount of work will be done on the entities fetched. On Thu, Jun 9, 2011 at 9:51 AM, Alfred Fuller arfuller+appeng...@google.com wrote: It does uses a lazy list to do asynchronous prefetching: http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/datastore/LazyList.java On Tue, Jun 7, 2011 at 3:19 AM, Anders blabl...@gmail.com wrote: I doubt that the difference can be that large. The performance test code uses the low-level PreparedQuery#asList call. The question is if the list (ListEntity) contains entities loaded with data or if the list returned has a lazy loading implementation so that the actual data from the the datastore only gets loaded when entity properties are accessed. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/WVhBRXBqMWFMZ3dK. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Deployments failing during Cloning application files (Python)
Recently my deployments have started to fail frequently during the Cloning X application files step. Usually I can get the deployment to succeed by rolling back and deploying again, but I'm not sure why it started happening. I'm at around 1100 application files right now. Is this a client-side or server-side issue? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/mg00XIqRsocJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: SLA credits and graceful degradation during service downtime
Hi Alexander, we are working on re-working the SLA to incorporate how we handle each of the APIs. Thanks for mentioning this specific circumstance as we should account for it! Greg On Thu, Jun 9, 2011 at 12:30 PM, Alexander Konovalenko alex...@gmail.comwrote: The prospective SLA [1] states that it does not apply to any performance issue related to apps that use and handle errors received from any API that is not an App Engine Datastore API. [1] http://code.google.com/appengine/sla.html I'd like my app to degrade gracefully when App Engine experiences problems. For instance, if the blobstore or the images service becomes unavailable, I'd rather catch any relevant exceptions and keep the site operational with limited functionality than spit 500 server errors for any page. Will that approach make my app ineligible for service credits according to the SLA if one of the App Engine services (such as the blobstore or the images service) experiences significant downtime? Will my app become eligible if it doesn't try to handle any GAE exceptions but just fails completely for every request until the broken service becomes available again? Thanks, -- Alexander -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
On Thu, Jun 9, 2011 at 2:56 PM, Jeff Schnitzer j...@infohazard.org wrote: Interesting - in retrospect, of course it isn't realloc overhead. There would be at most 9 reallocs, and there's no way that could take hundreds of milliseconds. I'm not sure that this optimization of Python will be as effective in Javaland. The pattern of fetching is totally different in Python - they fetch N entities at a time into a list, so it makes sense to turn that list into an async iterator behind the scenes. Java already has iterators, so people probably only use Lists when they want to iterate the dataset multiple times. It'll be a less common case. Funny because we actually don't have this feature in python (only iterators async prefetch). Ya, I would hope coders would use asIterable() when doing a single for loop. The real win for asList() async prefetch is in these situations: List l = q.asList(); // returns immediately other work/rpcs... ... use l ... and // Launches and runs both queries in parallel List l1 = q1.asList(); // returns immediately List l2 = d2.asList(); // returns immediately ... use l1 and l2 ... I have mixed feelings about converting the Objectify Query.list() method to return a lazy List instead of an eagerly-converted List of POJOs. It would mean the List is no longer serializable. I've always thought that the Query methods that return Iterator are there for performance; the Query methods that return List are there for convenience. In this case, seems like convenience (ie serializability) should trump. But I'm not certain of that. The lazy list implementation in the low level is both lazy and serializable. You should be able to do something similar in Objectify. Thanks for clearing this up! Jeff On Thu, Jun 9, 2011 at 10:29 AM, Alfred Fuller arfuller+appeng...@google.com wrote: If you are going to just iterate through a list with out doing any work, fetching everything up front is always going to be faster. However, we expect that you are going to be doing something with the entities you fetch. The lazy list tries to hid the cost of fetching the entities by doing it asynchronously while you are 'processing' the previous batch of entities. In my experiments (in Python) where work was actually being done (namely thread.sleep(X)) it gave a 10-15% speed up (depending on how much work you are doing, as the fetch time is a constant value). I would not have expected the difference to be this significant in Java though. We never know the # of results ahead of time so the difference couldn't be related to growing the List organically (as it always grows organically). One thing that should be noted, a wrapper that proactively converts entities (instead of doing the conversion on demand) will negate any benefit from async prefetching. Also a realistic benchmark should tak into account that some amount of work will be done on the entities fetched. On Thu, Jun 9, 2011 at 9:51 AM, Alfred Fuller arfuller+appeng...@google.com wrote: It does uses a lazy list to do asynchronous prefetching: http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/datastore/LazyList.java On Tue, Jun 7, 2011 at 3:19 AM, Anders blabl...@gmail.com wrote: I doubt that the difference can be that large. The performance test code uses the low-level PreparedQuery#asList call. The question is if the list (ListEntity) contains entities loaded with data or if the list returned has a lazy loading implementation so that the actual data from the the datastore only gets loaded when entity properties are accessed. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/WVhBRXBqMWFMZ3dK. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are
Re: [google-appengine] Re: Is the native API really so much faster than JDO and slim3?
On Thu, Jun 9, 2011 at 3:30 PM, Alfred Fuller
arfuller+appeng...@google.com wrote:
Funny because we actually don't have this feature in python (only iterators
async prefetch). Ya, I would hope coders would use asIterable() when doing a
single for loop. The real win for asList() async prefetch is in these
situations:
List l = q.asList(); // returns immediately
other work/rpcs...
... use l ...
and
// Launches and runs both queries in parallel
List l1 = q1.asList(); // returns immediately
List l2 = d2.asList(); // returns immediately
... use l1 and l2 ...
Ah, that makes total sense.
The lazy list implementation in the low level is both lazy and serializable.
You should be able to do something similar in Objectify.
This I don't understand. Looking at LazyList I see it references
QueryResultIteratorImpl and has no way of fully populating itself on
serialization. These classes might implement Serializable but you
sure can't serialize them! At least not if you want a useful set of
data on the other side.
For example, this works right now:
ListFoo myRemoteMethod() {
return ObjectifyService.begin().query(Foo.class).list();
}
But unless I'm missing something really obvious, this will not work:
ListEntity myRemoteMethod() {
return DatastoreServiceFactory.getDatastoreService.prepare(new
Query(Foo)).asList();
}
Jeff
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Register Article on GAE
Here is another piece of info from guy who worked 10 years for Google http://rethrick.com/#waving-goodbye And this relates to GAE Speed Here is something you've may have heard but never quite believed before: Google's vaunted scalable software infrastructure is obsolete. Don't get me wrong, their hardware and datacenters are the best in the world, and as far as I know, nobody is close to matching it. But the software stack on top of it is 10 years old, aging and designed for building search engines and crawlers. And it is well and truly obsolete. Protocol Buffers, BigTable and MapReduce are ancient, creaking dinosaurs compared to MessagePack, JSON, and Hadoop. And new projects like GWT, Closure and MegaStore are sluggish, overengineered Leviathans compared to fast, elegant tools like jQuery and mongoDB. Designed by engineers in a vacuum, rather than by developers who have need of tools. In the short time I've been outside Google I've created entire apps in Java in the space of a single workday. (Yes, you can program as quickly in Java as in Ruby or Python, if you understand your tools well.) I've gotten prototypes off the ground, shown it to people, or deployed them with hardly any barriers. On Jun 7, 9:59 am, Joshua Smith joshuaesm...@charter.net wrote: http://www.theregister.co.uk/2011/06/07/inside_google_app_engine/ Nice overview, and I didn't see any glaring errors. I'll be keeping a link to this to pass along whenever anyone asks me what GAE is (and isn't). An interesting observation was that many apps spill into non-GAE infrastructure when they need some functionality that GAE doesn't offer. That's true of several of our apps: one of my apps interfaces with S3 heavily because I wrote it before BlobStore existed; another uses GAE only as a massively scalable front-end lobby and scheduler, for access to non-GAE servers (in a private data center, and in EC2 when the private data center fills up). Also, it was interesting to see the coverage of App Scale. I was familiar with AETyphoon, but not App Scale. Nice to know there are options if google makes some really bad decisions about their pricing model in the coming months. -Joshua -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Register Article on GAE
Here is another piece of info from guy who worked for Google http://rethrick.com/#waving-goodbye And this relates to GAE Speed Here is something you've may have heard but never quite believed before: Google's vaunted scalable software infrastructure is obsolete. Don't get me wrong, their hardware and datacenters are the best in the world, and as far as I know, nobody is close to matching it. But the software stack on top of it is 10 years old, aging and designed for building search engines and crawlers. And it is well and truly obsolete. Protocol Buffers, BigTable and MapReduce are ancient, creaking dinosaurs compared to MessagePack, JSON, and Hadoop. And new projects like GWT, Closure and MegaStore are sluggish, overengineered Leviathans compared to fast, elegant tools like jQuery and mongoDB. Designed by engineers in a vacuum, rather than by developers who have need of tools. In the short time I've been outside Google I've created entire apps in Java in the space of a single workday. (Yes, you can program as quickly in Java as in Ruby or Python, if you understand your tools well.) I've gotten prototypes off the ground, shown it to people, or deployed them with hardly any barriers. On Jun 7, 9:59 am, Joshua Smith joshuaesm...@charter.net wrote: http://www.theregister.co.uk/2011/06/07/inside_google_app_engine/ Nice overview, and I didn't see any glaring errors. I'll be keeping a link to this to pass along whenever anyone asks me what GAE is (and isn't). An interesting observation was that many apps spill into non-GAE infrastructure when they need some functionality that GAE doesn't offer. That's true of several of our apps: one of my apps interfaces with S3 heavily because I wrote it before BlobStore existed; another uses GAE only as a massively scalable front-end lobby and scheduler, for access to non-GAE servers (in a private data center, and in EC2 when the private data center fills up). Also, it was interesting to see the coverage of App Scale. I was familiar with AETyphoon, but not App Scale. Nice to know there are options if google makes some really bad decisions about their pricing model in the coming months. -Joshua -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Register Article on GAE
You're about two days late to the party. Read the earlier comments in this thread ... Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Fri, Jun 10, 2011 at 7:23 AM, Andrei gml...@gmail.com wrote: Here is another piece of info from guy who worked 10 years for Google http://rethrick.com/#waving-goodbye And this relates to GAE Speed Here is something you've may have heard but never quite believed before: Google's vaunted scalable software infrastructure is obsolete. Don't get me wrong, their hardware and datacenters are the best in the world, and as far as I know, nobody is close to matching it. But the software stack on top of it is 10 years old, aging and designed for building search engines and crawlers. And it is well and truly obsolete. Protocol Buffers, BigTable and MapReduce are ancient, creaking dinosaurs compared to MessagePack, JSON, and Hadoop. And new projects like GWT, Closure and MegaStore are sluggish, overengineered Leviathans compared to fast, elegant tools like jQuery and mongoDB. Designed by engineers in a vacuum, rather than by developers who have need of tools. In the short time I've been outside Google I've created entire apps in Java in the space of a single workday. (Yes, you can program as quickly in Java as in Ruby or Python, if you understand your tools well.) I've gotten prototypes off the ground, shown it to people, or deployed them with hardly any barriers. On Jun 7, 9:59 am, Joshua Smith joshuaesm...@charter.net wrote: http://www.theregister.co.uk/2011/06/07/inside_google_app_engine/ Nice overview, and I didn't see any glaring errors. I'll be keeping a link to this to pass along whenever anyone asks me what GAE is (and isn't). An interesting observation was that many apps spill into non-GAE infrastructure when they need some functionality that GAE doesn't offer. That's true of several of our apps: one of my apps interfaces with S3 heavily because I wrote it before BlobStore existed; another uses GAE only as a massively scalable front-end lobby and scheduler, for access to non-GAE servers (in a private data center, and in EC2 when the private data center fills up). Also, it was interesting to see the coverage of App Scale. I was familiar with AETyphoon, but not App Scale. Nice to know there are options if google makes some really bad decisions about their pricing model in the coming months. -Joshua -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] backends emergency
Hi, I am experiencing problem with backends right now. One of backend is not updating. Here what I see in the logs: ... 99% Will check again in 60 seconds. 99% Will check again in 60 seconds. 99% Will check again in 60 seconds. 99% Will check again in 60 seconds. I noticed that it appeared in the list of backends but when I press Start button first I was forwarded to generic error message screen, now red message The backend could not be started because there are no versions ready. appears. I noticed that though I am requesting backend of class B8, it is listed as B2 in the list I tried several times. I disabled and reenabled application, but it does not help, other backends starting up normally. The same backend is updated fine on other application. Quota is OK and I am not exceeding neither backends number limit nor total instances number limit. Also I noticed that several static instances of other backend are not started also and are not starting after updating backend Additional information: I have 3 backends: 1) 9 static instances, B1 (infinite loop) 2) 1 static instance, B8 3) 1 static instance, B8 One of B8 instance is not starting at all, also 3 instances of 1) are not running. I have just noticed that on other application instances of 1) are stuck and I do not see what is going on with them. Could someone please assist me asap as we need this to be up and running by the start of 4:00am UTC. If someone from GAE team could contact me directly I would provide app id and all other relevant information. thanks in advance, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Pq4BeDiATTsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Modelling Many to Many Relationship - Is the GAE Documentation's example appropriate for my model?
Hi David, I'm having some trouble understanding your reply class Like(db.Model): topic = db.StringProperty() #enumeration user = db.ReferenceProperty(User) By enumeration, do you mean a number representing the thing being liked? So for example topic = 5 (where 5 might mean Dogs)? Your key strategy can be (serial of topic + user.key().name()) Are you saying the key for a Like might be (Dogs + John Smith)? Forgive me, I might be too much of a newb to understand this. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pzkk-e4SqwIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: backends emergency
A little update .. Seems like B8 instance that was not updating updated after ~30 minutes of waiting. But some of small B1 instances of 1) backend are not running. This is not acceptable for me as each instance is linked to different task. I feel like this somehow is caused by GAE system to be overloaded or something. Status page says everything is Normal. Can someone provide more details on what is going on. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/I3vteqY-n4EJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Is the tool mentioned at IO 2009 for calculating index overhead available?
At this point in a Google IO talkhttp://www.youtube.com/watch?v=AgaL6NGpkB8t=51m36ssuch a tool is mentioned, and since it's 2 years later I thought it might be available, but I can't find it. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/imoSEG_uZp4J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Is the tool mentioned at IO 2009 for calculating index overhead available?
D'oh. Nope, this keeps getting pushed back. I'll try to get the priority raised on this feature. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Fri, Jun 10, 2011 at 10:31 AM, King sirhc...@gmail.com wrote: At this point in a Google IO talkhttp://www.youtube.com/watch?v=AgaL6NGpkB8t=51m36ssuch a tool is mentioned, and since it's 2 years later I thought it might be available, but I can't find it. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/imoSEG_uZp4J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: lots of 500s - service down?
One of my apps is randomly getting those errors too for the last day or so. I even put up a servelet that just says blank http://arachnoconstructionapp.appspot.com/blank Sometimes it loads up fast sometimes it throws a 500 or takes over a minute to load. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/rEuUht8gKtcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Modelling Many to Many Relationship - Is the GAE Documentation's example appropriate for my model?
Hi King, I'm learning this thing here too so don't worry :) On 9 June 2011 19:54, King sirhc...@gmail.com wrote: Hi David, I'm having some trouble understanding your reply class Like(db.Model): topic = db.StringProperty() #enumeration user = db.ReferenceProperty(User) By enumeration, do you mean a number representing the thing being liked? So for example topic = 5 (where 5 might mean Dogs)? All depends on how you are going to distribute your topics or subjects. My example was just a way to point out the relation, not actually what you should be doing. Some people like to store strings directly, others enumerations, others a parent+child relation. It all goes down to what you feel more confortable with. Your key strategy can be (serial of topic + user.key().name()) Are you saying the key for a Like might be (Dogs + John Smith)? Correct :) - That gives you _many_ users liking many _subjects_ in one Entity Kind There are going to be obvious implications on what other queries you might need, but given your initial requirements it seems you need to tend to distribute those models so you don't kill an indexing server with many variations. Forgive me, I might be too much of a newb to understand this. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pzkk-e4SqwIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Register Article on GAE
Dang, I was four hours late to the party. I wanted to mention that we'd already beat that horse. On Thu, Jun 9, 2011 at 20:06, Ikai Lan (Google) ika...@google.com wrote: You're about two days late to the party. Read the earlier comments in this thread ... Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Fri, Jun 10, 2011 at 7:23 AM, Andrei gml...@gmail.com wrote: Here is another piece of info from guy who worked 10 years for Google http://rethrick.com/#waving-goodbye And this relates to GAE Speed Here is something you've may have heard but never quite believed before: Google's vaunted scalable software infrastructure is obsolete. Don't get me wrong, their hardware and datacenters are the best in the world, and as far as I know, nobody is close to matching it. But the software stack on top of it is 10 years old, aging and designed for building search engines and crawlers. And it is well and truly obsolete. Protocol Buffers, BigTable and MapReduce are ancient, creaking dinosaurs compared to MessagePack, JSON, and Hadoop. And new projects like GWT, Closure and MegaStore are sluggish, overengineered Leviathans compared to fast, elegant tools like jQuery and mongoDB. Designed by engineers in a vacuum, rather than by developers who have need of tools. In the short time I've been outside Google I've created entire apps in Java in the space of a single workday. (Yes, you can program as quickly in Java as in Ruby or Python, if you understand your tools well.) I've gotten prototypes off the ground, shown it to people, or deployed them with hardly any barriers. On Jun 7, 9:59 am, Joshua Smith joshuaesm...@charter.net wrote: http://www.theregister.co.uk/2011/06/07/inside_google_app_engine/ Nice overview, and I didn't see any glaring errors. I'll be keeping a link to this to pass along whenever anyone asks me what GAE is (and isn't). An interesting observation was that many apps spill into non-GAE infrastructure when they need some functionality that GAE doesn't offer. That's true of several of our apps: one of my apps interfaces with S3 heavily because I wrote it before BlobStore existed; another uses GAE only as a massively scalable front-end lobby and scheduler, for access to non-GAE servers (in a private data center, and in EC2 when the private data center fills up). Also, it was interesting to see the coverage of App Scale. I was familiar with AETyphoon, but not App Scale. Nice to know there are options if google makes some really bad decisions about their pricing model in the coming months. -Joshua -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Is the tool mentioned at IO 2009 for calculating index overhead available?
Hey Ikai, I know I haven't mentioned this before, but seemed like a good spot to bring up: http://code.google.com/p/googleappengine/issues/detail?id=2740 :) Robert On Thu, Jun 9, 2011 at 22:46, Ikai Lan (Google) ika...@google.com wrote: D'oh. Nope, this keeps getting pushed back. I'll try to get the priority raised on this feature. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Fri, Jun 10, 2011 at 10:31 AM, King sirhc...@gmail.com wrote: At this point in a Google IO talk such a tool is mentioned, and since it's 2 years later I thought it might be available, but I can't find it. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/imoSEG_uZp4J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] SSL version
Why don't you just go to https://appspot.com and view the connection and certificate info? On Wed, Jun 8, 2011 at 22:38, Khang Nguyen nguyenkhang@gmail.com wrote: Hi, Does anybody know what version of SSL is being supported by Appspot (e.g SSL version 2.0, 3.0 or TLS version 1, ...) ? I tried to google but I could not find out any information about it ? Any response will be greatly appreciated ! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
