date:20110920

Hi,

You are getting that Exception because the class that is used is not in our 
Runtime environment, for a complete list of all the class we do support 
please visit: http://code.google.com/appengine/docs/java/jrewhitelist.html

As mentioned Try using the URLFetch to access a webservices, more 
information: http://code.google.com/appengine/docs/java/urlfetch/overview.html

Hope this helps!

Jose Montes de Oca

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine-java/-/s8mY5yDflDUJ.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] How to select out of a one to many property using datastore API

2011-09-20 Thread siliconeagle

I have an appengine app which has been running for about a year now, i
have mainly been using JDO queries until now, but i am trying to
collect stats and the queries are taking too long. I have the
following entity (Device)

public class Device implements Serializable{
...
@Persistent
private SetKey feeds;// Key for the Feed entity
...
}
So I want to get a count of how many Devices have a certain Feed. I
was doing it in JDOQL before as such (uses javax.jdo.Query):

Query query = pm.newQuery(select from Device where
feeds.contains(:feedKey));
MapString, Object paramsf = new HashMapString, Object();
paramsf.put(feedKey,feed.getId());
ListDevice results = (ListDevice) query.executeWithMap(paramsf);
Though this code times out now. I was trying to use the Datastore API
so I could set chunk size,etc to see if i could speed the query up or
use a cursor, but I am unsure how to search in a Set field. I was
trying this (uses com.google.appengine.api.datastore.Query)

Query query = new Query(Device);
query.addFilter(feeds, FilterOperator.IN, feed.getId());
query.setKeysOnly();
final FetchOptions fetchOptions =
FetchOptions.Builder.withPrefetchSize(100).chunkSize(100);
QueryResultListEntity results =
dss.prepare(query).asQueryResultList(fetchOptions);
Essentially i am unsure how to search in the one-to-many filed (feeds)
for a single key. Is it possible to index it somehow?

hope it makes sense

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Timeout exceptions in paypalx-gae-toolkit appearing today

2011-09-20 Thread Alain de Raynal

Hi everyone,
My integration with paypal adaptive payment was working fine until today...
I've used the paypalx-gae-toolkithttp://code.google.com/p/paypalx-gae-toolkit/
without
any problem for the past 9 months

Then suddenly this afternoon, I started getting Exception while processing
simple payments.
   Could not fetch URL: https://svcs.paypal.com/AdaptivePayments/Pay
   Timeout while fetching: https://svcs.paypal.com/AdaptivePayments/Pay

I contacted paypal, but I'm still waiting for an answer...

I'd love to hear from other developers using the paypalx-gae-toolkit...
Have you noticed anything strange today ?

Thanks in advance,
Alain

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] GAE+GWT2.4 [ERROR] java.lang.NoClassDefFoundError: javax/validation/Path

2011-09-20 Thread John Howe

Hi David,

The gwt-servlet-deps jar is contained in WEB-INF/lib.
Here's a complete listing

appengine-api-1.0-sdk-1.5.3.jar
appengine-api-labs-1.5.3.jar
appengine-jsr107cache-1.5.3.jar
datanucleus-appengine-1.0.9.final.jar
datanucleus-core-1.1.5.jar
datanucleus-jpa-1.1.5.jar
geronimo-jpa_3.0_spec-1.1.1.jar
geronimo-jta_1.1_spec-1.1.1.jar
gwt-servlet-deps.jar
gwt-servlet.jar
gxt-2.2.5-gwt22.jar
jdo2-api-2.3-eb.jar
jsr107cache-1.1.jar

gwt-servlet-deps is also on the class path.
Any more suggestions would be greatly and sincerely appreciated.
I'm currently dead in the water.

On Tue, Sep 20, 2011 at 11:24 AM, David Chandler drfibona...@google.comwrote:

 Make sure that validation-api is in your WEB-INF/lib. Add it via
 gwt-servlet-deps (see lib folder in GWT SDK distro).

 /dmc

 On Tue, Sep 20, 2011 at 12:46 AM, John Howe jhowe...@gmail.com wrote:

 I thought this was going to be easy.

 I'm trying to move a working GAE java app up to GWT 2.4.

 But when I do, I see the following stack trace on the first RPC call.

 At least one other guy has seen this while doing same and posted a query
 at

 http://groups.google.com/group/google-appengine/browse_frm/thread/a78f4c443bd3b249

 Any help, suggestions, or insight would be most appreciated!

 Thanks.


 The server is running at http://localhost:/
 mozilla/5.0 (windows nt 6.0) applewebkit/535.1 (khtml, like gecko)
 chrome/14.0.835.163 safari/535.1
 [ERROR] javax.servlet.ServletContext log: Exception while dispatching
 incoming RPC call
 java.lang.NoClassDefFoundError: javax/validation/Path
 at java.lang.ClassLoader.defineClass1(Native Method)
  at java.lang.ClassLoader.defineClass(Unknown Source)
  at java.security.SecureClassLoader.defineClass(Unknown Source)
 at java.net.URLClassLoader.defineClass(Unknown Source)
  at java.net.URLClassLoader.access$100(Unknown Source)
  at java.net.URLClassLoader$1.run(Unknown Source)
 at java.net.URLClassLoader$1.run(Unknown Source)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(Unknown Source)
 at java.lang.ClassLoader.loadClass(Unknown Source)
  at
 com.google.appengine.tools.development.IsolatedAppClassLoader.loadClass(IsolatedAppClassLoader.java:176)
  at java.lang.ClassLoader.loadClass(Unknown Source)
  at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Unknown Source)
  at
 com.google.gwt.user.server.rpc.SerializationPolicyLoader.loadFromStream(SerializationPolicyLoader.java:196)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.loadSerializationPolicy(RemoteServiceServlet.java:90)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:293)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:157)
  at
 com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:455)
  at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:237)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:206)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248)
  at
 com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
  at
 org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
 at
 com.sportzcasterbeta.server.ServiceFilterImpl.doFilter(ServiceFilterImpl.java:25)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.sportzcasterbeta.server.ServiceFilterImpl.doFilter(ServiceFilterImpl.java:25)
 at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:35)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:58)
 at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:122)
 at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.appengine.tools.development.BackendServersFilter.doFilter(BackendServersFilter.java:97)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at

Re: [appengine-java] GAE+GWT2.4 [ERROR] java.lang.NoClassDefFoundError: javax/validation/Path

2011-09-20 Thread John Howe

Hi David,

Evidently, I didn't force a project rebuild.

Your suggestion worked. So, thank you very much!!

Im not a java/GWT expert but I am puzzled why this apparently new step
became necessary for me as part of the migration from GWT 2.2 to GWT 2.4. My
app had been working just fine under the older GWT version for many months.

Thanks again

On Tue, Sep 20, 2011 at 9:43 PM, John Howe jhowe...@gmail.com wrote:

 Hi David,

 The gwt-servlet-deps jar is contained in WEB-INF/lib.
 Here's a complete listing

 appengine-api-1.0-sdk-1.5.3.jar
 appengine-api-labs-1.5.3.jar
 appengine-jsr107cache-1.5.3.jar
 datanucleus-appengine-1.0.9.final.jar
 datanucleus-core-1.1.5.jar
 datanucleus-jpa-1.1.5.jar
 geronimo-jpa_3.0_spec-1.1.1.jar
 geronimo-jta_1.1_spec-1.1.1.jar
 gwt-servlet-deps.jar
 gwt-servlet.jar
 gxt-2.2.5-gwt22.jar
 jdo2-api-2.3-eb.jar
 jsr107cache-1.1.jar

 gwt-servlet-deps is also on the class path.
 Any more suggestions would be greatly and sincerely appreciated.
 I'm currently dead in the water.

 On Tue, Sep 20, 2011 at 11:24 AM, David Chandler 
 drfibona...@google.comwrote:

 Make sure that validation-api is in your WEB-INF/lib. Add it via
 gwt-servlet-deps (see lib folder in GWT SDK distro).

 /dmc

 On Tue, Sep 20, 2011 at 12:46 AM, John Howe jhowe...@gmail.com wrote:

 I thought this was going to be easy.

 I'm trying to move a working GAE java app up to GWT 2.4.

 But when I do, I see the following stack trace on the first RPC call.

 At least one other guy has seen this while doing same and posted a query
 at

 http://groups.google.com/group/google-appengine/browse_frm/thread/a78f4c443bd3b249

 Any help, suggestions, or insight would be most appreciated!

 Thanks.


 The server is running at http://localhost:/
 mozilla/5.0 (windows nt 6.0) applewebkit/535.1 (khtml, like gecko)
 chrome/14.0.835.163 safari/535.1
 [ERROR] javax.servlet.ServletContext log: Exception while dispatching
 incoming RPC call
 java.lang.NoClassDefFoundError: javax/validation/Path
 at java.lang.ClassLoader.defineClass1(Native Method)
  at java.lang.ClassLoader.defineClass(Unknown Source)
  at java.security.SecureClassLoader.defineClass(Unknown Source)
 at java.net.URLClassLoader.defineClass(Unknown Source)
  at java.net.URLClassLoader.access$100(Unknown Source)
  at java.net.URLClassLoader$1.run(Unknown Source)
 at java.net.URLClassLoader$1.run(Unknown Source)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(Unknown Source)
 at java.lang.ClassLoader.loadClass(Unknown Source)
  at
 com.google.appengine.tools.development.IsolatedAppClassLoader.loadClass(IsolatedAppClassLoader.java:176)
  at java.lang.ClassLoader.loadClass(Unknown Source)
  at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Unknown Source)
  at
 com.google.gwt.user.server.rpc.SerializationPolicyLoader.loadFromStream(SerializationPolicyLoader.java:196)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.loadSerializationPolicy(RemoteServiceServlet.java:90)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:293)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:157)
  at
 com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:455)
  at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:237)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:206)
  at
 com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248)
  at
 com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
  at
 org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
 at
 com.sportzcasterbeta.server.ServiceFilterImpl.doFilter(ServiceFilterImpl.java:25)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.sportzcasterbeta.server.ServiceFilterImpl.doFilter(ServiceFilterImpl.java:25)
 at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:35)
  at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:58)
 at
 org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at
 com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
  at

[google-appengine] Re: How do you view data downloaded from your app?

2011-09-20 Thread Tim Hoffman

Probably the easiest thing to do , is upload your data to the local dev 
server,
Then you can run your app, or play with the data directly with the remote 
api

Rgds

Tim

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Wq3ipX9Qx04J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Interrogate a push queue from code?

2011-09-20 Thread Emlyn

Is there a way to interrogate a push queue from code?

ie: list the tasks in the queue, and maybe get the status of each task?

Or, is there a way, given a task object, to get the current status of the task?

-- 
Emlyn

http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
Buzz posts,
comments and all.
http://point7.wordpress.com - My blog
Find me on Facebook and Buzz

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: /_ah/login returns status 500

2011-09-20 Thread Lior Harsat

Hi Gabriel,

fix, worked for me as well. Thanx :-)

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/8qyAE1m3qg0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Billing Status: Payment Due

2011-09-20 Thread Yossy

Everyone

Thank you for your replay.
I changed credit card information.
But my billing status is not change.

I contacted to AppEngineBillingSupport. (http://code.google.com/
support/bin/request.py?contact_type=AppEngineBillingSupport)
There is nothing for it but to wait and see.

Thank you.

On 9月15日, 午前5:50, Philip philip.mates...@driggle.com wrote:
 I had the same issue. You have to edit the credit card and re-enter
 your cvc.

 On Sep 15, 7:20 am, Christopher cspeckrun@gmail.com wrote:







  We are noticing the same on several of our accounts for the app engine
  applications. Brandon, can you please explain further? I am logged
  into Google Checkout and do not see any option for retrying anywhere.
  Thanks.

  On Sep 15, 12:55 am, Brandon Wirtz drak...@digerat.com wrote:

   Log in to Check out and select re-try.

   -Original Message-
   From: google-appengine@googlegroups.com

   [mailto:google-appengine@googlegroups.com] On Behalf Of Yossy
   Sent: Wednesday, September 14, 2011 9:32 PM
   To: Google App Engine
   Subject: [google-appengine] Billing Status: Payment Due

   Hi,

   My application billing status became  Payment Due.
   My credit card has no problem.(I checked credit card company.) Please help
   and tell me where to contact.

   Thanks.

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Interrogate a push queue from code?

2011-09-20 Thread Rishi Arora

I have a scenario in my app where I'd like to be able to do that too.  But
instead of looking for an API to do that, I resorted to using a pull queue
instead.  A backend processes my pull queue a few times in a day, but
between those times, I'd like to be able to query the pull queue and gather
some statistics based on the payload in all the queued pull tasks.  I do
that by leasing the tasks for a short period of time, so I can peak into
them.  And then I don't delete the tasks.  GAE doesn't delete my tasks after
I lease them either, because I have set the retry count to default=0.  Using
pull queues also had the advantage for me because it allowed me to utilize
my backend within the free 9 hour quota, and free up those Instance Hours
from front-ends, and eventually result in a lower instance hour bill.


On Tue, Sep 20, 2011 at 2:09 AM, Emlyn emlynore...@gmail.com wrote:

 Is there a way to interrogate a push queue from code?

 ie: list the tasks in the queue, and maybe get the status of each task?

 Or, is there a way, given a task object, to get the current status of the
 task?

 --
 Emlyn

 http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
 Buzz posts,
 comments and all.
 http://point7.wordpress.com - My blog
 Find me on Facebook and Buzz

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE documentation can't be downloaded. 404 Error.

2011-09-20 Thread Ian Marshall

I've raised issue Nº 5939 Documentation download link is broken at

 
http://code.google.com/p/googleappengine/issues/detail?id=5939sort=-id%20prioritycolspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log

since I encountered this issue this morning.


On Sep 12, 10:27 pm, Ernesto Oltra ernestoka...@gmail.com wrote:
 The link in this page:
  http://code.google.com/intl/en/appengine/downloads.html

 to the new documentation:

 http://googleappengine.googlecode.com/files/google-appengine-docs-201...

 throws a 404 error.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Apps For Domains is a MAJOR failing of AppEngine

2011-09-20 Thread Joshua Smith

Greg,

While you have them on the phone, please ask them to figure out why the viewer
doesn't work with blobstore-served .docx files like this one:

https://docs.google.com/viewer?url=http%3A//www.mytowngovernment.org/download%3Fdocument%3Dag50b3duZ292ZXJubWVudHIVCxINRG9jdW1lbnRNb2RlbBiEh1EM

But it works if you download the .docx files and upload it back to the viewer:

http://www.mytowngovernment.org/download?document=ag50b3duZ292ZXJubWVudHIVCxINRG9jdW1lbnRNb2RlbBiEh1EM

On Sep 19, 2011, at 8:57 PM, Gregory D'alesandre wrote:

Well it is the right place to rant in so far as we do depend on Apps for
Domains and I can pass this feedback along internally to ensure they
understand the external perception.

Thanks for the feedback, I'll make sure it gets to the right folks...

Greg

On Mon, Sep 19, 2011 at 5:46 PM, Jeff Schnitzer j...@infohazard.org wrote:
On Mon, Sep 19, 2011 at 5:45 PM, Jeff Schnitzer j...@infohazard.org wrote:
I'd like to give a huge +1 to this. The Google Apps system is a trainwreck.
I made the terrible mistake of moving some of my personal domains to Google
Apps and now:

* I'm in some sort of Catch-22 that locks me out of my adwords account. Not
a big deal (it wasn't important) but I can't create a new adwords account
(seriously, not even by creating new Google Accounts) and I can't delete the
old one. I like to think I'm a smart guy, but this is worse than one of
those chinese puzzles because I'm not 100% certain there is a solution.
Support? Nonexistant.

That should be *adsense*, not adwords. But yeah.

Jeff

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

2011-09-20 Thread Mike Wesner

I don't think Ikai read your post...

Robert and I wanted to write a little HRD status site to track this
and get real data, but we haven't done so yet.  I have never seen the
replication take more than about 1s.  I think 1s will cover about four
9's, but that is just an educated guess.  Until we (the users)
actually measure this over time I don't think we can know for sure.

-Mike

On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
 I know that an index update in the HRD will typically be visible
 within a couple seconds.  That's the average case.  What is the
 worst-case?

 Assuming something in the datacenter goes wacky, how long might it
 take for an index to update?  Tens of seconds, minutes, hours, days?

 Thanks,
 Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Worst-case scenario for eventual consistency in the HRD?

I think Jeff was actually asking about index lag -- how long before
the indexes will be updated, not how long for the data to replicate.
I'd like to know this info too.


Robert






On Mon, Sep 19, 2011 at 20:55, Ikai Lan (Google) ika...@google.com wrote:
 I'll check for you, but FWIW here are the last numbers for master/slave I
 heard with regards to replication delay:
 - most of the time data is replicated within hundreds of milliseconds
 - when there is something wrong, mean time is 3 minutes, with an upper bound
 that is roughly 10 minutes
 If a data center goes offline, that's the window of writes you may lose on
 master/slave. On HRD you don't lose data.
 I'll double check with the datastore team to see if we have numbers, but it
 might not work the same way. When you do a write, a majority of datastore
 instances have to acknowledge receiving the write and having appended it to
 the write journal. Thus, if the primary datastore goes offline, application
 servers make RPCs to the other datastores and use the first response that
 comes back. The datastores that are running behind still try to catch up in
 the background and continue to apply writes from the journal. I suppose the
 number you're looking for here is: what is replication delay if a datastore
 isn't forced to catch up?

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai


 On Mon, Sep 19, 2011 at 5:16 PM, Jeff Schnitzer j...@infohazard.org wrote:

 I know that an index update in the HRD will typically be visible
 within a couple seconds.  That's the average case.  What is the
 worst-case?

 Assuming something in the datacenter goes wacky, how long might it
 take for an index to update?  Tens of seconds, minutes, hours, days?

 Thanks,
 Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

2011-09-20 Thread Mike Wesner

And then I went and used the word replication... i meant index lag.

On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
 I don't think Ikai read your post...

 Robert and I wanted to write a little HRD status site to track this
 and get real data, but we haven't done so yet.  I have never seen the
 replication take more than about 1s.  I think 1s will cover about four
 9's, but that is just an educated guess.  Until we (the users)
 actually measure this over time I don't think we can know for sure.

 -Mike

 On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:







  I know that an index update in the HRD will typically be visible
  within a couple seconds.  That's the average case.  What is the
  worst-case?

  Assuming something in the datacenter goes wacky, how long might it
  take for an index to update?  Tens of seconds, minutes, hours, days?

  Thanks,
  Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to find out Timedifference

2011-09-20 Thread Opcenter [N1] Support

Hi,
i am new in GAE.  I need to find out timedifference between two
dates. i am from java background. i tried with  java.util.Date package . it
didnt come correct. plz help me. How to find out time difference between two
dates.

 Thanks
   Noorul

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: gaeutilities sessions, plz to help a n00b

2011-09-20 Thread bowman.jos...@gmail.com

The purpose of restricting logins to one session is to avoid session 
hijacking. gaeutilities has features that help your site avoid session 
hijacking which have been made even easier with tools like Firesheep - 
http://codebutler.com/firesheep

Since (as of last I checked) you can't use ssl when using your own domains 
cookie sniffing is simple for appengine apps.

Sure, other libraries are faster, and if all you care about is performance, 
then I'd suggest using them. The only reason to choose gaeutilities is it 
was written with security prioritized over performance, therefore is more 
secure than the other libraries. Not to say it's secure, without ssl it's 
not truly secure, but it's much more difficult to spoof a gaeutilities 
session if configured correctly.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/XWaPWJ54gt8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Write Ops incurred during Model.put()

Hi,

AFAIK, when you do a put() it will update the hole entity. that been said it 
will also update all the indexes related to it.

Hope this helps!

Jose Montes de Oca

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/L9blWm9jwz8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Write Ops incurred during Model.put()

2011-09-20 Thread Simon Knott

Hi,

According to Alfred, who's a Googler who appears to know lots about the 
datastore, only the updated values cause index write operations - see 
https://groups.google.com/d/msg/google-appengine/mjnSqQWOfqU/cgPVeHbrR8oJ 
for more info.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/OcSq1WpF5noJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Write Ops incurred during Model.put()

Thanks for bringing up that.

Indexes updated are property changed related.

I think that clears alex question. ;)

FYI, if you have properties you will never filter or sort on in a query, 
consider *indexed=False* to avoid this write ops

Best,
Jose

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Q1T6WiOEPgUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

2011-09-20 Thread Ikai Lan (Google)

Well, indexes are just Bigtable rows, so replication lag does apply to them
as well.

--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai



On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:

 And then I went and used the word replication... i meant index lag.

 On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
  I don't think Ikai read your post...
 
  Robert and I wanted to write a little HRD status site to track this
  and get real data, but we haven't done so yet.  I have never seen the
  replication take more than about 1s.  I think 1s will cover about four
  9's, but that is just an educated guess.  Until we (the users)
  actually measure this over time I don't think we can know for sure.
 
  -Mike
 
  On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
 
 
 
 
 
 
 
   I know that an index update in the HRD will typically be visible
   within a couple seconds.  That's the average case.  What is the
   worst-case?
 
   Assuming something in the datacenter goes wacky, how long might it
   take for an index to update?  Tens of seconds, minutes, hours, days?
 
   Thanks,
   Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Frontend vs Backend Instances for Task/Crons processing

2011-09-20 Thread Jay Meydad

I have gathered as much information as possible from the new pricing FAQ, 
the optimization document by Alon Levi and some posts on the blogesphere.

Here is what I know:
* Frontend instances cost x2 compared to Backend
* According to the post-preview pricing FAQ:
   - Do Frontend instances handle Task Queues and Cron? Frontend instances 
handle Task Queue Requests by default. 
 
* According to Alon Levi's optimization tips article:
   - The default settings for the Task Queue are tuned for performance.
   - Use Backend in order to completely control the number of instances 
used for task execution

Currently my app uses a series of tasks for offline fetching  processing of 
data from 3rd party APIs (Twitter, Facebook). 
- A cron job creates several tasks for fetching data - each task perform a 
single urlfetch and storse the response json in a temp file. It then creates 
a different task that loads the json from the text file, processes it and 
creates  stores necessary data objects into the data store. I am not using 
Backends at the moment.

I am trying to figure out which option will be cheaper - should switch to 
using Backends in order to completely control the number of instances or 
should I rely on app engine's scheduler  use the performance knobs?

Does anyone have suggestion/tips/insight?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/sW0JLfAsoW8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE Full-Text Search Faceting

2011-09-20 Thread Christina Ilvento

Hi Bryce,

For the initial release we are not planning to include faceting,
however it's certainly on our minds for subsequent releases.

Thanks,
Christina

On Sep 15, 12:49 pm, Bryce C pandas...@gmail.com wrote:
 Will the official GAE Full-Text Search supportfaceting?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Google

2011-09-20 Thread Ryan Mattison

For being geniuses, your documentation and organizational skills are
less than desirable.  You choose to take on multiple IDEs/environments
and release them untested.  Try picking one, we all have Linux, Mac,
Windows.  State the person that wrote this documentation is on XXYY
version xy, so it must work as he expected (we are testing other
environments, but we know this one works!).  I've developed against a
load of sdks web, mobile, data, os, windows etc etc, mostly because I
like to have an extremely good time,can't pull my life together, need
money and lack the overall skills to be a engineer.  One thing is
certain though, I would rather jab my eyes out then work through the
next Google project.   I have to be employed in America, why do you
have to make it painful.   Not everyone gets off on digging through
scripts, open source code, and logs to decipher why this pos decided
to lock x file and not rebuild it  write nothing to a log etc.  Some
of us want to build something cool and go to the bar.

Lastly, why the verbosity?  Clear  concise patterns  code never seem
to be the aim.   Its not like the code needs to be clear it'll take a
pick ax to get through a relatively new Google environment anyway.
You may as well finish it off and let developers type less.

   Composite, C
   Calendar, Ca

Let me imagine I got the chemistry degree I  should have.

Alright, better now .. back to jabbing my eyes out.   Your responding/
discussion skills are way above bar though, so that is good !~

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] App serving HTTP 500 out of nowhere

2011-09-20 Thread Bernd Final

I've had this issue for 5 or 6 times already. My app is running for a
long time and I've had no problems so far, except this one:

Out of nowhere, 20-30 seconds latency and only serving HTTP 500 errors
for 99,9% of all requests. In most Servlets (yeah I'm using Java if
that matters) I'm accessing the memcache, sometimes the datastore
(Master/Slave). As you can imagine the app is totally unusable when
this happens. When it occured the first time I didn't to any special,
just waited for a couple of hours until the app recovered on it's own.

Dashboard Milliseconds/Request Chart:
Yesterday: http://bit.ly/pSF66E
Last 30 Days: http://bit.ly/nYdZ2S

When it happend again and again I tried to figure out what's going on,
luckily I had some kind of Admin-Servlet deployed which allowed me to
clear the entire memcache. Tried it, and voila, everything back to
normal.

- The memcache size was up to 1.3-1.5 MB in total (I did a rough
estimation with the ByteArrayOutputStream trick). 12 Objects are
stored in the memcache. Each Object is POJO which implements
Serializable and contains some Strings, Integers and ArrayLists.
- My first idea was to minimize the memory usage, because I though I
was hitting the 1MB limit. At this time I didn't know the limit is for
each object not for the whole cache, anyway I was able to reduce the
size by 30-40% so the size is now about 800kb - 1MB
- I'm running a cron job which cleans out garbage of the memcache.
- This is how I access the memcache on every request:
CacheManager.getInstance().getCacheFactory().createCache(Collections.emptyMap());
- I'm using put and get Methods for storing / retrieving,
- The key length is about 160 bytes


Unfortunately I was not able to solve this issue yet, I would be very
happy for ANY input.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Application Identity problem in Google App

2011-09-20 Thread Thein Htike

I'd like to use @Embeddable annotation. So, it suggest to define as
Application Identity. I created as the Data Nucleus Documentation. But
it still error. Here is the Category.java

import java.util.List;
import javax.persistence.CascadeType;
import javax.persistence.Embeddable;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.OneToMany;

@Entity
@Embeddable
public class Category{
 @Id
 @GeneratedValue(strategy=GenerationType.IDENTITY)
 private String id;

 private String name;

 @OneToMany(mappedBy=category,cascade=CascadeType.ALL)
 private ListProduct products;

 public ListProduct getProducts() {
 return products;
 }

public void setProducts(ListProduct products) {
this.products = products;
 }

 public Category() {
 }

 public Category(String id,String name){
  this.id=id;
  this.name=name;
 }

 public String getId() {
 return id;
 }

 public void setId(String id) {
 this.id = id;
 }

 public String getName() {
 return name;
 }

 public void setName(String name) {
 this.name = name;
 }
 }

Error is

Errors were encountered when initialising the specified MetaData.
See the nested exceptions for details SEVERE: Class entity.Category
has been specified with 1 primary key fields, but this class is using
nondurable identity and should be application identity.

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Communication between Django nonrel (GAE) and Android

2011-09-20 Thread Blue

Hi!

I'm building an application that has Android client application and
uses Django nonrel (GAE) as server. There will be some kind of web
based client application also, but it will do only basic stuff (some
kind of viewer).
Any recommendations how can i connect those two platforms?

Thanks for answers!
With kind regards,
Gregor

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Write Ops incurred during Model.put()

2011-09-20 Thread Alex Epshteyn

So the Write Ops value in the new dev_appserver's Datastore Viewer
applies only to creating new entities of a kind, updating the entity i
is presumably much cheaper if only a few properties are changed?

On Tue, Sep 20, 2011 at 12:27 PM, Simon Knott knott.si...@gmail.com wrote:
 Hi,

 According to Alfred, who's a Googler who appears to know lots about the
 datastore, only the updated values cause index write operations - see
 https://groups.google.com/d/msg/google-appengine/mjnSqQWOfqU/cgPVeHbrR8oJ
 for more info.

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/OcSq1WpF5noJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Frontend vs Backend Instances for Task/Crons processing

2011-09-20 Thread Jason Collins

It's pretty subjective, but we tried moving our task-driven frontend
load to dynamic backend instances. We found that, perhaps due to the
nature of our workload, the dynamic backend pools never shrunk - e.g.,
if I configured 5 instances, 5 were always active. This had the effect
of costing a lot more instance-hours of billing than simply using
frontend instances, so we rolled back. Additionally, there would have
been the extra overhead of manually monitoring to determine if 5
instances is the appropriate number to be processing our workload in a
timely fashion (yes, I realize my conflicting statements here of
wanting to be able to control my instance-hours while at the same time
having some automation around the number of instances to serve the
load).

For us to successfully use backends, I think we'd need more dials.
E.g., I'd like to be able to tell the backend scheduler to not start a
new instance until the incoming request queues have a certain level of
pressure (e.g., # of tasks waiting, or estimated wait time in queue)
and conversely, some way to tell the scheduler to ramp down instances
when this pressure becomes too low.

j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

I'm doing a lot of work lately with data that requires a large degree
of transactional consistency.  One pattern I've found that makes some
of the pain of HRD eventuality go away is to add an extra entity that
uses your query field as a natural key.  This really requires global
transactions to work (as announced, it's in trusted testing, wheee!)
but here's an example:

Say you associate a facebook id with an account.  In M/S, you'd
probably have something like this:

class User {
@Id Long id;
long fbId;
...
}

...and then when a request arrives with a facebook id, you would query
for the user record.  No user record?  Create one.  With eventual
consistency, this creates a larger window (with M/S it was small)
where you can get duplicate Users for the same fbId.

The solution to transactional integrity and strong consistency is to
add a FbId entity:

class FbId {
@Id String fbId;
long userId;
}

I've now got several of these mapping entities in place now.  Using
global transactions to create the FbId and the User at the same time,
it seems to solve consistency issues entirely.  I don't know how it
will perform yet under load, but obviously there's not heavy
contention in this situation so I would be surprised if the 2pc hurt
much.

I'm starting to notice several of these FbId-type mapping objects
showing up in my code as a way to force queries (for unique items)
into strong consistency.  I'm guessing you could do this for
multi-item queries using a list property instead:

Instead of query(Thing.class).filter(color, someColor), you could
instead keep updating an entity like this:

class ColorThings {
   @Id String color;
   ListKeyThing things;
}

...which feels upside-down but really has a lot of advantages.  If you
put ColorThings in memcache, it's like a query cache which actually
updates properly.

Is anyone else noticing their code being pushed into this pattern by the HRD?

Jeff

On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.com wrote:
 Well, indexes are just Bigtable rows, so replication lag does apply to them
 as well.
 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai


 On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:

 And then I went and used the word replication... i meant index lag.

 On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
  I don't think Ikai read your post...
 
  Robert and I wanted to write a little HRD status site to track this
  and get real data, but we haven't done so yet.  I have never seen the
  replication take more than about 1s.  I think 1s will cover about four
  9's, but that is just an educated guess.  Until we (the users)
  actually measure this over time I don't think we can know for sure.
 
  -Mike
 
  On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
 
 
 
 
 
 
 
   I know that an index update in the HRD will typically be visible
   within a couple seconds.  That's the average case.  What is the
   worst-case?
 
   Assuming something in the datacenter goes wacky, how long might it
   take for an index to update?  Tens of seconds, minutes, hours, days?
 
   Thanks,
   Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Google

Wow. I've found the GAE docs to be pretty nice.  What some pain?  Try
the docs for:

 * Amazon Elastic Beanstalk
 * Paypal
 * Google Checkout
 * Amazon Payments
 * require.js

...just to name a few that have given me grief lately.  Paypal gets
the poo-colored star here.  They are truly a testament to how hard it
is to unseat a market-leading incumbent, even when they suck.

Jeff

On Mon, Sep 19, 2011 at 8:48 PM, Ryan Mattison rmattis...@gmail.com wrote:
 For being geniuses, your documentation and organizational skills are
 less than desirable.  You choose to take on multiple IDEs/environments
 and release them untested.  Try picking one, we all have Linux, Mac,
 Windows.  State the person that wrote this documentation is on XXYY
 version xy, so it must work as he expected (we are testing other
 environments, but we know this one works!).  I've developed against a
 load of sdks web, mobile, data, os, windows etc etc, mostly because I
 like to have an extremely good time,can't pull my life together, need
 money and lack the overall skills to be a engineer.  One thing is
 certain though, I would rather jab my eyes out then work through the
 next Google project.   I have to be employed in America, why do you
 have to make it painful.   Not everyone gets off on digging through
 scripts, open source code, and logs to decipher why this pos decided
 to lock x file and not rebuild it  write nothing to a log etc.  Some
 of us want to build something cool and go to the bar.

 Lastly, why the verbosity?  Clear  concise patterns  code never seem
 to be the aim.   Its not like the code needs to be clear it'll take a
 pick ax to get through a relatively new Google environment anyway.
 You may as well finish it off and let developers type less.

   Composite, C
   Calendar, Ca

 Let me imagine I got the chemistry degree I  should have.

 Alright, better now .. back to jabbing my eyes out.   Your responding/
 discussion skills are way above bar though, so that is good !~

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

2011-09-20 Thread Alfred Fuller

An interesting notion. Although you could also just
use ColorThings(key_name=color) as the parent entity for all the Things.
This way the list of things would be queriable directly (using
an ancestor query) and there would not be a limit on the number and size of
Things. They also exist next to each other in the underlying big table so
there is only one 'seek' to find them (which is the largest cost when
looking things up if you don't count serialization).

On Tue, Sep 20, 2011 at 12:37 PM, Jeff Schnitzer j...@infohazard.orgwrote:

 I'm doing a lot of work lately with data that requires a large degree
 of transactional consistency.  One pattern I've found that makes some
 of the pain of HRD eventuality go away is to add an extra entity that
 uses your query field as a natural key.  This really requires global
 transactions to work (as announced, it's in trusted testing, wheee!)
 but here's an example:

 Say you associate a facebook id with an account.  In M/S, you'd
 probably have something like this:

 class User {
@Id Long id;
long fbId;
...
 }

 ...and then when a request arrives with a facebook id, you would query
 for the user record.  No user record?  Create one.  With eventual
 consistency, this creates a larger window (with M/S it was small)
 where you can get duplicate Users for the same fbId.

 The solution to transactional integrity and strong consistency is to
 add a FbId entity:

 class FbId {
@Id String fbId;
long userId;
 }

 I've now got several of these mapping entities in place now.  Using
 global transactions to create the FbId and the User at the same time,
 it seems to solve consistency issues entirely.  I don't know how it
 will perform yet under load, but obviously there's not heavy
 contention in this situation so I would be surprised if the 2pc hurt
 much.

 I'm starting to notice several of these FbId-type mapping objects
 showing up in my code as a way to force queries (for unique items)
 into strong consistency.  I'm guessing you could do this for
 multi-item queries using a list property instead:

 Instead of query(Thing.class).filter(color, someColor), you could
 instead keep updating an entity like this:

 class ColorThings {
   @Id String color;
   ListKeyThing things;
 }

 ...which feels upside-down but really has a lot of advantages.  If you
 put ColorThings in memcache, it's like a query cache which actually
 updates properly.

 Is anyone else noticing their code being pushed into this pattern by the
 HRD?

 Jeff

 On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.com
 wrote:
  Well, indexes are just Bigtable rows, so replication lag does apply to
 them
  as well.
  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  plus.ikailan.com | twitter.com/ikai
 
 
  On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:
 
  And then I went and used the word replication... i meant index lag.
 
  On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
   I don't think Ikai read your post...
  
   Robert and I wanted to write a little HRD status site to track this
   and get real data, but we haven't done so yet.  I have never seen the
   replication take more than about 1s.  I think 1s will cover about four
   9's, but that is just an educated guess.  Until we (the users)
   actually measure this over time I don't think we can know for sure.
  
   -Mike
  
   On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
  
  
  
  
  
  
  
I know that an index update in the HRD will typically be visible
within a couple seconds.  That's the average case.  What is the
worst-case?
  
Assuming something in the datacenter goes wacky, how long might it
take for an index to update?  Tens of seconds, minutes, hours, days?
  
Thanks,
Jeff
 
  --
  You received this message because you are subscribed to the Google
 Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 
 
  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are

[google-appengine] Blobstore OutOfMemoryError

2011-09-20 Thread steflem

Hi,

today i ran the blobstore example from

http://code.google.com/intl/de-DE/appengine/docs/java/blobstore/overview.html#Complete_Sample_App

When i want to upload a large file (~700MB), i get the following
error:

20.09.2011 21:30:35 com.google.apphosting.utils.jetty.JettyLogger warn
WARNUNG: Error for /_ah/upload/
aglub19hcHBfaWRyGwsSFV9fQmxvYlVwbG9hZFNlc3Npb25fXxgCDA
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at
com.google.appengine.repackaged.com.google.common.io.ByteStreams.copy(ByteStreams.java:
172)
at
com.google.apphosting.utils.servlet.MultipartMimeUtils.parseMultipartRequest(MultipartMimeUtils.java:
39)
at
com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(UploadBlobServlet.java:
130)
at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access
$000(UploadBlobServlet.java:68)
at com.google.appengine.api.blobstore.dev.UploadBlobServlet
$1.run(UploadBlobServlet.java:97)
at java.security.AccessController.doPrivileged(Native Method)
at
com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobServlet.java:
94)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
511)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1166)
at
com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:
35)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:
58)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:
122)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.BackendServersFilter.doFilter(BackendServersFilter.java:
97)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
418)
at
com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:
70)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
152)
at com.google.appengine.tools.development.JettyContainerService
$ApiProxyHandler.handle(JettyContainerService.java:351)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
152)

What's wrong there?

Stefan

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Java Memcache Persistence Manager encapsulation

2011-09-20 Thread Daniel VE

Hello,

I'm new using GAE. I'm not sure if the way I'm developing my first app
is correct in the abstraction aproach and usage of this both, the
memcache and persistence manager. Can you give me any feedback? Is
that correct? Thank you!

This is what I do:

- PersistenLayer class - Singleton of PersistenManagerFactory to get
PersistenManagers

- CacheLayer class - Singleton Cache class with put and get methods

- PersistentCachedEntity - A base class to extend from that
encanpsulates the work with memcache and datastore, using the two
previous classes.

- MyClass - extending PersistentCachedEntity all my classes I want
that work as if objects always were in mem.


This let me works like that way:
--

MyClass a = MyClass.getById(70);
a.setAttributeA(value 1);
a.update();

MyClass b = new MyClass();
b.setAttributeA(value 1);
b.setAttributeB(value 2);
b.update();
int new_object_id = b.getId();





-- The code of PersistenCachedEntity class


public class PersistentCachedEntity implements Serializable{


private static final long serialVersionUID = 3622746186696241522L;

public PersistentCachedEntity(){}


public static PersistentCachedEntity getById(Class? what_class,
long id) throws Exception{

PersistentCachedEntity entity = getCached(what_class,id);

if(entity==null){
entity = getPersistent(what_class,id);
if(entity==null){
throw new Exception(Persistent Object of
+what_class.toString()+ not found. ID: +id);
}else{
cacheUpdate(entity);
}
}else{
System.out.println(From Memcache);
}

return entity;
}


public long update(){

try{
persistentUpdate(this);
cacheUpdate(this);
}catch(Exception e){
System.out.println(e);
}

return 1;
}


private static void cacheUpdate(PersistentCachedEntity entity) throws
Exception{
String memkey = entity.getClass().getSimpleName()
+_ID_+entity.getId();
CacheLayer.put(memkey,(Object)entity);
}

private static void persistentUpdate(PersistentCachedEntity entity){
PersistenceManager pm = 
PersistenceLayer.getPersistenceManager();
try {
pm.makePersistent(entity);
} finally {
pm.close();
}

}

private static PersistentCachedEntity getCached(Class? what_class,
long id){
String memkey = what_class.getSimpleName()+_ID_+id;
PersistentCachedEntity entity = (PersistentCachedEntity)
CacheLayer.get(memkey);
return entity;
}

private static PersistentCachedEntity getPersistent(Class?
what_class, long id){

PersistentCachedEntity entity;

try{
PersistenceManager pm = 
PersistenceLayer.getPersistenceManager();
entity = (PersistentCachedEntity)
pm.getObjectById(what_class,id);
pm.close();
}catch(JDOObjectNotFoundException exception){

entity = null;
}
return entity;
}

public Key getKey(){
return null;
}

public long getId() throws Exception{

Key key = getKey();

if(key!=null){
return key.getId();
}else{
throw new Exception(Persistent class has no valid ID. 
Not yet
persisted, update it first.);
}
}

}



---

 The MyClass class code


@PersistenceCapable
public class EntityA extends PersistentEntity{

//Stuff to make persistence works
///
private static final long serialVersionUID = 1L;

@PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Key key = null;

public static EntityA getById(int id) throws Exception{
return (EntityA) PersistentEntity.getById(EntityA.class, id);
}

public Key getKey() {
return key;
}


@Persistent
private String attribute;


public void setAttribute(String value) {
this.attribute = value;
}

public String getAttribute() {
return value;
}

}

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To

[google-appengine] Re: Java Memcache Persistence Manager encapsulation

2011-09-20 Thread Gerald Tan

I've worked on something similar, but found out I've totally wasted my time 
when i came across http://code.google.com/p/objectify-appengine/

Try it out, and don't look back

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/53wNpEmilP4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Java Memcache Persistence Manager encapsulation

2011-09-20 Thread Pascal Voitot Dev

Yes exactly and if you want something working with other DB also, look at
http://www.sienaproject.com
I'm lead dev of this project so don't hesitate to ask questions!

regards
Pascal

On Tue, Sep 20, 2011 at 10:17 PM, Gerald Tan woefulwab...@gmail.com wrote:

 I've worked on something similar, but found out I've totally wasted my time
 when i came across http://code.google.com/p/objectify-appengine/

 Try it out, and don't look back

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/53wNpEmilP4J.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to find out Timedifference

2011-09-20 Thread Gerald Tan

There's java.util.TimeZone

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/ydDXWXLPREEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

2011-09-20 Thread Alfred Fuller

Ikai is correct to think about replication in this case. In a single replica
you could have one of three states:

Applied - fully visible
Committed - has the log entry, but has yet to apply it
Missing - the log entry has yet to be replicated

Only in the first case is it visible to a global query. When you write
something, the log is committed to at least a majority of replicas. The
datastore returns success, then immediately tries to apply the write
everywhere it committed the log entry. It usually takes a couple hundred ms
to apply. This is why the majority of cases take O(100 ms) to become
visible. For a very small % of writes, the write either cannot commit to the
local replica or cannot be applied after the commit. In these cases the
datastore will still return success, but the write won't be visible until a
background process picks it up and applies it. In these case it can take
O(minutes) to be picked up and replicated/applied. If there is something
wrong in the replica you are querying (for example replication is backed up
or the bigtabale is unavailable or the background processes in that replica
are having issues), then it could take a deal longer (this becomes very very
unlikely very quickly, but not impossible). There really is no hard upper
bounds because distributed systems will have pieces that fail (and are
designed to still function when they do).

 - Alfred

On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.comwrote:

 Well, indexes are just Bigtable rows, so replication lag does apply to them
 as well.

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai



 On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:

 And then I went and used the word replication... i meant index lag.

 On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
  I don't think Ikai read your post...
 
  Robert and I wanted to write a little HRD status site to track this
  and get real data, but we haven't done so yet.  I have never seen the
  replication take more than about 1s.  I think 1s will cover about four
  9's, but that is just an educated guess.  Until we (the users)
  actually measure this over time I don't think we can know for sure.
 
  -Mike
 
  On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
 
 
 
 
 
 
 
   I know that an index update in the HRD will typically be visible
   within a couple seconds.  That's the average case.  What is the
   worst-case?
 
   Assuming something in the datacenter goes wacky, how long might it
   take for an index to update?  Tens of seconds, minutes, hours, days?
 
   Thanks,
   Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Frontend vs Backend Instances for Task/Crons processing

2011-09-20 Thread Rishi Arora

With backend instances, the full scheduling control can be achieved by
using pull queues.  So you could design your system to enqueue deferable
latency-tolerant tasks to a pull queue.  When front end instance determines
that enough tasks have been queued up in the pull queue, it sends a
trigger task to a push queue to wake up one or more backends, depending
upon cost considerations.  These backends start leasing tasks from the pull
queue until the queue is fully depleted.  After the last task is processed,
the backend stays idle for 15 minutes before being shutdown.  Driving a
backend solely through push queues makes the backend instance hours more
unpredictable, and such task queues are more suitable for front-ends.

I think I try to answer the backend / front-end question like this:  Is a
task latency sensitive?  Is the task predictably periodic?  If the latency
requirement for a task is around 50ms through a couple of seconds, and the
task is driven by user -facing requests, you should use a front-end.  If the
latencies of several minutes or hours can be tolerated, and the task is more
predictable in the future, then a backend is better.  One example is - lets
say my app connects to twitter and download my feeds every hour.  So, the
task of downloading a tweet can tolerate latencies of 60 minutes (I don't
need to see a tweet instantly - when it is published). So, I would configure
a backend to download my tweets periodically.  The number of backends would
depend upon how many people I follow.  If I follow just a handful of people
and downloading tweets will take just a few minutes every hour, then one
backend is sufficient.  If downloading tweets for million+ users, and it
takes several hours to do, if done serially, I'd configure multiple backends
to download them all in parallel within the one hour interval.

Also, remember that backends will cost the same as front ends starting
December 1st, when the 50% python discount expires.



On Tue, Sep 20, 2011 at 2:25 PM, Jason Collins jason.a.coll...@gmail.comwrote:

 It's pretty subjective, but we tried moving our task-driven frontend
 load to dynamic backend instances. We found that, perhaps due to the
 nature of our workload, the dynamic backend pools never shrunk - e.g.,
 if I configured 5 instances, 5 were always active. This had the effect
 of costing a lot more instance-hours of billing than simply using
 frontend instances, so we rolled back. Additionally, there would have
 been the extra overhead of manually monitoring to determine if 5
 instances is the appropriate number to be processing our workload in a
 timely fashion (yes, I realize my conflicting statements here of
 wanting to be able to control my instance-hours while at the same time
 having some automation around the number of instances to serve the
 load).

 For us to successfully use backends, I think we'd need more dials.
 E.g., I'd like to be able to tell the backend scheduler to not start a
 new instance until the incoming request queues have a certain level of
 pressure (e.g., # of tasks waiting, or estimated wait time in queue)
 and conversely, some way to tell the scheduler to ramp down instances
 when this pressure becomes too low.

 j

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

The problem with using a key-parent is that it limits to a single
index -- say I want to index Things by color and texture.

The downside of this multiple-thing index entity is that (like a
parent-key) it limits throughput.  And since there's a 2pc involved,
it probably limits throughput quite a lot...

Jeff

On Tue, Sep 20, 2011 at 1:03 PM, Alfred Fuller
arfuller+appeng...@google.com wrote:
 An interesting notion. Although you could also just
 use ColorThings(key_name=color) as the parent entity for all the Things.
 This way the list of things would be queriable directly (using
 an ancestor query) and there would not be a limit on the number and size of
 Things. They also exist next to each other in the underlying big table so
 there is only one 'seek' to find them (which is the largest cost when
 looking things up if you don't count serialization).

 On Tue, Sep 20, 2011 at 12:37 PM, Jeff Schnitzer j...@infohazard.org
 wrote:

 I'm doing a lot of work lately with data that requires a large degree
 of transactional consistency.  One pattern I've found that makes some
 of the pain of HRD eventuality go away is to add an extra entity that
 uses your query field as a natural key.  This really requires global
 transactions to work (as announced, it's in trusted testing, wheee!)
 but here's an example:

 Say you associate a facebook id with an account.  In M/S, you'd
 probably have something like this:

 class User {
    @Id Long id;
    long fbId;
    ...
 }

 ...and then when a request arrives with a facebook id, you would query
 for the user record.  No user record?  Create one.  With eventual
 consistency, this creates a larger window (with M/S it was small)
 where you can get duplicate Users for the same fbId.

 The solution to transactional integrity and strong consistency is to
 add a FbId entity:

 class FbId {
    @Id String fbId;
    long userId;
 }

 I've now got several of these mapping entities in place now.  Using
 global transactions to create the FbId and the User at the same time,
 it seems to solve consistency issues entirely.  I don't know how it
 will perform yet under load, but obviously there's not heavy
 contention in this situation so I would be surprised if the 2pc hurt
 much.

 I'm starting to notice several of these FbId-type mapping objects
 showing up in my code as a way to force queries (for unique items)
 into strong consistency.  I'm guessing you could do this for
 multi-item queries using a list property instead:

 Instead of query(Thing.class).filter(color, someColor), you could
 instead keep updating an entity like this:

 class ColorThings {
   @Id String color;
   ListKeyThing things;
 }

 ...which feels upside-down but really has a lot of advantages.  If you
 put ColorThings in memcache, it's like a query cache which actually
 updates properly.

 Is anyone else noticing their code being pushed into this pattern by the
 HRD?

 Jeff

 On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.com
 wrote:
  Well, indexes are just Bigtable rows, so replication lag does apply to
  them
  as well.
  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  plus.ikailan.com | twitter.com/ikai
 
 
  On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:
 
  And then I went and used the word replication... i meant index lag.
 
  On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
   I don't think Ikai read your post...
  
   Robert and I wanted to write a little HRD status site to track this
   and get real data, but we haven't done so yet.  I have never seen the
   replication take more than about 1s.  I think 1s will cover about
   four
   9's, but that is just an educated guess.  Until we (the users)
   actually measure this over time I don't think we can know for sure.
  
   -Mike
  
   On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
  
  
  
  
  
  
  
I know that an index update in the HRD will typically be visible
within a couple seconds.  That's the average case.  What is the
worst-case?
  
Assuming something in the datacenter goes wacky, how long might it
take for an index to update?  Tens of seconds, minutes, hours,
days?
  
Thanks,
Jeff
 
  --
  You received this message because you are subscribed to the Google
  Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 
 
  --
  You received this message because you are subscribed to the Google
  Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

Thanks... while I didn't follow it exactly, I get the gist of what's
going on.  Sounds like I should expect five- or six-sigma
probabilities of minute+ eventuality in global query indexes.

Jeff

On Tue, Sep 20, 2011 at 1:28 PM, Alfred Fuller
arfuller+appeng...@google.com wrote:
 Ikai is correct to think about replication in this case. In a single replica
 you could have one of three states:
 Applied - fully visible
 Committed - has the log entry, but has yet to apply it
 Missing - the log entry has yet to be replicated
 Only in the first case is it visible to a global query. When you write
 something, the log is committed to at least a majority of replicas. The
 datastore returns success, then immediately tries to apply the write
 everywhere it committed the log entry. It usually takes a couple hundred ms
 to apply. This is why the majority of cases take O(100 ms) to become
 visible. For a very small % of writes, the write either cannot commit to the
 local replica or cannot be applied after the commit. In these cases the
 datastore will still return success, but the write won't be visible until a
 background process picks it up and applies it. In these case it can take
 O(minutes) to be picked up and replicated/applied. If there is something
 wrong in the replica you are querying (for example replication is backed up
 or the bigtabale is unavailable or the background processes in that replica
 are having issues), then it could take a deal longer (this becomes very very
 unlikely very quickly, but not impossible). There really is no hard upper
 bounds because distributed systems will have pieces that fail (and are
 designed to still function when they do).
  - Alfred
 On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.com
 wrote:

 Well, indexes are just Bigtable rows, so replication lag does apply to
 them as well.
 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai


 On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:

 And then I went and used the word replication... i meant index lag.

 On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
  I don't think Ikai read your post...
 
  Robert and I wanted to write a little HRD status site to track this
  and get real data, but we haven't done so yet.  I have never seen the
  replication take more than about 1s.  I think 1s will cover about four
  9's, but that is just an educated guess.  Until we (the users)
  actually measure this over time I don't think we can know for sure.
 
  -Mike
 
  On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
 
 
 
 
 
 
 
   I know that an index update in the HRD will typically be visible
   within a couple seconds.  That's the average case.  What is the
   worst-case?
 
   Assuming something in the datacenter goes wacky, how long might it
   take for an index to update?  Tens of seconds, minutes, hours, days?
 
   Thanks,
   Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Java Memcache Persistence Manager encapsulation

2011-09-20 Thread Daniel VE

Thanks a lot!

On Sep 20, 10:19 pm, Pascal Voitot Dev pascal.voitot@gmail.com
wrote:
 Yes exactly and if you want something working with other DB also, look 
 athttp://www.sienaproject.com
 I'm lead dev of this project so don't hesitate to ask questions!

 regards
 Pascal







 On Tue, Sep 20, 2011 at 10:17 PM, Gerald Tan woefulwab...@gmail.com wrote:
  I've worked on something similar, but found out I've totally wasted my time
  when i came acrosshttp://code.google.com/p/objectify-appengine/

  Try it out, and don't look back

   --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/53wNpEmilP4J.

  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] I set the max idle instances to 3, but the number of idle instance is always 1.

2011-09-20 Thread Tapir

This make some requests load very slow.

What the max idle instances means? Is it means keep 3 idle
instances?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Faster than get_by_key_name

2011-09-20 Thread yamadaag

Hello.

I often use get_by_key_name to get entities from Datastore.
I'm looking for faster way to get entities, does anyone know?



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Faster than get_by_key_name

2011-09-20 Thread Steve Sherrie

get_by_key_name first computes a db.Key from the key_name and then 
fetches the entity by that key, so it's already the quickest method. 
There's no practical time cost in the computing of the key.


Steve Sherrie

On 11-09-20 06:16 PM, yamadaag wrote:

Hello.

I often use get_by_key_name to get entities from Datastore.
I'm looking for faster way to get entities, does anyone know?





--
You received this message because you are subscribed to the Google Groups Google 
App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Apps For Domains is a MAJOR failing of AppEngine

2011-09-20 Thread vlad

The right question to ask Greg is why did you tie GAE into GApps at all? My 
perception is that vast majority of GAE developers do not need and will 
never use GApps. Domain management is minor feature which you could have put 
on a Admin Console page. I always thought that coupling GAE to GApps was a 
very microsoft'y styled move. Bill is smiling on you guys.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/PHWLfZH8pKwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Some Best Practices for Apps For Domains with GAE [Was Apps For Domains is a MAJOR failing of AppEngine]

2011-09-20 Thread Brandon Wirtz

This started as a Response to Vlad, and then I decided to make it more of a
10 things to know about Apps For Domains, and I ended up with 11.

 

I'm not a Googler, but I can guess, that on average Apps For Domains makes
more money for Goog, than GAE does.  If Google can get you to sign up for
email on 20 users, that adds up to a lot of money, and the tracking cookies
on those users on adsense pages adds up to even more.  More data is more
money. 

 

Apps For Domains wasn't horrible until recently, the interface keeps getting
more GoDaddy like. Which may be Familiar to those of use with lots of
domains, but is a very bad thing.  I also don't think Apps For Domains likes
GAE users.  You can almost hear the Tech support people in that group moan
when they hear you have an issue with GAE. (at one point my subdomains all
disappeared and wouldn't let me redeploy them because they were in use and
Apps for Domains told me it was a Code Problem)

 

If you are setting up Apps for domain, I have over the last month or so
decided that you should follow some simple rules.

 

1.   Don't set the admin account to a user account you use for anything
else.  This makes the email notifications harder to see, but will save you
unified login headaches over, and over.  

2.   Pony up the $10 for a paid account. This sucks, especially for test
domains, but you don't get support otherwise. Sometimes you don't get
support even if you paid, but being able to at least call someone will make
your life less of a Pain when something goes wrong.

3.   Validate your domain using the DNS TXT entry not the HTML upload.
If your HTML file becomes unreachable you may suddenly be unable to connect.

4.   While you can have multiple domains with one account, as the
primary if you ever sell that domain you can't un-associate it, which is a
PITA (More for the buyer than the seller) so don't do this.

5.   Don't enable Multiple Logins on your Google Logins for any of
your Apps For Domains admin accounts.  While this seems to be well tested
for GMAIL and Adsense, sometimes sessions get messed up and you will find
that you are making changes to the wrong account.

6.   Use Incognito Windows when administrating your Apps For Domains
Accounts. Similar to above sessions don't seem to work write in Apps For
domain.

7.   Don't Administer accounts in IE9 or FireFox. Google seems to test
only against Chrome, and some of the Dynamic elements on the page don't
always line up in IE9 and FireFox.

8.   Don't even login to Apps For Domains via a Mobile Device or Android
Tablet.  It seems the Pre-Caching on these is a little over-zealous and will
often select all the options on some pages. (Awesome Sauce let me tell you)

9.   Don't use Root, Or Admin for the administrative Email User Name

10.   Don't Use the same Password across all your Apps For domains accounts.

11.   Login to your account altleast once a month, the TOS and deployed apps
change and sometimes if you miss a change things will just happen without
your knowledge.

 

 


Brandon Wirtz 
BlackWaterOps: President / Lead Mercenary 

Description: http://www.linkedin.com/img/signature/bg_slate_385x42.jpg



Work: 510-992-6548 
Toll Free: 866-400-4536 

IM: drak...@gmail.com (Google Talk) 
Skype: drakegreene 

 http://www.blackwaterops.com/ BlackWater Ops 




 

 

 

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

image001.jpg

[google-appengine] Re: Add User Problems

Did you login in first to appengine.google.com with the new account before 
adding it to the permission list ?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/izBtrCQsS6AJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: about error django 1.2 was requested, but 0.96.4.None is already in use

Are you adding that code in the beginning of your script handler your 
application use?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/AEteuU1eg70J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Why does appcfg.py request_logs not accept java as a valid runtime in app.yaml?

To download request logs for your java app use this command line instead:

/appengine-java-sdk/bin/appcfg.sh request_logs myapp/war mylogs.txt

More 
information: 
http://code.google.com/appengine/docs/java/tools/uploadinganapp.html#Downloading_Logs

Hope this helps!

Best,
Jose Montes de Oca

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/lXHg5X9FMGIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to add search term to the url of the handler?

Hi,

When using a form with POST, the attributes are not send thru the URL but 
are within the message body of the request. 

If you would like your app to react to an URL of the 
type: /searchhandler?search_string=[search string]  you will need to code 
the get() method for the handler. If you access that URL from your browsers, 
it will send the request as a GET method.

Hope this helps!

Jose Montes de Oca

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/9Wmk_8LPzUkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Faster than get_by_key_name

2011-09-20 Thread Stephen Johnson

If you're using HRD and using STRONG consistency, you can increase the speed
of the get by key by using EVENTUAL consistency. Of course with the possible
ramifications that incur with using EVENTUAL consistency.

Stephen

On Tue, Sep 20, 2011 at 3:21 PM, Steve Sherrie st...@wasteofpaper.comwrote:

get_by_key_name first computes a db.Key from the key_name and then fetches
the entity by that key, so it's already the quickest method. There's no
practical time cost in the computing of the key.

Steve Sherrie

On 11-09-20 06:16 PM, yamadaag wrote:

Hello.

I often use get_by_key_name to get entities from Datastore.
I'm looking for faster way to get entities, does anyone know?

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to
google-appengine@googlegroups.**comgoogle-appengine@googlegroups.com
.
To unsubscribe from this group, send email to
google-appengine+unsubscribe@**googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at http://groups.google.com/**
group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en
.

[google-appengine] Re: Add User Problems

2011-09-20 Thread Rori Stumpf

I don't remember... sorry.

On Sep 20, 7:26 pm, Jose Montes de Oca jfmontesde...@google.com
wrote:
 Did you login in first to appengine.google.com with the new account before
 adding it to the permission list ?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Some Best Practices for Apps For Domains with GAE [Was Apps For Domains is a MAJOR failing of AppEngine]

2011-09-20 Thread JH

For a while now I have been UNABLE to DELETE a google app engine app
from my google apps domain.  I had multiple old M/S apps that google
apps will never seem to let me delete...

On Sep 20, 6:25 pm, Brandon Wirtz drak...@digerat.com wrote:
 This started as a Response to Vlad, and then I decided to make it more of a
 10 things to know about Apps For Domains, and I ended up with 11.

 I'm not a Googler, but I can guess, that on average Apps For Domains makes
 more money for Goog, than GAE does.  If Google can get you to sign up for
 email on 20 users, that adds up to a lot of money, and the tracking cookies
 on those users on adsense pages adds up to even more.  More data is more
 money.

 Apps For Domains wasn't horrible until recently, the interface keeps getting
 more GoDaddy like. Which may be Familiar to those of use with lots of
 domains, but is a very bad thing.  I also don't think Apps For Domains likes
 GAE users.  You can almost hear the Tech support people in that group moan
 when they hear you have an issue with GAE. (at one point my subdomains all
 disappeared and wouldn't let me redeploy them because they were in use and
 Apps for Domains told me it was a Code Problem)

 If you are setting up Apps for domain, I have over the last month or so
 decided that you should follow some simple rules.

 1.       Don't set the admin account to a user account you use for anything
 else.  This makes the email notifications harder to see, but will save you
 unified login headaches over, and over.  

 2.       Pony up the $10 for a paid account. This sucks, especially for test
 domains, but you don't get support otherwise. Sometimes you don't get
 support even if you paid, but being able to at least call someone will make
 your life less of a Pain when something goes wrong.

 3.       Validate your domain using the DNS TXT entry not the HTML upload.
 If your HTML file becomes unreachable you may suddenly be unable to connect.

 4.       While you can have multiple domains with one account, as the
 primary if you ever sell that domain you can't un-associate it, which is a
 PITA (More for the buyer than the seller) so don't do this.

 5.       Don't enable Multiple Logins on your Google Logins for any of
 your Apps For Domains admin accounts.  While this seems to be well tested
 for GMAIL and Adsense, sometimes sessions get messed up and you will find
 that you are making changes to the wrong account.

 6.       Use Incognito Windows when administrating your Apps For Domains
 Accounts. Similar to above sessions don't seem to work write in Apps For
 domain.

 7.       Don't Administer accounts in IE9 or FireFox. Google seems to test
 only against Chrome, and some of the Dynamic elements on the page don't
 always line up in IE9 and FireFox.

 8.       Don't even login to Apps For Domains via a Mobile Device or Android
 Tablet.  It seems the Pre-Caching on these is a little over-zealous and will
 often select all the options on some pages. (Awesome Sauce let me tell you)

 9.       Don't use Root, Or Admin for the administrative Email User Name

 10.   Don't Use the same Password across all your Apps For domains accounts.

 11.   Login to your account altleast once a month, the TOS and deployed apps
 change and sometimes if you miss a change things will just happen without
 your knowledge.

 Brandon Wirtz
 BlackWaterOps: President / Lead Mercenary

 Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg

 Work: 510-992-6548
 Toll Free: 866-400-4536

 IM: drak...@gmail.com (Google Talk)
 Skype: drakegreene

  http://www.blackwaterops.com/ BlackWater Ops

  image001.jpg
  1KViewDownload

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: about error django 1.2 was requested, but 0.96.4.None is already in use

2011-09-20 Thread roberto.cr

as Jose has said, put the code you pasted here in the beginning of the
script
some stuff load django code without you knowing, like some stuff from
google.appengine.ext.webapp if I remember correctly

On Sep 20, 8:31 pm, Jose Montes de Oca jfmontesde...@google.com
wrote:
 Are you adding that code in the beginning of your script handler your
 application use?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

I've been using the same pattern as Jeff mentions for quite some time
-- even while I was on M/S.  I use it because it reduces my problems
to fetch by key scenarios, and I can build multiple specialized
indexes in this way.  Part of the reason I started doing this was
due to exploding indexes type issues; this lets me control the
explosion, and possibly even defer the writes in some cases.

It also allows you to avoid contention issues when the Things are
frequently updated, but the indexed values may not be.


Robert



On Tue, Sep 20, 2011 at 15:03, Alfred Fuller
arfuller+appeng...@google.com wrote:
 An interesting notion. Although you could also just
 use ColorThings(key_name=color) as the parent entity for all the Things.
 This way the list of things would be queriable directly (using
 an ancestor query) and there would not be a limit on the number and size of
 Things. They also exist next to each other in the underlying big table so
 there is only one 'seek' to find them (which is the largest cost when
 looking things up if you don't count serialization).

 On Tue, Sep 20, 2011 at 12:37 PM, Jeff Schnitzer j...@infohazard.org
 wrote:

 I'm doing a lot of work lately with data that requires a large degree
 of transactional consistency.  One pattern I've found that makes some
 of the pain of HRD eventuality go away is to add an extra entity that
 uses your query field as a natural key.  This really requires global
 transactions to work (as announced, it's in trusted testing, wheee!)
 but here's an example:

 Say you associate a facebook id with an account.  In M/S, you'd
 probably have something like this:

 class User {
    @Id Long id;
    long fbId;
    ...
 }

 ...and then when a request arrives with a facebook id, you would query
 for the user record.  No user record?  Create one.  With eventual
 consistency, this creates a larger window (with M/S it was small)
 where you can get duplicate Users for the same fbId.

 The solution to transactional integrity and strong consistency is to
 add a FbId entity:

 class FbId {
    @Id String fbId;
    long userId;
 }

 I've now got several of these mapping entities in place now.  Using
 global transactions to create the FbId and the User at the same time,
 it seems to solve consistency issues entirely.  I don't know how it
 will perform yet under load, but obviously there's not heavy
 contention in this situation so I would be surprised if the 2pc hurt
 much.

 I'm starting to notice several of these FbId-type mapping objects
 showing up in my code as a way to force queries (for unique items)
 into strong consistency.  I'm guessing you could do this for
 multi-item queries using a list property instead:

 Instead of query(Thing.class).filter(color, someColor), you could
 instead keep updating an entity like this:

 class ColorThings {
   @Id String color;
   ListKeyThing things;
 }

 ...which feels upside-down but really has a lot of advantages.  If you
 put ColorThings in memcache, it's like a query cache which actually
 updates properly.

 Is anyone else noticing their code being pushed into this pattern by the
 HRD?

 Jeff

 On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.com
 wrote:
  Well, indexes are just Bigtable rows, so replication lag does apply to
  them
  as well.
  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  plus.ikailan.com | twitter.com/ikai
 
 
  On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:
 
  And then I went and used the word replication... i meant index lag.
 
  On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
   I don't think Ikai read your post...
  
   Robert and I wanted to write a little HRD status site to track this
   and get real data, but we haven't done so yet.  I have never seen the
   replication take more than about 1s.  I think 1s will cover about
   four
   9's, but that is just an educated guess.  Until we (the users)
   actually measure this over time I don't think we can know for sure.
  
   -Mike
  
   On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
  
  
  
  
  
  
  
I know that an index update in the HRD will typically be visible
within a couple seconds.  That's the average case.  What is the
worst-case?
  
Assuming something in the datacenter goes wacky, how long might it
take for an index to update?  Tens of seconds, minutes, hours,
days?
  
Thanks,
Jeff
 
  --
  You received this message because you are subscribed to the Google
  Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 
 
  --
  You received this message because you are subscribed to the Google
  Groups
  Google App Engine group.
  To post to this group, send email to

Re: [google-appengine] Re: Worst-case scenario for eventual consistency in the HRD?

I get that indexes are just bigtable rows too, and that the normal
replication rules we all know and love apply, so I guess this boils
down to indexes being written separately from the entity.  Does the
index write apply to the same nodes, or possibly to different nodes?

Alfred, your next project idea: write some type of low-level
high-performance batcher providing crazy high write-rates to a single
entity group.  Perhaps with that you (or we) could come up with a
higher performance way to maintain global indexes.  ;)


Robert

Oh, for those wondering what this thread is about... we're just making
up words / phrases.





On Tue, Sep 20, 2011 at 15:28, Alfred Fuller
arfuller+appeng...@google.com wrote:
 Ikai is correct to think about replication in this case. In a single replica
 you could have one of three states:
 Applied - fully visible
 Committed - has the log entry, but has yet to apply it
 Missing - the log entry has yet to be replicated
 Only in the first case is it visible to a global query. When you write
 something, the log is committed to at least a majority of replicas. The
 datastore returns success, then immediately tries to apply the write
 everywhere it committed the log entry. It usually takes a couple hundred ms
 to apply. This is why the majority of cases take O(100 ms) to become
 visible. For a very small % of writes, the write either cannot commit to the
 local replica or cannot be applied after the commit. In these cases the
 datastore will still return success, but the write won't be visible until a
 background process picks it up and applies it. In these case it can take
 O(minutes) to be picked up and replicated/applied. If there is something
 wrong in the replica you are querying (for example replication is backed up
 or the bigtabale is unavailable or the background processes in that replica
 are having issues), then it could take a deal longer (this becomes very very
 unlikely very quickly, but not impossible). There really is no hard upper
 bounds because distributed systems will have pieces that fail (and are
 designed to still function when they do).
  - Alfred
 On Tue, Sep 20, 2011 at 10:10 AM, Ikai Lan (Google) ika...@google.com
 wrote:

 Well, indexes are just Bigtable rows, so replication lag does apply to
 them as well.
 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai


 On Tue, Sep 20, 2011 at 7:42 AM, Mike Wesner mbwes...@gmail.com wrote:

 And then I went and used the word replication... i meant index lag.

 On Sep 20, 9:40 am, Mike Wesner mbwes...@gmail.com wrote:
  I don't think Ikai read your post...
 
  Robert and I wanted to write a little HRD status site to track this
  and get real data, but we haven't done so yet.  I have never seen the
  replication take more than about 1s.  I think 1s will cover about four
  9's, but that is just an educated guess.  Until we (the users)
  actually measure this over time I don't think we can know for sure.
 
  -Mike
 
  On Sep 19, 7:16 pm, Jeff Schnitzer j...@infohazard.org wrote:
 
 
 
 
 
 
 
   I know that an index update in the HRD will typically be visible
   within a couple seconds.  That's the average case.  What is the
   worst-case?
 
   Assuming something in the datacenter goes wacky, how long might it
   take for an index to update?  Tens of seconds, minutes, hours, days?
 
   Thanks,
   Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Faster than get_by_key_name

2011-09-20 Thread stevep

I switched to EVENTUAL where ever possible, and realized significant
gains virtually every time. Worth working hard to manage it IMHO.

stevep

On Sep 20, 5:58 pm, Stephen Johnson onepagewo...@gmail.com wrote:
 If you're using HRD and using STRONG consistency, you can increase the speed
 of the get by key by using EVENTUAL consistency. Of course with the possible
 ramifications that incur with using EVENTUAL consistency.

 Stephen

 On Tue, Sep 20, 2011 at 3:21 PM, Steve Sherrie st...@wasteofpaper.comwrote:







  get_by_key_name first computes a db.Key from the key_name and then fetches
  the entity by that key, so it's already the quickest method. There's no
  practical time cost in the computing of the key.

  Steve Sherrie

  On 11-09-20 06:16 PM, yamadaag wrote:

  Hello.

  I often use get_by_key_name to get entities from Datastore.
  I'm looking for faster way to get entities, does anyone know?

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to 
  google-appengine@googlegroups.**comgoogle-appengine@googlegroups.com
  .
  To unsubscribe from this group, send email to
  google-appengine+unsubscribe@**googlegroups.comgoogle-appengine%2Bunsubscr 
  i...@googlegroups.com
  .
  For more options, visit this group athttp://groups.google.com/**
  group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en
  .

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Google

2011-09-20 Thread stevep

Agreed. I'd like to see more from Google about architectural
considerations, but compared to FBook Credit docs G's are heavenly.

stevep

On Sep 20, 12:45 pm, Jeff Schnitzer j...@infohazard.org wrote:
 Wow. I've found the GAE docs to be pretty nice.  What some pain?  Try
 the docs for:

  * Amazon Elastic Beanstalk
  * Paypal
  * Google Checkout
  * Amazon Payments
  * require.js

 ...just to name a few that have given me grief lately.  Paypal gets
 the poo-colored star here.  They are truly a testament to how hard it
 is to unseat a market-leading incumbent, even when they suck.

 Jeff







 On Mon, Sep 19, 2011 at 8:48 PM, Ryan Mattison rmattis...@gmail.com wrote:
  For being geniuses, your documentation and organizational skills are
  less than desirable.  You choose to take on multiple IDEs/environments
  and release them untested.  Try picking one, we all have Linux, Mac,
  Windows.  State the person that wrote this documentation is on XXYY
  version xy, so it must work as he expected (we are testing other
  environments, but we know this one works!).  I've developed against a
  load of sdks web, mobile, data, os, windows etc etc, mostly because I
  like to have an extremely good time,can't pull my life together, need
  money and lack the overall skills to be a engineer.  One thing is
  certain though, I would rather jab my eyes out then work through the
  next Google project.   I have to be employed in America, why do you
  have to make it painful.   Not everyone gets off on digging through
  scripts, open source code, and logs to decipher why this pos decided
  to lock x file and not rebuild it  write nothing to a log etc.  Some
  of us want to build something cool and go to the bar.

  Lastly, why the verbosity?  Clear  concise patterns  code never seem
  to be the aim.   Its not like the code needs to be clear it'll take a
  pick ax to get through a relatively new Google environment anyway.
  You may as well finish it off and let developers type less.

    Composite, C
    Calendar, Ca

  Let me imagine I got the chemistry degree I  should have.

  Alright, better now .. back to jabbing my eyes out.   Your responding/
  discussion skills are way above bar though, so that is good !~

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Some Best Practices for Apps For Domains with GAE [Was Apps For Domains is a MAJOR failing of AppEngine]

2011-09-20 Thread Sameer Lodha

Brandon,

Thanks for taking out time to share your experience  knowledge. This would
save us quite a lot of time  effort.

Sam

On Wed, Sep 21, 2011 at 4:55 AM, Brandon Wirtz drak...@digerat.com wrote:

 This started as a Response to Vlad, and then I decided to make it more of a
 10 things to know about Apps For Domains, and I ended up with 11.

 ** **

 I’m not a Googler, but I can guess, that on average Apps For Domains makes
 more money for Goog, than GAE does.  If Google can get you to sign up for
 email on 20 users, that adds up to a lot of money, and the tracking cookies
 on those users on adsense pages adds up to even more.  More data is more
 money. 

 ** **

 Apps For Domains wasn’t horrible until recently, the interface keeps
 getting more GoDaddy like. Which may be Familiar to those of use with lots
 of domains, but is a very bad thing.  I also don’t think Apps For Domains
 likes GAE users.  You can almost hear the Tech support people in that group
 moan when they hear you have an issue with GAE. (at one point my subdomains
 all disappeared and wouldn’t let me redeploy them because they were in use
 and Apps for Domains told me it was a Code Problem)

 ** **

 If you are setting up Apps for domain, I have over the last month or so
 decided that you should follow some simple rules…

 ** **

 **1.   **Don’t set the admin account to a user account you use for
 anything else.  This makes the email notifications harder to see, but will
 save you unified login headaches over, and over.  

 **2.   **Pony up the $10 for a paid account. This sucks, especially
 for test domains, but you don’t get support otherwise. Sometimes you don’t
 get support even if you paid, but being able to at least call someone will
 make your life less of a Pain when something goes wrong.

 **3.   **Validate your domain using the DNS TXT entry not the HTML
 upload. If your HTML file becomes unreachable you may suddenly be unable to
 connect.

 **4.   **While you can have multiple domains with one account, as the
 primary if you ever sell that domain you can’t un-associate it, which is a
 PITA (More for the buyer than the seller) so don’t do this.

 **5.   **Don’t enable “Multiple Logins” on your Google Logins for any
 of your Apps For Domains admin accounts.  While this seems to be well tested
 for GMAIL and Adsense, sometimes sessions get messed up and you will find
 that you are making changes to the wrong account.

 **6.   **Use Incognito Windows when administrating your Apps For
 Domains Accounts. Similar to above sessions don’t seem to work write in Apps
 For domain.

 **7.   **Don’t Administer accounts in IE9 or FireFox. Google seems to
 test only against Chrome, and some of the Dynamic elements on the page don’t
 always line up in IE9 and FireFox.

 **8.   **Don’t even login to Apps For Domains via a Mobile Device or
 Android Tablet.  It seems the Pre-Caching on these is a little over-zealous
 and will often select all the options on some pages. (Awesome Sauce let me
 tell you)

 **9.   **Don’t use Root, Or Admin for the administrative Email User
 Name

 **10.   **Don’t Use the same Password across all your Apps For domains
 accounts.

 **11.   **Login to your account altleast once a month, the TOS and
 deployed apps change and sometimes if you miss a change things will just
 happen without your knowledge.

 ** **

 ** **

 *Brandon Wirtz
 *BlackWaterOps: President / Lead Mercenary 

 [image: Description:
 http://www.linkedin.com/img/signature/bg_slate_385x42.jpg]

 *Work:* 510-992-6548
 *Toll Free:* 866-400-4536 

 *IM:* drak...@gmail.com (Google Talk)
 *Skype:* drakegreene 

 BlackWater Ops
 http://www.blackwaterops.com/

 ** **

 ** **

 ** **

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

image001.jpg

Re: [google-appengine] Google