[appengine-java] Re: HELP: My app is getting lots of HardDeadlineExceededError today.

2010-08-13 Thread SC2PlayerFinder

Used appstat and didn't notice anything out of the ordinary.  Still
thinking that the main issue here is exploding index and I'm not sure
how to get around that other than to cut search features from the
site, but that would more or less make the app not very useful.

On Aug 12, 10:19 pm, SC2PlayerFinder sc2playerfin...@gmail.com
wrote:
 Thanks for the response.  I wanted to get back to you sooner, but I
 have been scrambling to optimize the application.  I'm making some
 progress, however, it is still a relative CPU hog due to an exploding
 index and the frequent updates the application needs to make in order
 to keep the data current.  71000 CPU to save 100 records seems rather
 expensive?.

 My next step is to run appstat on it and see what is going on.

 Current version of the app is running a sc2pf6.appstat.com. The older
 version is running at sc2pf2.appspot.com.

 On Aug 9, 3:51 pm, Ikai L (Google) ika...@google.com wrote:



  I've got a few questions for you:

  1. What is your application ID?
  2. What are you doing in the tasks?
  3. Are you using transactions or entity groups?
  4. Are you writing to the same datastore entities?

  My guess is that you are running into write contention in the datastore.
  It'd be very helpful to understand what it is you're doing.

  App Engine will scale up the number of instances if you are returning web
  requests in less than 800ms (thought 400ms is optimal), but if you are
  writing to a small number of entities over and over it won't be able to
  resolve that bottleneck.

  On Mon, Aug 9, 2010 at 12:28 PM, SC2PlayerFinder
  sc2playerfin...@gmail.comwrote:

   Further explanation.  I had a large increase in users which caused a
   chain reaction. Tasks that used to take about 10-20 seconds started
   taking longer and using more CPU time. Interestingly, these tasks are
   doing the exact same thing, but under a higher load, they take longer
   and use more CPU time.  I can understand the process taking longer
   under load, but why would it use a significant amount more of CPU
   time?  I will admit that my code is not the most optimal, but then
   again isn't the main focus of GAE built in scalability and speed?  In
   short, I needed on the fly scalability and speed with an influx of new
   users and I don't feel like I got that with GAE?  Worse, there was no
   way for me to contact someone who could help me out.

   I'm left with the feeling that GAE does not do a good job of scaling
   up and it is kind of slow as well.  Creating and updating records
   seems painfully expensive, time out due to new instance creation
   happens way to often.  Now this is fine for a free app, but I'm going
   way over my daily free quota and it is not cheap. Maybe I should be
   using Phython and not Java?

   On Aug 6, 10:52 pm, SC2PlayerFinder sc2playerfin...@gmail.com wrote:
Application was running ok yesturday.  Still had about 10%
HardDeadlineExceededError, but the same task that executed yesturday
are not failing almost 100% of the time right now.  I even cut back
the amount of records it is processing from 100 to 10 and it is still
failing.

How do I get ahold of someone at Google, to see if there is something
wrong with the server my app is on or something. Why would the same
proccess consume more CPU time one day than another?

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine for Java group.
   To post to this group, send email to
   google-appengine-j...@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B
unsubscr...@googlegroups.com
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine-java?hl=en.

  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  Blog:http://googleappengine.blogspot.com
  Twitter:http://twitter.com/app_engine
  Reddit:http://www.reddit.com/r/appengine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] want to specify two different fields in JPA or clause

2010-08-13 Thread Pieter Coucke
You might want to check Query filters here:
http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Introducing_Queries
a query can only have one not-equal filter, and such a query cannot have
other inequality filters

A workaround can be to add two extra boolean properties hasContactGroupId
and hasContactId and query on this (or one boolean property
hasContactGroupOrContact).  One disadvantage here is that you need to
remember to update these fields every time the contact info changes.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Local Unit Testing with UserService: userId==null problem

2010-08-13 Thread Dmitriy T.
No, i didn't fix it. But now I create issue for this:
http://code.google.com/p/googleappengine/issues/detail?id=3579

On Aug 12, 1:37 pm, poe stefan.poe...@googlemail.com wrote:
 I have the same Problem, did you fix your problem? Please tell me how
 you did it.

 Thanks
 Poe

 On 30 Jun., 12:18, Dmitriy T. 403...@gmail.com wrote:

  I use LocalUnitTesting and recently change auth system and start
  using userService.getCurrentUser().getUserId() for identification.
  Problem is that its always return null in Local Testing
  (federatedIdentity==null too). For many reasons that unacceptable for
  me.

  Is that unfixable, or exist something like setUserId for
  LocalServiceTestHelper?



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Shared Datastore via Multiple App Versions Locally

2010-08-13 Thread millr
I should mention that I'm on Mac OS X for development.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Problems with OAuth

2010-08-13 Thread Alex Bertram
Resolved!

The final gotcha was that the authorization token returned by
AppEngine includes a '/' while the client library assumes that this
token will be alphanumeric.

http://code.google.com/p/oauth/issues/detail?id=182

Best,

Alex

On Aug 12, 12:04 am, Ikai L (Google) ika...@google.com wrote:
 You do need to register your application to get a consumer key and secret::

 http://code.google.com/apis/accounts/docs/OAuth.html#prepRegisterhttp://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto

 Here's a tool written by a friend of mine that I used to verify an OAuth
 setup, unfortunately it requires a Ruby on Rails setup ...

 http://github.com/episod/oauth-dancer

 I'm not familiar with .NET, but do you have any other OAuth clients you can
 try?





 On Wed, Aug 11, 2010 at 12:17 AM, Alex Bertram akbert...@gmail.com wrote:
  Hi, I'm trying to use the OAuth service but am running up against a brick
  wall. The client is a desktop application and I'm using the .NET code found
  herehttp://oauth.googlecode.com/svn/code/csharp/OAuthBase.cs

  I've only been able to get 400 Bad Request responses from my app, and the
  requests to _ah/OAuth* do not appear in the logs.

  I've tried both http and https.

  Do I need to register the consumer somewhere?

  Thanks for any pointers!

  Here are the raw request / response from Fiddler:

  POST /_ah/OAuthGetRequestToken HTTP/1.1
  User-Agent: Desktop Addin
  Content-Type: application/x-www-form-urlencoded
  Host: arctileserver.appspot.com
  Content-Length: 230
  Connection: Keep-Alive

  oauth_callback=http%3A%2F%2Farctileserver.appspot.comoauth_consumer_key=
  arctiles.comoauth_nonce=2173727

  HTTP/1.1 400 Bad Request
  Date: Wed, 11 Aug 2010 06:36:37 GMT
  Content-Type: text/html; charset=UTF-8
  Server: Google Frontend
  Content-Length: 273

  htmlhead
  meta http-equiv=content-type content=text/html;charset=utf-8
  title400 Bad Request/title
  /head
  body text=#00 bgcolor=#ff
  h1Error: Bad Request/h1
  h2Your client has issued a malformed or illegal request./h2
  h2/h2
  /body/html

   --
  You received this message because you are subscribed to the Google Groups
  Google App Engine for Java group.
  To post to this group, send email to
  google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B 
  unsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 Blog:http://googleappengine.blogspot.com
 Twitter:http://twitter.com/app_engine
 Reddit:http://www.reddit.com/r/appengine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Problem with one 2 many relationships

2010-08-13 Thread Aleksey Kiselev
Hi, group!!!

Sometimes my GAE application can't persist collection of child
elements.
I have following objects structure.
@PersistenceCapable(table = persons, identityType =
IdentityType.APPLICATION)
public class PersonEntity {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long id;

@Persistent
private String name;

@Persistent
private ListOfferEntity offers = new ArrayListOfferEntity();

// ... getter and setter
}

@PersistenceCapable(table = offers, identityType =
IdentityType.APPLICATION)
public class OfferEntity {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key id;

@Persistent
private ListChoosedElementEntity choosedElements = new
ArrayListChoosedElementEntity();

// ... getter and setter
}

@PersistenceCapable(table = offer_selections_ce, identityType =
IdentityType.APPLICATION)
public class ChoosedElementEntity {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private com.google.appengine.api.datastore.Key id;

@Persistent
private String filed;

// ... getter and setter
}

First of all I try to persist object of class PersonEntity. It is all
right.
PersistenceManager = null;
try {
pm = getPersistenceManager();
PersonEntity person = new PersonEntity();
//  set person fields
pm.makePersistent(person);
} catch (Throwable th) {
} finally {
if (null != pm) {
pm.close();
}
}

Then, I try to add some child elements to OfferEntity. And I am
thoroughly confused: Sometimes child object persists and sometimes
not. This behavior is very strange.
String offerId = ...;
PersistenceManager pm = null;
try {
pm = getPersistenceManager();
OfferEntity dbOffer = pm.getObjectById(OfferEntity.class,
KeyFactory.stringToKey(offerId));
if (null != dbOffer) {
ChoosedElementEntity element = new ChoosedElementEntity();
// set element fields
dbOffer.getChoosedElements().add(element);
log.log(Level.INFO, try to commit 'added choosed elements':
offerId= + dbOffer.getId() + , size:  +
dbOffer.getChoosedElements().size() + , elements:  +
dbOffer.getChoosedElements());
pm.makePersistent(dbOffer);
}
} catch (Throwable th) {
log.log(Level.SEVERE, Can't add choosed elements to offer entity:
offerId= + offerId, th);
throw new ServiceException(Can't add choosed elements to offer
entity: offerId= + offerId, th);
} finally {
if (null != pm) {
pm.close();
}
}

I try to analyze some child information before persist parent object
and I note, that from time to time child element has/hasn't value for
primary key.
Example:
try to commit 'added choosed elements': offerId=persons(1092001)/
offers(1), size: 1, elements:
[ChoosedElementEntity{id=persons(1092001)/offers(1)/
offer_selections_ce(5001), filed='filed765'}]
or
try to commit 'added choosed elements': offerId=persons(1092001)/
offers(1), size: 1, elements:  [ChoosedElementEntity{id=null,
filed='filed766'}]

Any help would be greatly appreciated. Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] update multiple one to many parent (JPA)

2010-08-13 Thread micwolf
Hi guys:

I currently running into issues that: second update for
inventoryDao.addRootCategory(category2); will throw exception that:

Caused by: java.lang.IllegalArgumentException: can't operate on
multiple entity groups in a single transaction. found both Element {
  type: CategoryEntity
  id: 1
}
 and Element {
  type: CategoryEntity
  id: 8
}

anyone can help?

///
public class CategoryEntity extends BaseEntity {
private static final long serialVersionUID = 4289692014710028046L;
private String categoryCode;
private int sortOrder;
//owned
@OneToMany(mappedBy = categoryEntity, cascade = CascadeType.ALL)
private ListProductEntity products = new
ArrayListProductEntity();
//unowned
@OneToMany(cascade = CascadeType.ALL)
private ListNameEntity categoryNames = new
ArrayListNameEntity();
//unowned
@OneToMany(cascade = CascadeType.ALL)
private ListImageEntity images = new ArrayListImageEntity();
//unowned
@OneToMany(cascade = CascadeType.ALL)
private ListAttributeEntity categoryAttributes = new
ArrayListAttributeEntity();
//unowned
@OneToMany(cascade = CascadeType.ALL)
private ListDescriptionEntity categoryDescriptions = new
ArrayListDescriptionEntity();

/

@Entity
@MappedSuperclass
public class BaseEntity implements Serializable {
private static final long serialVersionUID = -1042459872290397535L;
@Transient
public static SimpleDateFormat sdf = new SimpleDateFormat(dd-MM-
hh:mm:ss a);
@Temporal(TemporalType.TIMESTAMP)
private Date creationDate;
private String creationDateString = ;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Key key;

@Version
private long version;

public BaseEntity() {
setCreationDate(Calendar.getInstance().getTime());
}

/**
 * @see java.lang.Object#equals(Object)
 */
@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
BaseEntity other = (BaseEntity) obj;
if (getKey() == null) {
if (other.getKey() != null)
return false;
} else if (!getKey().equals(other.getKey())) {
return false;
}
if (getKey() != null  other.getKey() != null) {
if (getKey().getId() == other.getKey().getId()) {
return true;
} else {
return false;
}
}
return true;
}

public Date getCreationDate() {
return creationDate;
}

public String getCreationDateString() {
return creationDateString;
}

/**
 * @see java.lang.Object#hashCode()
 */
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((getKey() == null) ? 0 :
getKey().hashCode());
return result;
}

public void setCreationDate(Date creationDate) {
this.creationDate = creationDate;
if (null != creationDate) {
this.creationDateString = sdf.format(creationDate);
}
}

public void setCreationDateString(String creationDateString) {
this.creationDateString = creationDateString;
}

public long getVersion() {
return version;
}

public void setVersion(long version) {
this.version = version;
}

public Key getKey() {
return key;
}

public void setKey(Key key) {
this.key = key;
}

}
/
public CategoryEntity addRootCategory(CategoryEntity category) throws
GcommerceException {
if (null != category) {
if (null != category.getKey()) {
throw new
GcommerceEntityAlreadyPersistentException(CategoryEntity.class.getName());
}
try {
getEntityManager().getTransaction().begin();
getEntityManager().persist(category);
getEntityManager().getTransaction().commit();
return category;
} finally {
if 
(getEntityManager().getTransaction().isActive()) {

getEntityManager().getTransaction().rollback();
}
}
} 

[appengine-java] Re: EL works, but eclipse underlines red, and says can't find tag library - solutions?

2010-08-13 Thread somejava
Hi,

here is the post I've made with the solution
http://someprog.blogspot.com/2010/08/google-app-engine-can-not-find-tag.html.
Hope it will help you.

br, sergey

On Aug 9, 11:19 pm, Ikai L (Google) ika...@google.com wrote:
 You know, I've had Eclipse installed across several computers, and I'm not
 sure which versions run what, but I've had JSP work on about half of them.
 I've never looked too much into the cause. Do you have WTP installed? I want
 to say this should solve your issue, except I am fairly certain it works on
 only one of my Eclipse instances. On another note, I've always been pretty
 happy working with JSPs in IntelliJ IDEA and Netbeans. It's only been
 Eclipse that has issues.



 On Sun, Aug 8, 2010 at 2:28 AM, Lou lssay...@gmail.com wrote:
  Hello,

  I've got the following in my jsp in eclipse:

  %@ page language=java contentType=text/html; charset=ISO-8859-1
     pageEncoding=ISO-8859-1 isELIgnored=false %
  %@ taglib uri=http://java.sun.com/jstl/core_rt;  prefix=c %

  All the code like c:out in eclipse works, but
 http://java.sun.com/jstl/core_rt
  is underlined red, and eclipse says can not find the tag library
  descriptor forhttp://java.sun.com/jstl/core_rt;.

  I also don't get any code completion in eclipse, which I suspect is
  because of this.

  Does someone know how to make the errors go away? I'm also not sure
  about thehttp://java.sun.com/jstl/core_rturl as well, as I've seen
  some slightly different urls floating around on the internet.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine for Java group.
  To post to this group, send email to
  google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 Blog:http://googleappengine.blogspot.com
 Twitter:http://twitter.com/app_engine
 Reddit:http://www.reddit.com/r/appengine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: HELP: My app is getting lots of HardDeadlineExceededError today.

2010-08-13 Thread Ikai L (Google)
Saving 100 entities in a single request, especially if a TON of indexes will
be created is very expensive and will cause a lot of CPU to be consumed.
There probably is a design that will allow you to accomplish what you need
to do without this limitation. Can you post your use case?

On Thu, Aug 12, 2010 at 11:02 PM, SC2PlayerFinder sc2playerfin...@gmail.com
 wrote:


 Used appstat and didn't notice anything out of the ordinary.  Still
 thinking that the main issue here is exploding index and I'm not sure
 how to get around that other than to cut search features from the
 site, but that would more or less make the app not very useful.

 On Aug 12, 10:19 pm, SC2PlayerFinder sc2playerfin...@gmail.com
 wrote:
  Thanks for the response.  I wanted to get back to you sooner, but I
  have been scrambling to optimize the application.  I'm making some
  progress, however, it is still a relative CPU hog due to an exploding
  index and the frequent updates the application needs to make in order
  to keep the data current.  71000 CPU to save 100 records seems rather
  expensive?.
 
  My next step is to run appstat on it and see what is going on.
 
  Current version of the app is running a sc2pf6.appstat.com. The older
  version is running at sc2pf2.appspot.com.
 
  On Aug 9, 3:51 pm, Ikai L (Google) ika...@google.com wrote:
 
 
 
   I've got a few questions for you:
 
   1. What is your application ID?
   2. What are you doing in the tasks?
   3. Are you using transactions or entity groups?
   4. Are you writing to the same datastore entities?
 
   My guess is that you are running into write contention in the
 datastore.
   It'd be very helpful to understand what it is you're doing.
 
   App Engine will scale up the number of instances if you are returning
 web
   requests in less than 800ms (thought 400ms is optimal), but if you are
   writing to a small number of entities over and over it won't be able to
   resolve that bottleneck.
 
   On Mon, Aug 9, 2010 at 12:28 PM, SC2PlayerFinder
   sc2playerfin...@gmail.comwrote:
 
Further explanation.  I had a large increase in users which caused a
chain reaction. Tasks that used to take about 10-20 seconds started
taking longer and using more CPU time. Interestingly, these tasks are
doing the exact same thing, but under a higher load, they take longer
and use more CPU time.  I can understand the process taking longer
under load, but why would it use a significant amount more of CPU
time?  I will admit that my code is not the most optimal, but then
again isn't the main focus of GAE built in scalability and speed?  In
short, I needed on the fly scalability and speed with an influx of
 new
users and I don't feel like I got that with GAE?  Worse, there was no
way for me to contact someone who could help me out.
 
I'm left with the feeling that GAE does not do a good job of scaling
up and it is kind of slow as well.  Creating and updating records
seems painfully expensive, time out due to new instance creation
happens way to often.  Now this is fine for a free app, but I'm going
way over my daily free quota and it is not cheap. Maybe I should be
using Phython and not Java?
 
On Aug 6, 10:52 pm, SC2PlayerFinder sc2playerfin...@gmail.com
 wrote:
 Application was running ok yesturday.  Still had about 10%
 HardDeadlineExceededError, but the same task that executed
 yesturday
 are not failing almost 100% of the time right now.  I even cut back
 the amount of records it is processing from 100 to 10 and it is
 still
 failing.
 
 How do I get ahold of someone at Google, to see if there is
 something
 wrong with the server my app is on or something. Why would the same
 proccess consume more CPU time one day than another?
 
--
You received this message because you are subscribed to the Google
 Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%2B
 unsubscr...@googlegroups.com
.
For more options, visit this group at
   http://groups.google.com/group/google-appengine-java?hl=en.
 
   --
   Ikai Lan
   Developer Programs Engineer, Google App Engine
   Blog:http://googleappengine.blogspot.com
   Twitter:http://twitter.com/app_engine
   Reddit:http://www.reddit.com/r/appengine

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.





Re: [appengine-java] Shared Datastore via Multiple App Versions Locally

2010-08-13 Thread John Patterson
Your approach works fine for me on OS X.  But the datastore is held in  
memory so only one app can use it at a time.  If you make changes to  
the datastore you also need to wait until you see the log message  
Time to persist datastore... before shutting down the app.


On 13 Aug 2010, at 09:14, millr wrote:


Hello,

I am aware that it is possible to run different applications that
share the same datastore by using different version strings for the
two applications, as detailed here:
http://googleappengine.blogspot.com/2009/06/10-things-you-probably-didnt-know-about.html

Now that I have found this approach I am wondering how I could pool
the data on the development versions of the app that I am running
locally.

I am using JRuby for a rack app and have found that the datastore data
is stored in /WEB-INF/appengine-generated/local_db.bin, so I tried
creating a symlink from the original app to the same location within
the new app I wish to share the datastore with. This didn't work an
produced the following error:

INFO: Failed to load from the backing store, /WEB-INF/appengine-
generated/local_db.bin
java.io.StreamCorruptedException: invalid stream header: 626F6F6B
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:
783)

Does anyone know of ways I can share this data successfully on my
local versions of apps?

Thanks.

--
You received this message because you are subscribed to the Google  
Groups Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com 
.
To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com 
.
For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en 
.




--
You received this message because you are subscribed to the Google Groups Google 
App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: HELP: My app is getting lots of HardDeadlineExceededError today.

2010-08-13 Thread John Patterson


On 14 Aug 2010, at 00:24, Ikai L (Google) wrote:
 index and the frequent updates the application needs to make in  
order
 to keep the data current.  71000 CPU to save 100 records seems  
rather

 expensive?.


My App uses 600,000 CPUms to save 100 entities so I don't think that  
is expensive at all!  However updating an existing entity takes much  
less cpu, presumably because the datastore only updates those indexes  
that have changed.  I have split entities into parts that change often  
and parts that don't so updates are cheaper.


--
You received this message because you are subscribed to the Google Groups Google 
App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] How to send data back to client side code?

2010-08-13 Thread Sree ...
Am not sure this belongs to GWT or GAE or both ...

So here's my problem...

I have one client/Example.java [contains all GWT code] calls a methods on
server/SomeServiceImpl.java [Contains code to talk to dataStore]

Wat server/SomeServiceImpl.java does is Featch data from Employee.Java
  [this is class used to store data in datastore JPA]

So now my problem is.. as long as I send data in the form of String or
StringArray to client/Example.java am fine, but now I tried to send
ListEmployee to client/Example.java

Doing so throws me an error as below..

Validating newly compiled units
  [ERROR] Errors in
'file:/C:/XXX/XXX/src/xxx/client/GreetingServiceAsync.java'
 [ERROR] Line 18: No source code is available for type
xxx.server.Employee; did you forget to inherit a required module?
  [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/Example.java'
 [ERROR] Line 325: No source code is available for type
xxx.server.Employee; did you forget to inherit a required module?
  [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/GreetingService.java'
 [ERROR] Line 19: No source code is available for type
xxx.server.Employee; did you forget to inherit a required module?
   Finding entry point classes
  [ERROR] Unable to find type 'xxx.client.Example'
 [ERROR] Hint: Previous compiler errors may have made this type
unavailable
 [ERROR] Hint: Check the inheritance chain from your module; it may
not be inheriting a required module or a module may not be adding its source
path entries properly



-- 
-Thanks
-Srikanth.G
-Hyderabad

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Guestbook (as-is) works locally, but deployed on gae will give Error: Server Error at signin

2010-08-13 Thread Ikai L (Google)
Do the log files provide any information? What's the URL to your
application?

On Thu, Aug 12, 2010 at 3:48 AM, Simpatico gabri...@mysimpatico.com wrote:

 I've deployed the guestbook as it's provided in the demos folder but
 when I click to sign in I get:

 Error: Server Error
 The server encountered an error and could not complete your request.

 If the problem persists, please report your problem and mention this
 error message and the query that caused it.

 This is essentially what I've been getting throughout. Indeed the
 result is the same for the sample code:

 public class GuestbookServlet extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse
 resp)
  throws IOException {
UserService userService = UserServiceFactory.getUserService();
User user = userService.getCurrentUser();

if (user != null) {
resp.setContentType(text/plain);
resp.getWriter().println(Hello,  + user.getNickname());
} else {

 resp.sendRedirect(userService.createLoginURL(req.getRequestURI()));
}
}
 }

 I suspect it has to do with bug:

 http://code.google.com/p/googleappengine/issues/detail?id=3552can=5colspec=ID%20Type%20Status%20Priority%20Stars%20Owner%20Summary%20Log%20Component

 The bottom line is that I cannot deploy my application!

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




-- 
Ikai Lan
Developer Programs Engineer, Google App Engine
Blog: http://googleappengine.blogspot.com
Twitter: http://twitter.com/app_engine
Reddit: http://www.reddit.com/r/appengine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] How to send data back to client side code?

2010-08-13 Thread Alessandro Barbosa
Hi,

this is a GWT issue. The model class Employee must implements
the com.google.gwt.user.client.rpc.IsSerializable interface in order to be
able to be transcripted between server and client.

Best Regards,
Alessandro

2010/8/13 Sree ... gattasrika...@gmail.com

 Am not sure this belongs to GWT or GAE or both ...

 So here's my problem...

 I have one client/Example.java [contains all GWT code] calls a methods on
 server/SomeServiceImpl.java [Contains code to talk to dataStore]

 Wat server/SomeServiceImpl.java does is Featch data from Employee.Java
   [this is class used to store data in datastore JPA]

 So now my problem is.. as long as I send data in the form of String or
 StringArray to client/Example.java am fine, but now I tried to send
 ListEmployee to client/Example.java

 Doing so throws me an error as below..

 Validating newly compiled units
   [ERROR] Errors in
 'file:/C:/XXX/XXX/src/xxx/client/GreetingServiceAsync.java'
  [ERROR] Line 18: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
   [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/Example.java'
  [ERROR] Line 325: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
   [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/GreetingService.java'
  [ERROR] Line 19: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
Finding entry point classes
   [ERROR] Unable to find type 'xxx.client.Example'
  [ERROR] Hint: Previous compiler errors may have made this type
 unavailable
  [ERROR] Hint: Check the inheritance chain from your module; it may
 not be inheriting a required module or a module may not be adding its source
 path entries properly



 --
 -Thanks
 -Srikanth.G
 -Hyderabad

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] How to send data back to client side code?

2010-08-13 Thread Sree ...
Yes.. after some google search i came to know abt that only
http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Serializable_Objects

http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Serializable_ObjectsBut
am poor little developer so can you please help me out with a small example
say my class looks like
thishttp://code.google.com/appengine/docs/java/datastore/dataclasses.html#Child_Objects_and_Relationships

On Sat, Aug 14, 2010 at 1:35 AM, Alessandro Barbosa 
alessandro.barb...@gmail.com wrote:

 Hi,

 this is a GWT issue. The model class Employee must implements
 the com.google.gwt.user.client.rpc.IsSerializable interface in order to be
 able to be transcripted between server and client.

 Best Regards,
 Alessandro

 2010/8/13 Sree ... gattasrika...@gmail.com

  Am not sure this belongs to GWT or GAE or both ...

 So here's my problem...

 I have one client/Example.java [contains all GWT code] calls a methods on
 server/SomeServiceImpl.java [Contains code to talk to dataStore]

 Wat server/SomeServiceImpl.java does is Featch data from Employee.Java
   [this is class used to store data in datastore JPA]

 So now my problem is.. as long as I send data in the form of String or
 StringArray to client/Example.java am fine, but now I tried to send
 ListEmployee to client/Example.java

 Doing so throws me an error as below..

 Validating newly compiled units
   [ERROR] Errors in
 'file:/C:/XXX/XXX/src/xxx/client/GreetingServiceAsync.java'
  [ERROR] Line 18: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
   [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/Example.java'
  [ERROR] Line 325: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
   [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/GreetingService.java'
  [ERROR] Line 19: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
Finding entry point classes
   [ERROR] Unable to find type 'xxx.client.Example'
  [ERROR] Hint: Previous compiler errors may have made this type
 unavailable
  [ERROR] Hint: Check the inheritance chain from your module; it
 may not be inheriting a required module or a module may not be adding its
 source path entries properly



 --
 -Thanks
 -Srikanth.G
 -Hyderabad

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




-- 
-Thanks
-Srikanth.G
-Google India Ltd
-Hyderabad

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] How to send data back to client side code?

2010-08-13 Thread Alessandro Barbosa
Ok,

try this, move your class to some place inside /src/client and implement
IsSerializable.
I hope this be enough to solve the problem.


import ContactInfo;
// ... imports ...

@PersistenceCapable
public class Employee *implements
com.google.gwt.user.client.rpc.IsSerializable*{
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private ContactInfo myContactInfo;

// ... accessors ...
}

Best Regards,
Alessandro
2010/8/13 Sree ... gattasrika...@gmail.com

 Yes.. after some google search i came to know abt that only

 http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Serializable_Objects


 http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Serializable_ObjectsBut
 am poor little developer so can you please help me out with a small example
 say my class looks like 
 thishttp://code.google.com/appengine/docs/java/datastore/dataclasses.html#Child_Objects_and_Relationships

 On Sat, Aug 14, 2010 at 1:35 AM, Alessandro Barbosa 
 alessandro.barb...@gmail.com wrote:

 Hi,

 this is a GWT issue. The model class Employee must implements
 the com.google.gwt.user.client.rpc.IsSerializable interface in order to be
 able to be transcripted between server and client.

 Best Regards,
 Alessandro

 2010/8/13 Sree ... gattasrika...@gmail.com

  Am not sure this belongs to GWT or GAE or both ...

 So here's my problem...

 I have one client/Example.java [contains all GWT code] calls a methods on
 server/SomeServiceImpl.java [Contains code to talk to dataStore]

 Wat server/SomeServiceImpl.java does is Featch data from Employee.Java
   [this is class used to store data in datastore JPA]

 So now my problem is.. as long as I send data in the form of String or
 StringArray to client/Example.java am fine, but now I tried to send
 ListEmployee to client/Example.java

 Doing so throws me an error as below..

 Validating newly compiled units
   [ERROR] Errors in
 'file:/C:/XXX/XXX/src/xxx/client/GreetingServiceAsync.java'
  [ERROR] Line 18: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
   [ERROR] Errors in 'file:/C:/XXX/XXX/src/client/Example.java'
  [ERROR] Line 325: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
   [ERROR] Errors in
 'file:/C:/XXX/XXX/src/client/GreetingService.java'
  [ERROR] Line 19: No source code is available for type
 xxx.server.Employee; did you forget to inherit a required module?
Finding entry point classes
   [ERROR] Unable to find type 'xxx.client.Example'
  [ERROR] Hint: Previous compiler errors may have made this type
 unavailable
  [ERROR] Hint: Check the inheritance chain from your module; it
 may not be inheriting a required module or a module may not be adding its
 source path entries properly



 --
 -Thanks
 -Srikanth.G
 -Hyderabad

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




 --
 -Thanks
 -Srikanth.G
 -Google India Ltd
 -Hyderabad

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Data store low-level API and query filters

2010-08-13 Thread dominity
Hi, there!

Currently there is no possibility to use two inequality filters for
data retrieving.
I'm storing events with start/end dates. And I want to retrieve events
by date range.
Currently, I'm retrieving them with query with one filter for start
date, so retrieved events are older than start date.
If there is a lot of stored events, query takes a time. Is there any
possibility to reduce this time?
Any suggestions?

Best regards, Alexander.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Using JavaMail to send email to multiple recipients

2010-08-13 Thread Rodrigo
And for completeness, the email-related classes used are:

import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;

All of which exist in Sun's mail.jar I'm using, which is also in the
classpath BEFORE the AppEngine library (just in case).

The bug remains unfixed.

On Aug 12, 8:10 pm, Rodrigo ipi...@gmail.com wrote:
 Well... But I guess my question is:

 How do I override AppEngine's implementation of JavaMail so I can use
 Sun's??

 --

 BTW, this is the code I'm using:

         /**
          * Generic method to send an email
          */
         public static void sendEmail(String fromEmail, String fromName,
                         ListString to, ListString cc, ListString bcc,
                         String subject, String body) throws
                         AddressException, MessagingException, 
 UnsupportedEncodingException
 {

                 Properties props = new Properties();
                 Session session = Session.getDefaultInstance(props, null);
                 Message msg = new MimeMessage(session);
                 msg.setFrom(new InternetAddress(fromEmail, fromName));
                 for (String email : to) {
                         msg.addRecipient(Message.RecipientType.TO,
                                         new InternetAddress(email));
                 }
                 for (String email : cc) {
                         msg.addRecipient(Message.RecipientType.CC,
                                         new InternetAddress(email));
                 }
                 for (String email : bcc) {
                         msg.addRecipient(Message.RecipientType.BCC,
                                         new InternetAddress(email));
                 }
                 msg.setSubject(subject);
                 msg.setText(body);
                 Transport.send(msg);
         }

 On Aug 11, 2:02 pm, Nasif Noorudeen nas...@gmail.com wrote:

  Hi,
    can u specify the code's that you are currntly using, there is something
  is missing in your code, the implementaion of sun sample code is fine

  Thanks
  Nasif

  On Mon, Aug 9, 2010 at 3:41 PM, Rodrigo ipi...@gmail.com wrote:
   Hi,

   I'm trying to send a single email to multiple recipients, but when I
   add multiple recipients to a Message, it sends a different copy of the
   email to each recipient! I want a single email.

   Is there a way to get around this? I read in a previous post that
   Sun's implementation of JavaMail didn't have this problem and that we
   could override it somehow. How?

   I tried downloading a jar from here [1], put it in my classpath and
   the war/WEB-INF/lib, but it didn't make any difference. I also got the
   source code from Sun, but there's a bunch of classes that are
   unsupported by App Engine so I couldn't get it to work either.

   Appreciate any help! Thanks,
   -Rodrigo

   [1]http://download.java.net/maven/2/javax/mail/mail/

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine for Java group.
   To post to this group, send email to
   google-appengine-j...@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine-java?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Bulk loader and JPQL query

2010-08-13 Thread Deepika M
Ok. Thank you.

I'd like to know how to upload the list, in a way that can be used as
ArrayList in java programs. I had specified the type as list in the import
transform specification in the bulk loader configuration file and it is
uploaded as encoded strings.

When I retrieve the list as an Array List in the java program , [u'[', u'1',
u'4'] in the datastore viewer is displayed as [[, 1, 4] .

Thanks,
Deepika

On Mon, Aug 9, 2010 at 3:09 PM, Ikai L (Google) ika...@google.com wrote:

 u'1' means Unicode String 1


 On Thu, Aug 5, 2010 at 6:03 PM, Deepika M drift2elys...@gmail.com wrote:

 My list looks like this in the input csv file (I generated this list usign
 a Java program)
 

[appengine-java] Stats are 0ms in AppStats

2010-08-13 Thread Iain
I always get 0 ms when deployed, anyone know what the issue is?

real=848ms cpu=0ms api=0ms overhead=0ms (2 RPCs)

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: DatastoreFailureException: handle 3 not found

2010-08-13 Thread brendan
I think I might be encountering the same problem.  When I run each of
the tests individually they pass, but running as part of a test suite,
all tests that use Task Queue except the first one fails.

I'm using Deferrable tasks (described here
http://turbomanage.wordpress.com/2009/11/20/deferred-defer-is-a-thing-of-beauty/),
and the first time the code in the second tests tries to add a task to
the queue I get a TransactionalTaskException, which contains a
DatastoreFailureException with the message handle 1 not found;

My test cases extend this class shown at http://pastebin.com/bhuf9FdQ.
Perhaps I'm setting up the Local...TestConfig's incorrectly.

Brendan

On Aug 10, 4:22 am, Jorg jorg.ja...@accorto.com wrote:
 In the test environment, I have a stability problem with the local
 data store.
 When running the tests individually, they are ok, but running them in
 a test suite results in the error below for many of the tests.
 I read that the request come too fast, so I built in sleep(1000)
 between the tests, but that didnotmake a difference.  Any
 suggestions?

 The error I get:

 com.google.appengine.api.datastore.DatastoreFailureException:handle3notfound
                 at
 com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datast 
 oreApiHelper.java:
 51)
                 at
 com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(Datastor 
 eApiHelper.java:
 67)
                 at com.google.appengine.api.datastore.DatastoreServiceImpl
 $2.run(DatastoreServiceImpl.java:188)
                 at
 com.google.appengine.api.datastore.TransactionRunner.runInTransaction(Trans 
 actionRunner.java:
 30)
                 at
 com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServic 
 eImpl.java:
 176)
                 at
 com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServic 
 eImpl.java:
 156)
                 at
 com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServic 
 eImpl.java:
 148)
                 at com.accorto.gae.GaeDatastore.save(GaeDatastore.java:471)

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: HELP: My app is getting lots of HardDeadlineExceededError today.

2010-08-13 Thread SC2PlayerFinder

2 Main parts of the application.

Font End
Display StarCraft 2 Player 1v1 Ladder information from all Regions.
Search options: Order by (6 categories) filter by (Region) filter by
(Race). --  This is the cause of my exploding index.
Display 100 players per page.
I have implemented some cache use for this part, so it is not much of
an issue currently.

Back End (This is the big CPU Hog)
Load Player information from battle.net
http://us.battle.net/sc2/en/profile/388538/1/HuK/ladder/1794#current-rank
This is done by a task that gets the data, parses it and saves any
player changes.
There is 100 players per page, but usually less than 30 need to be
updated each scan.
Each save/update takes almost 1 second. .5 seconds with a version that
has NO INDEXES.

Currently the speed is not an issue, but the cost of CPU time for
Datastore operations is.
Ideally, I would like to support the almost 1 million players on
battle.net like sc2Ranks.com
But that would not be practical cost wise using the Datastore.
For now, I'm only supporting the top 10K players.

Don't get me wrong.  I really love GAE.  I have created a few small
apps that run fine.
However, this app is looking like it won't be practical for GAE due to
the Datastore?


On Aug 13, 1:24 pm, Ikai L (Google) ika...@google.com wrote:
 Saving 100 entities in a single request, especially if a TON of indexes will
 be created is very expensive and will cause a lot of CPU to be consumed.
 There probably is a design that will allow you to accomplish what you need
 to do without this limitation. Can you post your use case?

 On Thu, Aug 12, 2010 at 11:02 PM, SC2PlayerFinder sc2playerfin...@gmail.com





  wrote:

  Used appstat and didn't notice anything out of the ordinary.  Still
  thinking that the main issue here is exploding index and I'm not sure
  how to get around that other than to cut search features from the
  site, but that would more or less make the app not very useful.

  On Aug 12, 10:19 pm, SC2PlayerFinder sc2playerfin...@gmail.com
  wrote:
   Thanks for the response.  I wanted to get back to you sooner, but I
   have been scrambling to optimize the application.  I'm making some
   progress, however, it is still a relative CPU hog due to an exploding
   index and the frequent updates the application needs to make in order
   to keep the data current.  71000 CPU to save 100 records seems rather
   expensive?.

   My next step is to run appstat on it and see what is going on.

   Current version of the app is running a sc2pf6.appstat.com. The older
   version is running at sc2pf2.appspot.com.

   On Aug 9, 3:51 pm, Ikai L (Google) ika...@google.com wrote:

I've got a few questions for you:

1. What is your application ID?
2. What are you doing in the tasks?
3. Are you using transactions or entity groups?
4. Are you writing to the same datastore entities?

My guess is that you are running into write contention in the
  datastore.
It'd be very helpful to understand what it is you're doing.

App Engine will scale up the number of instances if you are returning
  web
requests in less than 800ms (thought 400ms is optimal), but if you are
writing to a small number of entities over and over it won't be able to
resolve that bottleneck.

On Mon, Aug 9, 2010 at 12:28 PM, SC2PlayerFinder
sc2playerfin...@gmail.comwrote:

 Further explanation.  I had a large increase in users which caused a
 chain reaction. Tasks that used to take about 10-20 seconds started
 taking longer and using more CPU time. Interestingly, these tasks are
 doing the exact same thing, but under a higher load, they take longer
 and use more CPU time.  I can understand the process taking longer
 under load, but why would it use a significant amount more of CPU
 time?  I will admit that my code is not the most optimal, but then
 again isn't the main focus of GAE built in scalability and speed?  In
 short, I needed on the fly scalability and speed with an influx of
  new
 users and I don't feel like I got that with GAE?  Worse, there was no
 way for me to contact someone who could help me out.

 I'm left with the feeling that GAE does not do a good job of scaling
 up and it is kind of slow as well.  Creating and updating records
 seems painfully expensive, time out due to new instance creation
 happens way to often.  Now this is fine for a free app, but I'm going
 way over my daily free quota and it is not cheap. Maybe I should be
 using Phython and not Java?

 On Aug 6, 10:52 pm, SC2PlayerFinder sc2playerfin...@gmail.com
  wrote:
  Application was running ok yesturday.  Still had about 10%
  HardDeadlineExceededError, but the same task that executed
  yesturday
  are not failing almost 100% of the time right now.  I even cut back
  the amount of records it is processing from 100 to 10 and it is
  still
  failing.

  How do I get ahold of someone at