[appengine-java] Re: how to save binary data in JDO
You're not limited to 1MB - you're just limited to using 1MB
entities. The GAEVFS link I sent you implies that it's files can span
multiple entities. As Mrityunjay stated, though, large storage
requires payment. See the quotas for details:
http://code.google.com/appengine/docs/quotas.html
Jake
On Mar 1, 12:43 pm, Valentino Hankypants f.hirs...@gmx.at wrote:
so i only can upload binary data up to 1 MB?
any possibility to upload binary data with GBs?
greatz
On 1 Mrz., 18:36, Jake jbrooko...@cast.org wrote:
Hey,
Saving binary data to the datastore isn't too difficult - you can just
use a Blob type and ensure that you don't break the 1MB capacity of a
single entity. You can see my class declaration below. The hard
part is converting a file upload to a byte[] and then from a byte[]
back into what you would consider a downloadable file. That
depends, somewhat, on your implementation. For example, I use the
Wicket architecture and all I need to do is
FileUpload f = fileUploadField.getFileUpload();
BinaryFileData b = new BinaryFileData(f.getBytes());
pm.makePersistent(b);
I presume you are not using Wicket, but some other framework. You'll
need to look into how it handles file uploads.
You may also find this interesting:http://code.google.com/p/gaevfs/
I haven't used it, but it skips the whole byte[] thing entirely.
Jake
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class BinaryFileData implements Serializable {
private static final long serialVersionUID = 1L;
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
@Persistent
private String name;
@Persistent
private String mimeType;
@Persistent(defaultFetchGroup=true)
private Blob data;
public BinaryFileData(byte[] bytes) {
this.data = new Blob(bytes);
}
public byte[] getData() {
return data.getBytes();
}
public void setData(byte[] d) {
this.data = new Blob(d);
}
}
On Mar 1, 11:25 am, Valentino Hankypants f.hirs...@gmx.at wrote:
hello together,
i started working with the app engine some days ago, and now i want to
save some binary data (pdf, etc.) from my app (eg by clicking a
button) in the storage system.
anybody who can help me doing this? (tutorials, etc.)
greatz
flo
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] local testing GAE storage
hello, if i save some data with JDO in the storage. Where is this data stored if i run my app locally? is there any possibility to reset this storage for testing? greatz flo -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] error deleting file in Blob Viewer
I tried to delete a file in Blob Viewer and every time, no matter if I'm attempting to delete one or several, I get this error: Server Error A server error has occurred. Return to Applications screen » -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] form to request being allowed to deploy the application multiple times
I've been Googling all over this forum and the documentation, but I can't find the form I need to submit to the GAE team. I am deploying multiple versions of my app (which are actually different apps) around the same datastore, and it has billing enabled. But I also need to deploy this stuff again around a separate datastore for testing and I don't want the GAE police to shut me down. I remember coming across a form I can fill out to request exemption in my case. Can someone please direct me to that? Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] local testing GAE storage
project folder\war\WEB-INF\appengine-generated\local_db.bin contains your database, you can remove it if the dev server is stopped On Tue, Mar 2, 2010 at 9:52 AM, Valentino Hankypants f.hirs...@gmx.atwrote: hello, if i save some data with JDO in the storage. Where is this data stored if i run my app locally? is there any possibility to reset this storage for testing? greatz flo -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: how to save binary data in JDO
hi jake,
thanks for your replies.
in my case i have jsp-page with a form. the user should have the
possibility to select a file. i used the input type=file
name=test directive from html.
after the user selected a file, he clicks a button. by clicking this
button the binary data of the selected file should be read and saved
in a blob in the GAE storage.
afterwards i'll give the user the possibility to open the uploaded
file by clicking a link...
therefore, should i use this gaevfs or not?
up to now my major problem is, how can i get the binary data from the
form in my servlet class where i handle the form actions? i got all
other informations by HttpServletRequest.getParameter(test);
greatz and hoping for some help ;-)
flo
On 2 Mrz., 15:24, Jake jbrooko...@cast.org wrote:
You're not limited to 1MB - you're just limited to using 1MB
entities. The GAEVFS link I sent you implies that it's files can span
multiple entities. As Mrityunjay stated, though, large storage
requires payment. See the quotas for
details:http://code.google.com/appengine/docs/quotas.html
Jake
On Mar 1, 12:43 pm, Valentino Hankypants f.hirs...@gmx.at wrote:
so i only can upload binary data up to 1 MB?
any possibility to upload binary data with GBs?
greatz
On 1 Mrz., 18:36, Jake jbrooko...@cast.org wrote:
Hey,
Saving binary data to the datastore isn't too difficult - you can just
use a Blob type and ensure that you don't break the 1MB capacity of a
single entity. You can see my class declaration below. The hard
part is converting a file upload to a byte[] and then from a byte[]
back into what you would consider a downloadable file. That
depends, somewhat, on your implementation. For example, I use the
Wicket architecture and all I need to do is
FileUpload f = fileUploadField.getFileUpload();
BinaryFileData b = new BinaryFileData(f.getBytes());
pm.makePersistent(b);
I presume you are not using Wicket, but some other framework. You'll
need to look into how it handles file uploads.
You may also find this interesting:http://code.google.com/p/gaevfs/
I haven't used it, but it skips the whole byte[] thing entirely.
Jake
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class BinaryFileData implements Serializable {
private static final long serialVersionUID = 1L;
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
@Persistent
private String name;
@Persistent
private String mimeType;
@Persistent(defaultFetchGroup=true)
private Blob data;
public BinaryFileData(byte[] bytes) {
this.data = new Blob(bytes);
}
public byte[] getData() {
return data.getBytes();
}
public void setData(byte[] d) {
this.data = new Blob(d);
}
}
On Mar 1, 11:25 am, Valentino Hankypants f.hirs...@gmx.at wrote:
hello together,
i started working with the app engine some days ago, and now i want to
save some binary data (pdf, etc.) from my app (eg by clicking a
button) in the storage system.
anybody who can help me doing this? (tutorials, etc.)
greatz
flo
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: local testing GAE storage
thx karel, worked fine greatz flo On 2 Mrz., 16:20, Karel Alvarez kalvar...@gmail.com wrote: project folder\war\WEB-INF\appengine-generated\local_db.bin contains your database, you can remove it if the dev server is stopped On Tue, Mar 2, 2010 at 9:52 AM, Valentino Hankypants f.hirs...@gmx.atwrote: hello, if i save some data with JDO in the storage. Where is this data stored if i run my app locally? is there any possibility to reset this storage for testing? greatz flo -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: datanucleus enhancer keeps running
See http://code.google.com/p/googleappengine/issues/detail?id=2115. This may
be the issue that you're having. It was recently fixed, and a version of GPE
with the fix is forthcoming.
On Thu, Feb 25, 2010 at 1:57 PM, Steve Pritchard steve...@gmail.com wrote:
On Feb 25, 6:39 am, haole mejoe...@gmail.com wrote:
i'm using eclipse 3.5.1, GPE 3.5 (1.2.0.v200912062003), GAE
1.3.1.v201002101412, GWT 2.0.3.v201002191036
datanucleusenhancer continuously runs over and over again
temporary workaround: turn off build automatically feature in eclipse
I have several projects with a dependency hierarchy. I found the
enhancer to be fragile and it often would not enhance the classes. I
had to do a 'clean' to make it happen. I could not find the pattern to
when it failed to enhance. It never looped on me so you may be seeing
a different problem.
About 3 weeks ago I had had enough and turned off the Enhancer
builder (the plugin yelled at me). In its place I added an ant task
to do the enhancement. I derived the ant task statements from the
Datanucleus site. Since that time things have been very stable and
the enhancer always finishes its job.
This is the essence of the ant task:
!--
--
!-- T A S K D E F
S --
!--
--
taskdef name=enhance
classname=com.google.appengine.tools.enhancer.EnhancerTask
classpath
pathelement path=${appengine.tools.classpath}/
pathelement path=${appengine.tools.lib}/
/classpath
/taskdef
!--
--
!-- T A R G E T
S --
!--
--
target name=enhance.cls
description=Performs Datnucleus JDO enhancement on compiled
data classes.
!-- echo message= path ${appengine.tools.classpath}/ --
!-- echo message= fileset ${proj.dir.gen.cls}/ --
enhance failonerror=true verbose=false
classpath
pathelement path=${appengine.tools.classpath}/
fileset dir=${proj.dir.war.lib} includes=*.jar/
pathelement path=${proj.dir.gen.cls}/
/classpath
fileset dir=${proj.dir.gen.cls} includes=**/*.class/
/enhance
/target
target name=enhance.war
description=Performs Datnucleus JDO enhancement on war
classes.
!-- echo message= path ${appengine.tools.classpath}/ --
!-- echo message= fileset ${proj.dir.gen.cls}/ --
enhance failonerror=true verbose=false
classpath
pathelement path=${appengine.tools.classpath}/
fileset dir=${proj.dir.war.lib} includes=*.jar/
pathelement path=${proj.dir.war.cls}/
/classpath
fileset dir=${proj.dir.war.cls} includes=**/*.class/
/enhance
/target
It may be worth a try.
Steve
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Expires header automatically added (bug?)
Yes, I am setting a cookie. The session leakage problem is interesting. I have a question though, does the Expires header have precedence over the LastModified/ETag headers? I would like to use the LastModified/ETag headers to implement conditional GET, are you sure that ETag has precedence over the Expires header? (I am using cache-control: private) btw, thanks for the answer. -g. On Mar 1, 8:39 pm, Ikai L (Google) ika...@google.com wrote: Are you setting a cookie? We force an expires header for any requests that have a set-cookie header. The reason for this is that many users access websites using HTTP proxies. Some proxies will cache the entire request, which may cause session leak (e.g. let you read someone else's email). On Mon, Mar 1, 2010 at 7:09 AM, George Moschovitis george.moschovi...@gmail.com wrote: I don NOT want to set an Expires header. I am just curious with the header is added (and messes up with my caching scheme) -g. Well, if you do not like what GAE sets as Expires value, why not set yours? On Mar 1, 11:18 am, George Moschovitis george.moschovi...@gmail.com wrote: Could you provide some code please? What kind of code should I provide? I do NOT set the Expires header in my code, and still GAE automatically adds the Expires header. -g. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Blobstore - delete record from apps console
I created a new post for this without realizing this thread existed. I also starred the issue. On Jan 13, 5:05 pm, Jason (Google) apija...@google.com wrote: Hi YONG. I'm seeing this too on my end. I see a more descriptive error message, however -- the Admin Console is basically reporting that billing isn't enabled for the application when it really is, and hence doesn't process the delete request. Please star this external issue and you'll be notified of updates. http://code.google.com/p/googleappengine/issues/detail?id=2622 - Jason On Tue, Jan 12, 2010 at 7:39 PM, YONG yongkia...@gmail.com wrote: What application ID are you using, and what kind of error do you see when you try to delete the blob? my app ID is yong8128. When I try to delete it from Admin Console, Blob Viewer, I got the Server Error A server error has occurred. Return to Applications screen » I had uploaded a few blob fileif delete using Blobstore API is OK. But when delete using Admin Console, Blob Viewer, got the above error. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: error deleting file in Blob Viewer
I found another thread relevant to this and it mentioned this issue: http://code.google.com/p/googleappengine/issues/detail?id=2622 So I starred that issue and left my info in a comment there. Is there no way to delete a blob from code? The documentation says this: If your app doesn't want to keep the blob, you should delete the blob immediately to prevent it from becoming orphaned. But I searched for delete on that page and didn't find a way to do it even though the implication is that there is a way. :) Thanks, Stephen On Mar 2, 9:00 am, Houston startup coder stephenh...@gmail.com wrote: I tried to delete a file in Blob Viewer and every time, no matter if I'm attempting to delete one or several, I get this error: Server Error A server error has occurred. Return to Applications screen » -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Multiple contains() in a query and exploding indexes
Hi
Some time ago, I asked how to use multiple contains in a query, and I got
some responses, that was great and I thank everybody for they help.
I am posting my findings and advance in hope it might be useful for somebody
trying to do the same.
I am trying to build a database with real state listings in my area, and
build some searches on it, the search is likely to have many fields, and
several of the fields the user can select multiple values. I chose to handle
the whole entity relationship myself instead of using the fancy features of
GAE, I had my reasons for it, but frustration with GAE it is a part of it,
regardless of that, the approach to searching might still apply if you chose
to use relationships from GAE.
I read somewhere in the docs that when you use contains in a query, it
internally it executes an equal sub-query for each of the values in the
list, (somebody care to confirm that?) so if you have several fields with
contains you might bump into the 30 sub-query constraint pretty fast.
So I choose to:
-execute the search by each one of the fields, and each ones of the selected
values sequentially, get only the ids, each one of this should hit only one
index, and be fast.
-add the results from each result to a memcache instance, using increment,
collect the ids in a list for later (there is no way to get all the keys in
the cache,that I found)
-collect the counts for each id in the list I got, and for each check the
count, if the count is equal to the number of queries, it means that entity
returned true for each of the queries and its an entity that I want to
return, i collect all the ides that are good results, and go to the
datastore to collect the full entities to return.
This process is expensive, and I still got to try it out with a a big set,
but executes sufficiently fast for my test set, of course I cache the result
until the user changes the search criteria (or expires).
Here is the code for the search method:
private ListIndexEntry buildResultsFor(SearchCriteria sc) {
ListIndexEntry result = new ArrayListIndexEntry();
// Price parsing
float minPrice = -1;
float maxPrice = -1;
if (sc.getMinPrice().length() 0) {
minPrice = Float.parseFloat(sc.getMinPrice());
}
if (sc.getMaxPrice().length() 0) {
maxPrice = Float.parseFloat(sc.getMaxPrice());
}
// Listing Status parsing
Long[] statusIds = null;
String[] statusNames = sc.getStatus();
if (statusNames != null statusNames.length 0) {
statusIds = getListingStatusIdsByNames(statusNames);
}
// House Types parsing
Long[] houseTypesIds = null;
String[] houseTypeNames = sc.getHouseType();
if (houseTypeNames != null houseTypeNames.length 0) {
houseTypesIds = getHouseTypeIdsByNames(houseTypeNames);
}
// THE search
MemcacheService cache = MemcacheServiceFactory.getMemcacheService();
SetString allIds = new HashSetString();
int condCount = 0;
MapObject, Long lastResults = null;
Long one = new Long(1);
// by price
if (minPrice 0 || maxPrice 0) {
condCount++;
ListString ids = indexService.getByPriceRange(minPrice, maxPrice);
allIds.addAll(ids);
lastResults = addToChache(cache, one, ids);
}
// by status
if (statusIds != null) {
condCount++;
for (int i = 0; i statusIds.length; i++) {
ListString listingByStatus =
indexService.getByListingStatus(statusIds[i]);
allIds.addAll(listingByStatus);
lastResults = addToChache(cache, one, listingByStatus);
}
}
// by house type
if (houseTypesIds != null) {
condCount++;
for (int i = 0; i houseTypesIds.length; i++) {
ListString listingByHT = indexService.getByHouseType(houseTypesIds[i]);
allIds.addAll(listingByHT);
lastResults = addToChache(cache, one, listingByHT);
}
}
// by Zip Code
String[] zipCodes = parseZipCode(sc.getZipCode());
if (zipCodes != null zipCodes.length 0) {
condCount++;
for (int i = 0; i zipCodes.length; i++) {
ListString listingByZ = indexService.getByZipCode(zipCodes[i]);
allIds.addAll(listingByZ);
lastResults = addToChache(cache, one, listingByZ);
}
}
if (lastResults != null) {
MapObject, Object counters =
cache.getAll(Arrays.asList(allIds.toArray()));
ListString ids = new ArrayListString();
for (Object listingNumber : counters.keySet()) {
String sCount = (String) counters.get(listingNumber);
long count = Long.parseLong(sCount);
if (count condCount) {
ids.add(listingNumber.toString());
if (ids.size()500){
break;
}
}
}
cache.clearAll();
if (ids.size() 0) {
result = indexService.getEntriesOn(ids);
}
}
return result;
}
hope it helps somebody
thanks
Karel
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: JPA 2.0 Support
The obvious question is ... what particular part of JPA2 are you requiring access to ? JPA2 adds on various ORM related mapping extensions, as well as JPQL Criteria, validation, and little else really. The ORM features are of no relevance for GAE/J clearly. DataNucleus 2.0 already implements JPA2, so it ought to be a matter of GAE/J to update their plugin to use it (or for people like you to contribute the updates necessary since it is open source). DN 2.0 isn't yet certified as JPA2 compliant since JPA2 is developed in secret, and the TCK is not readily available (we requested it a month ago and still nothing), though it passes our unit tests. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Multiple contains() in a query and exploding indexes
Hi Karel,
Keep in mind that each time you modify a list and put it in memcache
the whole list is serialized which is why you see it is expensive.
There is an efficient approach to merging queries that does not need
memcahe that Bret Slatkin called the zig zag method:
http://www.scribd.com/doc/16952419/Building-scalable-complex-apps-on-App-Engine
It does not require the results to be in memory either so will work
for large datasets. You just need to make sure all the queries are
sorted by the same property e.g. __key__
JD
On 2 Mar 2010, at 23:17, Karel Alvarez wrote:
Hi
Some time ago, I asked how to use multiple contains in a query, and
I got some responses, that was great and I thank everybody for they
help.
I am posting my findings and advance in hope it might be useful for
somebody trying to do the same.
I am trying to build a database with real state listings in my area,
and build some searches on it, the search is likely to have many
fields, and several of the fields the user can select multiple
values. I chose to handle the whole entity relationship myself
instead of using the fancy features of GAE, I had my reasons for it,
but frustration with GAE it is a part of it, regardless of that, the
approach to searching might still apply if you chose to use
relationships from GAE.
I read somewhere in the docs that when you use contains in a query,
it internally it executes an equal sub-query for each of the values
in the list, (somebody care to confirm that?) so if you have several
fields with contains you might bump into the 30 sub-query constraint
pretty fast.
So I choose to:
-execute the search by each one of the fields, and each ones of the
selected values sequentially, get only the ids, each one of this
should hit only one index, and be fast.
-add the results from each result to a memcache instance, using
increment, collect the ids in a list for later (there is no way to
get all the keys in the cache,that I found)
-collect the counts for each id in the list I got, and for each
check the count, if the count is equal to the number of queries, it
means that entity returned true for each of the queries and its an
entity that I want to return, i collect all the ides that are good
results, and go to the datastore to collect the full entities to
return.
This process is expensive, and I still got to try it out with a a
big set, but executes sufficiently fast for my test set, of course I
cache the result until the user changes the search criteria (or
expires).
Here is the code for the search method:
private ListIndexEntry buildResultsFor(SearchCriteria sc) {
ListIndexEntry result = new ArrayListIndexEntry();
// Price parsing
float minPrice = -1;
float maxPrice = -1;
if (sc.getMinPrice().length() 0) {
minPrice = Float.parseFloat(sc.getMinPrice());
}
if (sc.getMaxPrice().length() 0) {
maxPrice = Float.parseFloat(sc.getMaxPrice());
}
// Listing Status parsing
Long[] statusIds = null;
String[] statusNames = sc.getStatus();
if (statusNames != null statusNames.length 0) {
statusIds = getListingStatusIdsByNames(statusNames);
}
// House Types parsing
Long[] houseTypesIds = null;
String[] houseTypeNames = sc.getHouseType();
if (houseTypeNames != null houseTypeNames.length 0) {
houseTypesIds = getHouseTypeIdsByNames(houseTypeNames);
}
// THE search
MemcacheService cache =
MemcacheServiceFactory.getMemcacheService();
SetString allIds = new HashSetString();
int condCount = 0;
MapObject, Long lastResults = null;
Long one = new Long(1);
// by price
if (minPrice 0 || maxPrice 0) {
condCount++;
ListString ids = indexService.getByPriceRange(minPrice,
maxPrice);
allIds.addAll(ids);
lastResults = addToChache(cache, one, ids);
}
// by status
if (statusIds != null) {
condCount++;
for (int i = 0; i statusIds.length; i++) {
ListString listingByStatus =
indexService.getByListingStatus(statusIds[i]);
allIds.addAll(listingByStatus);
lastResults = addToChache(cache, one,
listingByStatus);
}
}
// by house type
if (houseTypesIds != null) {
condCount++;
for (int i = 0; i
Re: [appengine-java] Multiple contains() in a query and exploding indexes
The zig zag method will help you with the AND part of your problem but
the IN (OR) queries would need to be merged and ordered. If each
query is ordered by __key_ - java.util.PriorityQueue could
efficiently merge the parts of your IN query without needing to load
them all into memory. Then you could use the zig-zag method to
combine (AND) the different IN queries.
On 2 Mar 2010, at 23:17, Karel Alvarez wrote:
Hi
Some time ago, I asked how to use multiple contains in a query, and
I got some responses, that was great and I thank everybody for they
help.
I am posting my findings and advance in hope it might be useful for
somebody trying to do the same.
I am trying to build a database with real state listings in my area,
and build some searches on it, the search is likely to have many
fields, and several of the fields the user can select multiple
values. I chose to handle the whole entity relationship myself
instead of using the fancy features of GAE, I had my reasons for it,
but frustration with GAE it is a part of it, regardless of that, the
approach to searching might still apply if you chose to use
relationships from GAE.
I read somewhere in the docs that when you use contains in a query,
it internally it executes an equal sub-query for each of the values
in the list, (somebody care to confirm that?) so if you have several
fields with contains you might bump into the 30 sub-query constraint
pretty fast.
So I choose to:
-execute the search by each one of the fields, and each ones of the
selected values sequentially, get only the ids, each one of this
should hit only one index, and be fast.
-add the results from each result to a memcache instance, using
increment, collect the ids in a list for later (there is no way to
get all the keys in the cache,that I found)
-collect the counts for each id in the list I got, and for each
check the count, if the count is equal to the number of queries, it
means that entity returned true for each of the queries and its an
entity that I want to return, i collect all the ides that are good
results, and go to the datastore to collect the full entities to
return.
This process is expensive, and I still got to try it out with a a
big set, but executes sufficiently fast for my test set, of course I
cache the result until the user changes the search criteria (or
expires).
Here is the code for the search method:
private ListIndexEntry buildResultsFor(SearchCriteria sc) {
ListIndexEntry result = new ArrayListIndexEntry();
// Price parsing
float minPrice = -1;
float maxPrice = -1;
if (sc.getMinPrice().length() 0) {
minPrice = Float.parseFloat(sc.getMinPrice());
}
if (sc.getMaxPrice().length() 0) {
maxPrice = Float.parseFloat(sc.getMaxPrice());
}
// Listing Status parsing
Long[] statusIds = null;
String[] statusNames = sc.getStatus();
if (statusNames != null statusNames.length 0) {
statusIds = getListingStatusIdsByNames(statusNames);
}
// House Types parsing
Long[] houseTypesIds = null;
String[] houseTypeNames = sc.getHouseType();
if (houseTypeNames != null houseTypeNames.length 0) {
houseTypesIds = getHouseTypeIdsByNames(houseTypeNames);
}
// THE search
MemcacheService cache =
MemcacheServiceFactory.getMemcacheService();
SetString allIds = new HashSetString();
int condCount = 0;
MapObject, Long lastResults = null;
Long one = new Long(1);
// by price
if (minPrice 0 || maxPrice 0) {
condCount++;
ListString ids = indexService.getByPriceRange(minPrice,
maxPrice);
allIds.addAll(ids);
lastResults = addToChache(cache, one, ids);
}
// by status
if (statusIds != null) {
condCount++;
for (int i = 0; i statusIds.length; i++) {
ListString listingByStatus =
indexService.getByListingStatus(statusIds[i]);
allIds.addAll(listingByStatus);
lastResults = addToChache(cache, one,
listingByStatus);
}
}
// by house type
if (houseTypesIds != null) {
condCount++;
for (int i = 0; i houseTypesIds.length; i++) {
ListString listingByHT =
indexService.getByHouseType(houseTypesIds[i]);
allIds.addAll(listingByHT);
[appengine-java] App upload failed
Trying upload app: ./appcfg.sh update ../../../projects/appengine/myapp/www/ Reading application configuration data... Encountered a problem: null Please see the logs [/tmp/appcfgx2i014.log] for further information. an...@ast:/mnt/sda1/projects/appengine/myapp$ cat /tmp/ appcfgx2i014.log java.lang.NullPointerException at org.relaxng.datatype.helpers.DatatypeLibraryLoader$Service $Loader2.getResources(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader $Service.init(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaBuilder.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaSchemaFactory.newSchema(libgcj.so. 10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at com.google.appengine.tools.admin.Application.validateXml(Application.java: 319) at com.google.appengine.tools.admin.Application.init(Application.java: 87) at com.google.appengine.tools.admin.application.readaan...@ast:/mnt/ sda1/projects/appengine/myapp Same app was uploaded successfully from my home computer. Any suggestions? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Google App Engine - ZK Modal Window Freezes
Hello,
I am creating a search window with this code, everything works fine in
eclipse, but when i put it online
Window pesquisa = (Window)
Executions.createComponents(/principal/
comum/pesquisa.zul,winMain, map);
pesquisa.setParent(this);
pesquisa.doModal();
pesquisa.zul uses a listbox for browse records,
when i finally click on OK button
((WinMediumBasico) getFellow(Pesquisa).getParent()).
setMedium(MediumDAO.getInstance().
findById(((Medium)
lbMedium.getSelectedItem().getValue()).getId()));
this.onClose();
it don´t close the modal dialog.
public Medium findById(Key id) {
PersistenceManager pm = PMF.get().getPersistenceManager();
Medium medium = null;
try {
medium = pm.getObjectById(Medium.class,id);
} catch (NoResultException nr) {
nr.printStackTrace();
} catch (Exception ex) {
ex.printStackTrace();
} finally {
pm.close();
}
return medium;
}
I am not able to debug it, does someone can help ?
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Difficulty querying webserver through POSTing XML text
I am having difficulty POSTing and reading from a web server from GAE.
I can authenticate but the webserver does not accept the XML file I am
POSTing (which includes a query). I am not running the webserver in
question, so I have no insight into what might be wrong at that end
(or set it up differently).
First, a sanitized snippet of the curl statement that returns the
desired result (on OSX):
$ curl http://targetwebserver.org/servlet -u username:password -d '?
xml version=1.0 encoding=UTF-8 standalone=yes?Tag /
Tag'
That to me confirm the webserver works alright.
Below now a snippet of the code (sanitized and rough) that I cannot
get to work. Again, authentication seems to work (I am not getting
responseCode 401), but whatever I have tried, I cannot get over
responseCode 400 and a corresponding boilerplate error message from
the webserver.
- snip ---
String requestString = ?xml version=1.0 encoding=UTF-8
standalone=yes?Tag /Tag; // Also tried all sorts of
encoding, no luck
URL url = new URL(targetwebserver.org/servlet );
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod(POST);
conn.setDoOutput(true);
conn.setDoInput(true);
conn.addRequestProperty(Content-type, text/xml); // tried
application/xml as well, no luck
// Authenticate (seems to work OK)
String authData = username:password;
byte[] encodedAuthData = new
org.apache.commons.codec.binary.Base64().encode (authData.getBytes());
String authString = Basic ;
for (int i = 0; i encodedAuthData.length; i++)
authString += (char)encodedAuthData[i];
conn.addRequestProperty (Authorization, authString);
// Post request
PrintWriter pw = new PrintWriter(conn.getOutputStream()); // tried
OutputStreamWriter as well, no luck
pw.println(requestString);
pw.close();
responseCode = conn.getResponseCode();
if (responseCode == 200) {
InputStream in = conn.getInputStream();
BufferedReader rd = new BufferedReader(new
InputStreamReader(conn.getInputStream()));
String line; while ((line = rd.readLine()) != null)
{ log.info(line); }
}
- snip ---
Please have a look at this - help's appreciated!
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Datastore Error: No Source Attachment
Hello, I've been working on an app which uses the datastore, and this is the first time I've tested it out. When I attempted to add data, it didn't seem to work. So, once I use Eclipse to debug the app, it gives me an error saying that the source could not be found for a particular file (no source attachment). The file is named appengine-api-1.0- sdk-1.3.1.jar and resides in Eclipse's plugins directory (under where the Google Plugin for Eclipse is installed). I attempted to search for information regarding this error, but nothing of use turned up. So, I was hoping that somebody here could help me. I am using Eclipse with the Google App Engine plugin. GWT is installed, but I am not using it for this app. I am more than willing to provide any screenshots or other information needed to solve this problem. I would like to have it fixed as soon as possible. Thank you in advance! Sincerely, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Datastore Error: No Source Attachment
Hi Alex, this is not an error so to speak - its just Eclipse telling you that it cannot show you the source code for internal app engine libraries because they are not available. On 2 Mar 2010, at 12:47, ahawesome wrote: Hello, I've been working on an app which uses the datastore, and this is the first time I've tested it out. When I attempted to add data, it didn't seem to work. So, once I use Eclipse to debug the app, it gives me an error saying that the source could not be found for a particular file (no source attachment). The file is named appengine-api-1.0- sdk-1.3.1.jar and resides in Eclipse's plugins directory (under where the Google Plugin for Eclipse is installed). I attempted to search for information regarding this error, but nothing of use turned up. So, I was hoping that somebody here could help me. I am using Eclipse with the Google App Engine plugin. GWT is installed, but I am not using it for this app. I am more than willing to provide any screenshots or other information needed to solve this problem. I would like to have it fixed as soon as possible. Thank you in advance! Sincerely, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Request was aborted after waiting too long to attempt to service your request.
Hello, I'm developping an App (fullmetalgalaxy.com) with GWT/AppEngine and I've got the same error in production server: “Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please see http://code.google.com/appengine/docs/quotas.html for more details.” My app may be poorly designed and I'm using a long pooling technique for server push... nevertheless I don't understand how I can exceed 30 simultaneous dynamic request with only few client ! I though that web browser was able to handle only 2 simultaneous request for a single domain name, if I'm right how can I exceed 30 requests with only few clients (less than 10) ? Is there any other way to get this error ? I'm using http filter for a good part of my static resources (mainly to set cache lifetime), are these requests count as dynamics request ? If yes, do I have another way to set cache lifetime (at least for *.nocache.* and *.cache.* patern) My long pooling request last for 26 sec but spend most of there time in a “Thread.sleep()”... Are sleeping request count as one for all simultaneous request ? I guest yes, but what other choice do I have to do server push on AppEngine ? Apparently enabling billing let application scale to around 500 request per second... How many in simultaneous dynamic request ? If I don't exceed free quota, will enabling billing rise the max number of simultaneous request ? Thanks for your attention Vincent -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: error deleting file in Blob Viewer
If we cannot delete blobs, will we have to pay for them? On Mar 2, 10:15 am, Houston startup coder stephenh...@gmail.com wrote: I found another thread relevant to this and it mentioned this issue:http://code.google.com/p/googleappengine/issues/detail?id=2622 So I starred that issue and left my info in a comment there. Is there no way to delete a blob from code? The documentation says this: If your app doesn't want to keep the blob, you should delete the blob immediately to prevent it from becoming orphaned. But I searched for delete on that page and didn't find a way to do it even though the implication is that there is a way. :) Thanks, Stephen On Mar 2, 9:00 am, Houston startup coder stephenh...@gmail.com wrote: I tried to delete a file in Blob Viewer and every time, no matter if I'm attempting to delete one or several, I get this error: Server Error A server error has occurred. Return to Applications screen » -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Request was aborted after waiting too long to attempt to service your request.
On 3 Mar 2010, at 00:22, Kroc wrote: Hello, I'm developping an App (fullmetalgalaxy.com) with GWT/AppEngine and I've got the same error in production server: “Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please see http://code.google.com/appengine/docs/quotas.html for more details.” I have found I usually get this error when my app is loading and takes more than 10 seconds while another request come in. Does your app take longer than 10 seconds to load? My app may be poorly designed and I'm using a long pooling technique for server push... nevertheless I don't understand how I can exceed 30 simultaneous dynamic request with only few client ! I though that web browser was able to handle only 2 simultaneous request for a single domain name, if I'm right how can I exceed 30 requests with only few clients (less than 10) ? Is there any other way to get this error ? I'm using http filter for a good part of my static resources (mainly to set cache lifetime), are these requests count as dynamics request ? If yes, do I have another way to set cache lifetime (at least for *.nocache.* and *.cache.* patern) Set static resources and cache times in your appengine-web.xml. That way Google serves them directly without hitting your app. My long pooling request last for 26 sec but spend most of there time in a “Thread.sleep()”... Are sleeping request count as one for all simultaneous request ? I guest yes, but what other choice do I have to do server push on AppEngine ? I think there is no good solution on App Engine for server push. Polling would be the only option because as you say, simultaneous requests are limited. Apparently enabling billing let application scale to around 500 request per second... How many in simultaneous dynamic request ? If I don't exceed free quota, will enabling billing rise the max number of simultaneous request ? Its still 30 unless you get permission. Thanks for your attention Vincent -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: form to request being allowed to deploy the application multiple times
I finally found it in an old thread about multi tenant architecture not generally being allowed on GAE: http://groups.google.com/group/google-appengine-java/browse_thread/thread/6ae2e6737cbb4b40/f47f015099538467?lnk=gstq=multiple+site#f47f015099538467 This is the exception request form: http://code.google.com/support/bin/request.py?contact_type=AppEngineMultiInstanceExceptionRequest However, from other stuff I remember reading a long time ago, I guess this is not as much of a concern if I'm not deploying on multiple domains. I just want to be able to have multiple datastores (test, QA, prod, that sort of thing), and in order to use the Blobstore in each one I must have billing enabled, so I guess your automated detection wouldn't pick that up as a violation since I'm paying for the data, right? I just didn't want it to sniff out that I have the same code running around multiple datastores and shut me down thinking I was trying to use as much free quota as possible. Please let me know! Thanks so much, Stephen Huey On Mar 2, 9:16 am, Houston startup coder stephenh...@gmail.com wrote: I've been Googling all over this forum and the documentation, but I can't find the form I need to submit to the GAE team. I am deploying multiple versions of my app (which are actually different apps) around the same datastore, and it has billing enabled. But I also need to deploy this stuff again around a separate datastore for testing and I don't want the GAE police to shut me down. I remember coming across a form I can fill out to request exemption in my case. Can someone please direct me to that? Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Why is java.security.AccessController on the whitelist?
The purpose of AccessController is to perform an operation requiring
escalated permissions on behalf of code that doesn't have that permission.
For example, maybe you need to implement an authenticate(String user, String
password) function that needs to read from a password file that calling code
doesn't have permission to read from. The call stack might look something
like:
checkUser --- Untrusted caller doesn't have permission to read /etc/passwd
authenticate -- Your code does have permission.
AccessController.doPrivileged -- Escalate to your code's permission levels
readFromEtcPasswdAndCheck -- Now can do privileged read for untrusted code
In the context of GAE, all code in your application runs with the same
permissions. This means AccessController isn't needed, and you can write any
application without it.
However, we whitelist AccessController, because it improves compatibility.
For example, there are several Java libraries which need to use
AccessController when not running on GAE. By whitelisting AccessController,
we help these libraries run portably on GAE.
On Tue, Mar 2, 2010 at 4:20 AM, Yiming Li yiming...@umail.ucsb.edu wrote:
Hi All,
When I was looking at the whitelist of GAE Java, it is
interesting that I found java.security.AccessController class, and I
don't quite understand in what scenario we need to use this class.
On the other hand, it may introduce some security issues,
think about this code in a servlet:
public void doGet(HttpServletRequest req, final HttpServletResponse
resp)
throws IOException {
AccessController.doPrivileged(new PrivilegedAction() {
public Object run() {
File dir2 = new File(/etc);
String[] children2 = dir2.list();
if (children2 == null) {
}
else { for (int i=0; ichildren2.length; i++)
{
try {
resp.getWriter().println(children2[i]);
} catch (IOException e) {
e.printStackTrace();
}
}
Thread t= new Thread(){
public void run() {
try {
resp.getWriter().println(waaa);
} catch (IOException e) {
e.printStackTrace();
}
}
};t.run();
return null;
}
});
}
The result page will print all file names under /etc
directory, with waaa, which is the output of thread t.
But fortunately, this will happen only on development
server, and only if you have the permission to do view all files under
/etc directory(but if you run the dev server with root permission
accidentally, you can basically do anything).
So my point is with java.security.AccessController class,
you can easily get rid of the restriction of the sandbox, although
only on the dev server. Can anybody give me an example of using
java.security.AccessController in a legitimate way?
Thank you very much.
--
Yiming
MS student of CS Department @UCSB
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] App upload failed
It looks like you're using gcj. Can you use a different JVM, such as OpenJDK or Sun's JDK? On Tue, Mar 2, 2010 at 3:44 AM, ast anton.star...@gmail.com wrote: Trying upload app: ./appcfg.sh update ../../../projects/appengine/myapp/www/ Reading application configuration data... Encountered a problem: null Please see the logs [/tmp/appcfgx2i014.log] for further information. an...@ast:/mnt/sda1/projects/appengine/myapp$ cat /tmp/ appcfgx2i014.log java.lang.NullPointerException at org.relaxng.datatype.helpers.DatatypeLibraryLoader$Service $Loader2.getResources(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader $Service.init(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaBuilder.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaSchemaFactory.newSchema(libgcj.so. 10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at com.google.appengine.tools.admin.Application.validateXml(Application.java: 319) at com.google.appengine.tools.admin.Application.init(Application.java: 87) at com.google.appengine.tools.admin.application.readaan...@ast:/mnt/ sda1/projects/appengine/myapp Same app was uploaded successfully from my home computer. Any suggestions? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Eclipse hangs at startup, Ubuntu
The projects that cause the problem started life as Subclipse projects but I were converted to Subversive with the detach, share process. My eclipse install is a new one with Subversive installed but not Subclipse. However, the conversion to Subversive happened quite a while before the OS upgrade. I finally couldn't take it anymore and created a whole new workspace from scratch. A very painful solution. Jeff On Tue, Mar 2, 2010 at 8:17 AM, Rajeev Dayal rda...@google.com wrote: That message corresponds to GPE's attempt to update your project's war/WEB-INF/lib folder on project classpath initialization, which occurs on Eclipse startup. Sometimes, problems can occur if there is some sort of lock being held on the files in that directory (such as by a version control plugin). Do you have any version control plugins installed in Eclipse? We've actually made some changes so that the update of war/WEB-INF/lib occurs on classpath change instead of classpath init. That should alleviate this sporadic issue. The fix will be available in a forthcoming version of GPE. On Mon, Mar 1, 2010 at 2:34 AM, Jeff Schnitzer j...@infohazard.org wrote: Did anyone ever resolve this in a consistent way? I recently upgraded from OSX 10.5 to 10.6, and now my Eclipse (Cocoa 64-bit, as I was using before) hangs on most startups. I see the Updating MyProje... - 1.3.1 in the bottom right corner and the entire window is locked up. Only way out is to Force Quit. It doesn't happen on every startup. I can often get it working after a few tries. This was a brand-new OS install on a new HD. I did, however, copy my eclipse directory over by hand, and the workspace was imported as part of my user data. Jeff On Jan 14, 12:49 am, pgoetz pgo...@pgoetz.de wrote: Hi group, I have a problem with the Google Eclipse Plugin. I am developing an application for the Google App Engine (Eclipse JEE Galileo, Google Plugin 1.2.0, Google App Engine SDK 1.3.0, OS Ubuntu 9.10). Yesterday I reinstalled the Google Plugin and Google App Engine SDK after I got the same problem with an older version of the SDK. Today I started Eclipse and after the workspace comes up, I see the message Updating myproject/...ne - 1.3.0 in the status bar of Eclipse. I suspect that it means Updating myproject/Google App Engine - 1.3.0. With this message my whole Eclipse freezes and I can only kill the process. Does anybody know why that happens? And how can I prevent the plugin from performing the update? I had the same problem yesterday with the old setup and the solution was to remove the folders for the plugin from Eclipse plugins and features directories and to reinstall the plugin. Then it worked yesterday and crashed this morning. Thank you very much for your help! Greetings, Peter -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: lost sub entity
I've uploaded my app to google with the same result, so it's not a
problem with Eclipse or sdk version.
Gunnar
On 1 mar, 20:59, John Patterson jdpatter...@gmail.com wrote:
http://lmgtfy.com/?q=appengine+jdo+%22default+fetch+group%22
On 2 Mar 2010, at 02:50, Gunnar wrote:
I don't jnow what you mean with look into default fetch group!.
Please explain.
Gunnar
On 1 mar, 19:20, John Patterson jdpatter...@gmail.com wrote:
Did you look into default fetch group?
On 2 Mar 2010, at 01:04, Gunnar wrote:
Hi,
I followed Jakes advice to do e.setList(list) but no change.
I've also stepped through my code, but there is no exception!
Btw I use version 1.3.1 of the SDK.
Gunnar
On 1 mar, 15:28, John Patterson jdpatter...@gmail.com wrote:
It is unusual that you see no stack trace. Are you sure you are
not
catching it? You could step through the code line by line to see
what
happens after the line with the problem.
Could this be something to do with setting the fetch group to
default? That seems to be the problem with a lot of peoples JDO
code :)
On 1 Mar 2010, at 21:19, Jake wrote:
If I recall, JDO is picky when it comes to being aware of changes
made
to a persisted object. For example, changing fields directly
(object.field = newValue;) doesn't work - you need to use a
getter/
setter (object.setField(newValue);). Perhaps you are encountering
the
same issue here? Does the following type of thing work?
LIstSubEntity list = e.getMyList();
SubEntity first = list.remove(0);
list.add(first);
e.setMyList(list);
pm.makePersistent(e);
tx.commit();
Jake
On Feb 28, 11:05 am, Gunnar gunnar@gmail.com wrote:
Hi,
John and Karel, you are right about the placement of the commit
statement.
It worked in the test example because I only had one instance of
MyEntity.
I don't get any exception at all in the Eclipse console!
When I add a second instance of MyEntity if fails with
Transaction
is
not active as expected.
Now I've changed the code in the try statement like this, but
still
the first sub entity is lost and no exception is thrown!
(Of cause it would fail it the list is empty.)
ListMyEntity results = (ListMyEntity) query.execute();
if (results.iterator().hasNext()) {
tx.begin();
MyEntity e = results.iterator().next();
ListSubEntity list = e.getMyList();
SubEntity first = list.remove(0);
boolean ok = list.add(first);
if (!ok) {
System.err.println(could not add
first);
}
System.out.println(list);
pm.makePersistent(e);
tx.commit();
}
On 28 Feb, 02:51, John Patterson jdpatter...@gmail.com wrote:
This should be throwing an exception. The JavaDocs for
Transaction.commit() say
Commits the transaction. Whether this call succeeds or fails,
all
subsequent method invocations on this object will throw
IllegalStateException.
When something does not work as you expect the first place to
look
is
the logs under your application console.
On 28 Feb 2010, at 05:25, Karel Alvarez wrote:
dont you get any exceptions stacktrace in the server
console? it
would help... also you are calling commit() inside the for
loop,
although the transactions is started only once... that would
crash
in the second iteration in a normal db server, I am not sure
what
GAE does with it, in any case I dont think it is what you
intended...
On Sat, Feb 27, 2010 at 2:52 PM, Gunnar gunnar@gmail.com
wrote:
Hi,
I have problem with reordering a List.
I created a test with an entity called MyEntity which have a
ListSubEntity.
First I persist one instance of MyEntity with a list
containing 3
SubEntity.
This works fine. Then I call a reorder servlet that moves the
first
SubEntity to the last position in the list.
The result is that the first SubEntity is lost and the
datastore
only
contains 2 SubEntity instances.
What can be wrong?
Here is the reorder code:
package com.google.appengine.demo;
import java.io.IOException;
import java.util.List;
import javax.jdo.PersistenceManager;
import javax.jdo.Query;
import javax.jdo.Transaction;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@SuppressWarnings(serial)
public class ReorderServlet extends HttpServlet {
�...@suppresswarnings(unchecked)
public void doGet(HttpServletRequest req,
HttpServletResponse
resp)
throws IOException {
PersistenceManager pm =
PMF.get().getPersistenceManager();
Query query = pm.newQuery(MyEntity.class);
Transaction tx = pm.currentTransaction();
try {
[appengine-java] number of tasks in queue
Hello, In the near future will there be an API method to get the current number of tasks in a queue? Thanks for your time, Nichole -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] (Java) App Engine (Servlet) - Google Buzz
The Buzz API is documented here: http://code.google.com/apis/buzz/ http://code.google.com/apis/buzz/Just so you know - no write access yet. On Tue, Mar 2, 2010 at 12:52 PM, Perry Dykes pdy...@gmail.com wrote: I would like to augment my app engine java servlet application to write status updates to a specific user in google buzz or a group. Is there an api and example to start with? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] App Engine Code sending emails from Our Domain not from google.com or How to avoid Spoofing
Using the Google App Engine settings panel we have registered a new domain (e.g. OurDomain.com).The web of our application is now reachable directly from that domain however emails sent from our App Engine code are coming from Google.com instead of OurDomain.com. This then results in email security at Corporate America as seeing the email as Spoofing who it is really from. As expected the ‘Reply To’ does show up as OurDomain.com but the important part, the From string in the SMTP headers is from mail-yw0- f235.google.com Is it possible on the Google App Engine to actually have emails, sent via code, to originate from our domain to avoid spoofing? If so, what are the steps to configure this. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Google Accounts are killing my application!....
Hello Ikai, I guess many of us have the same problem. It would be good to have some general advice on that. Unfortunatly GAE only offers build in authentication for the admin users or for users within a apps domain. So what I did is a simple table with the users gmail addresses that are allowed to access my app. If I can retrieve a user from the content and his/her email corresponds to the one I have saved, I let them in. UserServiceFactory.getUserService().getCurrentUser() If not I redirect them to the Google login page: response.sendRedirect(UserServiceFactory.getUserService().createLoginURL(redirect)); This solution works quite well but somehow I do not like it. I have the strong feeling that I build something that must already be there somehow. Also users have not the option to add my site as trusted so they have to log in whenever they come back to my page. I do not know how to work around that. The only advantage is that I can have alternatively the possibility for users to create an extra account on my site... You mentioned oauth and somewhere I read about friendconnect. I can not see how exactly that fits into the framework. Why cant we just have a google authentication build into GAE. With a users list as we do for the account admins and a simple rule to throw into the web.xml. +some way for users to self-register if the application developer wants that.This way no-one needs to reinvent the wheel. Does such a thing maybe already exist? Or is it on the road-map? Or is there a good approach you would advice? Thanks, Toby On Feb 22, 8:09 pm, Ikai L (Google) ika...@google.com wrote: Providing a login inside a frame is a compromise you should never, ever make. You're essentially training your users to be victims of phishing attacks. By providing a login in a frame, you're essentially removing every single security mechanism browsers provide to attempt to ensure users that the site they are on is really the site they are on and not a password stealing site. This is why many companies go out of their way to provide OAuth:http://oauth.net- because this allows client developers a way to authenticate users against another site's identity mechanism without having users send their credentials to a potentially untrusted site itself. Granted, there's a bit of a disconnect on login, but this is a price we'll have to pay just because this is one of the failings of browser security. Savvy users have already caught on to this, and more and more mainstream users will as well. This is a stopgap - when browsers are able to provide native authentication mechanisms, we shouldn't have to do this anymore, but we have a ways to go before this sort of thing will exist. On Sun, Feb 21, 2010 at 3:21 PM, John V Denley johnvden...@googlemail.comwrote: The frame works fine when logging in. If its a security risk please elaborate, Im onlt using Google accounts because I dont really know how to do my own security, and Im guessing that even using google via a frame is more secure than trying to do it myself! When creating an account it does not take the user back to the original page as there is a total disconnect after the user clicks on the link in the email sent from google. Google have informed me that this is a known issue, but has a low priority (which is understandable). I have now created what I think is a reasonable compromise. Only time will tell if our potential customers are ok with the process! On Feb 18, 7:04 pm, Brian bwa...@gmail.com wrote: You shouldn't use a frame. It is a security problem, and right of google login code to break out of it. After they make a new account, if not using a frame, I believe it forwards the user back to the page they were trying to go to. Seems to work pretty well. On Feb 18, 8:40 am, John V Denley johnvden...@googlemail.com wrote: I have been trying to leverage google accounts for security for my users, but the way its working is really preventing useability within my application, its very frustrating Ive just spent the best part of the last week trying to get the google account login to work in an frame within my application. Ive run into a number of related issues (see other threads in the GWT group) which I have manage to work through finally. (Thanks to everyone who helped out and provided input) However, I have just tried clicking on the create an account now link which is what will be used by any new user who doesnt currently have a google account, but the account creation window has frame breakout code on it, which takes my users away from my application again, and then after clicking on the email link to confirm thier new account, the user is NOT taken back to my application but are just congratulated for creating a google account. The problem is that the user is then left thinking now what do i do? and several of the people
Re: [appengine-java] Re: Google Accounts are killing my application!....
It'll have to work similar to how OAuth works with a window redirect. The issue with domains is that Users will be giving their credentials to a potentially untrustworthy site, and this isn't something we want our users to do. We can certainly do more with authentication. For instance, if you are logged into your Google account and go to YouTube, if you click log in, it'll log you in and redirect you. The user sees a bit of a delay as the redirects take place, but it's otherwise invisible. We could probably do something like this if you have logged into the given application before. One issue is with revocation: we'd need to build something into the global Google Accounts infrastructure that allows for OAuth-like granting and revocation of access to specific applications. On Tue, Mar 2, 2010 at 1:53 PM, Toby toby.ro...@gmail.com wrote: Hello Ikai, I guess many of us have the same problem. It would be good to have some general advice on that. Unfortunatly GAE only offers build in authentication for the admin users or for users within a apps domain. So what I did is a simple table with the users gmail addresses that are allowed to access my app. If I can retrieve a user from the content and his/her email corresponds to the one I have saved, I let them in. UserServiceFactory.getUserService().getCurrentUser() If not I redirect them to the Google login page: response.sendRedirect(UserServiceFactory.getUserService().createLoginURL(redirect)); This solution works quite well but somehow I do not like it. I have the strong feeling that I build something that must already be there somehow. Also users have not the option to add my site as trusted so they have to log in whenever they come back to my page. I do not know how to work around that. The only advantage is that I can have alternatively the possibility for users to create an extra account on my site... You mentioned oauth and somewhere I read about friendconnect. I can not see how exactly that fits into the framework. Why cant we just have a google authentication build into GAE. With a users list as we do for the account admins and a simple rule to throw into the web.xml. +some way for users to self-register if the application developer wants that.This way no-one needs to reinvent the wheel. Does such a thing maybe already exist? Or is it on the road-map? Or is there a good approach you would advice? Thanks, Toby On Feb 22, 8:09 pm, Ikai L (Google) ika...@google.com wrote: Providing a login inside a frame is a compromise you should never, ever make. You're essentially training your users to be victims of phishing attacks. By providing a login in a frame, you're essentially removing every single security mechanism browsers provide to attempt to ensure users that the site they are on is really the site they are on and not a password stealing site. This is why many companies go out of their way to provide OAuth:http://oauth.net- because this allows client developers a way to authenticate users against another site's identity mechanism without having users send their credentials to a potentially untrusted site itself. Granted, there's a bit of a disconnect on login, but this is a price we'll have to pay just because this is one of the failings of browser security. Savvy users have already caught on to this, and more and more mainstream users will as well. This is a stopgap - when browsers are able to provide native authentication mechanisms, we shouldn't have to do this anymore, but we have a ways to go before this sort of thing will exist. On Sun, Feb 21, 2010 at 3:21 PM, John V Denley johnvden...@googlemail.comwrote: The frame works fine when logging in. If its a security risk please elaborate, Im onlt using Google accounts because I dont really know how to do my own security, and Im guessing that even using google via a frame is more secure than trying to do it myself! When creating an account it does not take the user back to the original page as there is a total disconnect after the user clicks on the link in the email sent from google. Google have informed me that this is a known issue, but has a low priority (which is understandable). I have now created what I think is a reasonable compromise. Only time will tell if our potential customers are ok with the process! On Feb 18, 7:04 pm, Brian bwa...@gmail.com wrote: You shouldn't use a frame. It is a security problem, and right of google login code to break out of it. After they make a new account, if not using a frame, I believe it forwards the user back to the page they were trying to go to. Seems to work pretty well. On Feb 18, 8:40 am, John V Denley johnvden...@googlemail.com wrote: I have been trying to leverage google accounts for security for my users, but the way its working is really preventing useability within my
[appengine-java] Re: Flash arcade game GAE based
Tim, But isn't it so expensive to use the AMF/HTTP comet technique on the appengine? CPU usage can be extremely high. Or you have to use AMF/ HTTP polling which is not so responsive for gaming. Thanks, Kaz On Mar 2, 7:43 pm, tsp...@green20now.com tsp...@green20now.com wrote: Take a look at GraniteDS or other AMF supporting tools. I am using GraniteDS to push data to a Flex client. It is fairly simple to setup so you can perform trsting and find the limitations which would impact your game. Tim Sent from my Verizon Wireless Phone - Reply message - From: Jeff Schnitzer j...@infohazard.org Date: Tue, Mar 2, 2010 3:50 AM Subject: [appengine-java] Flash arcade game GAE based To: google-appengine-java@googlegroups.com It would be extraordinarily difficult to use Appengine for this sort of project. You probably want a server framework that supports persistent connections and in-memory state that won't disappear when a memcache server is flushed. Here's a good starting point: http://en.wikipedia.org/wiki/List_of_game_engines Jeff On Sat, Feb 27, 2010 at 11:58 PM, Ahmed Khalifa derkhal...@gmail.com wrote: hi all, I am writing a multiplayer flash arcade game in actionscript .. i have discovered GAE recently and thought that it might be a very good choice for building and hosting my server .. however, i realize that arcade games need an almost realtime responsive capacity from the server .. besides the server has to be looping on receiving position object of Client A, storing it in a DB, Fetching Client B position object and sending it back .. this will result in a huge number of DB requests either storing, fetching or deleting which will quickly exhaust the CPU quota for the application .. So, I was wondering if any one had an idea or a reference to come around these two problems of real time response and CPU exhaustion by DB calls best regards, A. Khalifa -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Flash arcade game GAE based
FYI, I always use another messaging server along with appengine, on EC2 or something, which runs a socket relay server or Tornado (Python based long polling server). And I think WebSockets is the way to go for us in future ultimately :) Thanks, Kaz On Mar 3, 7:51 am, kazunori_279 kazunori...@gmail.com wrote: Tim, But isn't it so expensive to use the AMF/HTTP comet technique on the appengine? CPU usage can be extremely high. Or you have to use AMF/ HTTP polling which is not so responsive for gaming. Thanks, Kaz On Mar 2, 7:43 pm, tsp...@green20now.com tsp...@green20now.com wrote: Take a look at GraniteDS or other AMF supporting tools. I am using GraniteDS to push data to a Flex client. It is fairly simple to setup so you can perform trsting and find the limitations which would impact your game. Tim Sent from my Verizon Wireless Phone - Reply message - From: Jeff Schnitzer j...@infohazard.org Date: Tue, Mar 2, 2010 3:50 AM Subject: [appengine-java] Flash arcade game GAE based To: google-appengine-java@googlegroups.com It would be extraordinarily difficult to use Appengine for this sort of project. You probably want a server framework that supports persistent connections and in-memory state that won't disappear when a memcache server is flushed. Here's a good starting point: http://en.wikipedia.org/wiki/List_of_game_engines Jeff On Sat, Feb 27, 2010 at 11:58 PM, Ahmed Khalifa derkhal...@gmail.com wrote: hi all, I am writing a multiplayer flash arcade game in actionscript .. i have discovered GAE recently and thought that it might be a very good choice for building and hosting my server .. however, i realize that arcade games need an almost realtime responsive capacity from the server .. besides the server has to be looping on receiving position object of Client A, storing it in a DB, Fetching Client B position object and sending it back .. this will result in a huge number of DB requests either storing, fetching or deleting which will quickly exhaust the CPU quota for the application .. So, I was wondering if any one had an idea or a reference to come around these two problems of real time response and CPU exhaustion by DB calls best regards, A. Khalifa -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] GCacheException: Policy prevented put operation
I got few of those today Isn't it only for maintenance mode? com.google.appengine.api.memcache.stdimpl.GCacheException: Policy prevented put operation -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Request was aborted after waiting too long to attempt to service your request.
I have found I usually get this error when my app is loading and takes more than 10 seconds while another request come in. Does your app take longer than 10 seconds to load? Hum yes it may be this... But I don't know how to measure the loading time and how to reduce it. If a remove some useless jar file ? Vincent -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: App Engine cold starts and overly aggressive cycling
Hi.. I apologize for reviving this post, but this problem is really hitting me hard. I have an interactive fiction site (www.friedfiction.com) that has been experiencing some terrible response times over the past month. After investigating, it appears that my app is cold starting almost every two minutes. If someone is filling out a form or reading a story, and they've been idle for more than two minutes -- they have to wait approx 10-20 seconds for the app to cold start before it can process their request. This is giving my site a bad reputation. - What is your application ID? friedfic - How do you know it is being cycled out? You'll need to insert some code that only gets called when the app cold starts. Logging the servlet init - How much time of inactivity does it take before your application is cycled out? Every 2 minutes - What time or days does this seem to happen? All the time - What frameworks or libraries are you loading? GWT, AntiSamy On Jan 15, 6:32 pm, Ikai Lan i...@google.com wrote: Hey everybody, We've been seeing more and more reports of applications being cycled out overly aggressively, resulting in some folks implementing (discouraged) workarounds to keep their application from being cycled out. The primary symptom of this problem is that your application will see lots of loading requests that fire up a new JVM, which, as many of you know can take anywhere from a few seconds with naked servlets to as much as twenty seconds when loading something like Spring MVC, JRuby on Rails or Grails. In theory, there is enough capacity such that as long as you get some traffic every few hours, you should not be getting cycled out, but we have been seeing reports of applications being cycled after only a minute or less. To help us figure out if these are app specific issues or App Engine issues, can you post the following information if you believe this is happening to you? - What is your application ID? - How do you know it is being cycled out? You'll need to insert some code that only gets called when the app cold starts. - How much time of inactivity does it take before your application is cycled out? - What time or days does this seem to happen? - What frameworks or libraries are you loading? Any other information you can provide would be helpful. -- Ikai Lan Developer Programs Engineer, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Sporadic problems with very high response times
Looking at my App Engine logs, I see troubling results when viewing the response times for requests. In my current log set, the first 80 requests all complete in under 100 ms with less than 100 ms of cpu or api time. Then, oddly, the 83rd request, from the exact same client with the exact same request parameters, takes 7,192 ms to respond with 10,123 cpu ms (and 12 api ms). These kinds of spikes are dotted throughout my logs. They occur in less than 1% of cases, as far as I can tell, but the spikes are not just large; they're enormous. I know for a fact that the request parameters and returned data were identical to the requests several seconds before and after from the same client, but the request took about 20 times longer to serve. Does anyone know what causes these large spikes in response time, and can anyone share tricks to help alleviate these spikes? I know that it is somehow related to instantiating the JVM, but I don't know: - how to reduce the startup time of the JVM - how to predict when GAE will try to start a new JVM Thanks in advance for any advice, - Michael -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to use junit test in the newest SDK1.3.1
Thanks for starting this thread. I have my Junit tests working,
including my datastore tests, but I have a question regarding project
setup. I suppose this is more of an Eclipse question than anything
else, but perhaps you can help. I've worked in NetBeans prior, and I
can setup a 'test' folder with my unit test source. These classes are
excluded from the final war file. However, when I create a test folder
at the same level as my source folder:
Project
- src
- test
the only way I can make my tests work is to include test as a source
folder and add all the testing jars to my normal build path. This
means my test classes end up in the deployment.
Can I have my tests in the same project as my source, or do I need to
have a separate project?
Thanks for you help.
On Feb 11, 10:43 am, Krishna Caldas krishnacal...@gmail.com wrote:
Yes, I missed that method.
Maybe add this little trick on tutorial for now!?
Krishna
2010/2/11 Max Ross (Google) maxr+appeng...@google.com:
Great, glad to hear it! I forgot to add a method to LocalServiceTestHelper
to set a custom app id. My mistake. I'll make sure this gets added for the
next release so you don't need to provide your ownEnvironment
implementation.
Max
On Thu, Feb 11, 2010 at 10:30 AM, Krishna Caldas krishnacal...@gmail.com
wrote:
You're right!
I refactored my code and forget to annotate the new setUp method with
@Before.
Sorry for taking your time! It works now.
Thanks,
Krishna
2010/2/11 Max Ross (Google) maxr+appeng...@google.com:
Your code looks fine. Are you sure you're calling setUp() on the
LocalServiceTestHelper?
On Thu, Feb 11, 2010 at 10:11 AM, Krishna Caldas
krishnacal...@gmail.com
wrote:
Ooops.. missed your question!
It's just:
@Override
protectedEnvironmentnewEnvironment() {
return new TestEnvironment();
}
Thanks,
Krishna
2010/2/11 Max Ross (Google) maxr+appeng...@google.com:
Subclassing LocalServiceTestHelper and overriding newEnvironment()
should
work fine. What does your implementation of newEnvironment() look
like?
On Wed, Feb 10, 2010 at 7:35 PM, Krishna krishnacal...@gmail.com
wrote:
Ok!
But when I'm using transactions I'm getting:
java.lang.NullPointerException:NoAPIenvironmentisregisteredfor
thisthread.
at
com.google.appengine.api.datastore.DatastoreApiHelper.getCurrentAppId(Datas
toreApiHelper.java:
67)
at
com.google.appengine.api.datastore.DatastoreServiceImpl.beginTransaction(Da
tastoreServiceImpl.java:
270)
I've tried to extend LocalServiceTestHelper and overwrite
newEnvironment() with myEnvironment(who returns appId) but it
didn't work...
What's wrong?
Thanks,
Krishna
On Feb 10, 10:25 pm, Ikai L (Google) ika...@google.com wrote:
We've got a more simple interface for you now. Take a look:
http://code.google.com/appengine/docs/java/tools/localunittesting.html
2010/2/10 时空之蕊 skzr@gmail.com
I found the class
com.google.appengine.tools.development.ApiProxyLocalImpl is not
public,but before 1.3.1 it's public!
In the JUnit document:
import java.io.File;
import com.google.appengine.tools.development.ApiProxyLocalImpl;
import com.google.apphosting.api.ApiProxy;
ApiProxy.setDelegate(new ApiProxyLocalImpl(new File(.)){});
So I can't new a ApiProxyLocalImpl instance!
Any body know how to use JUnit?
Thanks :)
--
You received this message because you are subscribed to the
Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B
unsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Ikai Lan
Developer Programs Engineer, Google App
Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more
[appengine-java] 「sdc_internet_routing : NO_MATCHI NG_RESOURCE」 when trying to fetch URL via SDC
hello everyone I cannot fetch intranet resource via sdc. The following are my setting, Could someone kindly help me out: my localConfig.xml: entity sdcServerHostapps-secure-data-connector.google.com/sdcServerHost sdcServerPort443/sdcServerPort domainmydomain.com/domain usersecure-data-connector-user/user passwordmypwd/password agentIdagent_mydomain/agentId socksServerPort1080/socksServerPort sshd/etc/opt/google-secure-data-connector/openssh/start_sshd.sh/ sshd healthCheckGadgetUsersadmi...@mydomain.com/ healthCheckGadgetUsers /entity my resourceRules.xml: resourceRules rule repeatable=true ruleNum1/ruleNum agentIdagent_mydomain/agentId viewerEmail repeatable=trueadmi...@mydomain.com/ viewerEmail apps repeatable=true serviceAppEngine/service allowAnyAppIdtrue/allowAnyAppId /apps urlhttp://myintranetIP:8080/tomcat.gif/url urlMatchHOSTPORT/urlMatch /rule /resourceRules my java sourcecode: URL dataURL = new URL(http://myintranetIP:8080/tomcat.gif?X- secureDataConnectorDebug=text); HTTPRequest fetchreq = new HTTPRequest(dataURL); fetchreq.setHeader(new HTTPHeader(use_intranet,yes)); HTTPResponse fetchresp = fetcher.fetch(fetchreq); resp.setContentType(text/plain); resp.getWriter().print(new String(fetchresp.getContent())); Actual results: request_user : admi...@mydomain.com request_sdc_agent_domain : mydomain.com request_url : http://myintranetIP:8080/tomcat.gif?X-secureDataConnectorDebug=text request_service : AppEngine request_appId : myappid request_requestId : (Id = bf33d834f932802d) sdc_routing : Internet sdc_internet_routing : NO_MATCHING_RESOURCE response_sdc_status : ok response_content_size : 0 BTW: and I have open ports(8080 , 1080) in the firewall between sdc server and intranet webserver , I can fetch the intranet resource by executing a java program in sdc server. when use netstat in sdc server , I can see the connection between gae and my sdc server is ESTABLISHED. thanks in advance! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Flash arcade game GAE based
Kaz, Not sure about the CPU (we are still at the proof of concept stage for using GAE). We are using the polling aspect since our business requires updates with 5-15 minutes. Its not very time intensive like a game would be. Good luck, Tim Tim, But isn't it so expensive to use the AMF/HTTP comet technique on the appengine? CPU usage can be extremely high. Or you have to use AMF/ HTTP polling which is not so responsive for gaming. Thanks, Kaz On Mar 2, 7:43 pm, tsp...@green20now.com tsp...@green20now.com wrote: Take a look at GraniteDS or other AMF supporting tools. I am using GraniteDS to push data to a Flex client. It is fairly simple to setup so you can perform trsting and find the limitations which would impact your game. Tim Sent from my Verizon Wireless Phone - Reply message - From: Jeff Schnitzer j...@infohazard.org Date: Tue, Mar 2, 2010 3:50 AM Subject: [appengine-java] Flash arcade game GAE based To: google-appengine-java@googlegroups.com It would be extraordinarily difficult to use Appengine for this sort of project. You probably want a server framework that supports persistent connections and in-memory state that won't disappear when a memcache server is flushed. Here's a good starting point: http://en.wikipedia.org/wiki/List_of_game_engines Jeff On Sat, Feb 27, 2010 at 11:58 PM, Ahmed Khalifa derkhal...@gmail.com wrote: hi all, I am writing a multiplayer flash arcade game in actionscript .. i have discovered GAE recently and thought that it might be a very good choice for building and hosting my server .. however, i realize that arcade games need an almost realtime responsive capacity from the server .. besides the server has to be looping on receiving position object of Client A, storing it in a DB, Fetching Client B position object and sending it back .. this will result in a huge number of DB requests either storing, fetching or deleting which will quickly exhaust the CPU quota for the application .. So, I was wondering if any one had an idea or a reference to come around these two problems of real time response and CPU exhaustion by DB calls best regards, A. Khalifa -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to get testing working with anything other than the default queue
I'm using the workaround above and it's reading my queue.xml correctly; however, when I call runTask() I am getting a connection refused exception. Does the AppEngine test environment provide a servlet container or mock thereof for running the task servlet? The task servlet is configured in my web.xml and works fine in dev and prod, but not in unit tests, so I'm guessing I need to wire in a lightweight servlet container in test also... would appreciate any pointers. INFO: Local task queue initialized with base url http://localhost:8080 Mar 2, 2010 11:33:05 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused: connect Mar 2, 2010 11:33:05 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request Thank you, /dmc On Feb 16, 6:08 pm, Max Ross (Google) maxr+appeng...@google.com wrote: Sorry Will, we look for ./WEB-INF/queue.xml by default. Anyway glad you got it working with the workaround. I'll get this straightened out for the next release. Max On Tue, Feb 16, 2010 at 11:41AM, Will Bunker w...@thebunkers.com wrote: I put in queue.xml in the test directory and it didn't seem to find it. On Feb 16, 10:47am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com wrote: I see. In the test environment it looks in . by default so if you place queue.xml in the directory from which you're executing the test it should pick it up.I On Tue, Feb 16, 2010 at 10:35AM, Will Bunker w...@thebunkers.com wrote: I am saying that it is not reading queue.xml. I am testing to make sure a certain function puts x number tasks in a queue that is not the default. It doesn't load the queue.xml file unless I use your workaround (then it works great.) On Feb 16, 10:21am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com maxr%2bappeng...@google.com maxr%252bappeng...@google.com wrote: By default the LocalTaskQueueTestConfig configures the local task queue service to not automatically execute tasks: http://code.google.com/appengine/docs/java/tools/localunittesting/jav. .. Or are you saying it's not reading queue.xml? On Tue, Feb 16, 2010 at 10:16AM, Will Bunker w...@thebunkers.com wrote: Actually it doesn't seem to be reading the queue at all. It is coming from the standard directory in WAR, but doesn't seem to pick it up. I am on Mac OS if that makes any difference. On Feb 16, 9:35am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com maxr%2bappeng...@google.com maxr%252bappeng...@google.com maxr%2bappeng...@google.com maxr%252bappeng...@google.com maxr%252bappeng...@google.com maxr%25252bappeng...@google.com wrote: Hi Will, You're loading queue.xml from a different location in your unit tests? This is a use case I didn't thoroughly consider. I can certainly fix this for the next release but let me see if I can find a workaround for you. Thanks, Max -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com google-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%252bunsubscr...@googlegroups.com google-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%252bunsubscr...@googlegroups.com google-appengine-java%252bunsubscr...@googlegroups.comgoogle-appengine-java%25252bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com google-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%252bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more
Re: [appengine-java] Re: Flash arcade game GAE based
I want to do the same. Is there any good XMPP Java framework to include in my Applet? recomendations? Thanks NM On Tue, Mar 2, 2010 at 11:31 PM, tsp...@green20now.com wrote: Kaz, Not sure about the CPU (we are still at the proof of concept stage for using GAE). We are using the polling aspect since our business requires updates with 5-15 minutes. Its not very time intensive like a game would be. Good luck, Tim Tim, But isn't it so expensive to use the AMF/HTTP comet technique on the appengine? CPU usage can be extremely high. Or you have to use AMF/ HTTP polling which is not so responsive for gaming. Thanks, Kaz On Mar 2, 7:43 pm, tsp...@green20now.com tsp...@green20now.com wrote: Take a look at GraniteDS or other AMF supporting tools. I am using GraniteDS to push data to a Flex client. It is fairly simple to setup so you can perform trsting and find the limitations which would impact your game. Tim Sent from my Verizon Wireless Phone - Reply message - From: Jeff Schnitzer j...@infohazard.org Date: Tue, Mar 2, 2010 3:50 AM Subject: [appengine-java] Flash arcade game GAE based To: google-appengine-java@googlegroups.com It would be extraordinarily difficult to use Appengine for this sort of project. You probably want a server framework that supports persistent connections and in-memory state that won't disappear when a memcache server is flushed. Here's a good starting point: http://en.wikipedia.org/wiki/List_of_game_engines Jeff On Sat, Feb 27, 2010 at 11:58 PM, Ahmed Khalifa derkhal...@gmail.com wrote: hi all, I am writing a multiplayer flash arcade game in actionscript .. i have discovered GAE recently and thought that it might be a very good choice for building and hosting my server .. however, i realize that arcade games need an almost realtime responsive capacity from the server .. besides the server has to be looping on receiving position object of Client A, storing it in a DB, Fetching Client B position object and sending it back .. this will result in a huge number of DB requests either storing, fetching or deleting which will quickly exhaust the CPU quota for the application .. So, I was wondering if any one had an idea or a reference to come around these two problems of real time response and CPU exhaustion by DB calls best regards, A. Khalifa -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Difficulty querying webserver through POSTing XML text
Any takers? I've carried the code over to a regular Java Application
and it runs through without a hitch. Hmm, looks I am stuck with a GAE
bug?
On Mar 2, 6:54 am, Joa joachim.pfeif...@gmail.com wrote:
I am having difficulty POSTing and reading from a web server from GAE.
I can authenticate but the webserver does not accept the XML file I am
POSTing (which includes a query). I am not running the webserver in
question, so I have no insight into what might be wrong at that end
(or set it up differently).
First, a sanitized snippet of the curl statement that returns the
desired result (on OSX):
$ curlhttp://targetwebserver.org/servlet-u username:password -d '?
xml version=1.0 encoding=UTF-8 standalone=yes?Tag /
Tag'
That to me confirm the webserver works alright.
Below now a snippet of the code (sanitized and rough) that I cannot
get to work. Again, authentication seems to work (I am not getting
responseCode 401), but whatever I have tried, I cannot get over
responseCode 400 and a corresponding boilerplate error message from
the webserver.
- snip ---
String requestString = ?xml version=1.0 encoding=UTF-8
standalone=yes?Tag /Tag; // Also tried all sorts of
encoding, no luck
URL url = new URL(targetwebserver.org/servlet );
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod(POST);
conn.setDoOutput(true);
conn.setDoInput(true);
conn.addRequestProperty(Content-type, text/xml); // tried
application/xml as well, no luck
// Authenticate (seems to work OK)
String authData = username:password;
byte[] encodedAuthData = new
org.apache.commons.codec.binary.Base64().encode (authData.getBytes());
String authString = Basic ;
for (int i = 0; i encodedAuthData.length; i++)
authString += (char)encodedAuthData[i];
conn.addRequestProperty (Authorization, authString);
// Post request
PrintWriter pw = new PrintWriter(conn.getOutputStream()); // tried
OutputStreamWriter as well, no luck
pw.println(requestString);
pw.close();
responseCode = conn.getResponseCode();
if (responseCode == 200) {
InputStream in = conn.getInputStream();
BufferedReader rd = new BufferedReader(new
InputStreamReader(conn.getInputStream()));
String line; while ((line = rd.readLine()) != null)
{ log.info(line); }}
- snip ---
Please have a look at this - help's appreciated!
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Difficulty querying webserver through POSTing XML text
OK, please disregard. Issue resolved.
org.apache.commons.codec.binary.Base64().encode()
adds control characters \r\n to the end of the authorization string.
Although this should keep the response code add 401, it doesn't, so I
was looking at the wrong spot last night. Hope this helps somebody
else out who might run into a similar problem.
On Mar 2, 9:49 pm, Joa joachim.pfeif...@gmail.com wrote:
Any takers? I've carried the code over to a regular Java Application
and it runs through without a hitch. Hmm, looks I am stuck with a GAE
bug?
On Mar 2, 6:54 am, Joa joachim.pfeif...@gmail.com wrote:
I am having difficulty POSTing and reading from a web server from GAE.
I can authenticate but the webserver does not accept the XML file I am
POSTing (which includes a query). I am not running the webserver in
question, so I have no insight into what might be wrong at that end
(or set it up differently).
First, a sanitized snippet of the curl statement that returns the
desired result (on OSX):
$ curlhttp://targetwebserver.org/servlet-uusername:password -d '?
xml version=1.0 encoding=UTF-8 standalone=yes?Tag /
Tag'
That to me confirm the webserver works alright.
Below now a snippet of the code (sanitized and rough) that I cannot
get to work. Again, authentication seems to work (I am not getting
responseCode 401), but whatever I have tried, I cannot get over
responseCode 400 and a corresponding boilerplate error message from
the webserver.
- snip ---
String requestString = ?xml version=1.0 encoding=UTF-8
standalone=yes?Tag /Tag; // Also tried all sorts of
encoding, no luck
URL url = new URL(targetwebserver.org/servlet );
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod(POST);
conn.setDoOutput(true);
conn.setDoInput(true);
conn.addRequestProperty(Content-type, text/xml); // tried
application/xml as well, no luck
// Authenticate (seems to work OK)
String authData = username:password;
byte[] encodedAuthData = new
org.apache.commons.codec.binary.Base64().encode (authData.getBytes());
String authString = Basic ;
for (int i = 0; i encodedAuthData.length; i++)
authString += (char)encodedAuthData[i];
conn.addRequestProperty (Authorization, authString);
// Post request
PrintWriter pw = new PrintWriter(conn.getOutputStream()); // tried
OutputStreamWriter as well, no luck
pw.println(requestString);
pw.close();
responseCode = conn.getResponseCode();
if (responseCode == 200) {
InputStream in = conn.getInputStream();
BufferedReader rd = new BufferedReader(new
InputStreamReader(conn.getInputStream()));
String line; while ((line = rd.readLine()) != null)
{ log.info(line); }}
- snip ---
Please have a look at this - help's appreciated!
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: App upload failed
I removed gcj from the system, and now everything works, thanks! But, I can't understand why appengine chose GCJ rather than Sun's JDK. On 2 мар, 22:13, Toby Reyelts to...@google.com wrote: It looks like you're usinggcj gcj. Can you use a different JVM, such as OpenJDK or Sun's JDK? On Tue, Mar 2, 2010 at 3:44 AM, ast anton.star...@gmail.com wrote: Trying upload app: ./appcfg.sh update ../../../projects/appengine/myapp/www/ Reading application configuration data... Encountered a problem: null Please see the logs [/tmp/appcfgx2i014.log] for further information. an...@ast:/mnt/sda1/projects/appengine/myapp$ cat /tmp/ appcfgx2i014.log java.lang.NullPointerException at org.relaxng.datatype.helpers.DatatypeLibraryLoader$Service $Loader2.getResources(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader $Service.init(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaBuilder.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaSchemaFactory.newSchema(libgcj.so. 10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at com.google.appengine.tools.admin.Application.validateXml(Application.java: 319) at com.google.appengine.tools.admin.Application.init(Application.java: 87) at com.google.appengine.tools.admin.application.readaan...@ast:/mnt/ sda1/projects/appengine/myapp Same app was uploaded successfully from my home computer. Any suggestions? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How do you transfer a ListJdoObject from server to client?
Hello,
I have been wanting to know how to do this for a while now, but don't know
where to find information on how to do this. I simply want to know how to
setup my project so I can have a JDO object in the server package (so I
can use classes like com.google.appengine.api.datastore.Key and
javax.jdo.listener.StoreCallback), and request a list of those objects from
my client side code. Currently I store my JDO object in my client package.
So when a user clicks a button to retrieve all the People objects I can get
back a ListPeople. I'm sure these are some fundamentally basic steps I
just don't know to acheive. Do you need a persistent object on the server
side and a transfer object on the client? If so how do you send the data to
the transfer object. Thanks
currently have: (and want it instead in com.myapp.server but also to
retrieve say ListPeople )
in com.myapp.client
import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;
import com.google.gwt.user.client.rpc.IsSerializable;
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class People implements IsSerializable {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long pKeyRiderID;
@Persistent
private String lastName;
...
currently in com.myapp.server:
private ListPeople get() {
PersistenceManager pm = PMF.get().getPersistenceManager();
ListPeople result = null;
try {
String q = select from + People.class.getName();
result = new ArrayListPeople((ListPeople)
pm.newQuery(q).execute());
} finally {
pm.close();
}
return result;
}
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: multiple inserts/update/delete problem
thanks! On Mar 2, 12:28 pm, Chau Huynh cmhu...@gmail.com wrote: You can try searching the error in the group, it was QA many times, and explained in detail cant operate on multiple entity groups in a single transaction site:groups.google.comhttp://tinyurl.com/yeto4to On Tue, Mar 2, 2010 at 8:06 AM, legendlink gregc...@gmail.com wrote: hi, i have tried to insert multiple insert, using multiple pm.makePersistent calls, but exceprion always occurrs. exception thrown: javax.jdo.jdofataluserexception:illegal argument cant operate on multiple entity groups in a single transaction i wanted to try to do batch inserts. all of the entities should be successfully inserted because they are related to each other i also tried the pm.makePersistentAll but a single request to datastore is limited to 1mb. is there a workaround or am i doing it the wrong way? thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] elegant way of implementing sequence generator
hi, i wanted to have a sequence generator that increments by x value everytime it generates a value. if i would create the sequence generator by using the datastore, it is likely that data contention would occurr if there is high access times. i have looked into the sample code of max ross in the google code repository (SequenceExamplesJDO.java) and think this is limited to increment by 1 only and not increment by x value. if sharding technique is used, my concern is that i might not get the right sequence. what is the best/elegant way of doing sequence generator that increments x value? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[google-appengine] BlobProperty limit
Hi, Is it possible to increase the limit for blobproperty size upto 10 MB? We are not able to use the blobstore api as mentioned in the docs for sizes more then 1 MB because we want to do fine grained operations on the blob. What we are trying to store is a JSON datastructure(pickled) and not something like video. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Roadmap updates
The roadmap hasn't been updated in a while. Things have been pulled off it at a steady pace but nothing has gone in. Of the current 3 items remaining mapping across datasets looks interesting and hard, but alerting doesn't seem like rocket science and the dump and restore is mostly done. Are you guys running out of things to do? :-) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How many IPs would the urlfetch use?
You shouldn't use a security model based on ensuring that requests come from the app engine IP block even if it were possible; literally anyone could be running malicious code on appspot making this at best marginally more secure than allowing requests from any IP. There are countless techniques for doing real authentication. On Mar 1, 11:15 pm, Iap iap...@gmail.com wrote: Hi, I have a back-end web service which I would only allow the requests from my GAE application. So I want to check the IP if it comes from the GAE urlfetch. (plus some secret token) From my testing, there is only one IP: 64.233.172.18 was found. I am curious about if there are other IPs that might be used by the urlfetch? Because that only one IP seems to be weird for the so-called cloud architecture. Just for confirmation. My Test URL is:http://password-recovery.appspot.com/prs/test/urlfetchrequest (That will fetching thehttp://remote.12dt.com/which responses the requsting ip) Thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] BlobProperty limit
Is it possible to increase the limit for blobproperty size upto 10 MB? no. Split the Blob and store in 10 Separate entities. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91 HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please
seehttp://code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually much
faster now. Gary and our other devs have reworked it. I need updated
numbers, but they don't take 10s, probably more like 2s. We still have
some heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now with
the current beta that is in production, but it is low traffic at the
moment. We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last email, I have
made contact with a Developer Advocate and whatnot, which is fantastic. It
looks like we, as a business, will be able to have better contact with the
GAE team. We would very much like to continue working with you to figure
out what actions we can take and what provisioning we can do to make our
product successful and scale it as we grow in the near future. Gary Orser
will be replying to this thread soon with more findings from both our real
app code and a little test app we are using and which he will share with
you.
We plan on having a presence at Google I/O this year as we did at PyCon.
Hopefully we can even get setup in the demonstration area at I/O.
Thanks Nick for your help. Could we possibly setup a quick skype conf call
at some point?
-Mike Wesner
On Mar 1, 2010, at 1:13 PM, Michael Wesner wrote:
Nick,
If we (I work with Gary) require fairly heavy requests which run for
multiple seconds then it is not possible to get anywhere near 400 QPS.
The math used on the docs page only applies to 75ms requests.
(1000 ms/second / 75 ms/request) * 30 = 400 requests/second
so lets say each request takes 10 seconds (and ours, pulling data to EC2
for a heavy operation that we can't do on GAE could take that much since
we have to process and update some XML before sending it)
(1000 ms/second / 1 ms/request) * 30 = 3 requests/second
And that does not even take into account all the other traffic to our
application, nor the fact that many users could be doing this same heavy
operation at the same time. Our application will see spikes in this type
of activity also. The docs also mention that CPU heavy apps incur
penalties, which is vague and scary.
Great effort is put into doing things in the most efficient way possible,
but not everyones apps can do everything in 75ms. Most all of our service
calls are under 250ms. We do have a little overhead from our framework
which we are constantly working on improving. Our application is AMF
service/object based which is inherently heavy compared to simple web
requests. It limits the amount of memcache work we can do also, but we
are also working on improving our use of that.
We easily hit these boundaries during
Re: [google-appengine] Re: Parallel access to appengine
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91 HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually
much faster now. Gary and our other devs have reworked it. I need updated
numbers, but they don't take 10s, probably more like 2s. We still have some
heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now with
the current beta that is in production, but it is low traffic at the moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last email, I
have made contact with a Developer Advocate and whatnot, which is fantastic.
It looks like we, as a business, will be able to have better contact with
the GAE team. We would very much like to continue working with you to figure
out what actions we can take and what provisioning we can do to make our
product successful and scale it as we grow in the near future. Gary Orser
will be replying to this thread soon with more findings from both our real
app code and a little test app we are using and which he will share with
you.
We plan on having a presence at Google I/O this year as we did at
PyCon. Hopefully we can even get setup in the demonstration area at I/O.
Thanks Nick for your help. Could we possibly setup a quick skype conf
call at some point?
-Mike Wesner
On Mar 1, 2010, at 1:13 PM, Michael Wesner wrote:
Nick,
If we (I work with Gary) require fairly heavy requests which run for
multiple seconds then it is not possible to get anywhere near 400 QPS. The
math used on the docs page only applies to 75ms requests.
(1000 ms/second / 75 ms/request) * 30 = 400 requests/second
so lets say each request takes 10 seconds (and ours, pulling data to
EC2 for a heavy operation that we can't do on GAE could take that much since
we have to process and update some XML before sending it)
(1000 ms/second / 1 ms/request) * 30 = 3 requests/second
And that does not even take into account all the other traffic to our
application, nor the fact that many users could be doing this same heavy
operation at the same time. Our application will see spikes in this type of
activity also. The docs also mention that CPU heavy apps incur penalties,
which is vague and scary.
Great effort is put into doing things in the most efficient way
possible, but not everyones apps can do everything in 75ms. Most all of our
service calls are under 250ms. We do have a little overhead from our
framework which we are constantly working on improving. Our application is
AMF service/object based which is inherently heavy compared to simple web
requests. It limits
[google-appengine] I can't install my project
hello: I upload my project to app engine but i can't init data, when i run install i get below error; my install page is http://myappforum.appspot.com/install.jsp when i do it on my workspace it's work well.and less then 30 seconds. tip: I have less than 2000 data init. Error for /install.jsp com.google.apphosting.runtime.HardDeadlineExceededError: This request (bea8a6558c35cf56) started at 2010/03/02 14:04:51.484 UTC and was still executing at 2010/03/02 14:05:20.379 UTC. mail me thanks, sometimes i can't visit this website. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Strange issue with a where condition in JPA query
as usually the problem is in front of PC :) The error isn't in JPA but in some works made via reflection. :) 2010/2/27 Davide Cerbo davidece...@gmail.com excuse but i'm tired, the correct example is: t...@example.com - empty result set test @ example.com - works well test.example - empty result set test . exaple - empty result set testexample - works well the issue is only with dot char :) 2010/2/27 Davide Cerbo davidece...@gmail.com Hi all, I have a single query executed with JPA: select m from Message m where m.confirmed = true and m.name = ?1 The values of name field are sometime string with dot (.) and atpersand (@) char. When I use as param a value that contains this char the result is always empty. Example: t...@example - empty result set test @ example - works well test.example - empty result set test . exaple - empty result set testexample - works well Why? what happens? thanks in advance. Davide -- -- Davide Cerbo --- http://davide.cerbo.born-to-co.de http://jesty.it --- http://www.exmachina.ch + 39 329 70 81 927 --- -- -- Davide Cerbo --- http://davide.cerbo.born-to-co.de http://jesty.it --- http://www.exmachina.ch + 39 329 70 81 927 --- -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
But that's the point. I can not reach 30 active requests.
I can only reach 10 active requests without error.
Any ideas on how I can debug this?
Cheers, Gary.
On Mar 2, 7:05 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91 HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually
much faster now. Gary and our other devs have reworked it. I need updated
numbers, but they don't take 10s, probably more like 2s. We still have some
heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now with
the current beta that is in production, but it is low traffic at the moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last email, I
have made contact with a Developer Advocate and whatnot, which is fantastic.
It looks like we, as a business, will be able to have better contact with
the GAE team. We would very much like to continue working with you to figure
out what actions we can take and what provisioning we can do to make our
product successful and scale it as we grow in the near future. Gary Orser
will be replying to this thread soon with more findings from both our real
app code and a little test app we are using and which he will share with
you.
We plan on having a presence at Google I/O this year as we did at
PyCon. Hopefully we can even get setup in the demonstration area at I/O.
Thanks Nick for your help. Could we possibly setup a quick skype conf
call at some point?
-Mike Wesner
On Mar 1, 2010, at 1:13 PM, Michael Wesner wrote:
Nick,
If we (I work with Gary) require fairly heavy requests which run for
multiple seconds then it is not possible to get anywhere near 400 QPS. The
math used on the docs page only applies to 75ms requests.
(1000 ms/second / 75 ms/request) * 30 = 400 requests/second
so lets say each request takes 10 seconds (and ours, pulling data to
EC2 for a heavy operation that we can't do on GAE could take that much since
we have to process and update some XML before sending it)
(1000 ms/second / 1 ms/request) * 30 = 3 requests/second
And that does not even take into account all the other traffic to our
application, nor the fact that many users could be doing this same heavy
operation at the same time. Our application will see spikes in this type of
activity also. The docs also mention that CPU heavy apps incur penalties,
which is vague and scary.
Great effort is put into doing things in the most efficient way
possible, but not
Re: [google-appengine] Re: Parallel access to appengine
Are these threads you're using (at this point, it really seems like you
should post some simplified code to illustrate the issue at hand) waiting
for their response before trying to get again?
Posting some code to help recreate this issue will lead to a much faster
resolution.. as it stands.. I just know that someone on the internet has 10
threads that are hitting a dynamic request limit.
I also know that in the initial e-mail, when the request took longer to
return.. these threads were hitting a lower dynamic request limit (only 4
could run). This suggest that there is an important detail to how your
threads are doing their work.. and we would need that to provide useful
help.
Thanks for info.
On Tue, Mar 2, 2010 at 10:01 AM, Gary Orser garyor...@gmail.com wrote:
But that's the point. I can not reach 30 active requests.
I can only reach 10 active requests without error.
Any ideas on how I can debug this?
Cheers, Gary.
On Mar 2, 7:05 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas
for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91
HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com
wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually
much faster now. Gary and our other devs have reworked it. I need
updated
numbers, but they don't take 10s, probably more like 2s. We still have
some
heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now
with
the current beta that is in production, but it is low traffic at the
moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last
email, I
have made contact with a Developer Advocate and whatnot, which is
fantastic.
It looks like we, as a business, will be able to have better contact
with
the GAE team. We would very much like to continue working with you to
figure
out what actions we can take and what provisioning we can do to make
our
product successful and scale it as we grow in the near future. Gary
Orser
will be replying to this thread soon with more findings from both our
real
app code and a little test app we are using and which he will share
with
you.
We plan on having a presence at Google I/O this year as we did at
PyCon. Hopefully we can even get setup in the demonstration area at
I/O.
Thanks Nick for your help. Could we possibly setup a quick skype
conf
call at some point?
-Mike Wesner
On Mar 1, 2010, at 1:13 PM, Michael Wesner wrote:
Nick,
If we (I work with Gary) require fairly heavy requests which run
for
multiple
Re: [google-appengine] Roadmap updates
Hi Kenneth, We intend to update the roadmap in the near future. Rest assured we are definitely not running out of things to do. :) -Nick On Tue, Mar 2, 2010 at 1:43 PM, Kenneth goo...@kmacleod.ie wrote: The roadmap hasn't been updated in a while. Things have been pulled off it at a steady pace but nothing has gone in. Of the current 3 items remaining mapping across datasets looks interesting and hard, but alerting doesn't seem like rocket science and the dump and restore is mostly done. Are you guys running out of things to do? :-) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
But that's the point. I can not reach 30 active requests.
I can only reach 10 active requests without error.
Any ideas on how I can debug this?
Cheers, Gary.
On Mar 2, 7:05 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91 HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually
much faster now. Gary and our other devs have reworked it. I need updated
numbers, but they don't take 10s, probably more like 2s. We still have some
heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now with
the current beta that is in production, but it is low traffic at the moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last email, I
have made contact with a Developer Advocate and whatnot, which is fantastic.
It looks like we, as a business, will be able to have better contact with
the GAE team. We would very much like to continue working with you to figure
out what actions we can take and what provisioning we can do to make our
product successful and scale it as we grow in the near future. Gary Orser
will be replying to this thread soon with more findings from both our real
app code and a little test app we are using and which he will share with
you.
We plan on having a presence at Google I/O this year as we did at
PyCon. Hopefully we can even get setup in the demonstration area at I/O.
Thanks Nick for your help. Could we possibly setup a quick skype conf
call at some point?
-Mike Wesner
On Mar 1, 2010, at 1:13 PM, Michael Wesner wrote:
Nick,
If we (I work with Gary) require fairly heavy requests which run for
multiple seconds then it is not possible to get anywhere near 400 QPS. The
math used on the docs page only applies to 75ms requests.
(1000 ms/second / 75 ms/request) * 30 = 400 requests/second
so lets say each request takes 10 seconds (and ours, pulling data to
EC2 for a heavy operation that we can't do on GAE could take that much since
we have to process and update some XML before sending it)
(1000 ms/second / 1 ms/request) * 30 = 3 requests/second
And that does not even take into account all the other traffic to our
application, nor the fact that many users could be doing this same heavy
operation at the same time. Our application will see spikes in this type of
activity also. The docs also mention that CPU heavy apps incur penalties,
which is vague and scary.
Great effort is put into doing things in the most efficient way
possible, but not
[google-appengine] Trouble subclassing PolyModel
Among other methods, I'm trying to catch the put() method of both db.Model
and db.polymodel.PolyModel, so that I can set the entity in Memcache (after
it's been put into the Datastore).
I'm using the same common mixin class for both. The setup looks something
like this:
from google.appengine.ext.db.polymodel import PolyModel
from google.appengine.ext import db
class ModelMixin(object):
def __init__(self, *args, **kwds):
self.do_something_cool()
def do_something_cool(self):
...
def put(self, *args, **kwds):
super(ModelMixin, self).put(*args, **kwds)
memcache.set('EntityKey:'+self.key(), db.model_to_protobuf(self))
class MyModel(ModelMixin, db.Model):
def __init__(self, *args, **kwds):
db.Model.__init__(self, *args, **kwds)
ModelMixin.__init__(self, *args, **kwds)
class MyPolyModel(ModelMixin, PolyModel):
def __init__(self, *args, **kwds):
PolyModel.__init__(self, *args, **kwds)
ModelMixin.__init__(self, *args, **kwds)
Subclassing from MyModel is working fine, but when I try and subclass
MyPolyModel, eg:
class Animal(MyPolyModel):
...
class Cat(Animal):
...
I get this error on the dev appserver (I haven't tried it in production):
AttributeError: type object 'Animal' has no attribute '__root_class__'
Can someone suggest a way to fix this?
I'm also worried that, even if I get this fixed, the entity kind of those
subclasses of MyPolyModel will be 'MyPolyModel' instead of eg. 'Animal'
(because the direct subclass of PolyModel is MyPolyModel and not Animal). I
hope there's a way around that as well, and welcome suggestions.
Nick
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: appcfg.py error
I moved the remote_api handler to the top of my app.yaml file, but I got the same appcfg.py error. Just to clarify, when I call C:\Program Files (x86)\Google\google_appengine python appcfg.py upload_data --config_file=DataStoreLoader.py --filename=stimulus_set.csv --kind=DataStore face-eval --has_header I am calling appcfg.py with the config_file in my app-directory, which is face-eval in this case (a subdirectory in google_appengine). Also, the file that I want to uplpoad, stimulus_set.csv is also in the face-eval app directory, along with the DataStoreLoader class. The $PYTHON_LIB variable is at the head of the directory structure pointing the handler in app.yaml. I'm not sure if that variable is being interpreted correctly, causing the problem. I appreciate your help. On Mar 1, 7:01 pm, Eli Jones eli.jo...@gmail.com wrote: Is the remote_api handler at the very top of your app.yaml? if there is something like: - url: /.* script: myscript Above the remote_api handler.. then it won't work.On Mon, Mar 1, 2010 at 8:43 PM, pkhoosh pkho...@gmail.com wrote: I am trying to upload a csv file to a data store. I uploaded my app.yaml file with the following: - url: /remote_api script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py login: admin I wrote a python handler in my application directory (DataStoreLoader.py). However, I get the following when I upload: C:\Program Files (x86)\Google\google_appenginepython appcfg.py upload_data --config_file=DataStoreLoader.py -- filename=stimulus_set.csv --kind=DataStore face-eval --has_header Application: face-eval; version: 1. Usage: appcfg.py [options] upload_data directory appcfg.py: error: You must have google.appengine.ext.remote_api.handler assigned to an endpoint in app.yaml, or provide the url of the handler via the 'url' opt ion. Any help? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] how to work with blobs?
hello together, i want to upload and download binary data. so i have to use blobs. is there any tutorial or some instructions how i have to use blobs in GAE? perfectly, would be some instructions in java... greatz flo -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
Actually, 4 threads was before we optimized server side, and set up
the test environment.
I have a tarball, about 8mb, with the test environment. (django and
libraries, grrr)
What is the best way to post this? I don't see any file attachments
on groups.
Cheers, Gary
On Mar 2, 8:23 am, Eli Jones eli.jo...@gmail.com wrote:
Are these threads you're using (at this point, it really seems like you
should post some simplified code to illustrate the issue at hand) waiting
for their response before trying to get again?
Posting some code to help recreate this issue will lead to a much faster
resolution.. as it stands.. I just know that someone on the internet has 10
threads that are hitting a dynamic request limit.
I also know that in the initial e-mail, when the request took longer to
return.. these threads were hitting a lower dynamic request limit (only 4
could run). This suggest that there is an important detail to how your
threads are doing their work.. and we would need that to provide useful
help.
Thanks for info.
On Tue, Mar 2, 2010 at 10:01 AM, Gary Orser garyor...@gmail.com wrote:
But that's the point. I can not reach 30 active requests.
I can only reach 10 active requests without error.
Any ideas on how I can debug this?
Cheers, Gary.
On Mar 2, 7:05 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas
for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91
HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com
wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually
much faster now. Gary and our other devs have reworked it. I need
updated
numbers, but they don't take 10s, probably more like 2s. We still have
some
heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now
with
the current beta that is in production, but it is low traffic at the
moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last
email, I
have made contact with a Developer Advocate and whatnot, which is
fantastic.
It looks like we, as a business, will be able to have better contact
with
the GAE team. We would very much like to continue working with you to
figure
out what actions we can take and what provisioning we can do to make
our
product successful and scale it as we grow in the near future. Gary
Orser
will be replying to this thread soon with more findings from both our
real
app code and a little test app we are using and which he will share
with
you.
We plan on having a presence
[google-appengine] Re: The server encountered an error and could not complete your request.
Hi Waleed, You are not seeing any errors/warnings in the logs? There are a few possibilities here. If the request hits your app: * You could be hitting the 30 seconds limit (maybe because the input is too large or something). You should see an error in the logs. * Your code could also be raising an exception that you do not catch. You should see an error in the logs. * It is also possible that the return status of the request is set to 500 somehow. In this case, if your application could silently catch the exception without logging anything. But you should be able to debug it by logging info messages in your request handler. If your request does not hit your app: * You could be hitting the simultaneous requests limit. That's a warning in the logs, but it shouldn't be reproducible every time. Hope it helps, Marc On Mar 2, 3:09 am, Waleed wal...@ninua.com wrote: I'm getting this error for some requests: The server encountered an error and could not complete your request.pIf the problem persists, please A HREF=http:// code.google.com/appengine/community.htmlreport/A your problem and mention this error message and the query that caused it. It's repeatable and the error happens every time for that specific request I'm sending. I tested with another AE app and I get the same problem. The request doesn't seem to be hitting my app, but fails before that, so I can't do anything about it. When I submit the same request with different data in the POST body, it goes through and works well. My request is a simple POST with a blog feed in the body of the post. Nothing particularly unique about it. And as I mentioned earlier, it works for most feeds, except for a few where it breaks 100% of the time. How can I debug this? Can anyone shed some light? App id is: networkedblogs -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Parallel access to appengine
What I'm suggesting is.. You need to create a simple test setup that
recreates this dynamic request limit error.. (It definitely should not
take 8mb of code).
I will see if I can create a handler like the one you posted, deploy
it, and then run 30 seperate processes that keep getting from that
handler.. (I can write this up in less than 10kb or python code)...
My guess is this will work. Without seeing sample code.. I can't tell
where you may be going wrong (or where GAE may be breaking)
On 3/2/10, Gary Orser garyor...@gmail.com wrote:
Actually, 4 threads was before we optimized server side, and set up
the test environment.
I have a tarball, about 8mb, with the test environment. (django and
libraries, grrr)
What is the best way to post this? I don't see any file attachments
on groups.
Cheers, Gary
On Mar 2, 8:23 am, Eli Jones eli.jo...@gmail.com wrote:
Are these threads you're using (at this point, it really seems like you
should post some simplified code to illustrate the issue at hand) waiting
for their response before trying to get again?
Posting some code to help recreate this issue will lead to a much faster
resolution.. as it stands.. I just know that someone on the internet has
10
threads that are hitting a dynamic request limit.
I also know that in the initial e-mail, when the request took longer to
return.. these threads were hitting a lower dynamic request limit (only
4
could run). This suggest that there is an important detail to how your
threads are doing their work.. and we would need that to provide useful
help.
Thanks for info.
On Tue, Mar 2, 2010 at 10:01 AM, Gary Orser garyor...@gmail.com wrote:
But that's the point. I can not reach 30 active requests.
I can only reach 10 active requests without error.
Any ideas on how I can debug this?
Cheers, Gary.
On Mar 2, 7:05 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the
probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with
a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting
500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas
for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your
application
requires even higher quotas than the billing-enabled values
listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91
HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service
your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com
wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are
actually
much faster now. Gary and our other devs have reworked it. I need
updated
numbers, but they don't take 10s, probably more like 2s. We still
have
some
heavy ~5s services though, so the same issue exists with the
simul-req
stuff, just to less extent. We don't actually hit this limit much
now
with
the current beta that is in production, but it is low traffic at the
moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last
email, I
have
[google-appengine] Re: simultaneous dynamic request limit
Hi David, I don't have a precise answer, but I think you are going in the right direction. The idea is to minimize the response time of your most popular requests using memcache. Try to cache the html pages derived from the datastore queries. It is easy to drop the request time to a few hundred ms or even less that way (depending how much you can cache and if you use java or python). This approach works well if your users are interested in the same entities. For example a news site can easily cache its more popular articles that way. For entities/queries that are specific to each user, but might be reused in many pages, cache it as soon as possible! Hope it helps, Marc On Mar 2, 3:21 am, Waleed Abdulla wal...@ninua.com wrote: I got the same errors today on my dev app, which I'm the only user of. So it doesn't seem to be related to how much load the app has!! I've been noticing them on my production app as well on and off. Waleed On Mon, Mar 1, 2010 at 7:22 PM, Satoshi satoshi.nakaj...@gmail.com wrote: I've got the same warnings several time today too. The peak access rate was only 3.00 requests/sec, and the CPU time usage over the last 24 hours is 6% (1.08 CPU hours) out of 18.50 CPU hours (I am a paying customer). Satoshi On Mar 1, 6:51 pm, David dscri...@gmail.com wrote: I am losing sleep over this, so any help would be greatly appreciated! APP ID: conit-app01 Since our app released about a week ago, it has been getting an average of about 60 requests/second. On February 27, our app suddenly crashed and was down for several hours, with thousands of these errors appearing in the logs: Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please seehttp:// code.google.com/appengine/docs/quotas.htmlfor more details. Since getting this error, I filled out a request to increase this limit at: http://code.google.com/support/bin/request.py?contact_type=AppEngineC... This request was denied, because, your app has been using, over the past 24 hours, on average 60 QPS with a peak of ~135 QPS; thus you're well under the 500 QPS limit described above. Since this crash, I've also been working to decrease calls to the datastore, and I think our average CPU time has decreased around 30%. In the dashboard, one of our pages still appears yellow under the column Average CPU (API), with a speed of about 1100. This page is about 6% of the volume of our app. The other pages don't have any warnings. We are well within the limits of our billing. I would feel much better if I could understand the math/metrics that go into producing this error, so it doesn't happen again. How can I know if my page request times are low enough? If I add a new page with a higher CPU time, how can I know if it would make the app crash? Any help or references to details on this error would be appreciated. Thank you in advance. -David -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] non sms activation, then app creation issues
Hi all, I have recently had my apps account manually activated as I couldn't get the SMS sent (had already used the same number somewhere else) When I try to create an application, with an available identifier, I click save and I am returned to the same create application form, and the previously submitted identifier is no longer available. I cant spot a form to report this through, and searches on application identifier don't seem to contain anything with the same situation! ta, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] app upload failed
Trying upload my app: .../appengine-java-sdk-1.3.1/bin/appcfg.sh update www/ Reading application configuration data... Encountered a problem: null Please see the logs [/tmp/appcfgx2i014.log] for further information. an...@ast:/mnt/sda1/projects/appengine/warandmilitary$ cat /tmp/ appcfgx2i014.log java.lang.NullPointerException at org.relaxng.datatype.helpers.DatatypeLibraryLoader$Service $Loader2.getResources(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader $Service.init(libgcj.so.10) at org.relaxng.datatype.helpers.DatatypeLibraryLoader.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaBuilder.init(libgcj.so. 10) at gnu.xml.validation.xmlschema.XMLSchemaSchemaFactory.newSchema(libgcj.so. 10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at javax.xml.validation.SchemaFactory.newSchema(libgcj.so.10) at com.google.appengine.tools.admin.Application.validateXml(Application.java: 319) at com.google.appengine.tools.admin.Application.init(Application.java: 87) at com.google.appengine.tools.admin.application.readaan...@ast:/mnt/ sda1/projects/appengine/warandmilitary Same app was upload successfully from my home computer. Any suggestions? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Calling a web service from JSP
I am doing a project on RESTful web services on Google App Engine (Java) in Netbeans IDE. The problem is, after creating a RESTful web service in Netbeans, error comes while attempting to test the service using the option TEST RESTFUL WEB SERVICES. I get ERROR 400 or ERROR 500. The HTTP Error 500 says like this: Need to specify class name in environment or system property, or as an applet parameter or in an application resource file: java.naming.factory.initial I have created RESTful web service from a newly created database. Now, when I run the project, the JSP page I have created appears on the browser. But, I need to know WHAT THE CODE IS to CONNECT THE JSP page with the Service. Meaning, how can I call the web service from the JSP page or how do I connect the JSP page and the WEB SERVICE. For example, if I click on the SUBMIT button on the JSP output page on the browser, the details that I have entered on in the browser page must contact the HTTP Methods of the RESTful web service and update in the database correspondingly. So, is there any tutorial or any specific code to be inserted so that my project works flawlessly? URGENT!!! PLS HELP... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Installing Eclipse 3.5 Plugin from download
I don't have Internet access from my developing computer, so I have to install the Eclipse plugin from files. The archive provided at http://dl.google.com/eclipse/plugin/3.5/zips/gpe-e35-latest.zip didn't work. It seem that the jars for the features are not included. As a result I cannot open a new Google Web Project in Eclipse. I wold appretiate any help to make this working. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Problem changing sender email address in appengine email API
Our Python-based appengine app, StepRep, regularly sends 3500-4000 emails per day (e.g., alerts, account activaions, etc) to our app's subscribers using the appengine email API. In the past, we have used the gmail address steprep at myfrontsteps dot com as the sender (in the mail.mail_send() method) with no problems. This address is an administrator for the StepRep application. Recently, we attempted to change the sender address to a different StepRep administrator gmail address, steprep at steprep dot com. However, when we do this, the emails do not get sent, and steprep at steprep dot com gets suspended by Google. We cannot figure out why one StepRep administrator address (steprep at myfrontsteps dot com) works OK as the email sender, but another (steprep at steprep dot com) does not. Can anyone provide any insight into this problem? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] No Module Named Crypto.Hash - GAE
Hello All. I am stuck here am maybe someone knows the answer (All this on MAC OS- X btw) I've got some python GAE code that contains some Crypto usage. Eg. ... from Crypto.Hash import SHA256 from Crypto.Hash import HMAC from Crypto.Cipher import AES ... Now if I 1) run the code on my local box from the command line everything works fine. 2) deploy it to GAE and access it normall it all works fine. 3) run it via the GoogleAppEngineLauncher or Eclipse GAE/Python I get errors like from Crypto.Hash import SHA256 ImportError: No module named Crypto.Hash Now as far as I can see all the right paths are set etc etc. Has this happened to anyone else and how did you fix it. Wonderin if its some path order or something.. Any ideas? thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Help!! Marzia The phone number has been sent too many messages or has already been used to confirm an account.
Can someone in the cloud please help me out. I think I've done this already but it want confirmation... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Status Update on the February 24th App Engine Outage
I can't login into App Engine's administration console. When i try to login i have a 500 Server Error, url is : https://appengine.google.com/_ah/login?continue=https://appengine.google.com/ Any idea about this issue ?? On 25 fév, 21:14, App Engine Team appengine.nore...@gmail.com wrote: We continue to work diligently in the wake of yesterday's unforeseen App Engine outage on a number of issues, so we wanted to provide you an update with our current list of tasks. - Applications will not be charged for any App Engine resource usage that occurred during Feb 24th, 2010. - We are still working on fix for writes and transactional reads affecting a small number of datastore entity groups. We have determined that approximately 25 application IDs have been affected by this issue. If you feel you are having difficulties with certain entity groups in your application, please respond to this thread with your App ID. - We are working hard on putting together a detailed post mortem that we will be making public. We expect to have this available to you next week. Again we sincerely apologize for yesterday's service disruption. We take App Engine's reliability very seriously and we are confident we can use the hard lessons learned from yesterday's event to improve our offering to you. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 「sdc_internet_routing : NO_MATC HING_RESOURCE」 trying to fetch URL via SDC
hello everyone I cannot fetch intranet resource via sdc. my localConfig.xml: entity sdcServerHostapps-secure-data-connector.google.com/sdcServerHost sdcServerPort443/sdcServerPort domainmydomain.com/domain usersecure-data-connector-user/user passwordmypwd/password agentIdagent_mydomain/agentId socksServerPort1080/socksServerPort sshd/etc/opt/google-secure-data-connector/openssh/start_sshd.sh/ sshd healthCheckGadgetUsersadmin13...@mydomain.com/ healthCheckGadgetUsers /entity my resourceRules.xml: resourceRules rule repeatable=true ruleNum1/ruleNum agentIdagent_mydomain/agentId viewerEmail repeatable=trueadmin13...@mydomain.com/ viewerEmail apps repeatable=true serviceAppEngine/service allowAnyAppIdtrue/allowAnyAppId /apps urlhttp://myintranetIP:8080/tomcat.gif/url urlMatchHOSTPORT/urlMatch /rule /resourceRules my java soucecode: URL dataURL = new URL(http://myintranetIP:8080/tomcat.gif?X- secureDataConnectorDebug=text); HTTPRequest fetchreq = new HTTPRequest(dataURL); fetchreq.setHeader(new HTTPHeader(use_intranet,yes)); HTTPResponse fetchresp = fetcher.fetch(fetchreq); resp.setContentType(text/plain); resp.getWriter().print(new String(fetchresp.getContent())); and I have open ports(8080 , 1080) in the firewall between sdc server and intranet webserver. Actual results: request_user : admin13...@mydomain.com request_sdc_agent_domain : mydomain.com request_url : http://myintranetIP:8080/tomcat.gif?X-secureDataConnectorDebug=text request_service : AppEngine request_appId : myappid request_requestId : (Id = bf33d834f932802d) sdc_routing : Internet sdc_internet_routing : NO_MATCHING_RESOURCE response_sdc_status : ok response_content_size : 0 thanks in advance! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
Eli,
You have the python request server.
Here is the java client:
You'll have to get the libraries yourself.
Cheers, Gary
import java.util.ArrayList;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.params.ConnManagerParams;
import org.apache.http.conn.params.ConnPerRouteBean;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
public class Main
{
private Log log = LogFactory.getLog(Main.class);
// ADJUST: number of threads to make requests on
public static int NUM_PARALLEL_SECTION_REQUESTS = 20;
public static HttpParams httpParams = new BasicHttpParams();
{
httpParams.setBooleanParameter(http.protocol.expect-
continue, false);
// ADJUST: if this is included, will use as a
proxy port. Charles Proxy defaults to this port.
//httpParams.setParameter(http.route.default-proxy,
new HttpHost(localhost, ));
}
protected class GetSection implements CallableString
{
protected int index;
protected HttpClient client;
protected String URL;
public GetSection(int index, HttpClient client, String
URL)
{
this.index = index;
this.client = client;
this.URL = URL;}
public String call() throws Exception
{
HttpGet getSection = new
HttpGet(URL);HttpResponse respSection =
client.execute(getSection);
String foo =
IOUtils.toString(respSection.getEntity().getContent(), UTF-8);
return foo;
}}
public static void main(String[] args) throws Exception
{new Main().maint(args);
}
public void maint(String[] args) throws Exception{
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http,
PlainSocketFactory.getSocketFactory(), 80));
schemeRegistry.register(new Scheme(https,
SSLSocketFactory.getSocketFactory(), 443));
HttpParams params = new BasicHttpParams();
ConnManagerParams.setMaxTotalConnections(params,
NUM_PARALLEL_SECTION_REQUESTS);
ConnManagerParams.setMaxConnectionsPerRoute(params,
new ConnPerRouteBean(NUM_PARALLEL_SECTION_REQUESTS));
HttpProtocolParams.setVersion(params,
HttpVersion.HTTP_1_1);
ClientConnectionManager cm = new
ThreadSafeClientConnManager(params, schemeRegistry);
HttpClient client = new DefaultHttpClient(cm,
httpParams);
ExecutorService es =
Executors.newFixedThreadPool(NUM_PARALLEL_SECTION_REQUESTS);
// ADJUST: total number of requests to make.
int numSections = 100;
ArrayListFutureString futures = new
ArrayListFutureString(numSections);
log.info(queuing requests);
for (int i = 0; i numSections; i++)
{
// ADJUST: set a real hostname here
futures.add(es.submit(new GetSection(i,
client, http://yourappid.appspot.com/sit/; + Integer.toString(i;
// ADJUST: stagger initial requests with this
sleep
//Thread.sleep(200);
}
es.shutdown();
log.info(waiting for thread pool to finish);
while (!es.isTerminated())
Thread.sleep(500);
log.info(all requests queued);
try
{
for (FutureString future: futures)
future.get();
log.info(got all futures);
}
catch (ExecutionException e)
{
// TODO: not really sure what to do if
[google-appengine] server error on random occasions
I have my application deployed onto App Engine. When I access my application at times I get a server error. If I refresh the page the server error goes away. Another time the app worked perfectly for me but a friend based out of a different city tried to access it and it kept giving him the server error for about half an hour. And then he was able to access the application. The error - Error: Server Error - The server encountered an error and could not complete your request. If the problem persists, please report your problem and mention this error message and the query that caused it. I did a search and there were a couple of posts that mentioned app.yaml formatting issues. I work on the Java side and so far my web.xml seems to be well formatted. If the formatting is bad wouldn't the error at least be consistent. Not sure why it happens occasionally. Any explanations or inputs on this issue would be greatly appreciated. Manny -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Help!! Marzia The phone number has been sent too many messages or has already been used to confirm an account.
Hi, If you are having trouble with SMS verification, or want an additional account activated, please fill out the following form: http://appengine.google.com/waitlist/sms_issues (This is from the following FAQ http://code.google.com/appengine/kb/sms .html#error ) Once you fill out this form, you should receive access within a day or two. Happy coding, Nick Johnson On Tue, Mar 2, 2010 at 3:19 PM, rh0dium steven.kl...@gmail.com wrote: Can someone in the cloud please help me out. I think I've done this already but it want confirmation... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
Eli,
You have the python request server.
Here is the java client:
You'll have to get the libraries yourself.
Cheers, Gary
import java.util.ArrayList;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.params.ConnManagerParams;
import org.apache.http.conn.params.ConnPerRouteBean;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
public class Main
{
private Log log = LogFactory.getLog(Main.class);
// ADJUST: number of threads to make requests on
public static int NUM_PARALLEL_SECTION_REQUESTS = 20;
public static HttpParams httpParams = new BasicHttpParams();
{
httpParams.setBooleanParameter(http.protocol.expect-
continue, false);
// ADJUST: if this is included, will use as a
proxy port. Charles Proxy defaults to this port.
//httpParams.setParameter(http.route.default-proxy,
new HttpHost(localhost, ));
}
protected class GetSection implements CallableString
{
protected int index;
protected HttpClient client;
protected String URL;
public GetSection(int index, HttpClient client, String
URL)
{
this.index = index;
this.client = client;
this.URL = URL;}
public String call() throws Exception
{
HttpGet getSection = new
HttpGet(URL);HttpResponse respSection =
client.execute(getSection);
String foo =
IOUtils.toString(respSection.getEntity().getContent(), UTF-8);
return foo;
}}
public static void main(String[] args) throws Exception
{new Main().maint(args);
}
public void maint(String[] args) throws Exception{
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http,
PlainSocketFactory.getSocketFactory(), 80));
schemeRegistry.register(new Scheme(https,
SSLSocketFactory.getSocketFactory(), 443));
HttpParams params = new BasicHttpParams();
ConnManagerParams.setMaxTotalConnections(params,
NUM_PARALLEL_SECTION_REQUESTS);
ConnManagerParams.setMaxConnectionsPerRoute(params,
new ConnPerRouteBean(NUM_PARALLEL_SECTION_REQUESTS));
HttpProtocolParams.setVersion(params,
HttpVersion.HTTP_1_1);
ClientConnectionManager cm = new
ThreadSafeClientConnManager(params, schemeRegistry);
HttpClient client = new DefaultHttpClient(cm,
httpParams);
ExecutorService es =
Executors.newFixedThreadPool(NUM_PARALLEL_SECTION_REQUESTS);
// ADJUST: total number of requests to make.
int numSections = 100;
ArrayListFutureString futures = new
ArrayListFutureString(numSections);
log.info(queuing requests);
for (int i = 0; i numSections; i++)
{
// ADJUST: set a real hostname here
futures.add(es.submit(new GetSection(i,
client, http://yourappid.appspot.com/sit/; + Integer.toString(i;
// ADJUST: stagger initial requests with this
sleep
//Thread.sleep(200);
}
es.shutdown();
log.info(waiting for thread pool to finish);
while (!es.isTerminated())
Thread.sleep(500);
log.info(all requests queued);
try
{
for (FutureString future: futures)
future.get();
log.info(got all futures);
}
catch (ExecutionException e)
{
// TODO: not really sure what to do if
Re: [google-appengine] Re: Parallel access to appengine
Thanks Gary. I'll run this code through my mental debugger to see if
anything pops out.
I'm going to write my own more simplified test where I just create some
python code that hits the request handler.. and once it gets a response.. it
just tries again (For maybe 10 - 20 times in a row.)..
Then, just see if I can run 30 instances (or at least more than 10) of that
code at once.
On Tue, Mar 2, 2010 at 12:21 PM, Gary Orser garyor...@gmail.com wrote:
Eli,
You have the python request server.
Here is the java client:
You'll have to get the libraries yourself.
Cheers, Gary
import java.util.ArrayList;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.params.ConnManagerParams;
import org.apache.http.conn.params.ConnPerRouteBean;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
public class Main
{
private Log log = LogFactory.getLog(Main.class);
// ADJUST: number of threads to make requests on
public static int NUM_PARALLEL_SECTION_REQUESTS = 20;
public static HttpParams httpParams = new BasicHttpParams();
{
httpParams.setBooleanParameter(http.protocol.expect-
continue, false);
// ADJUST: if this is included, will use as a
proxy port. Charles Proxy defaults to this port.
//httpParams.setParameter(http.route.default-proxy,
new HttpHost(localhost, ));
}
protected class GetSection implements CallableString
{
protected int index;
protected HttpClient client;
protected String URL;
public GetSection(int index, HttpClient client, String
URL)
{
this.index = index;
this.client = client;
this.URL = URL;}
public String call() throws Exception
{
HttpGet getSection = new
HttpGet(URL);HttpResponse respSection =
client.execute(getSection);
String foo =
IOUtils.toString(respSection.getEntity().getContent(), UTF-8);
return foo;
}}
public static void main(String[] args) throws Exception
{new Main().maint(args);
}
public void maint(String[] args) throws Exception{
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http,
PlainSocketFactory.getSocketFactory(), 80));
schemeRegistry.register(new Scheme(https,
SSLSocketFactory.getSocketFactory(), 443));
HttpParams params = new BasicHttpParams();
ConnManagerParams.setMaxTotalConnections(params,
NUM_PARALLEL_SECTION_REQUESTS);
ConnManagerParams.setMaxConnectionsPerRoute(params,
new ConnPerRouteBean(NUM_PARALLEL_SECTION_REQUESTS));
HttpProtocolParams.setVersion(params,
HttpVersion.HTTP_1_1);
ClientConnectionManager cm = new
ThreadSafeClientConnManager(params, schemeRegistry);
HttpClient client = new DefaultHttpClient(cm,
httpParams);
ExecutorService es =
Executors.newFixedThreadPool(NUM_PARALLEL_SECTION_REQUESTS);
// ADJUST: total number of requests to make.
int numSections = 100;
ArrayListFutureString futures = new
ArrayListFutureString(numSections);
log.info(queuing requests);
for (int i = 0; i numSections; i++)
{
// ADJUST: set a real hostname here
futures.add(es.submit(new GetSection(i,
client, http://yourappid.appspot.com/sit/; + Integer.toString(i;
// ADJUST: stagger initial requests with this
sleep
//Thread.sleep(200);
}
es.shutdown();
[google-appengine] blobs, jsp and other problems
hello together, i started developing with google app engine. i want to develope an application where the user has the possibility to enter some data in a jsp page. this page contains normal information (eg author, title, etc) and two file upload fields (html) to upload an image and an pdf document. up to now i have no problems to save the normal data (author, title, etc) in the storage, this is done in the servlet class which processes the html-form out of the jsp page. My problems are: - how can i access the data from the image and the pdf file at the servlet class? - how can i create with this binary data a blob and save this blob in the cloud storage? - how can i download and open the pdf by clicking a link would be great if anybody can help me with my problems greatz flo -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] JDO relationship - Can a child object owned by many parents?
Hi, I am trying to see if below schema is possible in JDO. I read the
relationship documentation below but I was not sure if this schema
(Owned many-to-one ?) is possible. I'd appreciate your expert advise.
http://code.google.com/intl/en/appengine/docs/java/datastore/relationships.html
Example:
Child {
Key key;
}
ParentA {
Child child;
}
ParentB {
Child child;
}
And do this:
Child c = new Child();
ParentA a = new ParentA();
ParentB b = new ParentB();
a.setChild(c);
Eventually I want to load a object with object c automatically
fetched. Can I do it like this?
Parent A {
@Persistent(defaultFetchGroup = true)
Child child;
}
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: server error on random occasions
Your not alone I even got the same server error on Google own App Engine Status page at http://code.google.com/status/appengine My app also gets this error and it also takes a LONG time to load eve when its loading the minimum, and this is after a few refreshes so the excuse that its loading a new VM isnt valid. On Mar 2, 12:25 pm, Manny S manny.m...@gmail.com wrote: I have my application deployed onto App Engine. When I access my application at times I get a server error. If I refresh the page the server error goes away. Another time the app worked perfectly for me but a friend based out of a different city tried to access it and it kept giving him the server error for about half an hour. And then he was able to access the application. The error - Error: Server Error - The server encountered an error and could not complete your request. If the problem persists, please report your problem and mention this error message and the query that caused it. I did a search and there were a couple of posts that mentioned app.yaml formatting issues. I work on the Java side and so far my web.xml seems to be well formatted. If the formatting is bad wouldn't the error at least be consistent. Not sure why it happens occasionally. Any explanations or inputs on this issue would be greatly appreciated. Manny -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] App Engine Status Page Almost as Slow as my app
The app engine status page at http://code.google.com/status/appengine takes a WHILE to load and browse through the pages. Just like my app engine powered Java app. I got that please report your problem at the forms msg a few times while on that site so Im posting here. Anybody else using Java have a really slow app today and most days? Doubt this will make this past the moderators... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] JDO relationship - Can a child object owned by many parents?
If a Child object is owned by many Parents, then the Child is actually the
Parent, and this is possible. Parent-Child relationships are defined in the
Key of the objects themselves. If you decoded a Key, it would look something
like this:
Parent(1)/Child(5) - Some Child object with ID 5
Parent(2)/Child(4) - Some Child object with ID 6
This it is not possible for a Child to have multiple parents (though it is
possible to have multiple grandparents).
On Tue, Mar 2, 2010 at 10:45 AM, anjolight yo...@atsumistudio.com wrote:
Hi, I am trying to see if below schema is possible in JDO. I read the
relationship documentation below but I was not sure if this schema
(Owned many-to-one ?) is possible. I'd appreciate your expert advise.
http://code.google.com/intl/en/appengine/docs/java/datastore/relationships.html
Example:
Child {
Key key;
}
ParentA {
Child child;
}
ParentB {
Child child;
}
And do this:
Child c = new Child();
ParentA a = new ParentA();
ParentB b = new ParentB();
a.setChild(c);
Eventually I want to load a object with object c automatically
fetched. Can I do it like this?
Parent A {
@Persistent(defaultFetchGroup = true)
Child child;
}
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
http://googleappengine.blogspot.com | http://twitter.com/app_engine
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: JDO relationship - Can a child object owned by many parents?
I see that any persistent entity can have zero or one entity parent. But is it possible to have classes defined as above, with each child class instance having the option to have a parent of different type at run-time like the pseudo-code below? Child c1 = new Child(); ParentA a = new ParentA(); a.setChild(c1); Child c2 = new Child(); ParentB b = new ParentB(); b.setChild(c2); In other words, would c1 have entity group parent a, and c2 have entity group parent b? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Empty list for list property
Hi, When I set empty list to db.ListProperty(int), I got following error randomly. (The error doesn't seem to occur every time.) How can I deal with this error? class 'google.appengine.api.datastore_errors.BadValueError': May not use the empty list as a property value; property blocking is []. - Kazuho -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] User management and authentication
I don't know off the top of my head of any open source projects that will provide hierarchical ACLs for multiple domains. Are you working in Python or Java? I've never used it before, but I've heard chatter that Spring had some modules with similar functionality. If you do find a project that mostly meets your needs, please post it to the groups. I imagine there are probably others looking for a solution like this. On Sun, Feb 28, 2010 at 2:53 PM, andrew aute...@gmail.com wrote: I'm developing an application which I want to offer to companies as a service, and have a number of their employees be able to login and use it, but without forcing them all to get Google Accounts, or accounts in my Google Apps domain. I imagine this is a fairly common problem for anyone developing on GAE wanting to offer services to companies. I'd like help on solutions/code to manager those users and authenticate them, if possible without spending a lot of time writing a whole bunch of code to do that. I'd like to offer a company a hierarchy of organizations and roles, just like many Enterprise systems. I'd setup an entry for the company, and a number of levels of sub- groups within the company. Each level could have admins, who can assign admins for sub-levels, and create new sub-levels. I'd setup a number of levels, admin users, and optionally a number of end users. After that the idea is that they would take care of managing it all. Admins could login and setup further end users, admins, levels, remove them etc. All users authenticate before they can use the app. If the company has these roles and users in their internal systems then I would use that, accessing their system using LDAP or similar and leave it to them to manage the users Anyone doing that or know of open source code I could use to do it? But it's possible they don't have such an IT solution. Then I effectively need a user management system in my GAE app Anyone doing that or know of open source code I could use to do it? Another idea is an external service that takes care of all this for me and that I can access from GAE Anyone doing that or know of open source code I could use to do it? Other strategies or implementation ideas that will let me focus on my app, and not just managing users would be very appreciated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Empty list for list property
The error occurs after when I retrieve data from memcache. Here's how
to reproduce the issue. (Again, not happen every time. It happens
particular entity.)
class Dummy(db.Model):
blocking = db.ListProperty(int, indexed=False)
class CacheTest(webapp.RequestHandler):
def get(self, action):
if action == 'setup':
self.setup()
elif action == 'load':
self.load()
else:
return
def setup(self):
d = Dummy(blocking = [])
d.save()
memcache.set('dummy', d)
def load(self):
d = memcache.get('dummy') exception occurs here
On Mar 2, 11:45 am, naan kaz...@gmail.com wrote:
Hi,
When I set empty list to db.ListProperty(int), I got following error
randomly. (The error doesn't seem to occur every time.) How can I deal
with this error?
class 'google.appengine.api.datastore_errors.BadValueError': May not
use the empty list as a property value; property blocking is [].
- Kazuho
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: simultaneous dynamic request limit
The math is: (1000 ms/second / 75 ms/request) * 30 = 400 requests/second Google assumes that you can run all your requests in 75ms, which gives you the 400req/sec number. They say you should be able to do 30 simultaneous requests. We have been testing this lately and can only get 10, but that is an ongoing discussion. Interesting though, since one query takes ~500ms according to todays status page. Hard to do 75ms on a heavier application/request. :( http://code.google.com/status/appengine/detail/datastore/2010/03/02#ae-trust-detail-datastore-query-latency We are continuing to tune and reduce request length as always, but even with fairly low amounts of traffic we hit this limit. On Mar 2, 2010, at 11:07 AM, Marc Provost wrote: Hi David, I don't have a precise answer, but I think you are going in the right direction. The idea is to minimize the response time of your most popular requests using memcache. Try to cache the html pages derived from the datastore queries. It is easy to drop the request time to a few hundred ms or even less that way (depending how much you can cache and if you use java or python). This approach works well if your users are interested in the same entities. For example a news site can easily cache its more popular articles that way. For entities/queries that are specific to each user, but might be reused in many pages, cache it as soon as possible! Hope it helps, Marc On Mar 2, 3:21 am, Waleed Abdulla wal...@ninua.com wrote: I got the same errors today on my dev app, which I'm the only user of. So it doesn't seem to be related to how much load the app has!! I've been noticing them on my production app as well on and off. Waleed On Mon, Mar 1, 2010 at 7:22 PM, Satoshi satoshi.nakaj...@gmail.com wrote: I've got the same warnings several time today too. The peak access rate was only 3.00 requests/sec, and the CPU time usage over the last 24 hours is 6% (1.08 CPU hours) out of 18.50 CPU hours (I am a paying customer). Satoshi On Mar 1, 6:51 pm, David dscri...@gmail.com wrote: I am losing sleep over this, so any help would be greatly appreciated! APP ID: conit-app01 Since our app released about a week ago, it has been getting an average of about 60 requests/second. On February 27, our app suddenly crashed and was down for several hours, with thousands of these errors appearing in the logs: Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please seehttp:// code.google.com/appengine/docs/quotas.htmlfor more details. Since getting this error, I filled out a request to increase this limit at: http://code.google.com/support/bin/request.py?contact_type=AppEngineC... This request was denied, because, your app has been using, over the past 24 hours, on average 60 QPS with a peak of ~135 QPS; thus you're well under the 500 QPS limit described above. Since this crash, I've also been working to decrease calls to the datastore, and I think our average CPU time has decreased around 30%. In the dashboard, one of our pages still appears yellow under the column Average CPU (API), with a speed of about 1100. This page is about 6% of the volume of our app. The other pages don't have any warnings. We are well within the limits of our billing. I would feel much better if I could understand the math/metrics that go into producing this error, so it doesn't happen again. How can I know if my page request times are low enough? If I add a new page with a higher CPU time, how can I know if it would make the app crash? Any help or references to details on this error would be appreciated. Thank you in advance. -David -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at
[google-appengine] Re: urllib2.URLError: urlopen error [Errno 110] Connection timed out
greetings! can you confirm whether you're still having the same deployment problem? thanks for the stack trace as those are always helpful. one other comment i have is to be cautious when using Python 2.6 as App Engine currently only supports Python 2.5.x. let us know if you're still having problems. best regards, -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 Python Fundamentals, Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wesc+...@google.com developer relations :: google app engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] More developers
Hi all, Is there a way to increase the limit 15 developers? Thanks in advance -- Jairo Vasquez Moreno Mentez Developer www.mentez.com Medellin - Colombia -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: non sms activation, then app creation issues
what is/are the app identifier(s) you have already used? are you sure they don't show up in your list of applications now? (just go to http://appengine.google.com and check.) thanks, -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 Python Fundamentals, Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wesc+...@google.com developer relations :: google app engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App Engine Status Page Almost as Slow as my app
robert, are you still experiencing this issue? it seems to come up for me in On Mar 2, 5:51 pm, Robert Lancer robert.lan...@gmail.com wrote: The app engine status page athttp://code.google.com/status/appengine takes a WHILE to load and browse through the pages. Just like my app engine powered Java app. I got that please report your problem at the forms msg a few times while on that site so Im posting here. Anybody else using Java have a really slow app today and most days? Doubt this will make this past the moderators... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: appcfg.py error
just to verify... have you actually run the appcfg updater to upload your modified application? you'll need to update your app with the new app.yaml as well as uploading the loader script. the only way to do that is to first run appcfgy.py update dir. THEN you can run the uploader. On Mar 2, 10:03 am, pkhoosh pkho...@gmail.com wrote: I moved the remote_api handler to the top of my app.yaml file, but I got the sameappcfg.pyerror. Just to clarify, when I call C:\Program Files (x86)\Google\google_appengine pythonappcfg.pyupload_data --config_file=DataStoreLoader.py --filename=stimulus_set.csv --kind=DataStore face-eval --has_header I am callingappcfg.pywith the config_file in my app-directory, which is face-eval in this case (a subdirectory in google_appengine). Also, the file that I want to uplpoad, stimulus_set.csv is also in the face-eval app directory, along with the DataStoreLoader class. The $PYTHON_LIB variable is at the head of the directory structure pointing the handler in app.yaml. I'm not sure if that variable is being interpreted correctly, causing the problem. I appreciate your help. On Mar 1, 7:01 pm, Eli Jones eli.jo...@gmail.com wrote: Is the remote_api handler at the very top of your app.yaml? if there is something like: - url: /.* script: myscript Above the remote_api handler.. then it won't work.On Mon, Mar 1, 2010 at 8:43 PM, pkhoosh pkho...@gmail.com wrote: I am trying to upload a csv file to a data store. I uploaded my app.yaml file with the following: - url: /remote_api script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py login: admin I wrote a python handler in my application directory (DataStoreLoader.py). However, I get the following when I upload: C:\Program Files (x86)\Google\google_appenginepythonappcfg.py upload_data --config_file=DataStoreLoader.py -- filename=stimulus_set.csv --kind=DataStore face-eval --has_header Application: face-eval; version: 1. Usage:appcfg.py[options] upload_data directory appcfg.py:error: You must have google.appengine.ext.remote_api.handler assigned to an endpoint in app.yaml, or provide the url of the handler via the 'url' opt ion. Any help? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Parallel access to appengine
Hi All
Actually, the default limit is 30 active requests.
I can only reach 10 active requests without error.
Same here. I've been checking the instance count on my Task Queue
servlet and I've never seen it exceeds the 10 instance limit. There
are two other App Engine developers on my twitter TL who are also
experiencing the 10 limit. What's this? What's the relation between
30 active requests and the number of instances? Since an App Engine
Java instance can only run one request at a time, there should be 30
instances to handle 30 active requests.
Actually, it was 16 last year - I and another developer was observing
a 16 instance limit when you make your servlet busy with TQ or many
client requests. I've been thinking it must be a some kind of internal
safe valve. Is it possible to extend that by submitting a request to
Google? So that we can use TQ as a mean of large distributed batch
processing just like MapReduce.
Thanks,
Kaz
On Mar 3, 12:45 am, Gary Orser garyor...@gmail.com wrote:
But that's the point. I can not reach 30 active requests.
I can only reach 10 active requests without error.
Any ideas on how I can debug this?
Cheers, Gary.
On Mar 2, 7:05 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,
On Tue, Mar 2, 2010 at 1:54 PM, Wooble geoffsp...@gmail.com wrote:
The 500 requests per second number relies on the probably-unreasonable
assumption that each request can complete in ~75ms. Deliberately
making your requests take a whole 3 seconds each is, obviously, not
going to work. You can only have 10 instances active at a time by
default; if the pages you're serving actually take 3 seconds to
complete you'll need to optimize things a whole lot or be stuck with a
3.33 request/sec maximum.
Actually, the default limit is 30 active requests.
-Nick Johnson
On Mar 1, 11:33 pm, Gary Orser garyor...@gmail.com wrote:
Hi Nick,
Hmm, I was running tests on a billing enabled appspot today. 100
requests/test.
10 threads getting a URL with a 3 second sleep (to emulate
computation) on appspot, was the most I could get without getting 500
errors.
If I raised the thread pool beyond 10, I started getting errors??
That doesn't reconcile very well with this statement from the
appengine website.
Requests
The total number of requests to the app. The per-minute quotas for
application with billing enabled allow for up to 500 requests per
second--more than one billion requests per month. If your application
requires even higher quotas than the billing-enabled values listed
below, you can request an increase in these limits here.
Is there some billing setting that affects this?
Cheers, Gary
PS. dead simple request handler.
import time
from django import http
def sit(req):
time.sleep(3)
return http.HttpResponse('foo')
errors are:
03-01 04:15PM 48.177 /sit/91 500 10019ms 0cpu_ms 0kb gzip(gfe)
153.90.236.210 - - [01/Mar/2010:16:15:58 -0800] GET /sit/91 HTTP/1.1
500 0 - gzip(gfe) .appspot.com
W 03-01 04:15PM 58.197
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please seehttp://
code.google.com/appengine/docs/quotas.htmlfor more details.
On Mar 1, 2:36 pm, Michael Wesner mike.wes...@webfilings.com wrote:
Correction/addition to my last email.
It turns out that our requests for this EC2 pull thing are actually
much faster now. Gary and our other devs have reworked it. I need
updated
numbers, but they don't take 10s, probably more like 2s. We still have
some
heavy ~5s services though, so the same issue exists with the simul-req
stuff, just to less extent. We don't actually hit this limit much now
with
the current beta that is in production, but it is low traffic at the
moment.
We are just getting ready to ramp up heavily.
I asked Nick what we should do, well just today after my last email, I
have made contact with a Developer Advocate and whatnot, which is
fantastic.
It looks like we, as a business, will be able to have better contact
with
the GAE team. We would very much like to continue working with you to
figure
out what actions we can take and what provisioning we can do to make our
product successful and scale it as we grow in the near future. Gary Orser
will be replying to this thread soon with more findings from both our real
app code and a little test app we are using and which he will share with
you.
We plan on having a presence at Google I/O this year as we did at
PyCon. Hopefully we can even get setup in the demonstration area at I/O.
Thanks Nick for your help. Could we possibly setup a quick
[google-appengine] Re: App Engine Status Page Almost as Slow as my app
robert, are you still experiencing a slow response? i was able to get it to come up in about 5-6s (not speedy but it does return). also, at this time, 11:40p GMT/UTC, we are experiencing some datastore issues that are being investigated. -wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 Python Fundamentals, Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wesc+...@google.com developer relations :: google app engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
