[appengine-java] Performance optimization of query from datastore (cpu_ms/api_cpu_ms showing red)
Related to post:
http://groups.google.com/group/google-appengine-java/browse_thread/thread/f3fa2b04d2fa6a3
In a test app I have created, I am able to query only 50 rows per
request (for 1 entity with 10 properties only) from the datastore by
using low-level api without getting any warnings for cpu_ms /
api_cpu_ms usage.
However if I try to query more rows (again only 1 entity being
inserted), the cpu gets used for more than 1s thereby generating a
warning or error from GAE.
Stats from GAE log:
To retrieve 50 records - /viewData.action?offset=0range=50 200 142ms
861cpu_ms 678api_cpu_ms (everything ok)
To retrieve 100 records - /viewData.action?offset=0range=100 200
261ms 1653cpu_ms 1338api_cpu_ms (cpu_ms / api_cpu_ms in red)
Is there a more performant way to query the datastore for more records
in one request? Thanks in advance.
P.S: This is a test app and billing is not enabled.
Code in servlet:
Query query = persistenceManager.newQuery(SELECT FROM +
DetailsBean.class.getName());
query.setOrdering(crDate asc);
query.setRange(offset, range);
data = (ListDetailsBean) query.execute();
Code of Entity bean:
package com.pojo;
import java.io.Serializable;
import java.util.Date;
import java.util.List;
import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.NotPersistent;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class DetailsBean implements Serializable {
private static final long serialVersionUID = 1L;
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long slno;
@Persistent
private long column1;
@Persistent
private String column2;
@Persistent
private double column3;
@Persistent
private double column4;
@Persistent
private double column5;
@Persistent
private float column6;
@Persistent
private double column7;
@Persistent
private double column8;
@Persistent
private Date crDate;
@Persistent
private Date modDate;
@NotPersistent
private String message;
@NotPersistent
private int n;
@NotPersistent
private long offset;
@NotPersistent
private long range;
@NotPersistent
private List data;
public DetailsBean() {
}
public DetailsBean(long v1, String v2, double v3, double v4, double
v5,
float v6,
double v7, double v8) {
this.column1 = v1;
this.column2 = v2;
this.column3 = v3;
this.column4 = v4;
this.column5 = v5;
this.column6 = v6;
this.column7 = v7;
this.column8 = v8;
Date now = new Date();
this.crDate = now;
this.modDate = now;
}
public Long getSlno() {
return slno;
}
public void setSlno(Long slno) {
this.slno = slno;
}
public long getColumn1() {
return column1;
}
public void setColumn1(long column1) {
this.column1 = column1;
}
public String getColumn2() {
return column2;
}
public void setColumn2(String column2) {
this.column2 = column2;
}
public double getColumn3() {
return column3;
}
public void setColumn3(double column3) {
this.column3 = column3;
}
public double getColumn4() {
return column4;
}
public void setColumn4(double column4) {
this.column4 = column4;
}
public double getColumn5() {
return column5;
}
public void setColumn5(double column5) {
this.column5 = column5;
}
public float getColumn6() {
return column6;
}
public void setColumn6(float column6) {
this.column6 = column6;
}
public double getColumn7() {
return column7;
}
public void setColumn7(double column7) {
this.column7 = column7;
}
public double getColumn8() {
return column8;
}
public void setColumn8(double column8) {
this.column8 = column8;
}
public Date getCrDate() {
return crDate;
}
public void setCrDate(Date crDate) {
[appengine-java] Performance optimization of insert into datastore (cpu_ms/api_cpu_ms showing red)
In a test app I have created, I am able to insert only 2 rows per
request (for 1 entity with 10 properties only) into the datastore by
using low-level api without getting any warnings for cpu_ms /
api_cpu_ms usage.
However if I try to insert 3 rows or more (again only 1 entity being
inserted), the cpu gets used for more than 1s thereby generating a
warning or error from GAE.
Stats from GAE log:
For inserting 2 records - /insert.do 200 77ms 655cpu_ms 643api_cpu_ms
(everything ok)
For inserting 3 records - /insert.do 200 134ms 979cpu_ms
964api_cpu_ms (yellow warning)
For inserting 10 records - /insert.do 200 178ms 3249cpu_ms
3216api_cpu_ms (red warning)
How to optimize the ‘cpu_ms and api_cpu_ms’ usage? Can GAE perform
only as many operations as 2 inserts into datastore if we want to keep
within the warning levels?
Can you please provide some feedback as to whether I am doing anything
wrong in code or have not used a good practice to achieve better
performance. Thanks in advance.
Code in servlet (parsing and split operations have been benchmarked
and they are not causing any adverse impact on performance):
Pattern p1 = Pattern.compile(,); // created as servlet instance
variable
public void doPost(HttpServletRequest req, HttpServletResponse res)
throws ServletException, IOException
{
try {
DatastoreService datastore =
DatastoreServiceFactory.getDatastoreService();
ListEntity entity = new ArrayListEntity();
for (int i = 0; i n; i++) {
String[] record =
p1.split(1,tester,1.0,1.0,1.0,1.0,1.0,1.0);
data1 = Long.parseLong(record[0]);
data2 = record[1];
data3 = Double.parseDouble(record[2]);
data4 = Double.parseDouble(record[3]);
data5 = Double.parseDouble(record[4]);
data6 = Double.parseDouble(record[5]);
data7 = Double.parseDouble(record[6]);
data8 = Float.parseFloat(record[7]);
Entity e = new
Entity(DetailsBean.class.getSimpleName());
e.setProperty(column1, i);
e.setProperty(column2, data2);
e.setProperty(column3, i + data3);
e.setProperty(column4, i + data4);
e.setProperty(column5, i + data5);
e.setProperty(column6, i + data6);
e.setProperty(column7, i + data7);
e.setProperty(column8, i + data8);
e.setProperty(crDate, new Date());
e.setProperty(modDate, new Date());
entity.add(e);
}
datastore.put(entity);
} catch (Exception e) {
e.printStackTrace();
} finally {
persistenceManager.close();
}
}
Entity bean code:
package com.pojo;
import java.io.Serializable;
import java.util.Date;
import java.util.List;
import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.NotPersistent;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class DetailsBean implements Serializable {
private static final long serialVersionUID = 1L;
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long slno;
@Persistent
private long column1;
@Persistent
private String column2;
@Persistent
private double column3;
@Persistent
private double column4;
@Persistent
private double column5;
@Persistent
private float column6;
@Persistent
private double column7;
@Persistent
private double column8;
@Persistent
private Date crDate;
@Persistent
private Date modDate;
@NotPersistent
private String message;
@NotPersistent
private int n;
@NotPersistent
private long offset;
@NotPersistent
private long range;
@NotPersistent
private List data;
public DetailsBean() {
}
public DetailsBean(long v1, String v2, double v3, double v4, double
v5,
float v6,
double v7, double v8) {
this.column1 = v1;
this.column2 = v2;
[appengine-java] Re: AppEngine no longer shows any content
200 instead of 500 is nice. However all Jars were included in the deployment. The deployment works in general but fails occasionally. I normally get this error whent there is no Groovy installed on the webserver. If it helps perhaps there is one of the servers in the farm missing a runtime. But yes, there are infact to problems. one that i get 200 instead of 500. and the second that i get 500 in the first place. The application was up an running again without any new deployments being done, just later that night. /Morten On Aug 21, 3:10 pm, Don Schwarz schwa...@google.com wrote: To be clear, I meant that the fact that you're getting a 0-byte response here (it is a 500, not a 200, correct?) will be fixed in the next release. I don't know the cause of the actual exception. Are you sure you're including all of the necessary jars? Are there any other relevant excepions in your logs (perhaps logged at WARNING) ? On Thu, Aug 20, 2009 at 2:01 PM, Don Schwarz schwa...@google.com wrote: This will be fixed in the next release. Sorry about the inconvenience. On Thu, Aug 20, 2009 at 1:58 PM, DrMorten morten.dalgaard.niel...@gmail.com wrote: / returns HTTP-code 200 but no content (0 byte response), when I check the logs I get the following stacktrace: Error for / java.lang.NoClassDefFoundError: org/codehaus/groovy/antlr/ AntlrParserPlugin at org.codehaus.groovy.antlr.AntlrParserPluginFactory.createParserPlugin (AntlrParserPluginFactory.java:27) at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:247) at org.codehaus.groovy.control.CompilationUnit$1.call (CompilationUnit.java:160) at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits (CompilationUnit.java:798) at org.codehaus.groovy.control.CompilationUnit.compile (CompilationUnit.java:464) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java: 278) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java: 249) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java: 244) at org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine.compileGroovyPage (GroovyPagesTemplateEngine.java:462) at org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine.buildPageMetaInfo (GroovyPagesTemplateEngine.java:427) at org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine.createTemplate (GroovyPagesTemplateEngine.java:309) at org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine.createTemplateWithResource (GroovyPagesTemplateEngine.java:293) at org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine.createTemplate (GroovyPagesTemplateEngine.java:183) at org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine.createTemplate (GroovyPagesTemplateEngine.java:198) at org.codehaus.groovy.grails.web.servlet.view.GroovyPageView.renderWithTemplateEngine (GroovyPageView.java:104) at org.codehaus.groovy.grails.web.servlet.view.GroovyPageView.renderMergedOutputModel (GroovyPageView.java:86) at org.springframework.web.servlet.view.AbstractView.render (AbstractView.java:257) at org.codehaus.groovy.grails.web.mapping.filter.UrlMappingsFilter.renderViewForUrlMappingInfo (UrlMappingsFilter.java:235) at org.codehaus.groovy.grails.web.mapping.filter.UrlMappingsFilter.doFilterInternal (UrlMappingsFilter.java:188) at org.springframework.web.filter.OncePerRequestFilter.doFilter (OncePerRequestFilter.java:76) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.obtainContent (GrailsPageFilter.java:221) at org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.doFilter (GrailsPageFilter.java:126) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at org.codehaus.groovy.grails.web.servlet.filter.GrailsReloadServletFilter.doFilterInternal (GrailsReloadServletFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter (OncePerRequestFilter.java:76) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at org.codehaus.groovy.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal (GrailsWebRequestFilter.java:65) at org.springframework.web.filter.OncePerRequestFilter.doFilter (OncePerRequestFilter.java:76) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at org.codehaus.groovy.grails.plugins.resourcesfirst.ResourcesFirstFilter.doFilter (ResourcesFirstFilter.java:45) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
[appengine-java] Re: Kawa servlet
This is a known problem with the dev_appserver that we're working on. (Classes loaded by custom classloaders aren't granted the appropriate permissions). Unfortunately, there's not a workaround for this, and you'll need to test against our production servers. On Fri, Aug 21, 2009 at 9:31 AM, tetsujin1979 tetsujin1...@gmail.comwrote: Thank you, I double checked the web.xml and the xquery scripts are being passed to the KawaPageServlet class: servlet servlet-nameKawaPageServlet/servlet-name servlet-classgnu.kawa.servlet.KawaPageServlet/servlet-class /servlet servlet-mapping servlet-nameKawaPageServlet/servlet-name url-pattern*.xquery/url-pattern /servlet-mapping I've added the scripts to the appengine-web.xml file with the following: static-files exclude path=/**.xquery / /static-files now, it appears that the scripts are being passed correctly, but I'm getting the following permission exception: java.lang.ExceptionInInitializerError at sun.misc.Unsafe.ensureClassInitialized(Native Method) at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(Unknown Source) at sun.reflect.ReflectionFactory.newFieldAccessor(Unknown Source) at java.lang.reflect.Field.acquireFieldAccessor(Unknown Source) at java.lang.reflect.Field.getFieldAccessor(Unknown Source) at java.lang.reflect.Field.get(Unknown Source) at gnu.expr.ModuleContext.makeInstance(ModuleContext.java:60) at gnu.expr.ModuleContext.findInstance(ModuleContext.java:83) at gnu.kawa.servlet.KawaPageServlet.getModule(KawaPageServlet.java: 206) at gnu.kawa.servlet.KawaPageServlet.run(KawaPageServlet.java:46) at gnu.kawa.servlet.KawaServlet.doGet(KawaServlet.java:68) snip Caused by: internal error at gnu.expr.Compilation.setupLiterals(Compilation.java:2802) at temp.(temp.xquery) ... 36 more Caused by: java.security.AccessControlException: access denied (java.lang.RuntimePermission accessDeclaredMembers) at java.security.AccessControlContext.checkPermission(Unknown Source) at java.security.AccessController.checkPermission(Unknown Source) at java.lang.SecurityManager.checkPermission(Unknown Source) at com.google.appengine.tools.development.DevAppServerFactory $CustomSecurityManager.checkPermission(DevAppServerFactory.java:128) is it possible to grant the accessDeclaredMembers permission? On Aug 20, 5:15 pm, Toby Reyelts to...@google.com wrote: Can you double check your servlet mappings and make sure they are correct? Also, you'll need to modify your appengine-web.xml to exclude your scripts from static-files so they aren't served by default by our optimized static content servers. See our docs on static and resource files http://code.google.com/appengine/docs/java/config/appconfig.html#Stat... . On Wed, Aug 19, 2009 at 6:55 AM, tetsujin1979 tetsujin1...@gmail.com wrote: sorry, I should have added that I've checked for this on the Will it run on GAE page, and it's not listed On Aug 19, 11:54 am, tetsujin1979 tetsujin1...@gmail.com wrote: Hi, I have a Kawa -http://www.gnu.org/software/kawa/-based servlet that uses XQuery to query some xml documents to produce an output, using the instructions here - http://www.gnu.org/software/qexo/simple-xquery-webapp.html This runs fine on tomcat, but when I've tried to run the same web app on the GAE, any request just returns the raw xquery page, with no processing done. According to this page - http://www.gnu.org/software/kawa/server/auto-servlet.html - KawaPageServlet automatically compiles a script into a Java class. The class is internal to the server, and is not written out to disk so could it be that the class is not getting compiled? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Performance optimization of insert into datastore (cpu_ms/api_cpu_ms showing red)
What is your app id? Do you have a large number of indexes defined for this
entity kind?
FYI, it's certainly a good idea to optimize for performance, but I wouldn't
worry too much about the particular point at which warnings appear in the
request logs. These are just guidelines to let you know which requests are
consuming the most CPU. If you're only planning to update these entities on
a fraction of your requests then the overall CPU cost should not be too
high. However, if your application is doing a lot of writes then by all
means, try to optimize this (e.g. by reducing the number of indexes, or
perhaps by reducing the number of columns if you don't query them
independently).
On Fri, Aug 21, 2009 at 6:55 AM, sree sraj...@gmail.com wrote:
In a test app I have created, I am able to insert only 2 rows per
request (for 1 entity with 10 properties only) into the datastore by
using low-level api without getting any warnings for cpu_ms /
api_cpu_ms usage.
However if I try to insert 3 rows or more (again only 1 entity being
inserted), the cpu gets used for more than 1s thereby generating a
warning or error from GAE.
Stats from GAE log:
For inserting 2 records - /insert.do 200 77ms 655cpu_ms 643api_cpu_ms
(everything ok)
For inserting 3 records - /insert.do 200 134ms 979cpu_ms
964api_cpu_ms (yellow warning)
For inserting 10 records - /insert.do 200 178ms 3249cpu_ms
3216api_cpu_ms (red warning)
How to optimize the ‘cpu_ms and api_cpu_ms’ usage? Can GAE perform
only as many operations as 2 inserts into datastore if we want to keep
within the warning levels?
Can you please provide some feedback as to whether I am doing anything
wrong in code or have not used a good practice to achieve better
performance. Thanks in advance.
Code in servlet (parsing and split operations have been benchmarked
and they are not causing any adverse impact on performance):
Pattern p1 = Pattern.compile(,); // created as servlet instance
variable
public void doPost(HttpServletRequest req, HttpServletResponse res)
throws ServletException, IOException
{
try {
DatastoreService datastore =
DatastoreServiceFactory.getDatastoreService();
ListEntity entity = new ArrayListEntity();
for (int i = 0; i n; i++) {
String[] record =
p1.split(1,tester,1.0,1.0,1.0,1.0,1.0,1.0);
data1 = Long.parseLong(record[0]);
data2 = record[1];
data3 = Double.parseDouble(record[2]);
data4 = Double.parseDouble(record[3]);
data5 = Double.parseDouble(record[4]);
data6 = Double.parseDouble(record[5]);
data7 = Double.parseDouble(record[6]);
data8 = Float.parseFloat(record[7]);
Entity e = new
Entity(DetailsBean.class.getSimpleName());
e.setProperty(column1, i);
e.setProperty(column2, data2);
e.setProperty(column3, i + data3);
e.setProperty(column4, i + data4);
e.setProperty(column5, i + data5);
e.setProperty(column6, i + data6);
e.setProperty(column7, i + data7);
e.setProperty(column8, i + data8);
e.setProperty(crDate, new Date());
e.setProperty(modDate, new Date());
entity.add(e);
}
datastore.put(entity);
} catch (Exception e) {
e.printStackTrace();
} finally {
persistenceManager.close();
}
}
Entity bean code:
package com.pojo;
import java.io.Serializable;
import java.util.Date;
import java.util.List;
import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.NotPersistent;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class DetailsBean implements Serializable {
private static final long serialVersionUID = 1L;
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long slno;
@Persistent
private long column1;
@Persistent
private String column2;
@Persistent
private double column3;
@Persistent
private double column4;
@Persistent
private double column5;
@Persistent
[appengine-java] Re: Local transactions do not rollback
That's crazy, John. Is all the data still there or only some of it? Also, how is your transaction being demarcated? On Aug 21, 1:41 am, jd jdpatter...@gmail.com wrote: Hi, I am using the local sdk datastore to save several thousand of objects and when an exception occurs the data is not rolled-back and the data is still in the datastore. I can see in the log INFO: Time to persist datastore: 1140 ms which seems to indicate the data is being flushed to disk. Is this documented somewhere? Thanks, John --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: query performance
I agree with Iain: it depends on how many items are in the lists. But for large lists, I don't think it would work out very well. Are the values in listA and listB stored? That might give you more options. On Aug 20, 8:15 pm, Ray Li ray.lee@gmail.com wrote: Hi, I have a query filter on an entity Person that person.name be in listA and person.country in listB. As far as I can see, there're 2 options: 1. Create a query select t from Person t where t.name = :listAElement and t.country = :listBElement and run it listA.size() * listB.size() times, then combine the result sets. 2. Create a query select t from Person t where t.name =:listAElement and run it once, then for each entity in the resultset, check if its country is in listB. For option 1, I am not sure about querying the datastore too many times will case a serious performance issue. For option 2, I may have to get all results back, may be several several thousands, and this may be not achievable, is it? Any help is highly appreciated. Thanks, Ray --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Problem deploying application with jsps
Hi Abhinav,
Yes, you are right - we really should just invoke javac.exe on Windows. We
were assuming that javac would not be present in the bin directory on a
Windows machine. Feel free to file a bug about this if you'd like - it's not
too hard to fix, but as it's a corner case, it's not going to be at the top
of our priority list.
Thanks,
Rajeev
On Fri, Aug 21, 2009 at 1:29 AM, Abhinav Lele abhinav.l...@gmail.comwrote:
Hi,
I found the problem, albeit it is a silly one. I had a plugin named
kawigiedit ( @ Topcoder ) which had created a javac folder in the bin
directory. So when eclipse tried to deploy to appengine it tried to open a
folder instead of the binary. I removed the directory and things worked just
fine. This brings one thing to my mind, shouldn't the eclipse plugin evoke
javac.exe instead of javac which might be appropriate for *nix systems.
--
Abhinav
On Thu, Aug 20, 2009 at 11:58 PM, Abhinav Lele abhinav.l...@gmail.comwrote:
Hi Keith,
You were right. I created a Java Project and ran the following code
package testB;
import java.io.IOException;
public class testJ {
/**
* @param args
* @throws IOException
*/
public static void main(String[] args) throws IOException {
ProcessBuilder proc = new
ProcessBuilder(D:\\usr\\dev\\jdk16_12\\bin\\javac
);
proc.start();
}
}
and I got a Exception in thread main java.io.IOException: Cannot run
program D:\usr\dev\jdk16_12\bin\javac: CreateProcess error=5, Access is
denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at testB.testJ.main(testJ.java:15)
Caused by: java.io.IOException: CreateProcess error=5, Access is denied
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.init(ProcessImpl.java:81)
at java.lang.ProcessImpl.start(ProcessImpl.java:30)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 1 more
I'll try and google this out. If someone has solution for this please let
me know.
--
Abhinav
On Thu, Aug 20, 2009 at 8:37 PM, Keith Platfoot kplatf...@google.comwrote:
Hm, that is odd. Since you're able to use javac from the command line
and via appcfg, I'm assuming the problem does lie with Eclipse and the
permissions it was launched with.
Would it be possible for you to write a small test Java app that uses
ProcessBuilderhttp://java.sun.com/j2se/1.5.0/docs/api/java/lang/ProcessBuilder.htmlto
invoke javac? If your test app fails with an Access Denied error, then
at least we can rule out the plugin and App Engine as the source of the
problem. If that is indeed the case, you could try installing another JDK
and Eclipse instance using your user account, and see if that fixes things.
Keith
On Wed, Aug 19, 2009 at 2:35 PM, Abhinav Lele abhinav.l...@gmail.comwrote:
Hi Keith,
Well I was able to upload it using appcfg from the command line :) ..
Though I wonder why eclipse is getting a access denied error.
My user account has administrator privileges.
--
Abhinav
On Wed, Aug 19, 2009 at 8:49 PM, Keith Platfoot
kplatf...@google.comwrote:
Hi Abhinav,
I suspect the problem may be exactly as the error message implies:
access to javac is being denied because of its configured permissions.
To test that theory, could you try running javac from a command prompt,
using the path from the exception message? If you are able to run it,
javac
should display its usage options (since you didn't specify any arguments).
However, if you also get an Access Denied error from the command line,
then
the problem is not with Eclipse or the plugin, but with the permissions on
javac. One scenario that might produce this error would be if you
installed
the JDK as an Administrator and then run javac (either manually from the
command line or indirectly via Eclipse) as a regular (non-admin) user.
Let me know if you are able to reproduce the problem from the command
line. Thanks,
Keith
On Wed, Aug 19, 2009 at 5:03 AM, Abhinav Lele
abhinav.l...@gmail.comwrote:
Hi,
I am using Eclipse 3.5 ( Galilieo ) on Windows XP to deploy my
application. It runs fine on my local machine but when I try to deploy
it to
appengine it is giving me the following error.
Creating staging directory
Scanning for jsp files.
Compiling jsp files.
Compiling java files.
java.io.IOException: Cannot run program
D:\usr\dev\jdk16_12\bin\javac: CreateProcess error=5, Access is denied
Debugging information may be found in C:\Documents and
Settings\user\Local Settings\Temp\appengine-deploy6738175632278023548.log
appengine-deploy6738175632278023548.log
Unable to upload:
java.io.IOException: Cannot run program
D:\usr\dev\jdk16_12\bin\javac: CreateProcess error=5, Access is denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at
com.google.appengine.tools.admin.Application.startProcess(Application.java:434)
at
[appengine-java] Re: Local transactions do not rollback
Also, which version of the sdk are you using? Up until the most recent release the local datastore was not actually transactional. On Fri, Aug 21, 2009 at 8:37 AM, objectuser kevin.k.le...@gmail.com wrote: That's crazy, John. Is all the data still there or only some of it? Also, how is your transaction being demarcated? On Aug 21, 1:41 am, jd jdpatter...@gmail.com wrote: Hi, I am using the local sdk datastore to save several thousand of objects and when an exception occurs the data is not rolled-back and the data is still in the datastore. I can see in the log INFO: Time to persist datastore: 1140 ms which seems to indicate the data is being flushed to disk. Is this documented somewhere? Thanks, John --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] ClassCastException: [Object cannot be cast to [[Object
Hi I'm new to GAE, JSF and JDO, so I've probably made a mistake... But I don't have the same behaviour on my developemebt environment and on GAE servers. I've made a very simple application (hardly more than a Hello world). At this point it's like a guest book. You write a message, and the list of all messages is displayed. Everything works fine on my dev environment (I've published my code on code.google.com) But on the GAE servers, the creation of a new message fails with /faces/index.jsp java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [[Ljava.lang.Object; at javax.faces.component.UIComponentBase.processRestoreState (UIComponentBase.java:1054) at javax.faces.component.UIComponentBase.processRestoreState (UIComponentBase.java:1044) at javax.faces.component.UIComponentBase.processRestoreState (UIComponentBase.java:1044) at com.sun.faces.application.StateManagerImpl.restoreView (StateManagerImpl.java:336) ... Any idea? I suspect JSF 1.1 but I have no clue actually. Since I expect the same behaviour between the development environment and the production environment, I have filed a bug report, where you can have more details. http://code.google.com/p/googleappengine/issues/detail?id=2009 (it's a pure coincidence if the issue number is current year ;-) ) Régis --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: GAE Roadmap: Hibernate, sql etc
Hi Marc. Yes, there is a product roadmap available: http://code.google.com/appengine/docs/roadmap.html As leszek and Andy have said, App Engine is a fundamentally different beast than your typical J2EE server + RDBMS and you will have to compromise some convenience in order to achieve the level of scalability that App Engine offers. That said, App Engine for Java was designed from the beginning with Java standards in mind, including the servlet API, JDO/JPA, JCache, and so forth to ease the learning curve for J2EE developers, and we are continually looking at ways to make the developer experience even better. If you have any specific suggestions, I encourage you to post new feature requests to the public tracker which we use, in part, to prioritize upcoming features. http://code.google.com/p/googleappengine/issues/list - Jason On Thu, Aug 20, 2009 at 12:34 AM, mschipperheyn m.schipperh...@gmail.comwrote: Hi all, I'm just wondering if there is a roadmap available and if and when Hibernate, sql etc will be supported. Will there come a time soon when choosing GAE will no longer be a choice of if we use it, we have to dumb down the application because x (e.g. Hibernate), y (e.g. SQL) and z (e.g. Lucene, file system access) can't be used anymore? The concept is great but it seems that the limitations are gigantic. And for a high end application I feel using the plain vanilla usually doesn't cut it. project Has anyone done any big java-projects yet on this platform? Cheers, Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Problem deploying application with jsps
Hi, Added a bug report http://code.google.com/p/googleappengine/issues/detail?id=2010 . Please enhance the bug report if needed. -- Abhinav On Fri, Aug 21, 2009 at 9:06 PM, Rajeev Dayal rda...@google.com wrote: Yes, you are right - we really should just invoke javac.exe on Windows. We were assuming that javac would not be present in the bin directory on a Windows machine. Feel free to file a bug about this if you'd like - it's not too hard to fix, but as it's a corner case, it's not going to be at the top of our priority list. -- Ted Turner http://www.brainyquote.com/quotes/authors/t/ted_turner.html - Sports is like a war without the killing. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Problem issuing an HTTP GET request to an App Engine Page
Update: The issue appears to be in the page I am attempting to connect
to, not the fact that it is an App Engine Page. This is the code I
have, does anyone know of something I am missing that is necessary to
return information?
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
resp.setContentType(text/html);
PrintWriter out = resp.getWriter();
out.println(html);
out.println(headtitle Request Type: GET /title/head);
out.println(body);
out.println(Hello, world. This is a response to a get
request.);
out.println(/body/html);
out.flush();
out.close();
}
On Aug 20, 3:53 pm, Lisa lwhite.n...@gmail.com wrote:
Could anyone explain why issuing an HTTP GET request to an App Engine
Page would fail?
The following code accessing a page causes an IOException with an
Unknown error message:
String urlstring = http://et-demo.appspot.com/subscribe;;
URL url = new URL(urlstring);
BufferedReader reader = new BufferedReader(new InputStreamReader
(url.openStream()));
And the following code works (all I changed is the url string):
String urlstring = http://www.google.com;;
URL url = new URL(urlstring);
BufferedReader reader = new BufferedReader(new InputStreamReader
(url.openStream()));
The page I am trying to access does exist and when I go to it in my
web browser it is pretty fast. Does anyone know why this would
happen? Does it have something to do with App Engine?
Thanks,
Lisa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: ClassCastException: [Object cannot be cast to [[Object
Do you have the source code for the version of JSF you're using? A quick look at 1.1.4http://www.google.com/codesearch/p?hl=ensa=Ncd=2ct=rc#k7e2Oio8pzY/myfaces-core-1.1.4/source/javax/faces/component/UIComponentBase.javaq=javax.faces.component.UIComponentBase%20processRestoreStatedoesn't show a cast to Object[][]. Also, what binary of JSF are you deploying with? On Fri, Aug 21, 2009 at 1:55 PM, Régis Décamps deca...@users.sf.net wrote: Hi I'm new to GAE, JSF and JDO, so I've probably made a mistake... But I don't have the same behaviour on my developemebt environment and on GAE servers. I've made a very simple application (hardly more than a Hello world). At this point it's like a guest book. You write a message, and the list of all messages is displayed. Everything works fine on my dev environment (I've published my code on code.google.com) But on the GAE servers, the creation of a new message fails with /faces/index.jsp java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [[Ljava.lang.Object; at javax.faces.component.UIComponentBase.processRestoreState (UIComponentBase.java:1054) at javax.faces.component.UIComponentBase.processRestoreState (UIComponentBase.java:1044) at javax.faces.component.UIComponentBase.processRestoreState (UIComponentBase.java:1044) at com.sun.faces.application.StateManagerImpl.restoreView (StateManagerImpl.java:336) ... Any idea? I suspect JSF 1.1 but I have no clue actually. Since I expect the same behaviour between the development environment and the production environment, I have filed a bug report, where you can have more details. http://code.google.com/p/googleappengine/issues/detail?id=2009 (it's a pure coincidence if the issue number is current year ;-) ) Régis --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: how to store raster data
Hi Ivan. Currently, any data written to App Engine's datastore can be at most 1 MB, so if you're planning to store large files in App Engine, you'll have to split them into 1 MB chunks and store these chunks as separate entities. There are third-party abstraction frameworks that do this for you, notably: http://code.google.com/p/gaevfs/ Note that support for serving and storing large files is on the App Engine roadmap: http://code.google.com/appengine/docs/roadmap.html - Jason On Thu, Aug 20, 2009 at 6:27 AM, Ivan Lucena ivluc...@gmail.com wrote: Hi there, Is there any available API for storing large raster data in GAE? By raster data I mean geo-referenced satellite sensors data. By large I mean several years of observation. Thanks in advance, Ivan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Unable to upload
I get the message Email xx and password do not match when I try to deploy to AppEngine. However I use the same username and password to log into the appengine website successfully. I get the same error message whether I deploy using the Netbeans plug-in or appcfg from the command line. Also, I have created two applications but they do not appear when I log into appengine.google.com. I have verified that the application names were really created by checking their (non)availibility. Does anyone know why my username/password won't work for upload when it works for log in? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Relation indexing using a single list property - proposal
imagine you have a 100 000 data set.
each data has got 4 properties that you would like to use for
filtering.
then, you will have one Kind with your data, and 4 children Kinds for
the properties to be filtered.
you can then query each children kind in different queries. in this
case, 4 queries.
each query will give a maximum of 1000 entities.
for each query result, you will get the parent. and then, associate
those 4 lists of parents (remove duplicate, apply logic, ...).
if you want to get a complete result, you will add extra query to
solve the 1000 entities limit.
is this correct ?
On appengine, it seems to be the best way to use complex filter. but
requires a lot of queries.
On Aug 20, 7:24 pm, Juraj Vitko juraj.vi...@gmail.com wrote:
Hi, yes, exactly - using filteredTableEntry.getKey().getParent() --
that way they can be updated in a single transaction too.
The filteredTableEntry can be created like new Entity
('FT_User_isClient', parentUserKey).
When any of the primary record (User in this example) data changes
(eg. the isClient field, or a value in one of the sortable columns), I
assume the fastest way to update the filtered entry is to overwrite it
with a new one, instead of fetching it first (I may be wrong here).
The only caveat is that when you generate a new unique suffix for the
new entry (which overwrites the previous one), the record might 'jump'
in the UI, ie in a Grid that is currently paged through.
On Aug 20, 6:59 pm, Philippe philippe.cr...@gmail.com wrote:
looks interesting, but I do not understand all.
how do you get back to your data (if I understood, you query on
specific kinds that are your pre-filtered value).
Do you get your data using ancestor ?
If you have time to give a short example, it will be great !
thank you,
Philippe
On Aug 20, 3:04 pm, Juraj Vitko juraj.vi...@gmail.com wrote:
So guys thanks for the good IRC chat:)
Basically even if this was implemented, it would be too costly on
resources.
What I needed, was to change my thinking to better fit App Engine. The
good thing is, that if one optimizes for App Engine, then his app will
run faster _everywhere_.
Basically, the best method for being able to show filtered data in a
Grid that can be sorted _and_ paged on any column, one needs to pre-
filter the data (upon created/update/delete) into separate kinds. This
filtered kind then can be sorted on using the default indexes, because
filter was already applied. Also, the data stored in this filtered
kind is not the exact copy of the original data, but rather its string
representation with unique suffixes to support lexicographical
ordering and no-equality-restart pagination.
The advantage of this is also that a much more sophisticated filtering
can be used, not just the built-in query operators.
One has to add logic to support recomputation of the filtered kind if
the filter code or sorting-enabled columns change as the app is
evolving, but that's a one time job and can be reused if done
properly.
On Aug 19, 6:01 am, Juraj Vitko juraj.vi...@gmail.com wrote:
sorry! correction of: filter column is named 'f', value is named 'v'
the 'f' and 'v' are actually a the same property now, so:
select from RelationIndex where f == 'roleA' f == 'registered' f
== '{User}' f == '(lastName') f 'a.Smith:xJ8mFc9r' order by f
asc;
like that.
On Aug 19, 3:49 am, Juraj Vitko juraj.vi...@gmail.com wrote:
On a query with equality filter on a list property, would it be
possible to:
1) _not_ discard the sort order on that property,
2) allow for inequality test on smallest/highest value of that list
property (similar to ordering)
That way it would be possible to implement relation indexes using a
table with a single list property (no custom indexes required!)
The property would look something like: [ a.val:unique, filter1,
filter2, filterN, z.val:unique ]
The two 'val' is the same value to be lexicographically ordered (so
numeric types must be converted to appropriate strings).
The a. and z. prefixes would be actually the lowest and highest
printable ascii chars, just so that they are always the lowest and
highest value of the list property, and can be used in asc and desc
ordering.
The filters would be used for equality filtering.
Then this query would be possible:
select from kind where prop == filter1 prop == filter2 prop
'a.someval:unique' order by prop asc
(the optional greater-than operator is there to support paging)
The sort order would _not_ be dropped only if specifically requested
by the user, as to not incur needless overhead on current queries.
A real-use example of a relation index table shared by multiple kinds:
select from RelationIndex where f == 'roleA' f == 'registered' f
== '{User}' f ==
[google-appengine] weird sorting error with inequality filters
I'm getting the following error: BadArgumentError: First ordering property must be the same as inequality filter property, if specified for this query; received distance, expected cost when i run the following code: search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(distance) Did anyone know about this? Its not in the docs (atleast from what i've read)...please tell me there's a way around this weird restriction! I can't really think of a way to restructure my app cause the inequality filter is basically a necessity at this point i'd love to be able to sort my results by something other than cost... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore current time
All GAE servers have GMT/UTC as there current time. The times across different servers may very a few seconds because they are distributed. 2009/8/20 herbie 4whi...@o2.co.uk: How do I calculate the elapsed time since model instance was first stored in the datastore. My model has a property like this: date_added = db.DateTimeProperty(auto_now_add=True) So according to the docs, date_added is set to the current time the first time its put(). But what is the 'current time'? Is it the just the system time of whatever server the app instance is running on? Or is there a synchronised datastore 'current time' so that something like elapsed_time = 'current time' - date_added will always return the correct value irrespective of whatever server it is running on? Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Questions about the preferred way to handle multiple apps/accounts
Hi Brandon, You have a couple of options here. You can build your app to support multiple domains from a single App Engine instance, by checking the Host header, in which case your users only need to add your app to their Google Apps domain, a straightforward process. You can also create an app for each customer, then add them as administrators of the app (if appropriate). If you want to go with the option of creating an app per customer, please contact me offline and we can discuss the best way to go about this. -Nick Johnson On Wed, Aug 19, 2009 at 7:32 PM, Brandon N. Wirtz drak...@digerat.comwrote: We have built an app that allows a web site to move portions of their web site to Google AppEngine, with minimal configuration on their part. This is great for WordPress, and Movable type users who would like to move their images off site, but don't want to mess with their CMS. Think of it like s3 for image hosting light, or Akamai site proxying made cheap. My Question is, should each user of our app software have their own account for the deployment, or should we have multiple accounts we use to set up their accounts, or should we have one account and request more accounts every time we reach our App Quota? The latter would be the easiest for us as we wouldn't have to log in and out to upgrade or monitor our clients health, but if we sell as many copies as it looks like we are going to, I didn't want to pick a method that would not be the one Google wanted us to use. Thanks in advance. -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Chrome display issue
Hi Mark, You'll probably have more luck getting this answered on the GWT groups. -Nick Johnson On Wed, Aug 19, 2009 at 6:05 PM, Mark Pope markp...@gmail.com wrote: I wrote a GWT app using the Eclipse IDE plugin. The app has a form dialog that includes a TextArea widget. User entered data in the TextArea widget is persisted from the form and later displayed in a Label widget. The data 'appear' to be missing newline characters showing one big concatenated string. I printed out the byte representation of the persisted data and the newline characters are there. This only happens in the Chrome browser and the G1 Android browser. Unfortunately these are my target web browsers. -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: app engine post get method. - python
Hi cloudg,
In order to handle POST requests to your handler, you need to define a
'post' method that handles requests of that type. Any HTTP methods that you
have not defined handlers for will return a 405 response.
-Nick Johnson
On Wed, Aug 19, 2009 at 8:24 PM, cloudg gparuch...@gmail.com wrote:
Hi all,
Iam trying to post data using fetchurl and then use the request
method, i get an error with HTTP 405 code. Both get and post do not
work, but what works is if i hardcode the values in the URL like ?
myhost=cool, it works. Please suggest.
To post data: (y.py)
-
from google.appengine.api import urlfetch
from google.appengine.api import apiproxy_stub_map
from google.appengine.api import urlfetch_stub
import urllib
url = http://myurl/;
apiproxy_stub_map.apiproxy = apiproxy_stub_map.APIProxyStubMap()
apiproxy_stub_map.apiproxy.RegisterStub
('urlfetch',urlfetch_stub.URLFetchServiceStub())
form_fields = {
myhost: cool,
mysubject: aid,
email_address: t...@testing.com
}
form_data = urllib.urlencode(form_fields)
result = urlfetch.fetch(url=url,
payload=form_data,
method=urlfetch.POST,
headers={'Content-Type': 'application/x-www-
form-urlencoded'})
-
Read data after post (x.py)
-
from google.appengine.api import users
from google.appengine.ext import webapp
from google.appengine.api import mail
from google.appengine.ext.webapp.util import run_wsgi_app
class MainPage(webapp.RequestHandler):
def get(self):
user = users.get_current_user()
host = self.request.get(myhost)
subj = self.request.get(mysubject)
to_addr = 't...@tet.com'
application = webapp.WSGIApplication(
[('/', MainPage)],
debug=True)
def main():
run_wsgi_app(application)
if __name__ == __main__:
main()
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: weird sorting error with inequality filters
This cookbook recipe explains the problem, in the context of date ranges: http://www.google.com.au/url?sa=tsource=webct=rescd=4url=http%3A%2F%2Fappengine-cookbook.appspot.com%2Frecipe%2Fhow-to-query-by-date-rangeei=DnuOStn7EY__kAWv3s27Cgusg=AFQjCNGKYfrrm9dYm7L43cZw8hvwK9R2mQsig2=xuGt41_VuFei-jvrp1uVBw I'm not sure if the comments are valid. 2009/8/21 iceanfire iceanf...@gmail.com I'm getting the following error: BadArgumentError: First ordering property must be the same as inequality filter property, if specified for this query; received distance, expected cost when i run the following code: search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(distance) Did anyone know about this? Its not in the docs (atleast from what i've read)...please tell me there's a way around this weird restriction! I can't really think of a way to restructure my app cause the inequality filter is basically a necessity at this point i'd love to be able to sort my results by something other than cost... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: weird sorting error with inequality filters
Sorry, ignore me :) 2009/8/21 Scott Ellis sje...@gmail.com This cookbook recipe explains the problem, in the context of date ranges: http://www.google.com.au/url?sa=tsource=webct=rescd=4url=http%3A%2F%2Fappengine-cookbook.appspot.com%2Frecipe%2Fhow-to-query-by-date-rangeei=DnuOStn7EY__kAWv3s27Cgusg=AFQjCNGKYfrrm9dYm7L43cZw8hvwK9R2mQsig2=xuGt41_VuFei-jvrp1uVBw I'm not sure if the comments are valid. 2009/8/21 iceanfire iceanf...@gmail.com I'm getting the following error: BadArgumentError: First ordering property must be the same as inequality filter property, if specified for this query; received distance, expected cost when i run the following code: search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(distance) Did anyone know about this? Its not in the docs (atleast from what i've read)...please tell me there's a way around this weird restriction! I can't really think of a way to restructure my app cause the inequality filter is basically a necessity at this point i'd love to be able to sort my results by something other than cost... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: weird sorting error with inequality filters
here : http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Restrictions_on_Queries check : Properties In Inequality Filters Must Be Sorted Before Other Sort Orders I think that you should do search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(cost) search_query.order(distance) On Aug 21, 10:58 am, iceanfire iceanf...@gmail.com wrote: I'm getting the following error: BadArgumentError: First ordering property must be the same as inequality filter property, if specified for this query; received distance, expected cost when i run the following code: search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(distance) Did anyone know about this? Its not in the docs (atleast from what i've read)...please tell me there's a way around this weird restriction! I can't really think of a way to restructure my app cause the inequality filter is basically a necessity at this point i'd love to be able to sort my results by something other than cost... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Source-Code Version Control
You can try unfuddle.com for hosted SVN. -V --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] How to upload a document in to app engine and download that document
Hi All, I have one form from that user needs to upload Document on click of submit I needs to upload into App Engine and I have to download that uploaded document *Example:* I have Emp. Ref form, from that form user will fill all the data(text filed) and browse one .doc document, and on click of submit that page on back end text field data are inserted tnto spreadsheet(I have done this part ) and I need to download the uploaded document, how can I atchieve that, is it possible through Google App engine ?, if it is yes please help me how to do that Thnaks in Advance SKSK --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Source-Code Version Control
Hi All, I am starting to work on a project with a couple of my friends and we would like to use an Internet based VCS for sharing and maintaining code. We do not have a server environment at the moment, so a client- server system does not work. Do any of you know if Google provides any service to help us in this regard? Or in general, do you know of any good Internet based source control systems? Thanks for the help. Vicson. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to inherit from a template of a template?
Let's assume the following template folders and files in case you are
Twitter developer:
/layout/base.html
/layout/header.html
/layout/footer.html
/shared/tweetSingle.html
/shared/pagination.html
/tweets/tweets.html
Below is the content for each file:
tweets.html
{% extends ../layout/base.html %}
{% block tweets %}
{% for tweet in tweets %}
{% include ../shared/tweetSingle.html %}
{% endfor %}
{% include ../shared/pagination.html %}
{% endblock %}
base.html
{% include ../layout/header.html %}
{% block tweets %}{% endblock %}
{% include ../layout/footer.html %}
header.html
html
head
/head
body
footer.html
/body
/html
tweetSingle.html
div{{ tweet.content }}/div
pagination.html
div
span style=float:righta href{{ LINK_NEXT }}Next/a/span
spana href{{ LINK_PREVIOUS }}Previous/a/span
/div
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Google App Engine Application Ports
Dear friends, It seems to me google apps engine can only run applications that are requested on port 80. It is possible to host a service on google apps engine on a port other than port 80. For example port 25. I will like my service to be called from port 25. If possible, How can I accomplish this? Best Wishes, Leo. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to upload a document in to app engine and download that document
Have a look at this article http://code.google.com/appengine/articles/images.html It shows how to do it with images but other file types stored in GAE objects can be served the same way. Did you succeed to store the file in a GAE object? 2009/8/21 S K sksk...@gmail.com: Hi All, I have one form from that user needs to upload Document on click of submit I needs to upload into App Engine and I have to download that uploaded document Example: I have Emp. Ref form, from that form user will fill all the data(text filed) and browse one .doc document, and on click of submit that page on back end text field data are inserted tnto spreadsheet(I have done this part ) and I need to download the uploaded document, how can I atchieve that, is it possible through Google App engine ?, if it is yes please help me how to do that Thnaks in Advance SKSK --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] How to inherit from a template of a template?
Hi,
I'm building a website using app engine.
I would like to use multiple inheritance but I'm not allow to put
block inside a block.
Here is piece of code which should describe the problem well:
let's assume a .html file called meta_base.html:
!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
head
{% block meta_title %} {% endblock %}
/head
and now I have the file base.html (which inherits from meta_base).
{% extends meta_base.html %}
{% block meta_title %}
title
{% block title_content %} {% endblock}
/title
{% endblock %}
Now I would like my children html page to look like this:
{% extends base.html %}
{% block title_content %}
This is the title
{% endblock %}
But it does not work.
Any very descriptive answer will be greatly appreciated!!
thanks,
Rom
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Source-Code Version Control
Hi vicson, If your project is Open Source, you can use Google Code Project Hosting: http://code.google.com/hosting/createProject . If your project is not Open Source, there are a number of options. Git is a good decentralized source control system, which doesn't require a server - each working tree is also a complete repository. You can use any HTTP or SSH server as a repository you can all synchronize to, or you can synchronize between local or network filesystems. The Git Tutorial is a good introduction to git: http://www.kernel.org/pub/software/scm/git/docs/v1.2.6/tutorial.html If you want somewhere centralized to host your repository, there are a number of services. Nearly all of them charge at least a nominal amount if your project is not open source. I personally recommend github.com - their basic account for private repositories is $7/month for 5 private repositories and 1 collaborator, going up to $12 a month for 5 collaborators. Note that my recommendation of github is an entirely personal one, and shouldn't be construed as any sort of official support for it. :) -Nick Johnson On Fri, Aug 21, 2009 at 12:35 AM, vicson.v vicson@gmail.com wrote: Hi All, I am starting to work on a project with a couple of my friends and we would like to use an Internet based VCS for sharing and maintaining code. We do not have a server environment at the moment, so a client- server system does not work. Do any of you know if Google provides any service to help us in this regard? Or in general, do you know of any good Internet based source control systems? Thanks for the help. Vicson. -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore current time
Thanks On Aug 21, 10:23 am, djidjadji djidja...@gmail.com wrote: All GAE servers have GMT/UTC as there current time. The times across different servers may very a few seconds because they are distributed. 2009/8/20 herbie 4whi...@o2.co.uk: How do I calculate the elapsed time since model instance was first stored in the datastore. My model has a property like this: date_added = db.DateTimeProperty(auto_now_add=True) So according to the docs, date_added is set to the current time the first time its put(). But what is the 'current time'? Is it the just the system time of whatever server the app instance is running on? Or is there a synchronised datastore 'current time' so that something like elapsed_time = 'current time' - date_added will always return the correct value irrespective of whatever server it is running on? Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Friend Connect vs App Engine Google Accounts Services
Those are good resources Matthew. I would also add this server side walk-through from Google : http://code.google.com/apis/friendconnect/articles/serverside_integration.html For Brandon: That would have been neat if they integrated Friend Connect right into App Engine eh? :-) Maybe they'll do that in the future. Dale http://www.bookdope.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Is the datastore reliable?
2 weeks ago I added 1000 entries of type Tenant to two different applications and I downloaded the keys for those 1000 entries. (1000 entries per app) I used them for 1 week, adding related entries to the datastore and never had a problem. I was distracted for one week, and today added more entries and now some of those Tenant keys are no longer findable. I've added more than 400,000 Items that were linked to these, and unfortunately I did not keep a record of all the ones that were added (I was looking at performance testing.) Is this a common problem that others are seeing? Data that just disappears? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Source-Code Version Control
I've been using Mercurial, hosted on bitbucket.com. Neal Walters --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Massive datastore batch put, how to?
That is similar to what I am seeing. Writing to the datastore is VERY expensive. 130K items for me consumes nearly 6.5 hours of CPU. Not very efficient On Aug 16, 5:36 pm, Stakka henrik.lindqv...@gmail.com wrote: I implemented a rough version of my solution, and it seems to work up to ~15k entities. Above that I hit the undocumented transaction write limit you mention when trying to commit 36408 entities serialized into 24 blobs of 60 bytes: java.lang.IllegalArgumentException: datastore transaction or write too big. Well, the datastore seems fast enough for large dataset writes, but all the limitations really makes it troublesome to implement. Also the potentiallossofdataintegrity while processing in multiple requests/ tasks without transactions is risky. Costly too, an 15k entities upload comsumes about 30 minutes of CPU quota. On 15 Aug, 19:57, Juraj Vitko juraj.vi...@gmail.com wrote: I agree with everything you said. Just one thing to consider: by first storing the uploadeddata, then retrieving thatdatafor reprocessing and then storing the processeddataagain will consume additional resources / quotas of your app. GAE really appears to be designed for apps with very high read to write ratio. I would say, if you don't need to handle more than thousand of concurrent users, then you'd be better off renting a server. Into this I've factored additional hassles you may not know about yet, like index size and count limits, single entity group write limits, transaction limitations. All of these are possible to work around, but I have yet to see if those workarounds are feasible in terms of the final price I will be paying to run the app. On Aug 14, 9:24 pm, Stakka henrik.lindqv...@gmail.com wrote: Thanks for the tip, but why write a web app when Java Applets are required, that whouldn't be a good solution. Also, the uploaded file needs to be parsed in it's entirety (CRC check, value references, etc.), and it's not XML. I think I have to parse the file server-side, populate (Java) Entity objects and serialize as many I can into 1 MB blobs. When that is done, start a task that put the de-serialized entities in batches of 500 into the datastore. The response for the file upload request will have to contain some unique task URL that the browser can (AJAX) poll to display the progress. Before I commit to such a elaborate solution, I'll have to test the batch-put performance to see if GAE is even suitable for this kind of app.http://groups.google.com/group/google-appengine/browse_thread/thread/... Users of an online apps shouldn't have to wait hours for a simpledata import just because it's hosted at GAE. If the app where using an SQL database this would only take a minute. On Aug 14, 4:48 pm, Juraj Vitko juraj.vi...@gmail.com wrote: I think you need to write your own Flash or Java Applet based chunked uploader. Or use an existing one and let us know, so that we can use it too. On Aug 12, 11:36 pm, Stakka henrik.lindqv...@gmail.com wrote: I'am working on an browser based accounting app which has a feature to import ledger transactions through file uploads. I'am currently only running on the local dev server, but from what I've read datastore puts -- even batch -- is very slow and CPU (quota) intensive when deployed live. How do I overcome this problem if the user uploads a large file with thousands transaction? I've seen solutions where you batch put entities in chunks of 500. That only works if you run a custom upload tool on your computer, not from a browser since the request is limited to 30 seconds. Am I forced to use the Task Queue? But where do I store the raw uploaded file or the preferably parsed interim transaction entities when the task isn't executing? Funny App Engine has a 10 megabyte request (file upload) size limit when storing 10 megabyte worth of entities seems to be so hard. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Is the datastore reliable?
Hi Mark, What do you mean by 'not findable'? Are you saying that doing a db.get() on the key no longer returns anything? If so, are you positive you - or your code - didn't delete them at some point? Are you able to provide your App ID and keys for several of the entities, please? -Nick Johnson On Fri, Aug 21, 2009 at 3:54 PM, Mark Jones mark0...@gmail.com wrote: 2 weeks ago I added 1000 entries of type Tenant to two different applications and I downloaded the keys for those 1000 entries. (1000 entries per app) I used them for 1 week, adding related entries to the datastore and never had a problem. I was distracted for one week, and today added more entries and now some of those Tenant keys are no longer findable. I've added more than 400,000 Items that were linked to these, and unfortunately I did not keep a record of all the ones that were added (I was looking at performance testing.) Is this a common problem that others are seeing? Data that just disappears? -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine Application Ports
Even this is possible (which I dont think it is), AppEngine is pretty much designed to talk HTTP. You suggesting port 25 suggests wanting to talk SMTP, which AppEngine wont do anyway. On 21/08/2009, leo l...@fomatech.com wrote: Dear friends, It seems to me google apps engine can only run applications that are requested on port 80. It is possible to host a service on google apps engine on a port other than port 80. For example port 25. I will like my service to be called from port 25. If possible, How can I accomplish this? Best Wishes, Leo. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Cannot See List of Applications
Hello all, I have signed up for an account, created my first application ID, and uploaded my application to the App Engine. I am able to access the application via the expected appspot URL. However, when I log into the Console I still see the Create an Application button and have no way to view information such as logs and quotas for my existing application. I have considered the possibility that I the same address is being used for both a Gmail Account and a Google Account, but I double- checked and the account I am using has no access to Gmail. Does anyone have any suggestions? Thanks, -- Adam --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Friend Connect vs App Engine Google Accounts Services
Thanks for the help. Looking into server side integration now... On Aug 21, 6:03 am, dalenewman dalenew...@gmail.com wrote: Those are good resources Matthew. I would also add this server side walk-through from Google :http://code.google.com/apis/friendconnect/articles/serverside_integra... For Brandon: That would have been neat if they integrated Friend Connect right into App Engine eh? :-) Maybe they'll do that in the future. Dalehttp://www.bookdope.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: weird sorting error with inequality filters
Hi philippe, I did try the code you provided but that doesn't really work--all it does is rank the results in the order of cost. Infact, the result that is furthest away, ends up coming in first place, with the rest randomly scattered. Any other ideas? -thanks On Aug 21, 5:43 am, Philippe philippe.cr...@gmail.com wrote: here :http://code.google.com/appengine/docs/python/datastore/queriesandinde... check : Properties In Inequality Filters Must Be Sorted Before Other Sort Orders I think that you should do search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(cost) search_query.order(distance) On Aug 21, 10:58 am, iceanfire iceanf...@gmail.com wrote: I'm getting the following error: BadArgumentError: First ordering property must be the same as inequality filter property, if specified for this query; received distance, expected cost when i run the following code: search_query.filter(cost , min_price) search_query.filter(cost , max_price) search_query.order(distance) Did anyone know about this? Its not in the docs (atleast from what i've read)...please tell me there's a way around this weird restriction! I can't really think of a way to restructure my app cause the inequality filter is basically a necessity at this point i'd love to be able to sort my results by something other than cost... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: appcfg upload authentication problem
Hi John, Is the account that you are using to upload the app a Google Apps account? Also, what are the account type settings you are using for this application? If you wouldn't mind sending me your app ID it would simplify getting to the bottom of this issue. Thank you, Jeff On Mon, Aug 17, 2009 at 2:03 AM, John j...@netfm.org wrote: Hello, I have just loaded by app onto appengine but am having a problem using appcfg.py to upload data I am able to view my app running at appspot.com although without any data I can access my appengine app dashboard by going to https://appengine.google.com/a/mydomain my app is running at http://myapp.appspot.com I can upload my data to dev_appserver running locally using appcfg.py without any problems but when I try to upload my data to appengine I get an authentication failure The only change I make to my use of appcfg.py before uploading to appengine is by removing the --url=http://localhost:8000/remote_api any help on this is much appreciated, John. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine Application Ports
Well, I want to design a simple service that is used to synchronize two mail boxes on the internet. I will like to host this simple smtp service on google apps engine. I will use java to implement the smtp protocol. So you now see why I want port 25 open on my application. I think this possible with amazon ec2, but I will like to use google apps engine since this free. Is this possible ? Thanks, Leo. On Fri, Aug 21, 2009 at 11:50 PM, Barry Hunter barrybhun...@googlemail.comwrote: Even this is possible (which I dont think it is), AppEngine is pretty much designed to talk HTTP. You suggesting port 25 suggests wanting to talk SMTP, which AppEngine wont do anyway. On 21/08/2009, leo l...@fomatech.com wrote: Dear friends, It seems to me google apps engine can only run applications that are requested on port 80. It is possible to host a service on google apps engine on a port other than port 80. For example port 25. I will like my service to be called from port 25. If possible, How can I accomplish this? Best Wishes, Leo. -- Leonard Tchuta IT Management Consultant Fomatech IT Management Consulting Ltd www.fomatech.com l...@fomatech.com Mob: +86-15900755434 Tel: +86-21-64398772 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Inconsistent 404
In my app.yaml if have: - url: /ill/request/new/.* script: ill/request/new/main.py -- When I request the following: http://localhost:8080/ill/request/new/agRyY2xzcgwLEgZQYXRyb24YNAw I get the following: INFO 2009-08-21 18:27:42,381 dev_appserver.py:3029] GET /ill/ request/new/ag RyY2xzcgwLEgZQYXRyb24YNAw HTTP/1.1 200 - :) --- If I repeat the procedure with a different ending, such as: http://localhost:8080/ill/request/new/agRyY2xzcgwLEgZQYXRyb24YNgw I get the following: INFO 2009-08-21 19:03:08,569 dev_appserver.py:3029] GET /ill/ request/new/ag RyY2xzcgwLEgZQYXRyb24YNgw HTTP/1.1 404 - :( - If I repeat the request for the first item which worked the first time, it works fine, but anything else throws a 404 error. If I close the developement server and restart it, it lets the first pass through fine but then gives the 404 on anything that differs from the first pass. If I reverse the above procedure after restarting the developement server the results are reversed. Any thoughts would be greatly appreciated. Michael --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Massive datastore batch put, how to?
I wonder how Google thinks the app providers should bill their customers? A flat fee isn't feasible when a single user action costs soo much of the app profit. Limiting the number of actions (imports) is a possibility, but thats bad for the app's productiveness. Google should remove the cost of CPU for internal API calls. It might be more fair if they billed for the number of API calls in combination with the megabyte size of data in each call. That would also encurage good app design; Batch put whould be cheaper than many single puts. Small entities with few indices cheaper than large entities with many indices, etc,. Google say App Engine is great for small startups and scale to millions of users. But how will a startup afford to reach the scale where GAE shines. With regular hosting you atleast know the cost in advance, and during heavy load it's just slower. I really like GAE, I'd like to use it! But with all the missing JRE classes, many limits and the cost of datastore CPU I probably can't for my current project. ;( On 21 Aug, 16:57, Mark Jones mark0...@gmail.com wrote: That is similar to what I am seeing. Writing to the datastore is VERY expensive. 130K items for me consumes nearly 6.5 hours of CPU. Not very efficient On Aug 16, 5:36 pm, Stakka henrik.lindqv...@gmail.com wrote: I implemented a rough version of my solution, and it seems to work up to ~15k entities. Above that I hit the undocumented transaction write limit you mention when trying to commit 36408 entities serialized into 24 blobs of 60 bytes: java.lang.IllegalArgumentException: datastore transaction or write too big. Well, the datastore seems fast enough for large dataset writes, but all the limitations really makes it troublesome to implement. Also the potentiallossofdataintegrity while processing in multiple requests/ tasks without transactions is risky. Costly too, an 15k entities upload comsumes about 30 minutes of CPU quota. On 15 Aug, 19:57, Juraj Vitko juraj.vi...@gmail.com wrote: I agree with everything you said. Just one thing to consider: by first storing the uploadeddata, then retrieving thatdatafor reprocessing and then storing the processeddataagain will consume additional resources / quotas of your app. GAE really appears to be designed for apps with very high read to write ratio. I would say, if you don't need to handle more than thousand of concurrent users, then you'd be better off renting a server. Into this I've factored additional hassles you may not know about yet, like index size and count limits, single entity group write limits, transaction limitations. All of these are possible to work around, but I have yet to see if those workarounds are feasible in terms of the final price I will be paying to run the app. On Aug 14, 9:24 pm, Stakka henrik.lindqv...@gmail.com wrote: Thanks for the tip, but why write a web app when Java Applets are required, that whouldn't be a good solution. Also, the uploaded file needs to be parsed in it's entirety (CRC check, value references, etc.), and it's not XML. I think I have to parse the file server-side, populate (Java) Entity objects and serialize as many I can into 1 MB blobs. When that is done, start a task that put the de-serialized entities in batches of 500 into the datastore. The response for the file upload request will have to contain some unique task URL that the browser can (AJAX) poll to display the progress. Before I commit to such a elaborate solution, I'll have to test the batch-put performance to see if GAE is even suitable for this kind of app.http://groups.google.com/group/google-appengine/browse_thread/thread/... Users of an online apps shouldn't have to wait hours for a simpledata import just because it's hosted at GAE. If the app where using an SQL database this would only take a minute. On Aug 14, 4:48 pm, Juraj Vitko juraj.vi...@gmail.com wrote: I think you need to write your own Flash or Java Applet based chunked uploader. Or use an existing one and let us know, so that we can use it too. On Aug 12, 11:36 pm, Stakka henrik.lindqv...@gmail.com wrote: I'am working on an browser based accounting app which has a feature to import ledger transactions through file uploads. I'am currently only running on the local dev server, but from what I've read datastore puts -- even batch -- is very slow and CPU (quota) intensive when deployed live. How do I overcome this problem if the user uploads a large file with thousands transaction? I've seen solutions where you batch put entities in chunks of 500. That only works if you run a custom upload tool on your computer, not from a browser since the request is limited to 30 seconds. Am I forced to use the Task Queue? But where do I store the raw uploaded
[google-appengine] Re: Image Manipulation - JPEG Quality Setting
Hi MF, Unfortunately controlling the compression level for a JPEG is not currently possible in the images API. As you may know, the App Engine servers are not using PIL, but we use it in the SDK to simulate the server side functionality which is present in App Engine. This sounds like a good feature request, would you mind filing it here?: http://code.google.com/p/googleappengine/issues Thank you, Jeff On Tue, Aug 18, 2009 at 11:56 AM, MediumFidelity rzio...@gmail.com wrote: All- I have been playing around with the Python Image Library on GAE, and I was wondering if there is a way to specify the quality (compression level) of a JPEG image. I know you can set the output format (to convert from a PNG to JPEG for example), but can you control how compressed the JPEG is? Any help would be appreciated! Thanks, MF --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Requested Task_Queue Function: add_multi
Hi James, This sounds like a good idea. Would you mind filing a feature request so that we can track it and others can vote by starring? http://code.google.com/p/googleappengine/issues/list Thank you, Jeff On Wed, Aug 19, 2009 at 9:04 PM, James thelevybre...@gmail.com wrote: An add_multi function for the taskqueue would likely provide the same benefits we see from batching calls to the db and memcache. Of course, more introspection and taskqueue and functionality in general would be welcome, but 'add_multi' seems like low-hanging fruit. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] two applications two domains - please help
OK. I have a number of applications. I successfully set up one to be accessed via one of my domains. Now I have a second app I want to map top a different domain. I'm stumped. If I have one domain with a cname rcord pointing to ghs.google.com and another pointing to ghs.google.com, how does google know WHICH app it's meant to bring up? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Inconsistent 404
Although i doubt following is going to work, but give it a try - url: /ill/request/new/(.*) Either way, do you have a handler for each: agRyY2xzcgwLEgZQYXRyb24YNAw and agRyY2xzcgwLEgZQYXRyb24YNgw If not, i dont think you can treat the appengine way to handling urls similar to the pretty urls you can get in Zend Framework on RoR. What i would do is below: http://localhost:8080/ill/request/new?key=agRyY2xzcgwLEgZQYXRyb24YNAw --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Within appengine, how do I download the contents of a spreadsheet without having to write to disk?
Hi Dave,
Great find, yes it seems that the Documents API design is somewhat App
Engine un-friendly. I've filed a bug to remedy this.
http://code.google.com/p/gdata-python-client/issues/detail?id=278
My proposed solution, not yet tested, to replace the _DownloadFile method in
gdata.docs.service to something like the following
def _GetFile(self, uri, file_handle):
server_response = self.request('GET', uri)
if server_response.status != 200:
raise gdata.service.RequestError, {'status': server_response.status,
'reason': server_response.reason,
'body': server_response.read()}
file_handle.write(server_response.read())
def _DownloadFile(self, uri, file_path):
Downloads a file.
Args:
uri: string The full Export URL to download the file from.
file_path: string The full path to save the file to.
Raises:
RequestError: on error response from server.
f = open(file_path, 'wb')
try:
self._GetFile(uri, f)
except gdata.service.RequestError, e:
f.close()
raise e
f.flush()
f.close()
I haven't tested this yet but if you are feeling brave feel free to try it
out. You could then call _GetFile directly and pass in a StringIO object for
your file handle.
Happy coding,
Jeff
On Thu, Aug 20, 2009 at 6:12 AM, Dave dabo...@gmail.com wrote:
I'm using gdata.docs.service to download a list of a user's
spreadsheets using GetDocumentListFeed. I want to now let the user
select a specific spreadsheet so my app can download its contents, do
some manipulation, and then save a copy of the modified spreadsheet
back to their Google Docs account (with a new filename).
To grab the contents of their spreadsheet, I thought I could use the
Download method on the results of GetDocumentListFeed.
Unfortunately, this method requires a location to write the downloaded
file to, and AppEngine doesn't let its apps write files to disk (or so
I've read). So what is the recommended way to do this? I'd be happy
getting the contents of the file as a data structure in memory,
containing something like csv data. I don't need to deal with XML. Any
suggestions, or pointers to documentation, would be appreciated.
Thanks,
Dave
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: two applications two domains - please help
On Fri, Aug 21, 2009 at 5:32 PM, Kate wrote: OK. I have a number of applications. I successfully set up one to be accessed via one of my domains. Now I have a second app I want to map top a different domain. I'm stumped. If I have one domain with a cname rcord pointing to ghs.google.com and another pointing to ghs.google.com, how does google know WHICH app it's meant to bring up? Google Apps will look at the requested domain and load the app defined for that domain - which may be gmail, google calendar etc, or a App Engine app. That's why the domain must be registered with Google Apps. -- rodrigo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: two applications two domains - please help
by the URL. Consider it magic. It just works. On Aug 21, 2009, at 4:32 PM, Kate wrote: OK. I have a number of applications. I successfully set up one to be accessed via one of my domains. Now I have a second app I want to map top a different domain. I'm stumped. If I have one domain with a cname rcord pointing to ghs.google.com and another pointing to ghs.google.com, how does google know WHICH app it's meant to bring up? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] OAuth provider on app engine?
Hi, I am thinking about adding an api to my app engine app and it will need authentication. I thought I might as well try to go all the way and implement OAuth, so I was curious if anybody has tried this already. Actually, I use standard authentication (ie Google Accounts), and Google already has OAuth, so is there a way to use the standard Google OAuth entry points but with my app engine url as scope? That would be sweet! Cheers, Gijsbert --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: https support
google guys, I don't know if you guys realize the implication of not getting a static IP? if I have a domain (abc.com) I cannot map it to the appspot subdomain without an IP and I have to resort to a redirect. And search engines are not friendly to redirects. And no serious website will ever want to redirect to blah.appsot.com it just looks unprofessional. So this lack of static IP is preventing google app engine from becoming actually useful like EC2 for real business. And that's why there is no serious apps or serious ecommerce systems running on your GAE so far despite of all the hype.It's almost like selling a car without headlights i.e. sounds like a small miss but it will prevent the car you are selling from being popular. no serious person can drive only during the day time. Please don't think of this static IP problem from the techie point of view, think from your 'customers' perspective. This is a show stopper!GAE is such a good concept and can revolutionize the industry but this lack of static IP is such a spoiler! On Aug 6, 7:27 am, Tony fatd...@gmail.com wrote: Fair enough. I'm fine with we're working on solutions, as long as it's not sorry, wait for the world to upgrade to FF3/IE8. On Aug 6, 10:21 am, Nick Johnson (Google) nick.john...@google.com wrote: Hi Tony, We're looking into solutions. It's not as simple as it may first appear - in the case of a service like Amazon EC2, your server resides at only a single physical location, and so does theIPaddress you rent. In contrast, App Engine apps are served from IPs at Google datacenters around the globe. -Nick Johnson On Thu, Aug 6, 2009 at 3:18 PM, Tonyfatd...@gmail.com wrote: Why not charge a monthly fee for apps to get astaticIP(like Amazon does)? Scarcity of supply seems like a bit of a cop-out to me - it's not apparent that a majority of app engine apps require this support. The customers you're losing because of this, however, are customers that plan to process e-commerce transactions online without looking like a phishing scam. Customers that make money on your service are more likely to spend money on your service. On Aug 5, 3:55 am, Nick Johnson (Google) nick.john...@google.com wrote: Hi J Singh, Due to the way SSL works, this is not currently possible without allocating anIPaddress for each App Engine domain that would use SSL - which itself isn't very practical due to IPv4 address scarcity. The latest version of SSL supports using a singleIPaddress for multiple sites with different certificates, but browser support for this version is not yet nearly widespread enough to make it a practical alternative. -Nick Johnson On Wed, Aug 5, 2009 at 5:39 AM, J Singhj.si...@earlystageit.com wrote: For an appengine-based site,https://abc.appspot.comiscurrentlysupported buthttps://www.abc.comcannotbe supported. I know there is a technical hurdle to cross but didn't know if any techniques had been proposed for being able to usewww.abc.comwithSSLconnections? Thanks. J Singh Managing Director Early Stage IT (978) 760-2055 http://www.earlystageit.com -- Nick Johnson, Developer Programs Engineer, App Engine -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
