[appengine-java] Re: open source pdf engine for GAE
Hi Vik,
This should be possible with appengine and Google Docs, but it may
not be easy.
For any Google account you have 1GB of Google Docs usage free and
additional
storage is $0.25 per GB, so you could be creative with storing files
(securely) and or
moving them to other backup.
Here's how it could be done:
For 'creating a PDF' you would actually create or edit a Google
Docs document on the fly
and keep that stored in Google Docs. (Google Documents List API...
Creating a permanent
generic version of the document and making a copy and editing the copy
on the fly would be easiest.)
In the email to the donor, you would give them a url that they will
use to download the file as a pdf.
That url for the pdf would not be the Google Docs url, but would be
a url to your appengine application.
That specific delivery url would have to not use Google authentication
(though the rest of your urls would continue to). For that delivery
url, you would have to implement your own programatic authentication
and authorization for that delivery url and supply donor specific auth/
auth to the donor in an easy to use manner.
Behind the scenes in your app: Since your application is already
authenticated with Google
and authorized to access your Google Docs, you would read the PDF
export stream directly
into the response stream to the donor. A brief look at the Google
Documents List API
suggests that that's possible, but I haven't tried it.
Cheers,
Nichole
On May 29, 9:07 pm, Vik vik@gmail.com wrote:
Hie Nichole
My use case is different. We are a non profit and our systems wants to
implement a automatic appreciation certification mailing facility to the
blood donor after blood donation.
So, I need to create the pdf certificate on the fly and email it to the
blood donor. So the google docs approach wont help us.
Thankx and Regards
Vik
Founderhttp://www.sakshum.orghttp://blog.sakshum.org
On Mon, May 30, 2011 at 9:20 AM, Nichole nichole.k...@gmail.com wrote:
I haven't tried this programmatically, but you can export a Google
Docs document in pdf format.
If it's useful to store your files or a generic version of the file
which you can copy and specialize as a Google Doc, then export as pdf
might be a good way to create a PDF file.
I'd be interested in hearing how well that works for you if you do
implement it.
On May 20, 10:42 am, Vik vik@gmail.com wrote:
any further advise on this please?
Thankx and Regards
Vik
Founderhttp://www.sakshum.orghttp://blog.sakshum.org
On Wed, May 18, 2011 at 12:03 PM, Vik vik@gmail.com wrote:
Hie
Even below code fails for the same exception
ByteArrayOutputStream out = new ByteArrayOutputStream();
PDF pdf;
try {
pdf = new PDF(out);
log.info(#1);
pdf.setTitle(Using TextColumn and Paragraph classes);
pdf.setSubject(Examples);
pdf.setAuthor(Innovatics Inc.);
log.info(#2);
Page page = new Page(pdf, Letter.PORTRAIT);
pdf.flush();
Multipart mp = new MimeMultipart();
MimeBodyPart htmlPart = new MimeBodyPart();
htmlPart.setFileName(whatever.pdf);
log.info(#7);
htmlPart.setContent(out.toByteArray(), application/pdf);
mp.addBodyPart(htmlPart);
log.info(#8);
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
Message msg = new MimeMessage(session);
msg.setContent(mp);
msg.setFrom(new InternetAddress(vik@gmail.com));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress(vik@gmail.com));
msg.setSubject(testing PDF system);
Transport.send(msg);
Thankx and Regards
Vik
Founder
http://www.sakshum.org
http://blog.sakshum.org
On Wed, May 18, 2011 at 10:25 AM, Stephen Johnson
onepagewo...@gmail.comwrote:
Try not adding the image or the font. Just try to get as simple a PDF
working as possible.
On Tue, May 17, 2011 at 9:47 PM, Vik vik@gmail.com wrote:
Hie
I am doing exact same. Here is the code:
log.info(start of PDFTest:);
//OutputStream out = resp.getOutputStream();
ByteArrayOutputStream out = new ByteArrayOutputStream();
PDF pdf;
try {
pdf = new PDF(out);
log.info(#1);
pdf.setTitle(Using TextColumn and Paragraph classes);
pdf.setSubject(Examples);
pdf.setAuthor(Innovatics Inc.);
log.info(#2);
String fileName = images/share_facebook.png;
Image image1 = new Image(pdf, new BufferedInputStream(
getClass().getResourceAsStream(fileName)),
ImageType.PNG);
log.info(#3);
[appengine-java] Gmail IMAP and SMTP using OAuth
Hi Guys, Anyone knows about integrating Gmail IMAP and SMTP using OAuth on GAE. Both of them using sockets. Also if I use GAE Mail java API to send mails, from address(sender) becoming an issue as only the app developers email address can be added. Basically my app is integrated to Google APPS and I am looking to send mails on behalf of the logged in user. Any thoughts on this? Thanks in advance George Simon -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] [mapreduce] Global limitation on concurrent mapper jobs performance
We are testing the mass creation of a very large number of entities in the datastore (several billions). We use csv files (approx. 100 Mb each), uploaded into the blobstore, and run mapper jobs on them. Our goal : minimize the overall execution time (whatever the cost). There seems to be an overall performance limitation we can not overcome, even when playing with different parameters : . Set a high value to mapreduce.mapper.inputprocessingrate (for instance 1,000,000) . Set a high value to mapreduce.mapper.shardcount (for instance 20, or 50) . Launch concurrent mapper jobs in parallel (for instance 20 jobs, 1 job per file) The overall performance sticks around 500 entities/second. Is there a specific limitation related to the blobstore reads we should be aware of? Or has someone any tips about improving this perf? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Is Google planning to offer HR datastore migration feature
I would like to switch from M/S datastore to HR datastore, buut how should I do this? Is Google planning to offer HR datastore migration feature? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: FileWriteChannel writeChannel throwing java.io.IOException?
The error: 1. E2011-05-27 07:41:59.139 org.gonevertical.core.server.jdo.data.base64.WriteBase64 writeToBlob: writeToBlob(): Error 5: java.io.IOException 2. E2011-05-27 07:41:59.149 org.gonevertical.core.server.jdo.data.base64.WriteBase64 decode: com.google.appengine.api.datastore.DatastoreTimeoutException: The datastore operation timed out, or the data was temporarily unavailable. at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:46) at com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:98) at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:69) at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:71) at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:32) at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:69) at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:56) at com.google.appengine.api.datastore.QueryResultIteratorImpl.ensureLoaded(QueryResultIteratorImpl.java:156) at com.google.appengine.api.datastore.QueryResultIteratorImpl.hasNext(QueryResultIteratorImpl.java:65) at org.gonevertical.core.server.jdo.data.base64.WriteBase64.decode(WriteBase64.java:219) at org.gonevertical.core.server.jdo.data.base64.WriteBase64.decodeToBlob(WriteBase64.java:87) at org.gonevertical.core.server.jdo.data.BlobTmpJdo.writeTmpBlob(BlobTmpJdo.java:335) at org.gonevertical.core.server.jdo.data.BlobTmpJdo.writeTmpBlob(BlobTmpJdo.java:321) at org.gonevertical.core.server.servlet.Servlet_Task.startWriteToBlob(Servlet_Task.java:106) at org.gonevertical.core.server.servlet.Servlet_Task.doGet(Servlet_Task.java:92) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:97) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:260) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$2.handleRequest(RuntimePb.java:9669) at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:439) at com.google.net.rpc.impl.Server$RpcTask.runInContext(Server.java:573) at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:448) at com.google.tracing.TraceContext.runInContext(TraceContext.java:688) at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:326) at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:318) at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:446) at
[appengine-java] Channel API (Java) doesn't work locally, but in production mode.
Hey folks.
I know that I'm not the first one who is facing this problem.
There is a similar thread, but I don't think it's exactly the same
problem.
I'm working on a project including GAE + GWT.
We are using gwt-gae-channel (http://code.google.com/p/gwt-gae-
channel/) to send different messages from the server to the client
(basically we are implementing a real life game for mobile devices.)
I have read and understood the Channel API documentation and we are
implementing the channel creation in the same way as proposed.
Also gwt-gae-channel has been introduced on the client side like
described in gwt-gae-channel wiki.
The interesting part: In production mode (app is deployed to GAE)
everything works just fine (Users are able to chat, server sends
notifications, etc.)
However the channel doesn't work locally.
We use Maven and start our App locally with the maven command
gwt:run
Here is the first exception which occurs after creating the channel on
the server side:
[ERROR] 2011-05-28 11:00:48.523:WARN::/_ah/channel/dev
[ERROR]
com.google.appengine.api.channel.dev.LocalChannelFailureException:
Channel for application key null not found.
[ERROR] at
com.google.appengine.api.channel.dev.ChannelManager.getChannel(ChannelManager.java:
58)
However on server side the channel seems to be created.
When sending messages to a client there is no warning or exception.
The exception above occurs after creating the channel by some other
thread (I think it's the long-polling thread which works in the
background!?!?!?)
So this exception is not thrown by any of our code lines, but by some
other thread of the local test environment.
Here is what we do on server side when creating the channel (just the
important lines):
String clientId = getChannelIdFromUserKey(userKey); //userKey = the
datastore key which is then hashed to get a short client id used to
create the channel
String channelToken = channelService.createChannel(clientId); //the
token is then passed to the client
return channelToken;
The client receives this token (a gwt-dispatch command triggers the
channel creation and returns the token):
ChannelFactory.createChannel(channelToken, new
ChannelCreatedCallback()
{
@Override
public void onChannelCreated(Channel channel)
{
.
}
}
Which leads to:
[INFO] Socket error.
[INFO] Channel for application key null not found.
I really hope somebody out there can help me/us.
It is really, really important that we fix this bug soon, as we have
just few weeks left to finish the project (well, it's an university
thing ;) )
Kind regards,
Z
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Problem With JPA and Google Engine
Hello, i have a problem With JPA and Google Engine, i have one trial web application http://jocual.appspot.com/index.jsp and it works good with JDO but when i use JPA it report the next error: Caused by: java.lang.NoClassDefFoundError: javax/persistence/spi/ ProviderUtil at com.google.appengine.runtime.Request.process-2e0607ad3b5a947f(Request.java) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:132) at javax.persistence.Persistence.createFactory(Persistence.java:160) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java: 112) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java: 66) at com.presentacion.EMF.clinit(EMF.java:6) at com.presentacion.PersistenceActionJPA.execute(PersistenceActionJPA.java: 38) And i have a persistence.xml and all the anotations and all the jar´s incluyed in the proyect, i dont know what is happen? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Ghost entities. I can see entities in the 'Datastore Viewer' but they doesn't appear with a query
Hi, I have a very strange problem. In my database I have Entities I can see in the Datastore Viewer but do not appear in queries. It only happens with certain entities, but I have no way of knowing which. The solution to appear is very simple yet very strange. I edit the entity from the datastore viewer, and without any modifications, I click 'Save Entity'. Now automatically appear in the queries. This seems a index problem, but I have deleted all the index and have been recreated. The problem persists. It seems a serious problem in the datastore. The data is there, but I can not find them. This seems to occur sporadically and randomly in a block of entities that have been recorded at the same time. Example of entity 'Fichaje' author value: myem...@gmail.com type: string date -MM-DD HH:MM:SS value: 2011-05-19 20:01:00 type: gd:when With the query SELECT * FROM Fichaje WHERE author='myem...@gmail.com' the entity doesn't appear. But I edit the entity from datastore viewer and with no changes click in 'Save Entity'. Now automatically appear in the query. any ideas? Thank you very much Sorry for my bad English. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Getting STREAMED error whne trying to read InputStream
I am getting a STREAMED error when trying to read the InputStream from
a POSTed HttpServletRequest. It worked perfectly in 1.3, but it is not
working in 1.5. The following is the code:
public void doPost(HttpServletRequest request, HttpServletResponse
response)
throws IOException {
String posted_data = ;
//read all the contents of the posted data into a string
final char[] buffer = new char[0x1];
StringBuilder posted_data_builder = new StringBuilder();
BufferedReader in=request.getReader();
int d;
while((d=in.read()) != -1){
posted_data_builder.append((char)d);
}
posted_data = posted_data_builder.toString();
}
Am I doing something wrong, or is this a bug?
Thank you
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Problems uploading to app engine
I'm a developer on the application ridersofgilead, but when I try to use the toolbar inside eclipse to deploy the application I get this error Unable to update: java.io.IOException: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=ridersofgileadversion=3; 404 Not Found This application does not exist (app_id=u'ridersofgilead'). But when I run appcfg.sh update war and provide my email address and password at the script prompts it uploads correctly. If it matters the application is registered under a google apps domain and my email address is under a seperate google apps domain. Ryan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Checking if I understand transactions correctly
I am writing some code that needs to do a rollback on a secondary
object/table should the transaction fail, I believe this can be done
via entity groups, however I am not sure if this is how it would be
implemented. I have written some sample code to check if what I would
be doing is correct?
Would the following code ensure the Account object is never updated
if the insert of the TransactionRecord object fails.
public void addTransaction(String account, Double value, String
description, Date date) throws EntityNotFoundException {
DatastoreService datastore =
DatastoreServiceFactory.getDatastoreService();
int retries = 3;
while (true) {
Transaction txn = datastore.beginTransaction();
try {
// Update the bank balance
Key key = KeyFactory.createKey(Account,
account);
Entity e = datastore.get(key);
Double balance = (Double)
e.getProperty(balance);
balance += value;
e.setProperty(balance, value);
datastore.put(e);
// Record transaction details
Entity d = new Entity(TransactionRecord, key);
d.setProperty(account_key, key);
d.setProperty(date, date);
d.setProperty(value, value);
d.setProperty(description, description);
txn.commit();
break;
} catch (ConcurrentModificationException e) {
if (retries == 0)
throw e;
--retries;
} finally {
if (txn.isActive())
txn.rollback();
}
}
}
Thanks for any feedback!
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Would the following create problems for me in Google app engine?
I have a Java application that has approximately 100 users. In this application there is a table that would have the equivalent of 100,000 entities added per day, ie 100 users each doing 1000 inserts per day. From time to time I will need to output a CSV file that shows one months worth of entries for a particular user, ie would result in a file with 30,000 entries. If I understand correctly, The entities would be given an anscestor record to allow querying the transactions by user, and then filter them by month. Am I going to have timeout issues with querying by user+month, and for in the case where I need to export a month's worth of data? Any feedback much appreciated! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Unable to find the webapp directory Files\GoogleAppEngine\appengine- java-sdk-1.3.0\demos\guestbook\war
hi i have just installed the java sdk. after extracting the files from the appengine-java-sdk-1.3.0 jar, I am trying to start the server with at the command line with the following: C:\Program Files\GoogleAppEngine\appengine-java- sdk-1.3.0\bindev_appserver.cmd C:\Program Files\GoogleAppEngine \appengine-java-sdk-1.3.0\demos\guestbook\war I get the following error message: Unable to find the webapp directory Files\GoogleAppEngine\appengine- java-sdk-1.3.0\demos\guestbook\war can anyone help? Thx -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Unable to find the webapp directory
hi i have just installed the java sdk. after extracting the files from the appengine-java-sdk-1.3.0 jar, I am trying to start the server with at the command line with the following: C:\Program Files\GoogleAppEngine\appengine-java- sdk-1.3.0\bindev_appserver.cmd C:\Program Files\GoogleAppEngine \appengine-java-sdk-1.3.0\demos\guestbook\war I get the following error message: Unable to find the webapp directory Files\GoogleAppEngine\appengine- java-sdk-1.3.0\demos\guestbook\war can anyone help? Thx -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Simple data model
Hi there, I can not get my head around the datastore. Just want to have a list of users Entity: Users - name - Address - Read books // how to link to Books in low-level api Entiry: Books - History - Sports - Religion Should Books be in the Entity group, or by itself and then select from the Category list and add a property on user with a category. Entity = Object Ahh, I dont get it... help :-) Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[google-appengine] Re: Redirection problems after enabling OpenID
Hey Eric, In the request handler where you're redirected to after login with OpenId, make sure that handler doesn't cause the redirect loop. So look how you're checking for logged in user before redirecting to the login page again. Cheers, Rob On May 29, 6:40 pm, Eric Kolotyluk eric.koloty...@gmail.com wrote: I tried clearing the cookies and restarting the browser several times, but that did not seem to help. Cheers, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: FAQ for out of preview pricing changes
This change penalizes spiky apps in a huge way. Three years ago, when we first started with GAE, we did so for three reasons: 1) scalability 2) pay for what you use model, and 3) SIMPLICITY. Our app is very spiky - a single user will at times generate 50+ concurrent requests that need to be served quickly. We've spent three years working around the GAE to fine-tune the code and work within the scalability and other constraints, and it works remarkably well. There is not a chance that we could get as far if we had to use EC2 or other systems. The new pricing model has two implications for us: a) we will start paying mostly for idle instances and b) we lose the simplicity and predictability of the existing model. It seems that you're shifting the hard problem here back to the users, and that's a shame. As others have suggested, the new scheduler needs to have the flexibility to cater to small and tight apps - in not providing it you are penalizing the very people who embraced your tools first and worked hard around your early limitations and teething problems. Jan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] GAE/J persistence
Hi , I have this code: PersistenceManager pm = PMF.get(). getPersistenceManager(); and this jdoconfig.xml: ?xml version=1.0 encoding=utf-8? jdoconfig xmlns=http://java.sun.com/xml/ns/jdo/jdoconfig; xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; xsi:noNamespaceSchemaLocation=http://java.sun.com/xml/ns/jdo/ jdoconfig persistence-manager-factory name=transactions-optional property name=javax.jdo.PersistenceManagerFactoryClass value=org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory/ property name=javax.jdo.option.ConnectionURL value=appengine/ property name=javax.jdo.option.NontransactionalRead value=true/ property name=javax.jdo.option.NontransactionalWrite value=true/ property name=javax.jdo.option.RetainValues value=true/ property name=datanucleus.appengine.autoCreateDatastoreTxns value=true/ /persistence-manager-factory /jdoconfig I get an error regarding the jdoconfig.xml telling me that I have indicated transaction-optional which is a resource that is not available. Is there a resource that I need to add so that I can use it? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Detect mobile phone
Hi Ryan, How to use http://www.zytrax.com/tech/web/mobile_ids.html. Please give me demo about it ! Thanks ! On Thu, May 26, 2011 at 10:54 PM, Ryan Peden rya...@gmail.com wrote: The easiest way I can think of is to check to 'User-Agent' header. When you get a connection from a web browser on an Android phone, Android will appear somewhere in the user agent header. See http://www.zytrax.com/tech/web/mobile_ids.html for a good list of mobile user agent headers. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- *Best regards,* *--- * *Hồ Công Danh* Developer [M] : 094 7579 583 [E] : congdanh...@gmail.com linh...@viethungthai.vn -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: urlfetch_errors.DownloadError - ApplicationError 2 - What should I do?
I can't see the results for one of the 3 users now
So I guess I will try retrying the call for ApplicationError 2
situations.
Any ideas why these errors happen?
Also how can I identify them in python?
I use a try: except: block, probably I will parse the exception data
from sys.exc_info()
On May 28, 12:43 am, Kaan Soral kaanso...@gmail.com wrote:
class 'google.appengine.api.urlfetch_errors.DownloadError',
DownloadError('ApplicationError: 2 ',)
I am getting a lot of these, out of 800k users, 3 of them are my
accounts, and I see the results every day.
That's why I am skeptical about retrying the operation, considering
the possibility that the remote API call succeeds although I get this
error.
There is no easy way for me to check the results.
What do you think?
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What do you want to see answered in Greg's pricing FAQ?
It keeps quit from Google/Greg's side. Hope they announce more information shortly. Everybody is just speculating which is not good. On May 29, 11:51 pm, JH ja...@tickettrackit.com wrote: While things are still being decided I'd also like to inquire about roll over instance hours. If we switch to this new instance hour system, say I reserve 24 and only use 20. Can the 4 roll over to my next billing period? Also, from my previous post, maybe if I decline my SLA I could get 24 extra instance hours a day? (gotta give me credit for trying) 15 minutes idle time billed seems a bit odd, we can't be billed for what instance time we actually use? Finally, still curious how always on will work.. Thanks and great work on GAE On May 29, 12:20 pm, JH ja...@tickettrackit.com wrote: Being that an SLA typically just means I will receive a refund for down time, in other words a fraction of $9/month and $.08/hour, I'd like to propose an alternative model for people to have paid apps without an SLA. I am definitely interested in uptime, however a refund of my $.08/hour for 30 minutes downtime doesn't get me too excited. On May 12, 2:14 am, Kenneth kennet...@aladdinschools.com wrote: Greg mentioned he was putting together an FAQ so let's help him out! If you're going to answer this just put in your question into a single line, let's not try and answer them here or give opinions, there's plenty of other threads for that. I do understand that Google doesn't have answers to some these. Here's my list: 1) What is the time granularity of the instance pricing? ie if I have an instance up for 5 minutes, what am I charged, $0.08 / 60*5? 2) Will I be able to tune the scheduler myself, ie set it to performance or low cost, Will I be able to limit the min or max number of instances created (with the obvious impact on user experience)? 3) Python concurrency, will this require any code changes, do you have any estimates based on your testing of the number of well behaved requests per second a single instance will be able to handle for a given framework? 4) Database charges, when can you give us more details over what Max gave in the other thread, are you charging for deletes, what do you expect the ratio to be between the new pricing metric and the Datastore API calls metric we have today? 5) Will you be charging differently for instances that use different amounts of memory, since this seems to be the cost that you're going after that isn't charged for in the current model. Thanks, Kenneth -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] delete all data (tuples) for a kind(table) in google app engine datastore
Hello, If one clicks on Datastore Viewer, then click on the drop down list By kind and choose the kind (table) one wants to view, 20 entities (tuples) of one kind will be showed. There is a tick box to enable user to choose these 20 entities and a delete button can be used to delete these selected 20 tuples. This works for deleting 20 tuples. If i have 1 million tuples (entities), it is not realistic to delete all the tuples by 20 tuples each time. My question is how can one easily delete all tuples belongs to a kind? Is there any user interface that GAE provide to delete all tuples one time? Thanks! David -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Newbie question. How to deal with real world app new versions?
Hi there, I'm starting with GAE with python. I've made a few toy apps but now i'm working on a mobile app that has backend in GAE. Now the app is live and has a few thousand users. I need to implement new feature without breaking the app. What is the best way to keep developing safely without breaking the app for users? - Deploy a new version to the same app and keep the previous version active or - Copy the data from this app to a another one just for testing or - Do everything locally and only upload when ready Thanks for your times. Regards Luis Confraria -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: delete all data (tuples) for a kind(table) in google app engine datastore
I know it won't help much but you can increase the limit URL parameter to 200. You'll go ten times faster then :-) Otherwise you'll have to make a task and use a cursor. Francois -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Invalid runtime or the current user is not authorized to use it.
guest@calculate ~/APPENGINE/Go $ appcfg.py update ./myapp/ Application: graycardinalster; version: 1 Host: appengine.google.com Starting update of app: graycardinalster, version: 1 Scanning files on local disk. Error 400: --- begin server output --- Invalid runtime or the current user is not authorized to use it. --- end server output --- Runtime is go. Work properly on local host. Please, help my. Oleg. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Infinite loop backend gets stuck
Hi, I have a backend that is started once on /_ah/start request and just goes to infinite loop for data processing. From time to time it gets stuck inside the loop waiting for something forever. Problems: - I can not find a place where it gets stuck. Can someone suggest the way I can do this. - I do not see ability to restart instance programatically. Any ideas? More information: - backend is static; - i have no more infinite loops inside the main loop; - logging does not help as I have no idea how to log properly in order to identify the place of stuck -- Regards, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Succesful deployment, but pages on custom domain don't reflect the changes
Has anyone else run into this? I have a (mostly static content) application running on the python appengine. I can deploy my app without a problem. If I access the app on the appspot.com domain after deploying, the pages reflect the changes as expected (e.g mysite.appspot.com/ index.html) but the exact same pages accessed on my custom domain (www.mysite.com/index.html) are completely unaffected by the deployment. Of course I did my best to flush any cache I could think of, starting with the browser (...and checking that I was getting 200 statuses, not 304...), I use my phone to access the site over 3g and bypass any possible proxy cache, not to avail. I'm running out of ideas, if you have a hint, please share it! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Rudimentary New Pricing Analysis
I've been keeping an eye on AWS Elastic Beanstalk (http:// aws.amazon.com/elasticbeanstalk/) and looking forward to the day they roll out the Python version. With Beanstalk it seems Amazon is moving towards providing some of the auto-scaling functionality we've become accustomed to on GAE (and at no additional cost). It would certainly make the transition easier if it ever comes to that. Also EC2 does have a free usage tier for new customers. On May 29, 8:27 pm, Barry Hunter barrybhun...@gmail.com wrote: For what it worth, I dont think it fair to compare a GAE instance like for like to a VPS. they are very different 'beasts'. GAE in theory provides you much more. Google handle all the setup, configuration, provisioning, load balancing. Granted it does seem expensive, if you only using equivient of one VPS. But once you start need to manage multiple VPSs, provision load-balancers, re-provision in case of failure etc, make backups. All that happens 'magicically' with GAE. Dont forget you also get a free quota of instance hours too. On 30 May 2011 01:06, Ugorji ugo...@gmail.com wrote: The other threads don't address this in isolation. I'm hoping Greg can address this in his updated document. *On-demand pricing (using 1.2 GHz processor for baseline compute):* GAE Frontend: 128MB RAM, 1.2 GHz processor ... cost 0.08cents/IH GAE Backend: 256MB RAM, 1.2 GHz processor ... costs 0.16cents/IH *Compare With:* Amazon EC2 micro instance: 613MB RAM, ~1.2 GHz processor (burst to ~2.4GHz). -- COST: 0.02cents/IH -- SUMMARY: GAE costs 4X on front end with 1/4 RAM, and 8X on backend, with 1/2 RAM 1and1 DEDICATED server: 2 X 2.2 GHz CPU, 2GB RAM. -- COST: $60/month (roughly 0.08cents/IH) -- SUMMARY: GAE provides 1/8 RAM and 1/4 CPU for equal pricing *Closing* It's hard to understand or justify how the hosting prices can jump so aggressively that they are not even close to being competitive against other established players (like Amazon). It feels atypical of what we would expect from Google. ** Note that this comparison is just for the hosting prices. This is fair since the APIs are charged separately (so are not compared here). *References:* http://www.google.com/enterprise/appengine/appengine_pricing.html http://order.1and1.com/xml/order/ServerPremiumDualCoreM http://www.rackspace.com/cloud/cloud_hosting_products/servers/pricing/ http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/pricing/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Blocking IPs
I want to block some IPs from accessing my site. Is there a direct way of doing so in AppEngine? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Succesful deployment, but pages on custom domain don't reflect the changes
It sounds like it might be this issue. Please star it. http://code.google.com/p/googleappengine/issues/detail?id=5040 I have found that deleting the old version will force the new one to be served. BTW, using your phone is no guarantee you won't be going through a proxy cache. -Chris On Sat, May 28, 2011 at 9:44 PM, Hugo hhar...@gmail.com wrote: Has anyone else run into this? I have a (mostly static content) application running on the python appengine. I can deploy my app without a problem. If I access the app on the appspot.com domain after deploying, the pages reflect the changes as expected (e.g mysite.appspot.com/ index.html) but the exact same pages accessed on my custom domain (www.mysite.com/index.html) are completely unaffected by the deployment. Of course I did my best to flush any cache I could think of, starting with the browser (...and checking that I was getting 200 statuses, not 304...), I use my phone to access the site over 3g and bypass any possible proxy cache, not to avail. I'm running out of ideas, if you have a hint, please share it! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Blocking IPs
Look into using the dos.yaml file. It does exactly what you want. On May 30, 4:43 am, StudApps studa...@gmail.com wrote: I want to block some IPs from accessing my site. Is there a direct way of doing so in AppEngine? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: delete all data (tuples) for a kind(table) in google app engine datastore
Use Datastore Admin Bulk Delete http://code.google.com/appengine/docs/adminconsole/datastoreadmin.html#Deleting_Entities_in_Bulk -Chris On Mon, May 30, 2011 at 8:03 AM, Francois Masurel f.masu...@gmail.comwrote: I know it won't help much but you can increase the limit URL parameter to 200. You'll go ten times faster then :-) Otherwise you'll have to make a task and use a cursor. Francois -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Blocking IPs
My bad, it would probably help more to point you to this page: http://code.google.com/appengine/docs/python/config/dos.html On May 30, 4:43 am, StudApps studa...@gmail.com wrote: I want to block some IPs from accessing my site. Is there a direct way of doing so in AppEngine? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Newbie question. How to deal with real world app new versions?
Hi Luis, These are all viable options. Personally i would choose to upload a new version to AppSpot and have a test version of your mobile app point to 2.yourmobilebackend.appspot.com. That does mean that the temporary version uses the same datastore as your live version, so that could be an issue depending on the kind of update you're introducing. If you feel confident enough it all works, it's just a simple click to change the live version. Hope it helps. Regards, Rohan Chandiramani -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What do you want to see answered in Greg's pricing FAQ?
Although I was ignoring the $9/app discussion before, now I realize that* I want N environments as I develop my app: development, production, test, beta,* etc. And, I would like* all those environments to be identical to my production environment (ie: all be paid apps)*. Thus, to develop properly the cost is N x $9/month. I'm not enthusiastic about paying for paid app service level when most of these environments don't need that level of service. Yet, I don't want to discover that there is a difference between 'free' and 'paid' when I finally upload to my production environment. Paying a little to keep spammy apps away is ok, but I think the payment policy should be focused on the person paying the bill and not the apps that they generate. In fact, I would go further and say that google should really* encourage developers to get on the highest service level early in the development process. * In particular, always-on and high replication data should be encouraged at the prototype stage so developers can feel how fast their apps will be and ... and so* we can imagine what we can do with that enhanced capability!!Appengine developers that pay* is a market niche that google should really give incentives to so we can build and evangelize the platform!In contrast, right now with the current fee structure, I'm on the free tier and waiting each time my prototype is called as the instance is loaded ... I could pay for always-on but I feel a little silly doing that when the prototype is just for me -- the incentives are not aligned correctly... Dennis -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Newbie question. How to deal with real world app new versions?
- All versions share the same datastore data. - Copy some/all data locally and test everything you want, then you have the two options you have said. If you want to test something that can break your data, test it in other app. If you want only to test page X is working well in production too (once it works locally), upload other version (the users can't see that version until you want it) and go visit page X. - From the second point you have a great tip, when you deploy new features, do so in a new version, preload it visiting your site to start up some instances, and then switch the version in Dashboard. That way you prevent any downtimes. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Rudimentary New Pricing Analysis
I'm very aware that GAE and AWS are different beasts and understand the advantages of IAAS. My point still stands though. Let's forget GAE and AWS for a second. If someone said you can use IAAS at 4X to 8X the cost of PAAS, is it justifiable in your mind? Back in the day, we had dedicated servers and managed servers for a 25% premium. I don't know what the premium should be, but 4X to 8X premium sounds a bit high. Just something for the GAE team to think about. It would be nice to see some price justification - it makes it easier to continue to evangelise GAE. I can see an AWS ad as below: AWS hosting costs 1/8 the price of GAE, and gives you double the RAM. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: FAQ for out of preview pricing changes
Jan, I hope the GAE team corrects me if I'm wrong, but I wouldn't worry too much in your case, especially if you've tuned the requests to consume minimal CPU. The way things are right now, if each of your requests takes 50ms, (of which 10ms is actual CPU and the rest is waiting) then one GAE instance can serve 1000 / 50 = 20 requests a second. So you'll be paying (we all think) for three instances, plus the startup / shutdown overhead. What most of us are forgetting is that threading will be introduced to coincide with the pricing changes (hopefully :D), so when that happens one instance will actually serve 1000 / 10 = 100 requests a second. So things might not be as bad as you think, especially if you've optimized your app to consume less CPU than what I'm calculating. In this case you might not even notice the changed pricing from a CPU point of view. This is approximate, of course, but you get the idea. Things might not be as bad as you think, and your CPU optimizations will probably continue to serve you well. Let's wait a bit and see what the actual situation is. Sudhir On May 30, 12:05 pm, Jan Z/ Hapara jan.zawad...@gmail.com wrote: This change penalizes spiky apps in a huge way. Three years ago, when we first started with GAE, we did so for three reasons: 1) scalability 2) pay for what you use model, and 3) SIMPLICITY. Our app is very spiky - a single user will at times generate 50+ concurrent requests that need to be served quickly. We've spent three years working around the GAE to fine-tune the code and work within the scalability and other constraints, and it works remarkably well. There is not a chance that we could get as far if we had to use EC2 or other systems. The new pricing model has two implications for us: a) we will start paying mostly for idle instances and b) we lose the simplicity and predictability of the existing model. It seems that you're shifting the hard problem here back to the users, and that's a shame. As others have suggested, the new scheduler needs to have the flexibility to cater to small and tight apps - in not providing it you are penalizing the very people who embraced your tools first and worked hard around your early limitations and teething problems. Jan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: Detect mobile phone
Yeah, but Android could also be a tablet, which would be better than a NetBook in terms of resolution and capabilities. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of JH Sent: Wednesday, May 25, 2011 3:10 PM To: Google App Engine Subject: [google-appengine] Re: Detect mobile phone These guys have some good code: http://detectmobilebrowser.com/ On May 25, 3:45 am, Cong Danh Ho congdanh...@gmail.com wrote: Hi all, How to detect mobile phone when android connect to Google App Engine ! help me ! -- *Best regards,* *--- * *Hồ Công Danh* Developer [M] : 094 7579 583 [E] : congdanh...@gmail.com linh...@viethungthai.vn -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: URLFetch 1MB Request Limit
If you own the second system, you can build a protocol where your appengine system saves the blob to blobstore, notifies the second system that a job is ready, handing it a key. You then have the second system fetch the blob off the appengine. G -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Rudimentary New Pricing Analysis
I still think you are basing your comparsion on assumptions, ones yet to be proved. Its almost certainly better to wait until the promised price comparisons are available before jumping to conclusions. On 30 May 2011 19:12, Ugorji ugo...@gmail.com wrote: I'm very aware that GAE and AWS are different beasts and understand the advantages of IAAS. My point still stands though. Let's forget GAE and AWS for a second. If someone said you can use IAAS at 4X to 8X the cost of PAAS, is it justifiable in your mind? Back in the day, we had dedicated servers and managed servers for a 25% premium. I don't know what the premium should be, but 4X to 8X premium sounds a bit high. Just something for the GAE team to think about. It would be nice to see some price justification - it makes it easier to continue to evangelise GAE. I can see an AWS ad as below: AWS hosting costs 1/8 the price of GAE, and gives you double the RAM. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: URLFetch 1MB Request Limit
Or build a proxy somewhere that does the big fetch caches and provides as 2+
files.
GAE:URL Fetch Proxy
Proxy: URL Fetch Requested URL, Write to Local, return first 1000 bytes ,
(if File longer than 1000 Bytes return 1001 bytes)
GAE: If file len 1000 bytes {URL Fetch Proxy 2} Else Goto 10
Proxy: Return from file bytes 1001 to end, If file still longer than 1001
bytes return 1001 bytes
GAE: If file len 1000 bytes {URL Fetch Proxy 2} Else Goto 10
GAE 10: Append file bits. Do Stuff.
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Guy Tavor
Sent: Monday, May 30, 2011 12:03 PM
To: google-appengine@googlegroups.com
Subject: [google-appengine] Re: URLFetch 1MB Request Limit
If you own the second system, you can build a protocol where your appengine
system saves the blob to blobstore, notifies the second system that a job is
ready, handing it a key.
You then have the second system fetch the blob off the appengine.
G
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Rudimentary New Pricing Analysis
Just to clarify: I'm not making any assumptions here. I just printed out facts from the official websites, and asked Google for some price justification. ie. Amazon gives a micro instance with some specs for a price, Google gives a front end and backend instance with some specs for a price. I listed both, and said that a rudimentary (basic, undeveloped) analysis looking at the price alone shows a 4X to 8X increase for GAE while giving significantly less RAM. There's no assumption there. There's some listed prices, and a request for a justification from Google. If anything, an explanation by a non-googler on Google's behalf would be considered the assumptions. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Rudimentary New Pricing Analysis
I know all of this, I just wish app engine got cheaper... It's a hard choice, you really have to analyze per case. If only it got cheaper, it'd be a lot more suited to a wider range of projects. I could bet this factor isn't present in google's price analysis, while it definitely should. If we, as app engine developers who enjoy the system, start to think about pricing every now and then... something's not right. * Here's hope for anyone from the app engine team to read this humble opinion from a fellow GTUG member and app engine enthusiast! * On 30 maio, 16:16, Ugorji ugo...@gmail.com wrote: Just to clarify: I'm not making any assumptions here. I just printed out facts from the official websites, and asked Google for some price justification. ie. Amazon gives a micro instance with some specs for a price, Google gives a front end and backend instance with some specs for a price. I listed both, and said that a rudimentary (basic, undeveloped) analysis looking at the price alone shows a 4X to 8X increase for GAE while giving significantly less RAM. There's no assumption there. There's some listed prices, and a request for a justification from Google. If anything, an explanation by a non-googler on Google's behalf would be considered the assumptions. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Datastore best use, and queries..
All -
I'm a little new to app engine and JDO, but before I dive to far into
my project I wanted to ask a question on how to query for stored data.
I have some something that looks like this:
@PersistenceCapable
public class User{
@Persitant
private ListGroups
...
}
Is it possible to write a query to find all all users that belong to
some group given this format ? If so how would the query look ?
Thanks in advance.
..Chris
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore best use, and queries..
On Monday, May 30, 2011 1:23:44 PM UTC-7, Christopher Johnson wrote:
All -
I'm a little new to app engine and JDO, but before I dive to far into
my project I wanted to ask a question on how to query for stored data.
I have some something that looks like this:
@PersistenceCapable
public class User{
@Persitant
private ListGroups
...
}
Is it possible to write a query to find all all users that belong to
some group given this format ? If so how would the query look ?
It would be possible if Groups were a primitive type like a string or an
int, or if it were the key of another entity. (For starting out, I'd suggest
using Objectify since it's simpler than JDO for datastore access.)
- Brian
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] disable, delete, remove ID/Name in a kind (table)
Hello, I click on DataViewer to see my kinds (tables). No matter which kind (table) i look into and i always see the property (attribute) ID/ Name. I have not found any use of this attribute and really want to disable or remove or delete it. Is there any way to do so? Thanks! David -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Change property name of a kind (table)
Hello, Suppose i have a kind (table) called Person defined in GAE data store already, it has properties (attributes) like personId, age, location, phone number and contains one million tuples. Now i just want to change attribute personId to pId without affect anything else. Is there any easy way to do so? Thanks! David -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: scalable querying GAE data store
Hello Robert,
Thanks for your idea! I try to find the cursor coding example from the
internet, but what i have found is really little and GAE site does not
say much about it.
Do you know where can i find coding example in Python for cursor of
GAE?
Thanks!
David
On May 29, 5:31 am, Robert Kluin robert.kl...@gmail.com wrote:
HiDavid,
There is no way you're going to return a million entities in a
single request. You should look into cursors. Cursors allow you to
fetch results from a query, then resume fetching more results where
you left off. Cursors are nice because you can use them within the
same request to efficiently fetch a larger number of entities in
smaller batches, or you can use them across requests (by passing them
between requests). You could also potentially use the new files API
to dump your entities to the blobstore (maybe as CSV) then download
the blob.
http://code.google.com/appengine/docs/python/datastore/queryclass.htm...
http://code.google.com/appengine/docs/python/blobstore/overview.html#...
Within a request, you can use a cursor something like this:
query = YourKind.all().order('some_prop')
cursor = None
while True:
if cursor:
query.with_cursor(cursor)
entities = query.fetch(500)
# do stuff with entities
if len(entities) 500:
break
cursor = query.cursor()
Across requests you'd just need to pass the cursor to the next
request (perhaps in a form field, for example).
Robert
On Sat, May 28, 2011 at 04:06,DavidStoneminpeng2...@gmail.com wrote:
Hello,
Suppose i have 4,5 or even more kinds (tables) stored in GAE data
store and each table has quite a lot of data, let's say each table has
5 million rows (entities) or even more. Now i am facing a big problem
with generating and returning a big size result for a query. The
reason is that a request handler will time out if takes more than 30
second to generate the query result and i think there is a probably
10MB limitation for a http response to return the query result.
So i am wondering is there a way to send the query result as small
portion (data streaming) each time back to me.
I have read from this link:
http://code.google.com/appengine/docs/python/runtime.html
The link mentions App Engine collects all of the data the request
handler script writes to the standard output stream, then waits for
the script to exit. When the script exits, all of the output data is
sent to the user.
App Engine does not support sending data to the user's browser before
exiting the handler. Some web servers use this technique to stream
data to the user's browser over a period of time in response to a
single request. App Engine does not support this streaming technique.
As far as i understand their information is that there is no way to
send the query result as many small portions by time and each time
sends out for example 5000 rows.
At the same time, this link (http://arstechnica.com/web/news/2010/12/
app-engine-gets-streaming-api-and-longer-background-tasks.ars)
mentions there is a streaming API from GAE.
Now i am confused by these messages. Any idea or suggestion about this
issue would be well appreciated.
Another question is that once i have a query generating a lot of
result might cause time out to the request handler due to the 30
seconds time limitation. How could i overcome this problem?
Thank you very much for all the answers from you.
David
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.- Hide quoted text -
- Show quoted text -
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] disable, delete, remove ID/Name in a kind (table)
You can't. It's the primary key of the kind. The objects are fetched using this key. 2011/5/30 David Stone minpeng2...@gmail.com: Hello, I click on DataViewer to see my kinds (tables). No matter which kind (table) i look into and i always see the property (attribute) ID/ Name. I have not found any use of this attribute and really want to disable or remove or delete it. Is there any way to do so? Thanks! David -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: scalable querying GAE data store
Hey David,
I'm not sure where to point you for a simple example, but I'm sure
there are some out there. You just need to build your query, fetch
some results, grab the cursor, pass it into the same query, then fetch
more results, then repeat. Make sure you check the number of results
returned so that you know if you need to continue fetching more
results or not.
Here is one example of using cursors to continue processing results
(just the get_work method):
https://bitbucket.org/thebobert/slagg/src/a41cc67326ce/slagg/__init__.py#cl-324
The items you should pay special attention to are 'work_query' and
'continue_cursor.'
Robert
On Mon, May 30, 2011 at 17:27, David Stone minpeng2...@gmail.com wrote:
Hello Robert,
Thanks for your idea! I try to find the cursor coding example from the
internet, but what i have found is really little and GAE site does not
say much about it.
Do you know where can i find coding example in Python for cursor of
GAE?
Thanks!
David
On May 29, 5:31 am, Robert Kluin robert.kl...@gmail.com wrote:
HiDavid,
There is no way you're going to return a million entities in a
single request. You should look into cursors. Cursors allow you to
fetch results from a query, then resume fetching more results where
you left off. Cursors are nice because you can use them within the
same request to efficiently fetch a larger number of entities in
smaller batches, or you can use them across requests (by passing them
between requests). You could also potentially use the new files API
to dump your entities to the blobstore (maybe as CSV) then download
the blob.
http://code.google.com/appengine/docs/python/datastore/queryclass.htm...
http://code.google.com/appengine/docs/python/blobstore/overview.html#...
Within a request, you can use a cursor something like this:
query = YourKind.all().order('some_prop')
cursor = None
while True:
if cursor:
query.with_cursor(cursor)
entities = query.fetch(500)
# do stuff with entities
if len(entities) 500:
break
cursor = query.cursor()
Across requests you'd just need to pass the cursor to the next
request (perhaps in a form field, for example).
Robert
On Sat, May 28, 2011 at 04:06,DavidStoneminpeng2...@gmail.com wrote:
Hello,
Suppose i have 4,5 or even more kinds (tables) stored in GAE data
store and each table has quite a lot of data, let's say each table has
5 million rows (entities) or even more. Now i am facing a big problem
with generating and returning a big size result for a query. The
reason is that a request handler will time out if takes more than 30
second to generate the query result and i think there is a probably
10MB limitation for a http response to return the query result.
So i am wondering is there a way to send the query result as small
portion (data streaming) each time back to me.
I have read from this link:
http://code.google.com/appengine/docs/python/runtime.html
The link mentions App Engine collects all of the data the request
handler script writes to the standard output stream, then waits for
the script to exit. When the script exits, all of the output data is
sent to the user.
App Engine does not support sending data to the user's browser before
exiting the handler. Some web servers use this technique to stream
data to the user's browser over a period of time in response to a
single request. App Engine does not support this streaming technique.
As far as i understand their information is that there is no way to
send the query result as many small portions by time and each time
sends out for example 5000 rows.
At the same time, this link (http://arstechnica.com/web/news/2010/12/
app-engine-gets-streaming-api-and-longer-background-tasks.ars)
mentions there is a streaming API from GAE.
Now i am confused by these messages. Any idea or suggestion about this
issue would be well appreciated.
Another question is that once i have a query generating a lot of
result might cause time out to the request handler due to the 30
seconds time limitation. How could i overcome this problem?
Thank you very much for all the answers from you.
David
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.- Hide quoted text
-
- Show quoted text -
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
Re: [google-appengine] Invalid runtime or the current user is not authorized to use it.
Hi Oleg, You've currently got to sign up to request access to use the Go runtime on production. http://googleappengine.blogspot.com/2011/05/app-engine-150-release.html Sign up form: https://spreadsheets.google.com/a/google.com/spreadsheet/viewform?formkey=dGJ6LXlIYWk4MjhnM0dubUstUHFKVXc6MQ Robert On Sat, May 28, 2011 at 08:57, Oleg Puchinin graycardinals...@gmail.com wrote: guest@calculate ~/APPENGINE/Go $ appcfg.py update ./myapp/ Application: graycardinalster; version: 1 Host: appengine.google.com Starting update of app: graycardinalster, version: 1 Scanning files on local disk. Error 400: --- begin server output --- Invalid runtime or the current user is not authorized to use it. --- end server output --- Runtime is go. Work properly on local host. Please, help my. Oleg. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Infinite loop backend gets stuck
Hi Alex, What do you mean it 'gets stuck'? Are you able to include / show us any code? Make sure you checkout the logging flush functions, they'll probably be helpful. http://code.google.com/appengine/docs/python/backends/logserviceapi.html Robert On Sat, May 28, 2011 at 10:28, radzish radz...@gmail.com wrote: Hi, I have a backend that is started once on /_ah/start request and just goes to infinite loop for data processing. From time to time it gets stuck inside the loop waiting for something forever. Problems: - I can not find a place where it gets stuck. Can someone suggest the way I can do this. - I do not see ability to restart instance programatically. Any ideas? More information: - backend is static; - i have no more infinite loops inside the main loop; - logging does not help as I have no idea how to log properly in order to identify the place of stuck -- Regards, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Google maps stopped working
Check the maps docs / forums? http://code.google.com/apis/maps/index.html http://code.google.com/apis/maps/forum/ This is an App Engine forum. http://code.google.com/appengine/ On Sat, May 28, 2011 at 23:09, yossi Manor yosmano...@gmail.com wrote: I added a Google maps gadget to my friend's web site. Initially it worked but stopped. I tried to generate another code but it did not solve the problem. Here is the last code generated: !--Google maps Script -- script src=http://www.gmodules.com/ig/ifr?url=http://hosting.gmodules.com/ ig/gadgets/file/11428391296844949/driving- directions.xmlamp;up_fromLocation=amp;up_myLocations=285%20Flatbush %20Ave.%20extention%2C%20Brooklyn%2C%20NY %2011201amp;up_defaultDirectionsType=amp;up_autoExpand=amp;synd=openamp;w=800amp;h=75amp;title=Directions +by+Google +Mapsamp;brand=lightamp;lang=enamp;country=USamp;border=http%3A%2F %2Fwww.gmodules.com%2Fig%2Fimages%2Famp;output=js/script !--Google maps Script End -- Any suggestions? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Succesful deployment, but pages on custom domain don't reflect the changes
After staring the issue Chris mentioned, you might want to add some type of cache-buster to your static assets. There could be many different caches between you and your app Robert On Mon, May 30, 2011 at 10:01, Chris Copeland ch...@cope360.com wrote: It sounds like it might be this issue. Please star it. http://code.google.com/p/googleappengine/issues/detail?id=5040 I have found that deleting the old version will force the new one to be served. BTW, using your phone is no guarantee you won't be going through a proxy cache. -Chris On Sat, May 28, 2011 at 9:44 PM, Hugo hhar...@gmail.com wrote: Has anyone else run into this? I have a (mostly static content) application running on the python appengine. I can deploy my app without a problem. If I access the app on the appspot.com domain after deploying, the pages reflect the changes as expected (e.g mysite.appspot.com/ index.html) but the exact same pages accessed on my custom domain (www.mysite.com/index.html) are completely unaffected by the deployment. Of course I did my best to flush any cache I could think of, starting with the browser (...and checking that I was getting 200 statuses, not 304...), I use my phone to access the site over 3g and bypass any possible proxy cache, not to avail. I'm running out of ideas, if you have a hint, please share it! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: delete all data (tuples) for a kind(table) in google app engine datastore
Yes! 10x speed boost ;) If you're doing this for 1 million entities I'd also suggest some beer. On Mon, May 30, 2011 at 09:03, Francois Masurel f.masu...@gmail.com wrote: I know it won't help much but you can increase the limit URL parameter to 200. You'll go ten times faster then :-) Otherwise you'll have to make a task and use a cursor. Francois -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Change property name of a kind (table)
Not sure if there is any tool can help to do the migration. Usually we will use low level API to iterate through all existing entities and save. If your total number of entities is less than 100K, then GAE can finish it in 30s. If it's more, then you will need to use cursor + task queue or backend process. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Change property name of a kind (table)
Hi David,
Depends how you define easy. You'll need to loop over the data,
convert it, then re-put it. There are a couple ways to do this, but I
think the easiest to explain is the 'Expando two-step' method. First,
change your kind definition to the new schema (including removing the
old properties) also switch from a db.Model to a db.Expando. Then
loop over your entities, for each entity copy the value from the old
property to the new property, then db.put your entities. Change your
kind back to a db.Model.
I'd also suggest adding a 'version' property to your kind definition
to make this sort of thing easier in the future. Personally, I stick
a v_ = IntegerProperty(default=0) on all of my kinds then bump the
default each time I modify the schema. Makes finding stuff that needs
updated super easy.
There are several choices for how you loop over your data. The
write a handler and use task-queues, use task-queues plus deferred, or
use the map-reduce lib. The first method is pretty easy to outline:
def get(self):
query = KindToMigrate.all()
cursor = self.request.get('cursor')
if cursor:
query.with_cursor(cursor)
entities = query.fetch(batch_size) # maybe a couple hundred
if len(entities) = batch_size:
taskqueue.add(url=self.request.path,
params={'cursor': str(query.cursor())})
for entity in entities:
if hasattr(entity, 'old_property'):
entity.np = entity.old_property
delattr(entity, 'old_property')
db.put(entities)
I'd carefully review and test the before running it since I just
quickly typed it up. ;)
Robert
On Mon, May 30, 2011 at 17:18, David Stone minpeng2...@gmail.com wrote:
Hello,
Suppose i have a kind (table) called Person defined in GAE data
store already, it has properties (attributes) like personId, age,
location, phone number and contains one million tuples. Now i just
want to change attribute personId to pId without affect anything
else.
Is there any easy way to do so?
Thanks!
David
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
