Re: [appengine-java] Ghost rows in Datastore - cant get some rows in a queries until I resave them in the Datastore viewer
HI, I sent you the info by email. Thank you very much Shai On Thu, Feb 25, 2010 at 2:59 AM, Ikai L (Google) ika...@google.com wrote: Were you able to resolve this? What's your app-id? On Wed, Feb 17, 2010 at 2:18 PM, Shai levys...@gmail.com wrote: HI, I encountered something strange when running queries in my application so I went to the Datastore Viewer and reproduced it in there, maybe someone can help me out: I have a table with a string column named 'owner' and + int column names 'seesionId' (+ some irrelevant columns) Everything is reproducing in datastore viewer: When I make the query SELECT * FROM table where owner = '[owner]' I can see a row with the owner and a sessionId with value 7 So I tried: SELECT * FROM table where owner = '[owner]' and sessionId = 7 And the row didn't return ! So I open the row, didn't (!) change any thing and saved it. Now the same query did return the row I really don't understand it Can any one point me to what went wrong ? I have a lot of rows with the same problem and I really need to clean my datastore -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Problem with persistance
I think in your Offer class you should use getter/setter for mOfferDetail
(not a constructor to set the field).
2010/2/25 Anton Klotz dipl.ing.akl...@googlemail.com
Hi Jake,
thanks a lot for your answer.
After changing the mappedBy statement to child like this:
@Persistent(mappedBy = mOfferDetails)
private Offer mOffer;
and removing mappedBy from parent, Offer object seems to be persisted.
But when I get this object from the database with:
public Offer getOfferWithId (long id)
{
return pm.getObjectById(Offer.class, id);
}
and try to access child with
System.out.printf(String.format (Title %s,
mOffer.getOfferDetails().getOfferDescription() ));
I get following error:
java.lang.NullPointerException
at
com.sparradar.server.action.actioncommands.ShowOfferDetailsActionCommand.execute(ShowOfferDetailsActionCommand.java:
41)
at
info.rk.vaadinapp.manager.urldispatching.URLActionDispatcher.handleURI(URLActionDispatcher.java:
90)
at com.vaadin.ui.Window.handleURI(Window.java:358)
at
com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleURI(AbstractCommunicationManager.java:
1830)
at
com.vaadin.terminal.gwt.server.CommunicationManager.handleURI(CommunicationManager.java:
311)
at
com.vaadin.terminal.gwt.server.AbstractApplicationServlet.handleURI(AbstractApplicationServlet.java:
912)
at
com.vaadin.terminal.gwt.server.AbstractApplicationServlet.service(AbstractApplicationServlet.java:
471)
at
com.vaadin.terminal.gwt.server.GAEApplicationServlet.service(GAEApplicationServlet.java:
231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
487)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1093)
at
com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:
51)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:
121)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
360)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
405)
at
com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:
70)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
139)
at com.google.appengine.tools.development.JettyContainerService
$ApiProxyHandler.handle(JettyContainerService.java:352)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
506)
at org.mortbay.jetty.HttpConnection
$RequestHandler.headerComplete(HttpConnection.java:830)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:
396)
at org.mortbay.thread.BoundedThreadPool
$PoolThread.run(BoundedThreadPool.java:442)
I don't get the error if I try to access eg mOffer.getId(). So my
interpretation of this error is that either the child object was not
saved in the database, or it was not fetched from the database. I
don't know how to verify this.
Thanks,
Anton
On 24 Feb., 20:17, Jake jbrooko...@cast.org wrote:
http://code.google.com/appengine/docs/java/datastore/relationships.ht...
You create a bidirectional one-to-one relationship using fields on
both classes, with an annotation on the child class's field to declare
that the fields represent a bidirectional relationship. The field of
the child class must have a @Persistent annotation with the argument
mappedBy = ..., where the value is the name of the field on the
parent class.
From what I see, you have it backwards, with the annotation in the
parent class.
Jake
On Feb 24, 4:26 am, Anton Klotz
[appengine-java] Re: Deleting Entities By Query or the PersistenceManager's deletePersistentAll()
I wan't able to perform a relevant benchmark since PersistenceManager.deletePersistentAll() with 500 rows usually took more than 30 sec's. Also, I can't really ignore the fetch time so I ended up doing many many smaller delete by query I must say delete is highly expensive (cpu) and slow in GAE, I really think you guys should come up with an alternative solution for a cheap delete. Perhaps allowing us to control when the index rebuild should occur by the api. Thanks On Feb 25, 2:43 am, Ikai L (Google) ika...@google.com wrote: I'm not certain there's a big benefit either way, as you still have to remove all the indexes related to the entities. What have your benchmarks reported? On Wed, Feb 17, 2010 at 12:06 PM, Shai levys...@gmail.com wrote: Hi When I delete entities by query am I limited to 500 entities like when I am using PersistenceManager.deletePersistentAll() ? Also, if we ignore the object fetch time - what would consume less CPU time ? Can someone estimate if making many delete queries of smaller amounts of data would be better than getting all the data to for a big pm.deletePersistentAll() opertaion ? Thanks, Shai -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Problem with persistance
I think in your Offer class you should use getter/setter for mOfferDetail (not a constructor to set the field). Can't see why that would make the slightest difference. What does appear iffy is that if you have a bidirectional relation then the user is responsible for setting both sides. I only see Offer.mOfferDetails being set and not the other side. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Cron is not working
Hi to all, Crons in google app engine is not working for me. could you please help me. cron.xml - ?xml version=*1.0* encoding=*UTF-8*? cronentries cron url/*emailing*/url descriptionrepeated every one hour/description scheduleevery 1 hours/schedule /cron /cronentries * my application have the servlet for url emailing My admin console show status as follows Cron Job Schedule/Last Run/Last Status (All times are UTC) /emailing repeated every one hour every 1 hours (UTC) 2010-02-25 07:11:16 on time Too many continues Please anybody tell me what was my mistake. Thanks, Lakshmi. * -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Problem with persistance
2010/2/25 datanucleus andy_jeffer...@yahoo.com I think in your Offer class you should use getter/setter for mOfferDetail (not a constructor to set the field). Can't see why that would make the slightest difference. What does appear iffy is that if you have a bidirectional relation then the user is responsible for setting both sides. I only see Offer.mOfferDetails being set and not the other side. isn't setting Offer.mOfferDetails enough for making Offer.getOfferDetails() returning not null, even if it is a bidirectional relation? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] custom authentication when using Google Apps domain
It is very simple to know if you need to use google apps or not. Here is the article about it: http://code.google.com/appengine/articles/auth.html 1.) Do you want to authenticate users from a specific google apps domain? (Note: Saying yes will exclude gmail and general google accounts) 2.) Do you need to have a custom domain name? If neither of there is true then you don't need google apps. If either is true then you do. If you *just* want a custom domain name then you will need to create a free google apps account for your domain-name mappings; it will not need to have any users created (your users will not be in there because they will come from the google accounts authentication domain). So, to answer your question, create a free google apps account. Add your domain mapping, and never go in there again. Now, when it comes to security, and authorization, you will have to write your own using some kind of application-based security (like a filter). You will need to maintain your own roles (and membership). This will most likely mean creating security entities keyed to their google email address from the current logged in user. As I said above, you will not need to pay for that authentication service (even with a custom domain name). Good luck to you :) On Wed, Feb 24, 2010 at 11:54 PM, Stephan Hartmann hartm...@metamesh.de wrote: You can use Google Apps Standard accounts or common Google Accounts (e.g. Goggle mail) as well (you have to choose when you setup your app). If you choose not to use Google Apps Accounts for your app, you still can assign it a subdomain of your Google Apps domain. For implementing your own custom authentication i'd try http://securityfilter.sourceforge.net/ It has a deployment descriptor-like configuration. If you choose SecurityFilter, you should take care of static files because GAE will serve them directly bypassing servlet filters. Regards, Stephan 2010/2/24 Houston startup coder stephenh...@gmail.com We need users to be able to access our app on our domain, so I went into my Dashboard and clicked Add New URL to set this up, and then added a CNAME at GoDaddy. We use Google Apps for our domain internally, but do not want the users of our App Engine app accessing Mail, Documents, Sites, etc. However, my understanding is that the only way I could point a subdomain at our App Engine app was to use Google Apps and Add New URL. We're going to have an installed desktop application access our app via ClientLogin, and it's fine if that needs to talk to myapp.appspot.com in order to use HTTPS. The plan is for the installed application to hide the credentials it uses to safely communicate with our App Engine application behind the scenes. The only time our users will directly interact with the App Engine app is when they login via a web browser. For that, we'd like to use custom authentication because I don't want to have to pay $50/user/ year for them to access via a Google Apps Premier account. But it sounds like this means we can't safeguard portions of the site with the deployment descriptor if we're using custom authentication: http://code.google.com/appengine/docs/java/config/webxml.html#Security_and_Authentication Is this true? How can we handle this? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Problem with persistance
Hi Stephan,
no, this was not the solution. As I understand only lazy fetching is
supported by GAE. What do I have to do to get all child objects? Now I
wrote such procedure, to keep the persistanceManager open, but it did
not help either:
public OfferDetails getOfferWithId (long id)
{
OfferDetails offerDetail;
pm = PMF.get().getPersistenceManager();
try {
Offer offer = pm.getObjectById(Offer.class, id);
offerDetail = offer.getOfferDetails();
} finally {
pm.close();
}
return offerDetail;
On 25 Feb., 09:07, Stephan Hartmann hartm...@metamesh.de wrote:
I think in your Offer class you should use getter/setter for mOfferDetail
(not a constructor to set the field).
2010/2/25 Anton Klotz dipl.ing.akl...@googlemail.com
Hi Jake,
thanks a lot for your answer.
After changing the mappedBy statement to child like this:
�...@persistent(mappedBy = mOfferDetails)
private Offer mOffer;
and removing mappedBy from parent, Offer object seems to be persisted.
But when I get this object from the database with:
public Offer getOfferWithId (long id)
{
return pm.getObjectById(Offer.class, id);
}
and try to access child with
System.out.printf(String.format (Title %s,
mOffer.getOfferDetails().getOfferDescription() ));
I get following error:
java.lang.NullPointerException
at
com.sparradar.server.action.actioncommands.ShowOfferDetailsActionCommand.ex
ecute(ShowOfferDetailsActionCommand.java:
41)
at
info.rk.vaadinapp.manager.urldispatching.URLActionDispatcher.handleURI(URLA
ctionDispatcher.java:
90)
at com.vaadin.ui.Window.handleURI(Window.java:358)
at
com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleURI(Abstr
actCommunicationManager.java:
1830)
at
com.vaadin.terminal.gwt.server.CommunicationManager.handleURI(Communication
Manager.java:
311)
at
com.vaadin.terminal.gwt.server.AbstractApplicationServlet.handleURI(Abstrac
tApplicationServlet.java:
912)
at
com.vaadin.terminal.gwt.server.AbstractApplicationServlet.service(AbstractA
pplicationServlet.java:
471)
at
com.vaadin.terminal.gwt.server.GAEApplicationServlet.service(GAEApplication
Servlet.java:
231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
487)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1093)
at
com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFi
lter.java:
51)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans
actionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFile
Filter.java:
121)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
360)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
405)
at
com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEn
gineWebAppContext.java:
70)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
139)
at com.google.appengine.tools.development.JettyContainerService
$ApiProxyHandler.handle(JettyContainerService.java:352)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
506)
at org.mortbay.jetty.HttpConnection
$RequestHandler.headerComplete(HttpConnection.java:830)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:
396)
at org.mortbay.thread.BoundedThreadPool
$PoolThread.run(BoundedThreadPool.java:442)
I don't get the error if I try
Re: [appengine-java] Citical Security error in Accounts Java API: request.getUserPrincipal() gets wrong username/email
Yes, i implemented this already. I developped the widget in GAE in Java. There I used OAuth to authenticate against the gdata-api. Then I stored the combination of usern...@gappdomain.xxx and the OAuth accesskey in the GAE persistant storage. Everything works fine. So when i log in google apps the includes Gadget reads the logged in username (request.getUserPrincipal().getName() )) and searches with this username for the OAuth key. But than the problem: sometimes the username (request.getUserPrincipal().getName() ) is wrong, so i read the data (e.g. Contacts List) of the wrong user. So actuelly I don't need a solution for authentication but I need to know which user is actually really logged in to google apps. Any ideas? This should be a quite important topic for all developpers who want to enhance google apps with gae, isn' t it? thanks 2010/2/25 Ikai L (Google) ika...@google.com Probably OAuth, though I'm not sure how that will work with a gadget. In most places gadgets will include a user ID with the makeRequest. On Tue, Feb 23, 2010 at 12:23 PM, Thomas Schnocklake thomas.schnockl...@googlemail.com wrote: Thank you for your answer. So what would you suppose to use for authentication for a gadget that is places in google apps (e.g. gmail, google sites ) ? thanks thomas 2010/2/18 Ikai L (Google) ika...@google.com Yes, this seems to make sense. Being logged into Google Apps is independent of being logged into an App Engine application. They don't use the same cookie. App Engine's User service allows you to use Google logins, but not the Google Apps session. On Tue, Feb 16, 2010 at 3:29 AM, tsschnoc thomas.schnockl...@googlemail.com wrote: Hello, I use App Engine in my Google Apps domain and restricted the authentification of app engine to my apps domain. (see http://code.google.com/appengine/articles/auth.html ) I developed a widget and use this in multiple accounts of my google apps domain. When i switch from one account (of my apps domain) to the other, the former account is displayed in the widget (Java: request.getUserPrincipal().getName() ). So the gadget placed on my Google Apps Inbox displays data of a user different to the user logged in to google apps. I noticed that the problems goes away when i wait some minutes ( session expiration ?? ). see picture: http://picasaweb.google.com/lh/photo/QDcR2Lgk2xI2-UQ77BoGXw?feat=directlink -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Local datastore is empty after migrating to 1.3.1
I'm pretty sure the your datastore is empty message on the admin data viewer pages doesn't mean that his query, in his application code, returned no results. While I don't disagree with you that the data is still in there I'm pretty what you just described is documented in the behavior of indexes in the data-store pretty thoroughly. If you don't index a property (or don't store it) you can not sort or search for it. As an example of your data is there, but completely invisible, try changing your app-id in your appengine-web.xml and looking at the admin page data viewer after a restart. On Thu, Feb 25, 2010 at 12:14 AM, Valters Vingolds valt...@gmail.com wrote: I suspect the data is still there, but somehow not visible to your queries. For example, following happened in my case: I had a query that retrieves my (root) entities, but they are ordered by sequence_id . In 1.3.0., this query worked fine even if sequence_id is un-initialized (is null). With 1.3.1., somehow entities with null sequence_id do not show up in query results - and in development I had neglected to set this field at all. So, getting empty results back: and this may lead you to think that data has been lost. Though I found out that if I remove order by from the query, all results show up - the data is still there. Now, from correctness standpoint, excluding null entities from order by query is ok - as you can't compare against null (per SQL rules) - so such rows may be dropped. Anyway, what I did, I wrote a small migration script to load all my entities, initialize sequence_id = 0 and save them. And it's now working fine with 1.3.1. On Feb 17, 9:06 pm, mcrady mcr...@gmail.com wrote: Same here. If I switch back to 1.3.0 the local data is still there. Any workarounds to get to 1.3.1 with the local data? On Feb 13, 10:45 am, Brian bwa...@gmail.com wrote: My remote datastore kept the data, mylocalone was wiped. Pretty easy to make a script to load a DB with test data, not really a good idea to rely on yourlocaldatastore... On Feb 12, 6:32 pm, mably fm2...@mably.com wrote: Same thing happened to me. Got an emptylocaldatastore after updating to 1.3.1. Hopefully I didn't have much data in it. But it might not be the case for everybody. On 12 fév, 09:35, Al Murauski a.murau...@gmail.com wrote: The Development Console tells The datastore is empty after updating to 1.3.1. Any ideas why? I need mylocaldata back. The local_db file is not emoty and contain sthe data it contained before 1.3.1 migration. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Cron is not working
Hi. What is output to [admin console]-[Logs]-[Requests only]? please Confirm. thanks. 2010/2/25 Sowji sowji.ap...@gmail.com Hi to all, Crons in google app engine is not working for me. could you please help me. cron.xml - ? xml version=*1.0* encoding=*UTF-8*? cronentries cron url/*emailing*/url descriptionrepeated every one hour/description scheduleevery 1 hours/schedule /cron /cronentries * my application have the servlet for url emailing My admin console show status as follows Cron Job Schedule/Last Run/Last Status (All times are UTC) /emailing repeated every one hour every 1 hours (UTC) 2010-02-25 07:11:16 on time Too many continues Please anybody tell me what was my mistake. Thanks, Lakshmi. * -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Problem with persistance
You may want to look into the JDO fetch groups, hollow returns of objects, makeTransient, and other items having to do with what is fetched and what remains accessible after the persistence manager is closed. The DataNucleus documentation is a pretty good place to look at that. I'm somewhat new to this, but had similar troubles and found my answers there. Jake On Feb 25, 8:36 am, datanucleus andy_jeffer...@yahoo.com wrote: interpretation of this error is that either the child object was not saved in the database, or it was not fetched from the database. GAE/J has some DB viewer, so debugging if it is actually in the database is kinda step 1. Then look at the log, since that is why it exists -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Compare keys with JDO ?
Hy... I'm using a JDO Query to retrieve some objects of type Contact witch contains: Key key; Key parentKey; String name; ... parentKey is the key of the User that owns that Contacts. Si if want to retrieve all contacts that belong to a certain user i'd have to make a query that returns all Contact objects with parentKey == user.key But i have a problem comparing the keys if I pass them (the keys...) as String, it will retrieve all contacts. If i pass them as Key, it will retrieve nothing. So what is the problem ? I can't use JDO to make a Query that uses a key equality constraint ? Is it mandatory to use datastore Query? or at least can I compose the string query my self ? How ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Does GAE for Java Support JSF 1.1 or JSF 1.2?
Amr: I asked this question on StackOverflow and Jim Driscoll from the jsf team responded saying that JSF 1.2 should work as of build 13 if you set the context parameter com.sun.faces.enableMultiThreadedStartup to false in webl.xml. See here for his full response. http://stackoverflow.com/questions/732565/can-i-use-facelets-in-a-google-app-engine-app Puche: I'm interested in how you got around the jndi problem in JSF 2. I ended up having to recompile jsf 2.0.2 with a small change to disable the jndi startup stuff. I blogged about it here: http://digitaljoel.wordpress.com/2009/12/21/jsf-2-0-2-and-google-app-engine/ http://digitaljoel.wordpress.com/2009/12/21/jsf-2-0-2-and-google-app-engine/I don't think you see the problem if you are using an older SDK, but I'm not sure. I would be even more interested if you plan to move to EL 2. I couldn't get that to work for the life of me and ended up going back to 1.x. Thanks, Joel On Thu, Feb 25, 2010 at 10:22 AM, Puche pucher...@gmail.com wrote: I think so. Read this: http://javawords.com/2009/06/05/using-jsf-12-with-facelets-on-google-app-engine-for-java/ I'm though, trying to run JSF 2.0 on GAE. I'm almost done, however I'm having some issue as I describe here: http://groups.google.com/group/google-appengine-java/browse_frm/thread/d4d2c334818591ce/2a488383221e618d?hl=en#2a488383221e618d My source of info has been these: http://in.relation.to/14203.lace http://java.wildstartech.com/Java-Platform-Enterprise-Edition/JavaServer-Faces/sun-javaserver-faces-reference-implementation/configuring-jsf-20-to-run-on-the-google-appengine Please, if you go any further, let me know! Thanks in advance. On Feb 25, 2:06 pm, Amr amrdarwish1...@gmail.com wrote: Does GAE for Java Support JSF 1.1 or JSF 1.2? (A sample project is highly appreciated) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?
Jeff, point taken, but the original poster has been asking for three different requirements: - requirement to do all writes synchronously - sub-some-couple-hundred-millisecond writes - 12k entities being written This just won't scale well if it's common. Messaging users can be done asynchronously, as can the portion crediting friends. I understand the argument that you may want to do this during the lifecycle of the request so the original user gets some kind of feedback backed by a strongly consistent datastore, but this just isn't done. Feedback is usually faked out optimistically, assuming that the writes will all be successful with some cached layer being the only part of the stack being updated inside the request. Thinking of worse case scenarios is a good thought exercise, but it's also a bit too optimistic to design a product assuming all of a Users' friends will play the game and engineer to meet that unrealistic expectation. What are the standard and slightly non-standard use cases? I'd probably look at a solution where I can store the data somewhere associated with the original user for any users not already in the datastore, then retrieve and generate a score for any of that User's friends on first access. Facebook's developer admin tool has some pretty good statistics such as bounce rate, block rate and invitation accept rate that can be used to tune this design. Slightly off topic, but we've been asked before if it was possible to provide different levels of datastore consistency. In some cases I can see the tradeoffs making sense. On Wed, Feb 24, 2010 at 5:52 PM, Jeff Schnitzer j...@infohazard.org wrote: On Wed, Feb 24, 2010 at 1:06 PM, Ikai L (Google) ika...@google.com wrote: My point wasn't necessarily that it wasn't possible. makePersistentAll does use a batch write, and there are definitely sites that can do 12,000+ writes a second (and well above that), but I don't know of any that will attempt to do that in a single request. While it's an interesting thought exercise to see if BigTable can do it through App Engine's interface (hint: it can, globally, easily), I can't think of a single use case for a site to need to do this all the time and with the sub-second requirement. I think it's reasonable to ask why this design exists and why the requirements exist and rethink one or the other. It does seem to be a pretty extreme case, but it's not all that far fetched. It's possible for a Facebook user to have 5,000 friends. Perhaps a user wants to message all 5k of them. I could actually use this right ability now. I would like to add a game mechanic which, when you score some points, you also credit a portion of that to all of a user's friends. Worst case scenario is a 5,000 element read followed by a 5,000 element write. I'm probably going to skip this mechanic for now because I can't afford it - even with the average 200 or so friends. If I want it badly enough, I may ultimately need to move my scoring system offsite. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How expensive is a memcache instance?
Great answer. Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] URLFetchService example
I am trying to get a simple URLFetchService example to work, here is the code. URL url = new URL(http://www.google.com;); URLFetchService urlFetchService = URLFetchServiceFactory.getURLFetchService(); HTTPRequest httpRequest = new HTTPRequest(url,HTTPMethod.GET); HTTPResponse resp = urlFetchService.fetch(httpRequest); System.out.println(resp.toString()); Getting a nullpointer in the fetch method java.lang.NullPointerException at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java: 31) What am I missing? Please help.. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] RESTful web services on Google App Engine
Is it possible for me to create three RESTful web services in NetBeans, bundle them up, host on the cloud, and make communication between them? If so, Tell me how? This is for a project work -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] datanucleus enhancer keeps running
i'm using eclipse 3.5.1, GPE 3.5 (1.2.0.v200912062003), GAE 1.3.1.v201002101412, GWT 2.0.3.v201002191036 datanucleusenhancer continuously runs over and over again temporary workaround: turn off build automatically feature in eclipse -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: datanucleus enhancer keeps running
On Feb 25, 6:39 am, haole mejoe...@gmail.com wrote:
i'm using eclipse 3.5.1, GPE 3.5 (1.2.0.v200912062003), GAE
1.3.1.v201002101412, GWT 2.0.3.v201002191036
datanucleusenhancer continuously runs over and over again
temporary workaround: turn off build automatically feature in eclipse
I have several projects with a dependency hierarchy. I found the
enhancer to be fragile and it often would not enhance the classes. I
had to do a 'clean' to make it happen. I could not find the pattern to
when it failed to enhance. It never looped on me so you may be seeing
a different problem.
About 3 weeks ago I had had enough and turned off the Enhancer
builder (the plugin yelled at me). In its place I added an ant task
to do the enhancement. I derived the ant task statements from the
Datanucleus site. Since that time things have been very stable and
the enhancer always finishes its job.
This is the essence of the ant task:
!--
--
!-- T A S K D E F
S --
!--
--
taskdef name=enhance
classname=com.google.appengine.tools.enhancer.EnhancerTask
classpath
pathelement path=${appengine.tools.classpath}/
pathelement path=${appengine.tools.lib}/
/classpath
/taskdef
!--
--
!-- T A R G E T
S --
!--
--
target name=enhance.cls
description=Performs Datnucleus JDO enhancement on compiled
data classes.
!-- echo message= path ${appengine.tools.classpath}/ --
!-- echo message= fileset ${proj.dir.gen.cls}/ --
enhance failonerror=true verbose=false
classpath
pathelement path=${appengine.tools.classpath}/
fileset dir=${proj.dir.war.lib} includes=*.jar/
pathelement path=${proj.dir.gen.cls}/
/classpath
fileset dir=${proj.dir.gen.cls} includes=**/*.class/
/enhance
/target
target name=enhance.war
description=Performs Datnucleus JDO enhancement on war
classes.
!-- echo message= path ${appengine.tools.classpath}/ --
!-- echo message= fileset ${proj.dir.gen.cls}/ --
enhance failonerror=true verbose=false
classpath
pathelement path=${appengine.tools.classpath}/
fileset dir=${proj.dir.war.lib} includes=*.jar/
pathelement path=${proj.dir.war.cls}/
/classpath
fileset dir=${proj.dir.war.cls} includes=**/*.class/
/enhance
/target
It may be worth a try.
Steve
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?
I don't think the original poster had a requirement for synchronous writes; he just didn't want to do the writes asynchronously because it involved a lot more code. I'm also perfectly fine with asynchronous writes and a very lax interpretation of consistency. I don't even mind writing extra code. The thing I worry about is the feasibility of a heavy write load and the total cost of it. Unfortunately I really can't describe in detail what I want to do (I normally laugh at this kind of secrecy, but in this case it's warranted). For the game mechanic I'm thinking about, the average-case scenario is not very far from the worst-case scenario. Just a little detail: * There is no requirement that all of a user's friends must be playing the game or even have installed the app to receive points. Welcome to the world of social gaming, you can play without even without knowing it! * There are *lots* of FB users that have 1k friends. Probably millions. More active FB users are likely to have more friends... and more likely to use my app. * Points can be assigned to multiple layers, so the # of updates is (layers * friends). * Tens of thousands of people play this game. It could become hundreds of thousands very soon. If I'm lucky, millions. I would love to implement this game mechanic, but I just can't. Asynchronous or not, it's *way* too expensive on appengine. When it comes time to implement this feature (and it's going to come, I can see the winds blowing), I'm probably going to have to move my scoring system out of appengine. Which is a bit ironic, because one of the main advantages of appengine is scalability. I would *love* to see some sort of super-super-lax and super-super-cheap consistency option for BigTable. Or even an alternative key/value datastore that simply works like a persistent version of memcached. Something that would let me sustain 10k writes/sec without bankrupting me. Jeff On Thu, Feb 25, 2010 at 11:16 AM, Ikai L (Google) ika...@google.com wrote: Jeff, point taken, but the original poster has been asking for three different requirements: - requirement to do all writes synchronously - sub-some-couple-hundred-millisecond writes - 12k entities being written This just won't scale well if it's common. Messaging users can be done asynchronously, as can the portion crediting friends. I understand the argument that you may want to do this during the lifecycle of the request so the original user gets some kind of feedback backed by a strongly consistent datastore, but this just isn't done. Feedback is usually faked out optimistically, assuming that the writes will all be successful with some cached layer being the only part of the stack being updated inside the request. Thinking of worse case scenarios is a good thought exercise, but it's also a bit too optimistic to design a product assuming all of a Users' friends will play the game and engineer to meet that unrealistic expectation. What are the standard and slightly non-standard use cases? I'd probably look at a solution where I can store the data somewhere associated with the original user for any users not already in the datastore, then retrieve and generate a score for any of that User's friends on first access. Facebook's developer admin tool has some pretty good statistics such as bounce rate, block rate and invitation accept rate that can be used to tune this design. Slightly off topic, but we've been asked before if it was possible to provide different levels of datastore consistency. In some cases I can see the tradeoffs making sense. On Wed, Feb 24, 2010 at 5:52 PM, Jeff Schnitzer j...@infohazard.org wrote: On Wed, Feb 24, 2010 at 1:06 PM, Ikai L (Google) ika...@google.com wrote: My point wasn't necessarily that it wasn't possible. makePersistentAll does use a batch write, and there are definitely sites that can do 12,000+ writes a second (and well above that), but I don't know of any that will attempt to do that in a single request. While it's an interesting thought exercise to see if BigTable can do it through App Engine's interface (hint: it can, globally, easily), I can't think of a single use case for a site to need to do this all the time and with the sub-second requirement. I think it's reasonable to ask why this design exists and why the requirements exist and rethink one or the other. It does seem to be a pretty extreme case, but it's not all that far fetched. It's possible for a Facebook user to have 5,000 friends. Perhaps a user wants to message all 5k of them. I could actually use this right ability now. I would like to add a game mechanic which, when you score some points, you also credit a portion of that to all of a user's friends. Worst case scenario is a 5,000 element read followed by a 5,000 element write. I'm probably going to skip this mechanic for now because I can't afford it - even with the average 200 or so friends. If I
[appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?
I think there is a way to grab big chunks of oprations, put them in a queue to be done asynchronously and that would be it. My take would be that using proxies it would be easy to queue any long operation transparently. I've done that with EJBs in the past, I don't see the reason a QueingProxy couldn't be written. Cheers, Guillermo. On 25 feb, 17:02, Jeff Schnitzer j...@infohazard.org wrote: I don't think the original poster had a requirement for synchronous writes; he just didn't want to do the writes asynchronously because it involved a lot more code. I'm also perfectly fine with asynchronous writes and a very lax interpretation of consistency. I don't even mind writing extra code. The thing I worry about is the feasibility of a heavy write load and the total cost of it. Unfortunately I really can't describe in detail what I want to do (I normally laugh at this kind of secrecy, but in this case it's warranted). For the game mechanic I'm thinking about, the average-case scenario is not very far from the worst-case scenario. Just a little detail: * There is no requirement that all of a user's friends must be playing the game or even have installed the app to receive points. Welcome to the world of social gaming, you can play without even without knowing it! * There are *lots* of FB users that have 1k friends. Probably millions. More active FB users are likely to have more friends... and more likely to use my app. * Points can be assigned to multiple layers, so the # of updates is (layers * friends). * Tens of thousands of people play this game. It could become hundreds of thousands very soon. If I'm lucky, millions. I would love to implement this game mechanic, but I just can't. Asynchronous or not, it's *way* too expensive on appengine. When it comes time to implement this feature (and it's going to come, I can see the winds blowing), I'm probably going to have to move my scoring system out of appengine. Which is a bit ironic, because one of the main advantages of appengine is scalability. I would *love* to see some sort of super-super-lax and super-super-cheap consistency option for BigTable. Or even an alternative key/value datastore that simply works like a persistent version of memcached. Something that would let me sustain 10k writes/sec without bankrupting me. Jeff On Thu, Feb 25, 2010 at 11:16 AM, Ikai L (Google) ika...@google.com wrote: Jeff, point taken, but the original poster has been asking for three different requirements: - requirement to do all writes synchronously - sub-some-couple-hundred-millisecond writes - 12k entities being written This just won't scale well if it's common. Messaging users can be done asynchronously, as can the portion crediting friends. I understand the argument that you may want to do this during the lifecycle of the request so the original user gets some kind of feedback backed by a strongly consistent datastore, but this just isn't done. Feedback is usually faked out optimistically, assuming that the writes will all be successful with some cached layer being the only part of the stack being updated inside the request. Thinking of worse case scenarios is a good thought exercise, but it's also a bit too optimistic to design a product assuming all of a Users' friends will play the game and engineer to meet that unrealistic expectation. What are the standard and slightly non-standard use cases? I'd probably look at a solution where I can store the data somewhere associated with the original user for any users not already in the datastore, then retrieve and generate a score for any of that User's friends on first access. Facebook's developer admin tool has some pretty good statistics such as bounce rate, block rate and invitation accept rate that can be used to tune this design. Slightly off topic, but we've been asked before if it was possible to provide different levels of datastore consistency. In some cases I can see the tradeoffs making sense. On Wed, Feb 24, 2010 at 5:52 PM, Jeff Schnitzer j...@infohazard.org wrote: On Wed, Feb 24, 2010 at 1:06 PM, Ikai L (Google) ika...@google.com wrote: My point wasn't necessarily that it wasn't possible. makePersistentAll does use a batch write, and there are definitely sites that can do 12,000+ writes a second (and well above that), but I don't know of any that will attempt to do that in a single request. While it's an interesting thought exercise to see if BigTable can do it through App Engine's interface (hint: it can, globally, easily), I can't think of a single use case for a site to need to do this all the time and with the sub-second requirement. I think it's reasonable to ask why this design exists and why the requirements exist and rethink one or the other. It does seem to be a pretty extreme case, but it's not all that far fetched. It's possible
Re: [appengine-java] Difference between AJAX search and Google.com
You mentioned reading forums - have you checked this forum? http://groups.google.com/group/Google-AJAX-Search-API http://groups.google.com/group/Google-AJAX-Search-APIThis is a group for App Engine related questions. On Thu, Feb 25, 2010 at 4:26 AM, Peter Hulsen hul...@gmail.com wrote: Hi, When I search via my custom search at http://www.google.com/cse/home?cx=partner-pub-8067732287343894:aiqtt2-g1ck (e.g., mayer) with site restriction www.peert.com, I get less results then when searching via http://www.google.com (e.g., site:peert.com mayer). The same happens when using the AJAX API for Web Search. I read similar posts in the help forums and this group but did not found a satisfying answer. How can this happen? Thanks. Regards, Peter Hulsen -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?
Hi I wrote importing geoip database in JSF. It has like 100k entries. Besides appengine and datastore performance is awful and importing took around 1,5h of total CPU its fairly easy to use TaskQueue. If there is above average processing to be done then i suggest separating task into smaller batches and process them asynchronously. Margus On Thu, Feb 25, 2010 at 3:52 AM, Jeff Schnitzer j...@infohazard.org wrote: On Wed, Feb 24, 2010 at 1:06 PM, Ikai L (Google) ika...@google.com wrote: My point wasn't necessarily that it wasn't possible. makePersistentAll does use a batch write, and there are definitely sites that can do 12,000+ writes a second (and well above that), but I don't know of any that will attempt to do that in a single request. While it's an interesting thought exercise to see if BigTable can do it through App Engine's interface (hint: it can, globally, easily), I can't think of a single use case for a site to need to do this all the time and with the sub-second requirement. I think it's reasonable to ask why this design exists and why the requirements exist and rethink one or the other. It does seem to be a pretty extreme case, but it's not all that far fetched. It's possible for a Facebook user to have 5,000 friends. Perhaps a user wants to message all 5k of them. I could actually use this right ability now. I would like to add a game mechanic which, when you score some points, you also credit a portion of that to all of a user's friends. Worst case scenario is a 5,000 element read followed by a 5,000 element write. I'm probably going to skip this mechanic for now because I can't afford it - even with the average 200 or so friends. If I want it badly enough, I may ultimately need to move my scoring system offsite. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Serialized field is ignored in JDO
Two classes are defined:
--
@PersistenceCapable(identityType=IdentityType.DATASTORE,
detachable=true)
public class TestSerialized {
@Persistent
private String nonSerialized;
@PrimaryKey() @Persistent(valueStrategy=IdGeneratorStrategy.IDENTITY)
private Key primaryKey;
@Persistent @Serialized
private SerializedObj serializedObj;
}
public class SerializedObj implements Serializable{
/**
*
*/
private static final long serialVersionUID = 6258869487732411716L;
private String value;
public SerializedObj() {
}
public SerializedObj(String value) {
this.value = value;
}
}
--
When a TestSerialized is persisted like below, the nonSerialized
field works fine but the serializedObj field is not saved to the
datastore, and looking into the development console, the field/column
is not even created:
PersistenceManager pm = PMF.get().getPersistenceManager();
try{
pm.currentTransaction().begin();
TestSerialized draft = new TestSerialized();
draft.setSerializedObj(new SerializedObj(dd));
pm.makePersistent(draft);
pm.currentTransaction().commit();
}finally{
if(pm.currentTransaction().isActive()){
pm.currentTransaction().rollback();
}
pm.close();
}
@Persistent(serialized=true) didn't help either.
Am I missing something?
I am using SDK 1.3.1 .
Thanks.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?
Guillermo, Taskqueue items can only be 10K (http://code.google.com/appengine/docs/java/taskqueue/overview.html#Quotas_and_Limits). The basic idea is that if you have more data than that you put it into an entity (in the data-store) and have the task pull it out and process it. It might be that you can persist those 12K entities in a lot of large entities (that are still under 1mByte each), but that is a lot of work for something that will still probably fail. I guess it all depends where the cost is on the puts (indexing, raw writes by bytes, number of items). And if your comeback is memcache, well, I won't even start discussing using a non-persistent, volatile, store like that for temporary storage while you write them to the datastore... in batches using the taskqueue/cron/etc. Really, there needs to be something that can handle the write volume. On Thu, Feb 25, 2010 at 12:08 PM, Guillermo Schwarz guillermo.schw...@gmail.com wrote: I think there is a way to grab big chunks of oprations, put them in a queue to be done asynchronously and that would be it. My take would be that using proxies it would be easy to queue any long operation transparently. I've done that with EJBs in the past, I don't see the reason a QueingProxy couldn't be written. Cheers, Guillermo. On 25 feb, 17:02, Jeff Schnitzer j...@infohazard.org wrote: I don't think the original poster had a requirement for synchronous writes; he just didn't want to do the writes asynchronously because it involved a lot more code. I'm also perfectly fine with asynchronous writes and a very lax interpretation of consistency. I don't even mind writing extra code. The thing I worry about is the feasibility of a heavy write load and the total cost of it. Unfortunately I really can't describe in detail what I want to do (I normally laugh at this kind of secrecy, but in this case it's warranted). For the game mechanic I'm thinking about, the average-case scenario is not very far from the worst-case scenario. Just a little detail: * There is no requirement that all of a user's friends must be playing the game or even have installed the app to receive points. Welcome to the world of social gaming, you can play without even without knowing it! * There are *lots* of FB users that have 1k friends. Probably millions. More active FB users are likely to have more friends... and more likely to use my app. * Points can be assigned to multiple layers, so the # of updates is (layers * friends). * Tens of thousands of people play this game. It could become hundreds of thousands very soon. If I'm lucky, millions. I would love to implement this game mechanic, but I just can't. Asynchronous or not, it's *way* too expensive on appengine. When it comes time to implement this feature (and it's going to come, I can see the winds blowing), I'm probably going to have to move my scoring system out of appengine. Which is a bit ironic, because one of the main advantages of appengine is scalability. I would *love* to see some sort of super-super-lax and super-super-cheap consistency option for BigTable. Or even an alternative key/value datastore that simply works like a persistent version of memcached. Something that would let me sustain 10k writes/sec without bankrupting me. Jeff On Thu, Feb 25, 2010 at 11:16 AM, Ikai L (Google) ika...@google.com wrote: Jeff, point taken, but the original poster has been asking for three different requirements: - requirement to do all writes synchronously - sub-some-couple-hundred-millisecond writes - 12k entities being written This just won't scale well if it's common. Messaging users can be done asynchronously, as can the portion crediting friends. I understand the argument that you may want to do this during the lifecycle of the request so the original user gets some kind of feedback backed by a strongly consistent datastore, but this just isn't done. Feedback is usually faked out optimistically, assuming that the writes will all be successful with some cached layer being the only part of the stack being updated inside the request. Thinking of worse case scenarios is a good thought exercise, but it's also a bit too optimistic to design a product assuming all of a Users' friends will play the game and engineer to meet that unrealistic expectation. What are the standard and slightly non-standard use cases? I'd probably look at a solution where I can store the data somewhere associated with the original user for any users not already in the datastore, then retrieve and generate a score for any of that User's friends on first access. Facebook's developer admin tool has some pretty good statistics such as bounce rate, block rate and invitation accept rate that can be used to tune this design. Slightly off topic, but we've been asked before if it was possible to provide different levels of datastore consistency. In some cases
Re: [appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?
We have an issue for an asynchronous write API for the datastore: http://code.google.com/p/googleappengine/issues/detail?id=2817 This is something that can fit into that model. On Thu, Feb 25, 2010 at 12:26 PM, Scott Hernandez scotthernan...@gmail.comwrote: Guillermo, Taskqueue items can only be 10K ( http://code.google.com/appengine/docs/java/taskqueue/overview.html#Quotas_and_Limits ). The basic idea is that if you have more data than that you put it into an entity (in the data-store) and have the task pull it out and process it. It might be that you can persist those 12K entities in a lot of large entities (that are still under 1mByte each), but that is a lot of work for something that will still probably fail. I guess it all depends where the cost is on the puts (indexing, raw writes by bytes, number of items). And if your comeback is memcache, well, I won't even start discussing using a non-persistent, volatile, store like that for temporary storage while you write them to the datastore... in batches using the taskqueue/cron/etc. Really, there needs to be something that can handle the write volume. On Thu, Feb 25, 2010 at 12:08 PM, Guillermo Schwarz guillermo.schw...@gmail.com wrote: I think there is a way to grab big chunks of oprations, put them in a queue to be done asynchronously and that would be it. My take would be that using proxies it would be easy to queue any long operation transparently. I've done that with EJBs in the past, I don't see the reason a QueingProxy couldn't be written. Cheers, Guillermo. On 25 feb, 17:02, Jeff Schnitzer j...@infohazard.org wrote: I don't think the original poster had a requirement for synchronous writes; he just didn't want to do the writes asynchronously because it involved a lot more code. I'm also perfectly fine with asynchronous writes and a very lax interpretation of consistency. I don't even mind writing extra code. The thing I worry about is the feasibility of a heavy write load and the total cost of it. Unfortunately I really can't describe in detail what I want to do (I normally laugh at this kind of secrecy, but in this case it's warranted). For the game mechanic I'm thinking about, the average-case scenario is not very far from the worst-case scenario. Just a little detail: * There is no requirement that all of a user's friends must be playing the game or even have installed the app to receive points. Welcome to the world of social gaming, you can play without even without knowing it! * There are *lots* of FB users that have 1k friends. Probably millions. More active FB users are likely to have more friends... and more likely to use my app. * Points can be assigned to multiple layers, so the # of updates is (layers * friends). * Tens of thousands of people play this game. It could become hundreds of thousands very soon. If I'm lucky, millions. I would love to implement this game mechanic, but I just can't. Asynchronous or not, it's *way* too expensive on appengine. When it comes time to implement this feature (and it's going to come, I can see the winds blowing), I'm probably going to have to move my scoring system out of appengine. Which is a bit ironic, because one of the main advantages of appengine is scalability. I would *love* to see some sort of super-super-lax and super-super-cheap consistency option for BigTable. Or even an alternative key/value datastore that simply works like a persistent version of memcached. Something that would let me sustain 10k writes/sec without bankrupting me. Jeff On Thu, Feb 25, 2010 at 11:16 AM, Ikai L (Google) ika...@google.com wrote: Jeff, point taken, but the original poster has been asking for three different requirements: - requirement to do all writes synchronously - sub-some-couple-hundred-millisecond writes - 12k entities being written This just won't scale well if it's common. Messaging users can be done asynchronously, as can the portion crediting friends. I understand the argument that you may want to do this during the lifecycle of the request so the original user gets some kind of feedback backed by a strongly consistent datastore, but this just isn't done. Feedback is usually faked out optimistically, assuming that the writes will all be successful with some cached layer being the only part of the stack being updated inside the request. Thinking of worse case scenarios is a good thought exercise, but it's also a bit too optimistic to design a product assuming all of a Users' friends will play the game and engineer to meet that unrealistic expectation. What are the standard and slightly non-standard use cases? I'd probably look at a solution where I can store the data somewhere associated with the original user for any users not already in the datastore, then retrieve and generate a
[appengine-java] Re: Billing currently disabled on all App Engine applications
As of 11am February 25th, billing is now re-enabled. On Feb 24, 11:30 pm, App Engine Team appengine.nore...@gmail.com wrote: At 3pm February 24th, we have temporarily disabled billing on all App Engine applications in the wake of the unexpected morning outage. While billing is disabled, your application will continue to use up to your existing budget values, but we will not issue charges as a result of that usage. We will also not be charging for resource usage before and during the outage on the 24th. We will update this thread once billing has been re-enabled. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Passing parameters in a Task
I wanted to use Queue tasks on my GAE application. I need to pass two parameters to my URL servlet /runTasks. So i am doing this. But when I print the values for start and end, it prints null for start and 15 for end. Am i doing anything wrong? Is this the right way to pass the parameters? TaskOptions to = null; to = TaskOptions.Builder.param(start, 0); to = TaskOptions.Builder.param(end, 15); to = to.url(/runTasks); queue.add(to); Thanks for the help -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Possible to cause instance recycling?
Hey all, I know that instance recycling times are not ideal - that's not my problem here. I'm wondering if it is possible to unintentionally cause GAE to drop your instance and restart it? I'm using the Wicket framework. Everything works locally, but when I deployed today, I experienced 5 instance restarts in the span of three minutes - each restart is 15-20 seconds. I'm not seeing any exceptions in the logs. My quotas, load, etc are all fine (practically non-existent). Application behavior is perfect - just with delays. The only strenuous thing I'm doing is loading/processing a 350K XML document into a Java object when the instance starts. AppID: http://jacob-brookover.appspot.com Look at the Library and Start Reading the first two pages of Call of the Wild. This project is a port of an existing project that has gone through pretty rigorous testing, usage. Thanks! Jake -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Passing parameters in a Task
Perfect , thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: uniqueness control ideas
Just wanted to say that I see that what I'm trying to do is just
fundamentally wrong and will decidedly cause serious problems in the
longer term. I suppose it is best to check for uniqueness in the
service layer.
On Feb 24, 12:05 pm, atomi at...@verizon.net wrote:
I've been trying to devise a way to maintain uniqueness among different
entities by storing some kind of UUID based on the fields I want unique for
that entity Using JDO, and a GenericDAO (groovy)
Here is the method for put() in my GenericDAO:
def put(Object object) {
Collection objects = list();
Boolean exists = false;
objects.each {if(it.getUniqueKey() == object.getUniqueKey()) {exists =
true}}
if(!exists) {
PersistenceManager pm = getPersistenceManager();
Transaction tx = pm.currentTransaction();
try {
tx.begin();
pm.makePersistent(object);
tx.commit();} finally {
if (tx.isActive()) {
tx.rollback();}
pm.close();
}
}
}
Where the uniqueKey field is a *unique string based on fields I need to be
unique*.
My question is what is good way to create this unique key string using for
example multiple string fields that will be consistent? My first inclination
was to use an MD5 but I see that datanucleus provides value generators that
might work better? Ideas? Or am I just going about this the wrong way?
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Task queue not starting the action when expected
Hello, We have been using task queues for a few months without problem. We handle actions that must execute right away. At the time of the release of 1.3.1, we noticed that sometimes, some tasks will take 1 to 2 minutes to start, but in the task queue, the ETA shows as a date in the past (like 0:01:12 ago), usually 1 second or so after the creation time. This is not a consistent issue. I might works perfectly (task being kicked in within 1 second of creation) for several times and suddenly, the next task takes 1.5 minutes to get started. Our task queue is empty when we see this issue. Our queue is setup with a max rate of 5/s and a bucket size of 1. Any idea of what we can do to work around or solve this new issue? Jerome -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Cloud2db (Universal database for cloud computing)
Dear member: Cloud2db http://www.cloud2db.com/ is pleased to announce the availability of Cloud2db Server which provides a standards-based abstraction layer over Google Datastore (Bigtable). This product provides you with performance and scalability of GAE along with structure, standards and interoperability of RDBMS, SQL and JDBC. With Cloud2db, you will be able to manage your data on Google Datastore by using established concepts of RDBMS, SQL and JDBC, and thereby preserving your existing investments in tools, technologies, frameworks and skills. Here are Cloud2db features: - Manage data on Google Datastore using familiar concepts of RDBMD, SQL - Referential integrity (Primary Keys, Foreign Keys) - Role Based Security - Joins (Inner Join, Left Outer Join, Theta Join, Cross Join) - Subqueries (Exists, Not Exists, In) - DDL DML - Transactions - ANSI SQL stored functions - Views - BLOB and CLOB support - Use any JDBC compliant tools and frameworks in the market to interact with the Google Datastore. For example. - Squirrel SQL (database management) - Power Architect (data modeling) - Jasper Reports (reporting) - Hibernate (Object To Relational mapping) - All JDBC compliant tools - Port exiting RDBMS instances to Google Datastore Please visit our website http://www.cloud2db.com and try out the free trial version of our product by registering at http://cloud2db.appspot.com/website/registration.html. Please visit google group Cloud2db http://groups.google.com/group/cloud2db to share your feedback and thoughts. Sincerely, Sandeep Sathaye Founder and Chief Architect Cloud2db -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] dumb question: does the development console list Text?
I'm probably doing something wrong, but I don't see the Text fields of
persisted objects when listing the entities from the Development Console.
E.g., using JDO, I have Text field foo, as declared below:
@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable =
true)
public class LocationData implements Serializable {
...
@Persistent(defaultFetchGroup = true)
Text foo;
...
Should I expect the viewer to display the contents of foo, or should I
assume that the viewer simply does not list it since it may be (and actually
is) quite long (500 char)?
Thanks.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Date Range Query
ListProperty is simply a collection. You can simply persist a list of GAE primitive type into datastore On Feb 25, 9:07 pm, Steve Pritchard steve...@gmail.com wrote: This post is very helpful found athttp://appengine-cookbook.appspot.com/recipe/how-to-query-by-date-range Often, you need to store, and retrieve, data that needs to match an effective date period, containing a start date and end date. ... You can handle your query if you add another field in your Kind, a ListProperty containing both the start date and end date. For example, add a ListProperty named date_list, and populate it like this: date_list = [start_date, end_date]. My Question. In Java, what constitutes a ListProperty. I assume ArrayList. I am hoping that an int[] also counts because this has much less overhead to store. Does anybody know or know where the rules for ListProperty might be found. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Date Range Query
Be careful of the start-end date query trick - it does not work. There was a bug in the SDK that made it appear to work. an entity will only match the query if it has an individual value for that property that matches all of the inequality filters http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html You can store any Collection (but not array) and it will become a multi valued property. All collection types are returned as an ArrayList - even sets etc. JD On 26 Feb 2010, at 13:38, Max wrote: ListProperty is simply a collection. You can simply persist a list of GAE primitive type into datastore On Feb 25, 9:07 pm, Steve Pritchard steve...@gmail.com wrote: This post is very helpful found athttp://appengine-cookbook.appspot.com/recipe/how-to-query-by-date-range Often, you need to store, and retrieve, data that needs to match an effective date period, containing a start date and end date. ... You can handle your query if you add another field in your Kind, a ListProperty containing both the start date and end date. For example, add a ListProperty named date_list, and populate it like this: date_list = [start_date, end_date]. My Question. In Java, what constitutes a ListProperty. I assume ArrayList. I am hoping that an int[] also counts because this has much less overhead to store. Does anybody know or know where the rules for ListProperty might be found. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
