[appengine-java] Re: Mail Service - get User which belongs to email address
hi Lars Would this help you/ http://code.google.com/appengine/docs/java/users/ On Jul 26, 10:19 pm, vogella wrote: > Hi, > > I have a small Todo application written. I would like to use the Mail > service to receive new todos items via email. > > I can get the sender of the email via getFrom() and would like to map > this to the user in my application and create a new Todo for this > user. > > Is it possible to retrieve the user based on an email from AppEngine? > If yes it would be great if you can give me a pointer where to look. > > Best regards, Lars > > --- > I originally posted this message > tohttp://groups.google.com/group/google-appengine > before realizing that there is a Java specific group. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Date, Calendar question (multiple instances of web application behaviour) clarification
Hello, I am sorry for my late answer. I am trying to implement kind of authorization system to access certain service. This service is only available for example X times per Y seconds (x = 5, y = 60 * 15) each request to this service is identified by uniqueID so I know who is asking the service I use memcache to find out how many requests there were in last Y secs. If number of requests is lower or equal to X I try to put another different entry to cache which contains current date (current app instance date). Subsequential requests retrieve this date from memcache and use it to compute remaining time to next RESET. RESET is meant to be time when first cache entry expires and thus reset number of attemps to request my service. (more simple) I put some date to memcache for example (currentDate + 15 mins), next time I retriev this date, but it can be different app instance than which made the entry. If I do some computation like (dateFromMemcache.toMilisecs - currentAppInstanceDate.toMilisecs) this may vary for app instance. If different for different instace is only few milisecs or secs, it's ok. But if it is more than 1 min or even more that would be problem. Any simple idea how to cope with this ? > On Jul 22, 3:04 am, "Ikai L (Google)" wrote: > It should just be whatever the date is on the current instance. In general, > do not ever rely on dates to be synchronized. Clock skew is a reality of > distributed computing, and you'll have to work around it. What exactly is > the problem you're trying to solve? > > > > On Sun, Jul 18, 2010 at 11:23 PM, Marcus Brody wrote: > > Good day, > > > I would like to ask about following. > > > How is java.util.Date synchronized in case there is more than 1 > > instance of my web application. > > Example: > > > WebAppInstance01 : I create somewhere new Date() > > > at the same time > > WebAppInstance02: I create somewhere new Date() > > > Is there some time shift ? Or those both app request some "central > > authority" ? > > > Thank you, > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog:http://googleappengine.blogspot.com > Twitter:http://twitter.com/app_engine > Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Dance Dance Robot error / Channel API
Thanks, Don. It works like a charm after I include below line in html. On Jul 26, 11:03 pm, Don Schwarz wrote: > Yes, this feature isn't launched in the production server yet. > > You can try it out in the 1.3.5 DevAppServer, but it requires a slightly > workaround: > > Extract the "apphosting/tools/dev-channel-js.js" file from > appengine-api-1.0-sdk-1.3.5.jar and move it to /_ah/channel/jsapi in your > war directory (as if it were a static file). > > Your HTML should then use
[appengine-java] Mail Service - get User which belongs to email address
Hi, I have a small Todo application written. I would like to use the Mail service to receive new todos items via email. I can get the sender of the email via getFrom() and would like to map this to the user in my application and create a new Todo for this user. Is it possible to retrieve the user based on an email from AppEngine? If yes it would be great if you can give me a pointer where to look. Best regards, Lars --- I originally posted this message to http://groups.google.com/group/google-appengine before realizing that there is a Java specific group. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Instance startup/shutdown and sessions
Thanks for the update Jake. On Jul 26, 8:24 pm, Jake wrote: > Hey, > > Yes, sessions are stored in the datastore, make use of the memcache, > and behave as expected regardless of which instances are serving your > user. However, you still face the same 1MB limit and this system, > obviously, only applies to values saved in the session; any class or > application variables are not maintained. > > Jake > > On Jul 26, 7:02 am, Marcel Overdijk wrote: > > > > > So GAE makes sure the preserve session data between shutting down/ > > starting up instances. Nice. > > > On Jul 26, 12:44 pm, Shawn Brown wrote: > > > > > What happens if between logged in user navigates to another page and > > > > the GAE instance was shutdown? > > > > I understand a new instance is started but what happened to the > > > > session data? > > > > AFAIK session data is persisted to big table. > > > > > Also, how is startup/shutdown of instances related to session-timeout > > > > in web.xml? > > > > > > > > > 30 > > > > > > > > AFAIK there is no connection. An instance will not be kept warm > > > because there is an open session. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
On 27 Jul 2010, at 00:24, Bill wrote: This would completely disregard my business model. A Domain is not an Element, nor vice-versa. It's something analogous to making both classes Elephant and Corn implement the same interface because both have Ears. Ha ha well you would not actually need to have Domain implement Element to set up the same parent hierarchy. In fact a parent can be any Key - even one that does not exist. Not sure about JDO, but in Twig you can set the parent when you store the Element like this: datasore.store() .instances(someElements) .parent(theDomainOrElement) .returnKeysNow(); So the parent does not actually need to be referenced by the child or visa versa. And as you say, "this limits the concurrency of changes to Elements to a few writes per second per Domain" which would be very bad in an enterprise system. It all depends on the granularity of your element groups. If you can break them down sufficiently then you can still use transactions. I would advise using either the top level Element or a lower level element as your root entity. If you must use a lower level element you would need to remove the list of child keys from a Element to stop the cascade of updates. Every child has a reference to its parent encoded in its key which can be used to efficiently return the list of children using an ancestor query. Are domains added and removed often? If not then is it sufficient to check that the domain exists before working on your elements outside of the transaction? Just never delete Domains - mark them as deleted instead. No, they're not added and removed often, and as it happens the Domain does have an "active" flag. Nonetheless, I think you're missing the point. If I have to interleave a constraint check with transaction elements inside my destructive operations, I am defeating the principle of separation of concerns. I already have to manually verify "foreign key" relationships, which is blends concerns badly enough! Sounds like you have architected your solution to do too much in a single transaction which is just not performant (or possible) with the datastore. Time to re-think how you demarcate transactions. A succinct explanation for a transaction is that it is supposed to make different operations involving multiple resources atomic to the level of isolation specified. If you can only have one resource, or one small group of very tightly coupled resources, then transactions give you almost no benefit. At this point the only thing I can trust the transaction layer to do is to provide an optimistic locking version. Highly unsatisfactory, this is! The datastore is not an RDBMS. There are certain domain models which simply will not work on the datastore. If you cannot break down you data into small independent chunks your app will not work. I appreciate your goal of not compromising your data model and architecture for the practicalities of low-level concerns - but it just won't work here. GAE makes it impossible to ignore that fact that you are writing for a distributed system. You have to front up to these problems such as sharding and distributed transactions from day one. Even load operations on a non existent entities (e.g. Domain) are included in the transaction. Yeah. Why on earth? You can read a record a billion times and the it will not "clobber" an update. The load cannot clobber - its when you depend on the result of that load to do a write that you would hit a problem. In order to check that there is no existing entity with the same key before writing an entity the load must be a part of the transaction or another request could store an entity with the same key before you committed - clobber. Let me put it this way. I'm not looking for a way to hack this into working. I've already hacked it into working by adding an extra, very hacky layer to my architecture. What I would like is some sort of explanation, and hopefully some indication that at some point this is going to be fixed. I'd even be interested in helping out with said fix. Google, can you provide me with any answers? As I pointed out in my earlier reply - there is an RDBMS offering on the road map: http://code.google.com/appengine/business/roadmap.html That sounds like the "fix" you are looking for unless you can bash your data model into a shape more suited to a distributed system. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Instance startup/shutdown and sessions
Hey, Yes, sessions are stored in the datastore, make use of the memcache, and behave as expected regardless of which instances are serving your user. However, you still face the same 1MB limit and this system, obviously, only applies to values saved in the session; any class or application variables are not maintained. Jake On Jul 26, 7:02 am, Marcel Overdijk wrote: > So GAE makes sure the preserve session data between shutting down/ > starting up instances. Nice. > > On Jul 26, 12:44 pm, Shawn Brown wrote: > > > > What happens if between logged in user navigates to another page and > > > the GAE instance was shutdown? > > > I understand a new instance is started but what happened to the > > > session data? > > > AFAIK session data is persisted to big table. > > > > Also, how is startup/shutdown of instances related to session-timeout > > > in web.xml? > > > > > > > 30 > > > > > > AFAIK there is no connection. An instance will not be kept warm > > because there is an open session. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
Hi John, > Are all Element "trees" in the same domain? Yes they are. > If so then you could make > Domain implement Element and use it as the root of an element > hierarchy. This would completely disregard my business model. A Domain is not an Element, nor vice-versa. It's something analogous to making both classes Elephant and Corn implement the same interface because both have Ears. And as you say, "this limits the concurrency of changes to Elements to a few writes per second per Domain" which would be very bad in an enterprise system. > > Do you update more than one element in a transaction? If not then an > unowned relationship between entities would be fine. Yes, I do. A parent Element knows the order of its children and must be updated accordingly. Even were it not the case, I still want operations to be transactional, because without a transaction, optimistic locking will not function. > Are domains added and removed often? If not then is it sufficient to > check that the domain exists before working on your elements outside > of the transaction? Just never delete Domains - mark them as deleted > instead. No, they're not added and removed often, and as it happens the Domain does have an "active" flag. Nonetheless, I think you're missing the point. If I have to interleave a constraint check with transaction elements inside my destructive operations, I am defeating the principle of separation of concerns. I already have to manually verify "foreign key" relationships, which is blends concerns badly enough! A succinct explanation for a transaction is that it is supposed to make different operations involving multiple resources atomic to the level of isolation specified. If you can only have one resource, or one small group of very tightly coupled resources, then transactions give you almost no benefit. At this point the only thing I can trust the transaction layer to do is to provide an optimistic locking version. Highly unsatisfactory, this is! > Even load > operations on a non existent entities (e.g. Domain) are included in > the transaction. Yeah. Why on earth? You can read a record a billion times and the it will not "clobber" an update. Let me put it this way. I'm not looking for a way to hack this into working. I've already hacked it into working by adding an extra, very hacky layer to my architecture. What I would like is some sort of explanation, and hopefully some indication that at some point this is going to be fixed. I'd even be interested in helping out with said fix. Google, can you provide me with any answers? On Jul 26, 10:04 am, John Patterson wrote: > On 26 Jul 2010, at 19:21, Bill wrote: > > > > > Good thought, but unforunately Elements already have a parent key in > > this manner. Elements are hierarchical, so that an Element's parent > > is another parent. > > > Can an entity have more than one parent? I haven't tried that... > > An Entity can only have one parent as it is encoded into its key as a > path of ancestors leading back to the root entity. > > Are all Element "trees" in the same domain? If so then you could make > Domain implement Element and use it as the root of an element > hierarchy. But this limits the concurrency of changes to Elements to > a few writes per second per Domain. > > Do you update more than one element in a transaction? If not then an > unowned relationship between entities would be fine. > > Are domains added and removed often? If not then is it sufficient to > check that the domain exists before working on your elements outside > of the transaction? Just never delete Domains - mark them as deleted > instead. > > Your original problem "Cannot operate on more than one" was caused > by trying to load an element in a different entity group. Even load > operations on a non existent entities (e.g. Domain) are included in > the transaction. That is how you can guarantee you don't clobber > existing data when you do a put(...) in a transaction. > > You can think of starting a transaction as creating a timestamp on the > root entity - any changes to any element in the entire entity group > after that time and before the commit will cause a concurrency > exception. I think of transactions on non-existing roots as making a > kind of temporary place-holder root to which the timestamp is added. > See this for more details: > > http://www.youtube.com/watch?v=tx5gdoNpcZM > > If non of those work arounds is sufficient then perhaps you need > distributed transactions. The Slim3 project implements distributed > transactions using the task queue and there was a project with the aim > of making a generic solution: > > http://www.youtube.com/watch?v=IOtBlzsx0m8 > > Not sure what happened to that though. > > It is possible that you need a full RDBMS - in which case you will > either need to patiently wait for the release of the App Engine one > recently announce
[appengine-java] Discussion on will-it-play-in-app-engine
It is misleading to suggest DWR 3.0RC1 is fully compatible with google app engine. See here: http://groups.google.com/group/google-appengine-java/browse_thread/thread/8db782c90827d825?pli=1 and here: http://old.nabble.com/DWR3RC1-threads-to23074328.html#a23074328 for more info -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Is possible a query inside an arrayList?
Sorry...i have posted multiple times same content.
My mistake.
On 26 Lug, 17:08, salvatore wrote:
> :-(
> As you can see here
>
> http://code.google.com/intl/it-IT/appengine/docs/java/javadoc/com/goo...
> Gae low level API don't support, query.setParameter(...)
> there are some other solutions?
> (tank's for the answer Tony)
>
> On 25 Lug, 16:14, Tony Qiu wrote:
>
>
>
> > Of course you can do it.
> > there is a keyword "member of" in JP-QL. you can use the JP-QL below
> > to do what you want.
>
> > Query query = new Query("select o from user o where :tag member of
> > o.tags");
> > query.setParameter("tag","abc")
>
> > On 7月25日, 下午8时28分,salvatore wrote:
>
> > > Hi i've problem
> > > How a can query a data inside an ArrayList?
> > > I use low level Datastore API
> > > the data is structured like this
> > > entity (kind 'User')
> > > -String username
> > > -String pass
> > > -ArrayList tags
> > > I want to query all the user with a specific role
> > > my code
> > > //...
> > > DatastoreService ds=
> > > DatastoreServiceFactory.getDatastoreService();
> > > Query query = new Query("User");
> > > query.addFilter("tag", Query.FilterOperator.EQUAL,
> > > "admin"); // <-HERE
> > > PreparedQuery pq = ds.prepare(query);
> > > Iterable it = pq.asIterable();
> > > //...
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Is possible a query inside an arrayList?
:-(
As you can see here
http://code.google.com/intl/it-IT/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Query.html
Gae low level API don't support, query.setParameter(...)
there are some other solutions?
(tank's for the answer Tony)
On 25 Lug, 16:14, Tony Qiu wrote:
> Of course you can do it.
> there is a keyword "member of" in JP-QL. you can use the JP-QL below
> to do what you want.
>
> Query query = new Query("select o from user o where :tag member of
> o.tags");
> query.setParameter("tag","abc")
>
> On 7月25日, 下午8时28分,salvatore wrote:
>
> > Hi i've problem
> > How a can query a data inside an ArrayList?
> > I use low level Datastore API
> > the data is structured like this
> > entity (kind 'User')
> > -String username
> > -String pass
> > -ArrayList tags
> > I want to query all the user with a specific role
> > my code
> > //...
> > DatastoreService ds=
> > DatastoreServiceFactory.getDatastoreService();
> > Query query = new Query("User");
> > query.addFilter("tag", Query.FilterOperator.EQUAL,
> > "admin"); // <-HERE
> > PreparedQuery pq = ds.prepare(query);
> > Iterable it = pq.asIterable();
> > //...
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Is possible a query inside an arrayList?
:-(
As you can see here
http://code.google.com/intl/it-IT/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Query.html
Gae low level API don't support, query.setParameter(...)
there are some other solutions?
(tank's for the answer Tony)
On 25 Lug, 16:14, Tony Qiu wrote:
> Of course you can do it.
> there is a keyword "member of" in JP-QL. you can use the JP-QL below
> to do what you want.
>
> Query query = new Query("select o from user o where :tag member of
> o.tags");
> query.setParameter("tag","abc")
>
> On 7月25日, 下午8时28分,salvatore wrote:
>
> > Hi i've problem
> > How a can query a data inside an ArrayList?
> > I use low level Datastore API
> > the data is structured like this
> > entity (kind 'User')
> > -String username
> > -String pass
> > -ArrayList tags
> > I want to query all the user with a specific role
> > my code
> > //...
> > DatastoreService ds=
> > DatastoreServiceFactory.getDatastoreService();
> > Query query = new Query("User");
> > query.addFilter("tag", Query.FilterOperator.EQUAL,
> > "admin"); // <-HERE
> > PreparedQuery pq = ds.prepare(query);
> > Iterable it = pq.asIterable();
> > //...
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Is possible a query inside an arrayList?
:-(
As you can see here
http://code.google.com/intl/it-IT/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Query.html
Gae low level API don't support, query.setParameter(...)
there are some other solutions?
(tank's for the answer Tony)
On 25 Lug, 16:14, Tony Qiu wrote:
> Of course you can do it.
> there is a keyword "member of" in JP-QL. you can use the JP-QL below
> to do what you want.
>
> Query query = new Query("select o from user o where :tag member of
> o.tags");
> query.setParameter("tag","abc")
>
> On 7月25日, 下午8时28分,salvatore wrote:
>
> > Hi i've problem
> > How a can query a data inside an ArrayList?
> > I use low level Datastore API
> > the data is structured like this
> > entity (kind 'User')
> > -String username
> > -String pass
> > -ArrayList tags
> > I want to query all the user with a specific role
> > my code
> > //...
> > DatastoreService ds=
> > DatastoreServiceFactory.getDatastoreService();
> > Query query = new Query("User");
> > query.addFilter("tag", Query.FilterOperator.EQUAL,
> > "admin"); // <-HERE
> > PreparedQuery pq = ds.prepare(query);
> > Iterable it = pq.asIterable();
> > //...
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Dance Dance Robot error / Channel API
Yes, this feature isn't launched in the production server yet. You can try it out in the 1.3.5 DevAppServer, but it requires a slightly workaround: Extract the "apphosting/tools/dev-channel-js.js" file from appengine-api-1.0-sdk-1.3.5.jar and move it to /_ah/channel/jsapi in your war directory (as if it were a static file). Your HTML should then use
Re: [appengine-java] Channel api / Dance dance robot: what is ApplicationError: 2:
The Channel API is not yet publicly launched. However, most of the DevAppServer support was in 1.3.5. Here's a workaround if you want to test it out: Extract the "apphosting/tools/dev-channel-js.js" file from appengine-api-1.0-sdk-1.3.5.jar and move it to /_ah/channel/jsapi in your war directory (as if it were a static file). Your HTML should then use
Re: [appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
On 26 Jul 2010, at 19:21, Bill wrote: Good thought, but unforunately Elements already have a parent key in this manner. Elements are hierarchical, so that an Element's parent is another parent. Can an entity have more than one parent? I haven't tried that... An Entity can only have one parent as it is encoded into its key as a path of ancestors leading back to the root entity. Are all Element "trees" in the same domain? If so then you could make Domain implement Element and use it as the root of an element hierarchy. But this limits the concurrency of changes to Elements to a few writes per second per Domain. Do you update more than one element in a transaction? If not then an unowned relationship between entities would be fine. Are domains added and removed often? If not then is it sufficient to check that the domain exists before working on your elements outside of the transaction? Just never delete Domains - mark them as deleted instead. Your original problem "Cannot operate on more than one" was caused by trying to load an element in a different entity group. Even load operations on a non existent entities (e.g. Domain) are included in the transaction. That is how you can guarantee you don't clobber existing data when you do a put(...) in a transaction. You can think of starting a transaction as creating a timestamp on the root entity - any changes to any element in the entire entity group after that time and before the commit will cause a concurrency exception. I think of transactions on non-existing roots as making a kind of temporary place-holder root to which the timestamp is added. See this for more details: http://www.youtube.com/watch?v=tx5gdoNpcZM If non of those work arounds is sufficient then perhaps you need distributed transactions. The Slim3 project implements distributed transactions using the task queue and there was a project with the aim of making a generic solution: http://www.youtube.com/watch?v=IOtBlzsx0m8 Not sure what happened to that though. It is possible that you need a full RDBMS - in which case you will either need to patiently wait for the release of the App Engine one recently announced or run your own on a hosted server. Just don't expect the same scalability as you get from the datastore. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
I think it's not possible to have more than one parent entity. I'm sure this is not possible using Keys, I don't know if it's the case for Strings but I think the same applies there. Adding another parent to the code shown (which should work) results in an exception using datastore Keys. I think the only workaround for your problem is doing the check for the domain key outside transactions, if this does not lead to consistency issues. Regards Lorenzo On Jul 26, 2:26 pm, Bill wrote: > * should read, "Elements are hierarchical, so that an Element's parent > is another Element, or else it is a root." -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
* should read, "Elements are hierarchical, so that an Element's parent is another Element, or else it is a root." -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
Good thought, but unforunately Elements already have a parent key in
this manner. Elements are hierarchical, so that an Element's parent
is another parent.
Can an entity have more than one parent? I haven't tried that...
For more reference, here are all the fields that might be of interest
to this problem:
@PersistenceCapable ( identityType = IdentityType.APPLICATION )
@Version (
strategy=VersionStrategy.VERSION_NUMBER,
extensions = { @Extension ( vendorName = "datanucleus", key = "field-
name", value = "version" ) }
)
public class Element {
@Persistent ( valueStrategy = IdGeneratorStrategy.IDENTITY )
@PrimaryKey
@Extension ( vendorName = "dataNucleus", key = "gae.encoded-pk",
value="true" )
@Immutable
private String key ;
@Persistent
@Extension ( vendorName = "datanucleus", key = "gae.parent-pk", value
= "true" )
@Immutable
private String parentKey ;
private long version ;
@Persistent
@Immutable
private String domain ;
// etc., etc., etc.
}
On Jul 26, 8:14 am, "l.denardo" wrote:
> Well, this is one of the major (maybe THE) drawback in
> appengine...apps architecture must be designed for the platform much
> more than with other kind of solutions.
>
> Thinking a bit more about the original problem, there's a simple
> solution for all: you can simply put your Domain Key in each element,
> and mark it as a parent for the element as stated in
>
> http://code.google.com/intl/it/appengine/docs/java/datastore/transact...
>
> class Element{
>
> �...@persistent
> @Extension(vendorName="datanucleus", key="gae.parent-pk",
> value="true")
> private Key domainKey;
>
> }
>
> This way all elements are in the same group of a domain, and
> transaction should work as long as you don't manipulate elements from
> multiple domains in a single transaction.
>
> Sorry for not stating it on first place, but I forgot abuot it
>
> On 26 Lug, 13:28, Bill wrote:
>
>
>
> > Hi Lorenzo,
>
> > Certainly, this would work from a technical perspective. However, it
> > is absolutely not right for the problem. For a variety of reasons,
> > Domain and Element need to have a completely unowned relationship.
> > I'll explain some of my high-level situation:
>
> > The Domain will be administered by a DomainAdministrator. This will
> > be either me, or someone personally known to me. For security
> > reasons, the DomainAdministrator has *no access* to the Elements that
> > "belong" to the Domain. The DomainAdministrator will also assign
> > authenticated users to what we can, for now, call the
> > ElementAdministrator role.
>
> > An ElementAdministrator has access to administer Elements within the
> > context of assigned Domains. There may be thousands of Elements
> > within a single Domain. At no time should all the Elements within a
> > Domain ever be selected in a single operation. There would be no
> > point! Also, the security model I've designed will fragment Elements
> > further such that a User who is a member of the Domain will have
> > access to only some of the Elements in that Domain.
>
> > In fact, one of the primary reasons I'm building the Element the way I
> > am is that Elements will be part of the fine-grained security model
> > itself. If a particular piece of data is selected in the system, I
> > need to check the user's profile's security elements against the
> > security elements baked into the data. This would necessitate a
> > datastore check, possibly in the middle of a transaction.
>
> > If I have to manually twig the transaction on or off because of entity
> > groups, I'm defeating years of conventional wisdom on "separation of
> > concerns". I've designed my architecture the way I always have -- I
> > want to be able to bring on junior developers who don't have to make
> > decisions on low-level things like transactions and security. To
> > separate this concern, I'm using the extremely nice @Transactional
> > annotation within spring. If any service that inherits from my base
> > entity service interface is called on its "update()" method, for
> > instance, I want the whole thing to be transactional. It deeply pains
> > me that I could easily build this kind of a separated-concern solution
> > ten years ago with clunky hand-built libraries for a Weblogic
> > production environment, but I can't do it in 2010 with Google App
> > Engine.
>
> > The fundamental problem with the concept of the entity group as a
> > restriction on transactional operations is that it mandates a
> > hierarchy on all operations. To be part of the entity group and
> > thence any operation in the transaction, it says, there must be a hard-
> > and-fast "owned" relationship between the entities. This looks very
> > nice on paper, but the problem is that if you're modelling the real
> > world, real world entities do not have nice hierarchical
> > relationships. For a very nice
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
Well, this is one of the major (maybe THE) drawback in
appengine...apps architecture must be designed for the platform much
more than with other kind of solutions.
Thinking a bit more about the original problem, there's a simple
solution for all: you can simply put your Domain Key in each element,
and mark it as a parent for the element as stated in
http://code.google.com/intl/it/appengine/docs/java/datastore/transactions.html
class Element{
@Persistent
@Extension(vendorName="datanucleus", key="gae.parent-pk",
value="true")
private Key domainKey;
}
This way all elements are in the same group of a domain, and
transaction should work as long as you don't manipulate elements from
multiple domains in a single transaction.
Sorry for not stating it on first place, but I forgot abuot it
On 26 Lug, 13:28, Bill wrote:
> Hi Lorenzo,
>
> Certainly, this would work from a technical perspective. However, it
> is absolutely not right for the problem. For a variety of reasons,
> Domain and Element need to have a completely unowned relationship.
> I'll explain some of my high-level situation:
>
> The Domain will be administered by a DomainAdministrator. This will
> be either me, or someone personally known to me. For security
> reasons, the DomainAdministrator has *no access* to the Elements that
> "belong" to the Domain. The DomainAdministrator will also assign
> authenticated users to what we can, for now, call the
> ElementAdministrator role.
>
> An ElementAdministrator has access to administer Elements within the
> context of assigned Domains. There may be thousands of Elements
> within a single Domain. At no time should all the Elements within a
> Domain ever be selected in a single operation. There would be no
> point! Also, the security model I've designed will fragment Elements
> further such that a User who is a member of the Domain will have
> access to only some of the Elements in that Domain.
>
> In fact, one of the primary reasons I'm building the Element the way I
> am is that Elements will be part of the fine-grained security model
> itself. If a particular piece of data is selected in the system, I
> need to check the user's profile's security elements against the
> security elements baked into the data. This would necessitate a
> datastore check, possibly in the middle of a transaction.
>
> If I have to manually twig the transaction on or off because of entity
> groups, I'm defeating years of conventional wisdom on "separation of
> concerns". I've designed my architecture the way I always have -- I
> want to be able to bring on junior developers who don't have to make
> decisions on low-level things like transactions and security. To
> separate this concern, I'm using the extremely nice @Transactional
> annotation within spring. If any service that inherits from my base
> entity service interface is called on its "update()" method, for
> instance, I want the whole thing to be transactional. It deeply pains
> me that I could easily build this kind of a separated-concern solution
> ten years ago with clunky hand-built libraries for a Weblogic
> production environment, but I can't do it in 2010 with Google App
> Engine.
>
> The fundamental problem with the concept of the entity group as a
> restriction on transactional operations is that it mandates a
> hierarchy on all operations. To be part of the entity group and
> thence any operation in the transaction, it says, there must be a hard-
> and-fast "owned" relationship between the entities. This looks very
> nice on paper, but the problem is that if you're modelling the real
> world, real world entities do not have nice hierarchical
> relationships. For a very nice explanation of why this is a very
> serious problem in solution design, see Christopher Alexander's
> classic "A City Is Not A Tree" (it's available online for free).
>
> I've hacked around this for now, with a ten pound sledge hammer and a
> rusty hacksaw. Using spring interceptors, I've built in an "extra-
> transactional check" layer around my services, outside the aegis of
> the transactional advisor. This is highly unsatisfactory because now
> my developers are going to have to keep track of what is, and what
> isn't part of the transaction. The concern is no longer neatly
> separated. This is de-evolution.
>
> I absolutely love what leap forward Google App Engine represents in
> terms of building applications. OTOH, the restrictions of entity
> groups on transaction is a very real obstacle to building "serious"
> business applications!
>
> Thanks,
>
> - Bill
>
> On Jul 26, 3:22 am, "l.denardo" wrote:
>
> > As far as I know there are two possible solutions:
>
> > *Review the architecture inserting Elements as a list property of your
> > domain, something like
>
> > @PersistenceCapable
> > class Domain{
> > private ArrayList elements;
>
> > }
>
> > This works correctly and you get the benefit to have a cascasing
> > delete on elements when yo
[appengine-java] Re: Reading blobs larger an 1MB
You need to read your Blob's content in several chunks that are no larger than the limit. Combining this with XML file compression can also make sense. On Jul 25, 9:07 pm, korey_sed wrote: > I am stumped. No matter what API call I try to read a large blob, I > get the following: > > Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: > ApplicationError: 6: Blob fetch size too large. > > I have tried using the BlobStoreInputStream, > BlobStoreService.fetchData, and even BlobStoreService.serve with a > fetch URL which also fails since URLs also have a 1MB limit. My > latest attempt was using fetchData setting the length to > MAX_BLOB_FETCH_SIZE. > > how are you suppose to read large data out of the blob store? all the > examples are using serve which does not help me. > > I have a 2MB XML file that I am trying to fetch and parse after it is > uploaded. > > any help is much appreciated. Code examples are even more > appreciated. > > Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: getting a java.lang.NullPointerException when running example query code on datastore
Hi Fixou,
Try replacing this line:
Query query = Device.getPersistenceManager().newQuery(sqlFetchAll);
with this:
Query query = Device.getPersistenceManager().newQuery(Device.class);
eliminating the "String sqlFetchAll" line. I use this approach and it
works for me.
On Jul 25, 3:23 pm, Fixou wrote:
> Here is the code i am trying to run, where "Device" is a
> persistententity stored in the datastore.
>
> @SuppressWarnings("unchecked")
> public static List findAll() {
> String sqlFetchAll = "select from " + Device.class.getName();
> Query query = Device.getPersistenceManager().newQuery(sqlFetchAll);
> List Devices = (List) query.execute();
> return Devices;
>
> }
>
> here is what i am getting:
> SEVERE: [Ljava.lang.StackTraceElement;@35389244
> java.lang.NullPointerException
> at
> org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.prepareDatas
> toreMapping(PersistenceCapableMapping.java:
> 173)
> at
> org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.initialize(P
> ersistenceCapableMapping.java:
> 117)
> at
> org.datanucleus.store.mapped.mapping.MappingFactory.createMapping(MappingFa
> ctory.java:
> 97)
> at
> org.datanucleus.store.mapped.mapping.AbstractMappingManager.getMapping(Abst
> ractMappingManager.java:
> 248)
> at
> org.datanucleus.store.appengine.DatastoreTable.initializeNonPK(DatastoreTab
> le.java:
> 322)
> at
> org.datanucleus.store.appengine.DatastoreTable.buildMapping(DatastoreTable.
> java:
> 288)
> at
> org.datanucleus.store.appengine.DatastoreManager.buildStoreData(DatastoreMa
> nager.java:
> 479)
> at
> org.datanucleus.store.appengine.DatastoreManager.newStoreData(DatastoreMana
> ger.java:
> 437)
> at
> org.datanucleus.store.AbstractStoreManager.addClasses(AbstractStoreManager.
> java:
> 788)
> at
> org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.ja
> va:
> 759)
> at
> org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedSto
> reManager.java:
> 358)
> at
> org.datanucleus.store.appengine.DatastoreManager.getDatastoreClass(Datastor
> eManager.java:
> 709)
> at
> org.datanucleus.store.appengine.query.DatastoreQuery.performExecute(Datasto
> reQuery.java:
> 214)
> at
> org.datanucleus.store.appengine.query.JDOQLQuery.performExecute(JDOQLQuery.
> java:
> 89)
> at org.datanucleus.store.query.Query.executeQuery(Query.java:1489)
> at org.datanucleus.store.query.Query.executeWithArray(Query.java:
> 1371)
> at org.datanucleus.store.query.Query.execute(Query.java:1344)
> at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:221)
> at com.fxbeckers.test.Device.findAll(Device.java:233)
> at com.fxbeckers.test.DevicesManager.findWinners(DevicesManager:112)
>
> Does anybody have a clue? i've been stuck on this for 3 days!
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
Hi Lorenzo,
Certainly, this would work from a technical perspective. However, it
is absolutely not right for the problem. For a variety of reasons,
Domain and Element need to have a completely unowned relationship.
I'll explain some of my high-level situation:
The Domain will be administered by a DomainAdministrator. This will
be either me, or someone personally known to me. For security
reasons, the DomainAdministrator has *no access* to the Elements that
"belong" to the Domain. The DomainAdministrator will also assign
authenticated users to what we can, for now, call the
ElementAdministrator role.
An ElementAdministrator has access to administer Elements within the
context of assigned Domains. There may be thousands of Elements
within a single Domain. At no time should all the Elements within a
Domain ever be selected in a single operation. There would be no
point! Also, the security model I've designed will fragment Elements
further such that a User who is a member of the Domain will have
access to only some of the Elements in that Domain.
In fact, one of the primary reasons I'm building the Element the way I
am is that Elements will be part of the fine-grained security model
itself. If a particular piece of data is selected in the system, I
need to check the user's profile's security elements against the
security elements baked into the data. This would necessitate a
datastore check, possibly in the middle of a transaction.
If I have to manually twig the transaction on or off because of entity
groups, I'm defeating years of conventional wisdom on "separation of
concerns". I've designed my architecture the way I always have -- I
want to be able to bring on junior developers who don't have to make
decisions on low-level things like transactions and security. To
separate this concern, I'm using the extremely nice @Transactional
annotation within spring. If any service that inherits from my base
entity service interface is called on its "update()" method, for
instance, I want the whole thing to be transactional. It deeply pains
me that I could easily build this kind of a separated-concern solution
ten years ago with clunky hand-built libraries for a Weblogic
production environment, but I can't do it in 2010 with Google App
Engine.
The fundamental problem with the concept of the entity group as a
restriction on transactional operations is that it mandates a
hierarchy on all operations. To be part of the entity group and
thence any operation in the transaction, it says, there must be a hard-
and-fast "owned" relationship between the entities. This looks very
nice on paper, but the problem is that if you're modelling the real
world, real world entities do not have nice hierarchical
relationships. For a very nice explanation of why this is a very
serious problem in solution design, see Christopher Alexander's
classic "A City Is Not A Tree" (it's available online for free).
I've hacked around this for now, with a ten pound sledge hammer and a
rusty hacksaw. Using spring interceptors, I've built in an "extra-
transactional check" layer around my services, outside the aegis of
the transactional advisor. This is highly unsatisfactory because now
my developers are going to have to keep track of what is, and what
isn't part of the transaction. The concern is no longer neatly
separated. This is de-evolution.
I absolutely love what leap forward Google App Engine represents in
terms of building applications. OTOH, the restrictions of entity
groups on transaction is a very real obstacle to building "serious"
business applications!
Thanks,
- Bill
On Jul 26, 3:22 am, "l.denardo" wrote:
> As far as I know there are two possible solutions:
>
> *Review the architecture inserting Elements as a list property of your
> domain, something like
>
> @PersistenceCapable
> class Domain{
> private ArrayList elements;
>
> }
>
> This works correctly and you get the benefit to have a cascasing
> delete on elements when you delete the Domain they're part of.
> This does not seem to have significant drawbacks, as documentation
> says somewhere Elements are fetched from datastore only if you access
> the list, and not if you just read the Domain without actually going
> thru the list.
> Elements are also queriable without reading the actual Domain they're
> in.
> The only thing this architecture seems to impede is finding the domain
> an element belongs to, given the element, without having complex
> iterations.
>
> * Check if domain exists outside the transaction.
> If your domains are not deleted during an Element insert this should
> work.
>
> I've gone the first path for my application and benefits in terms of
> code complexity and maintenance have been worth the effort, anyway it
> depends on your business logic if that can be right.
>
> Regards
> Lorenzo
>
> On Jul 24, 4:04 pm, Bill wrote:
>
>
>
> > Hello,
>
> > I'm quite frustrated with entity groups. They seem to make the
> > concept of a tran
[appengine-java] Re: Instance startup/shutdown and sessions
So GAE makes sure the preserve session data between shutting down/ starting up instances. Nice. On Jul 26, 12:44 pm, Shawn Brown wrote: > > What happens if between logged in user navigates to another page and > > the GAE instance was shutdown? > > I understand a new instance is started but what happened to the > > session data? > > AFAIK session data is persisted to big table. > > > Also, how is startup/shutdown of instances related to session-timeout > > in web.xml? > > > > > 30 > > > > AFAIK there is no connection. An instance will not be kept warm > because there is an open session. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Instance startup/shutdown and sessions
> What happens if between logged in user navigates to another page and > the GAE instance was shutdown? > I understand a new instance is started but what happened to the > session data? AFAIK session data is persisted to big table. > Also, how is startup/shutdown of instances related to session-timeout > in web.xml? > > > 30 > AFAIK there is no connection. An instance will not be kept warm because there is an open session. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Instance startup/shutdown and sessions
I like to store the logged in user (custom; not Google User Api) of my app in the session. What happens if between logged in user navigates to another page and the GAE instance was shutdown? I understand a new instance is started but what happened to the session data? Also, how is startup/shutdown of instances related to session-timeout in web.xml? 30 -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Highly Frustrated By Entity Groups / Transactions
As far as I know there are two possible solutions:
*Review the architecture inserting Elements as a list property of your
domain, something like
@PersistenceCapable
class Domain{
private ArrayList elements;
}
This works correctly and you get the benefit to have a cascasing
delete on elements when you delete the Domain they're part of.
This does not seem to have significant drawbacks, as documentation
says somewhere Elements are fetched from datastore only if you access
the list, and not if you just read the Domain without actually going
thru the list.
Elements are also queriable without reading the actual Domain they're
in.
The only thing this architecture seems to impede is finding the domain
an element belongs to, given the element, without having complex
iterations.
* Check if domain exists outside the transaction.
If your domains are not deleted during an Element insert this should
work.
I've gone the first path for my application and benefits in terms of
code complexity and maintenance have been worth the effort, anyway it
depends on your business logic if that can be right.
Regards
Lorenzo
On Jul 24, 4:04 pm, Bill wrote:
> Hello,
>
> I'm quite frustrated with entity groups. They seem to make the
> concept of a transaction largely pointless.
>
> The specific problem I face at this moment is this. Since the
> datastore has no concept of foreign key constraints, I need to do
> bounds checking. Here's the case:
>
> There exists a Domain type, and an Element type. An Element requires
> a reference to a pre-existing Domain in order to function, per my
> business rules. Domains exist independently of Elements, and are
> administered separately. I would characterize their relationship as
> "Element has a reference to a required Domain's primary key" rather
> than "Element is a child of Domain".
>
> When creating an Element, I need to check that the reference to the
> assigned Domain is not bogus. I do this in my service layer, which is
> also where I want to determine transactionality.
>
> The algorithm goes like this:
>
> * start transaction. Per my current architecture I cannot start this
> transaction any later without creating an entirely new non-
> transactional business delegate layer that feels like absolute
> overkill.
> * retrieve Domain by key. If no such Domain is found, throw an error
> * insert Element
> * commit
>
> Unfortunately, this causes the dreaded "can't operate on multiple
> entity groups in a single transaction" exception. I'm absolutely
> hornswaggled by this, since I'm not "operating" on the Domain. I just
> want to check that it exists! If this were a destructive change I
> were making, I'd kind of -- barely -- see the argument here, but why
> on earth is this a problem? Am I missing something?
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
