Re: Invalidating Objects
Hi, this sounds familiar. We have severe problems with vanishing snapshots, but only on one installation. Unfortunately, we found no way to reproduce this on our testing systems. There was a thread about this a while ago, but no real solution: http://lists.apple.com/archives/Webobjects-dev/2010/Mar/msg00330.html Otherwise, what you describe is exactly what we are seeing. We also use invalidate...(), the nulled out EO is never refetched, the only way to recover is to restart the app. Well, if you would be able to reproduce this in a small defined scenario, that would be extremely helpful. Marc On 18.05.2010, at 23:24, Andrew Lindesay wrote: Hello; As part of a built-in change notification facility, I am using EOObjectStoreCoordinator.invalidateObjectsWithGlobalIDs(..) with the relevant OSC locked. All use of other editing contexts are locked and unlocked correctly. Occasionally I seem to end up with disappearing snapshots where all the values of the snapshot go to NULL -- I can only imagine this would be linked to my use of 'invalidate..'. Other new editing contexts then see the NULLed-out EO and the EO does not seem to re-fault in that instance. When that corrupted EO is used in some sort of qualifier, EOF will treat the primary-key qualifier as looking for NULL. Any ideas where to look or has anybody else had this problem and has any pointers? cheers. ___ Andrew Lindesay www.silvereye.co.nz ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/yoda%40schli.ch This email sent to y...@schli.ch ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Invalidating Objects
On May 18, 2010, at 5:24 PM, Andrew Lindesay wrote: Hello; As part of a built-in change notification facility, I am using EOObjectStoreCoordinator.invalidateObjectsWithGlobalIDs(..) with the relevant OSC locked. All use of other editing contexts are locked and unlocked correctly. Occasionally I seem to end up with disappearing snapshots where all the values of the snapshot go to NULL -- I can only imagine this would be linked to my use of 'invalidate..'. Other new editing contexts then see the NULLed-out EO and the EO does not seem to re-fault in that instance. When that corrupted EO is used in some sort of qualifier, EOF will treat the primary-key qualifier as looking for NULL. Any ideas where to look or has anybody else had this problem and has any pointers? cheers. ___ Andrew Lindesay www.silvereye.co.nz Hi Andrew! I've seen similar problems, but I don't think it was related to invalidating objects. It appeared related to the use of nested editing contexts. There was a discussion about it around two months ago... http://www.mail-archive.com/webobjects-dev@lists.apple.com/msg36101.html Using a nested editing context anywhere? Ramsey ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Invalidating Objects
Hello Ramsey Marc Thanks for getting back to me. I've seen similar problems...Using a nested editing context anywhere? I _was_ until they seemed to causing some other problems so I have removed use of that now. I don't think the nested ECs would have been used on the instances where the problem was manifesting itself though. Otherwise, what you describe is exactly what we are seeing. We also use invalidate...(), the nulled out EO is never refetched, the only way to recover is to restart the app. What I have done is to use a lock across the work of handling R-R cycles and the change notification (the only place where the invalidate is actioned). In this way, if the issue is one of concurrency with regular EC use then I should see this issue go away for human-facing instances which are doing any EC-work outside of the R-R cycles. It's still a fair way off a production deploy, but I will let you know if this resolves the issue. cheers. ___ Andrew Lindesay www.silvereye.co.nz ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Invalidating Objects
What I have done is to use a lock across the work of handling R-R cycles and the change notification (the only place where the invalidate is actioned). In this way, if the issue is one of concurrency with regular EC use then I should see this issue go away for human-facing instances which are doing any EC-work outside of the R-R cycles. It's still a fair way off a production deploy, but I will let you know if this resolves the issue. in wonder's we take a dbc lock during the background queue processing, then do a refreshing fetch of the affected EO so that it updated the snapshots. you really don't want to ever do an .invalidate() because if you any EO's in an modified state, they'll be messed up. ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Invalidating Objects
Hello Mike; I think I was _just_ doing fetches at one point, but I suspect that there was a change in 5.3 - 5.4 in that invalidate...() in 5.3 did not drop to-manys from caches in EOF and 5.4 seemed to. In 5.3 fresh fetch with pre-pretches into to-manys did re-cache the to-many, but in 5.4 it did not seem to. I obviously am unable to look and see what it is actually doing, but that is my observation and the only way I could get it to function for to-many's is to use invalidate... –– I presume that change notification in PW is functioning seamlessly under high load in 5.4 without invalidate and for to-many relationships? Do you take out a DBC lock for the duration of the change notification process for all DBC's which are involved in the changes? I guess I should go take a look... cheers. What I have done is to use a lock across the work of handling R-R cycles and the change notification (the only place where the invalidate is actioned). In this way, if the issue is one of concurrency with regular EC use then I should see this issue go away for human-facing instances which are doing any EC-work outside of the R-R cycles. It's still a fair way off a production deploy, but I will let you know if this resolves the issue. in wonder's we take a dbc lock during the background queue processing, then do a refreshing fetch of the affected EO so that it updated the snapshots. you really don't want to ever do an .invalidate() because if you any EO's in an modified state, they'll be messed up. ___ Andrew Lindesay www.silvereye.co.nz ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Invalidating Objects
Does this have any bearing on your problem?
/**
* Clears snapshot the relaationship of a given enterprise so it
will be read again when next accessed.
* @param eo enterprise object
* @param relationshipName relationship name
*/
public static void
clearSnapshotForRelationshipNamed(EOEnterpriseObject eo, String
relationshipName) {
EOEditingContext ec = eo.editingContext();
EOModel model = EOUtilities.entityForObject(ec, eo).model();
EOGlobalID gid = ec.globalIDForObject(eo);
EODatabaseContext dbc =
EODatabaseContext.registeredDatabaseContextForModel(model, ec);
EODatabase database = dbc.database();
ERXEOControlUtilities.clearSnapshotForRelationshipNamedInDatabase(eo,
relationshipName, database);
}
That is what I use to re-fresh to-many relationships. The only time I
use invalidate is when I know that an EO with a snapshot has been
deleted from the database.
Chuck
On May 20, 2010, at 3:53 PM, Andrew Lindesay wrote:
Hello Mike;
I think I was _just_ doing fetches at one point, but I suspect that
there was a change in 5.3 - 5.4 in that invalidate...() in 5.3
did not drop to-manys from caches in EOF and 5.4 seemed to. In 5.3
fresh fetch with pre-pretches into to-manys did re-cache the to-
many, but in 5.4 it did not seem to. I obviously am unable to look
and see what it is actually doing, but that is my observation and
the only way I could get it to function for to-many's is to use
invalidate... –– I presume that change notification in PW is
functioning seamlessly under high load in 5.4 without invalidate and
for to-many relationships? Do you take out a DBC lock for the
duration of the change notification process for all DBC's which are
involved in the changes? I guess I should go take a look...
cheers.
What I have done is to use a lock across the work of handling R-R
cycles and the change notification (the only place where the
invalidate is actioned). In this way, if the issue is one of
concurrency with regular EC use then I should see this issue go
away for human-facing instances which are doing any EC-work
outside of the R-R cycles. It's still a fair way off a production
deploy, but I will let you know if this resolves the issue.
in wonder's we take a dbc lock during the background queue
processing, then do a refreshing fetch of the affected EO so that
it updated the snapshots. you really don't want to ever do
an .invalidate() because if you any EO's in an modified state,
they'll be messed up.
___
Andrew Lindesay
www.silvereye.co.nz
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village.net
This email sent to ch...@global-village.net
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
RE: Invalidating Objects
Hi Chuck; That looks perfect thanks; I will try it out soon! cheers. Does this have any bearing on your problem? ... That is what I use to re-fresh to-many relationships. The only time I use invalidate is when I know that an EO with a snapshot has been deleted from the database. ___ Andrew Lindesay www.silvereye.co.nz ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Invalidating Objects
Hello; As part of a built-in change notification facility, I am using EOObjectStoreCoordinator.invalidateObjectsWithGlobalIDs(..) with the relevant OSC locked. All use of other editing contexts are locked and unlocked correctly. Occasionally I seem to end up with disappearing snapshots where all the values of the snapshot go to NULL -- I can only imagine this would be linked to my use of 'invalidate..'. Other new editing contexts then see the NULLed-out EO and the EO does not seem to re-fault in that instance. When that corrupted EO is used in some sort of qualifier, EOF will treat the primary-key qualifier as looking for NULL. Any ideas where to look or has anybody else had this problem and has any pointers? cheers. ___ Andrew Lindesay www.silvereye.co.nz ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Invalidating objects asynchronously
On 09/09/2008, at 10:56 AM, Mike Schrag wrote:
More specifically:
/**
* This is Kelly Hawks' fix for the missing to one relationship.
* Delegate on EODatabaseContext that gets called when a to-one
fault cannot find its data in
* the database. The object that is returned is a cleared fault.
* We raise here to restore the functionality that existed prior
to WebObjects 4.5.
* Whenever a fault fails for a globalID (i.e. the object is NOT
found in the database), we raise
* an [EMAIL PROTECTED]
com.webobjects.eoaccess.EOObjectNotAvailableException
EOObjectNotAvailableException}. br
* If you have entities you don't really care about, you can set
the system property
*
codeer.extensions.ERXDatabaseContextDelegate.tolerantEntityPattern/
code to a regular expression
* that will be tested against the GID entity name. If it
matches, then only an error will be logged
* but no exception will be thrown.
*
* @param context database context
* @param object object that is firing the fault for a given to-
one relationship
* @param gid global id that wasn't found in the database.
*/
public boolean
databaseContextFailedToFetchObject(EODatabaseContext context, Object
object, EOGlobalID gid) {
Ah, thank you! That always bugged me that it would throw an exception
when the restricting qualifier had kicked in.
Now I can have peace ;-)
er.extensions.ERXDatabaseContextDelegate.tolerantEntityPattern=\\w+
with regards,
--
Lachlan Deck
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
On 3-Sep-08, at 4:16 PM, Mike Schrag wrote: We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. Yes, with an asterisk. As Travis said, use Project Wonder and the ERXRemoteSynchronizer with JGroups support ... The JMS change notification framework is known to have problems under load. I have tested the remote synchronizer under VERY heavy load tests (500-1000 insanely clicking updating multiple instances) and verified for accuracy of results (accuracy defined as final state in all instances matches the database). If JMS is invalidating objects, it's going to be screwed -- you CANNOT invalidate and expect a valid graph state ... We take a performance hit, but in the case where an EO is faulted into the snapshot cache and then modified in a remote instance, we force a refresh from the DB. There is really almost no other way to end up in a safe state (I tried LOTS of approaches -- I rewrote this entire process in 3 different ways until I found all the crazy cases that work). We also properly update inverse relationships (which every other change notification system that I've seen does incorrectly). I'm not saying you can't end up in a potentially wonky state, because this ultimately is impossible to get 100% right, but I've tested all the other popular ones and I can tell you for a 100% fact that they get it wrong. You WILL get deadlocks and you WILL get incorrect graph state. Wonder's is the only one that has was able to get through my butt-wuppin' load test and survive, and I've been using this in a multiple instance application in production for almost 2 years without any graph state exceptions. As far as sharding your EO's, I definitely agree ... If your data lends itself to sharding, then this is far more ideal (and dependable ... if you get it right). However, there are some cases where you have globally collaborative data that does not lend itself to sharding, or even if it does, you may want to have a shard on multiple instances, which puts you back where you started. There are also often applications that require users to cross shards, which means you have to have a system for dealing with redirecting the user to another instance mid-flight. All completely possible to do, but none of these solutions are trivial (but then we wouldn't have jobs if they were :) ). ms Mike, Thanks very much for the reply. As always, your knowledge and help is invaluable. We're getting deeper and deeper down the rabbit hole here... deployment date is looming, and we've spent almost two weeks on this one bug that is really really hurting. I have a few very specific questions before we move forward, any help is greatly appreciated. 1. We're thinking that the only way to fix this is to start using Project Wonder and the ERXRemoteSynchronizer for change notifications. I'm still a little concerned that it won't help our specific case. (here is the scary scenario) We get an NSArray of EO's from a relationship, and are enumerating over them. A change notification comes in relating to one of the EO's in the array (it's actually been deleted by another user on another instance, which happens all the time) When the object in question is re-fetched from the database, it turns out to be deleted(!) and is then null, and is then very bad for us. Will ERXRemoteSynchronizer handle this is some nice way, or will the object still resolve to null in the middle of the enumeration? (if it's still null, we need to think of some other solution...) (so if it's still going to get nulled in the middle of the Enumeration - what the heck can we do?) 2. We ditched JGroups in favor of JMS as our message transport so we can use the built-in JMS providers in our J2EE deploys. (which is much nicer) We're thinking that we can write an interface for the constructor of ERXRemoteSynchronizer so that it can use a JGroups or JMS provider transparently. In your reply above you said that JMS will not work, ever. Can you please expand on this? I was under the impression that JMS is used and relied upon heavily in the J2EE world. Will JMS + ERXRemoteSynchronizer be OK? (which I would think is the case, as JMS just delivers messages, the Wonder framework decides what to do about the message). Thanks once again, -Dan Grec ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/dan%40stratahealth.com This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http
Re: Invalidating objects asynchronously
We have started down the rabbit hole as well. Taking a look at Jgroups (http://www.jgroups.org) it looks like Jgroups can be configured to use JMS as a transport mechanism. Not sure which version of Jgroups is bundled with Wonder but it seems like it might be possible without having to write your own interface. The only thing I think adding JMS into the picture would add is persistent messages which may not even be necessary unless you never restart your apps apps. As we learn more we will pass it on. Dov Rosenberg On 9/8/08 7:47 PM, Dan Grec [EMAIL PROTECTED] wrote: On 3-Sep-08, at 4:16 PM, Mike Schrag wrote: We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. Yes, with an asterisk. As Travis said, use Project Wonder and the ERXRemoteSynchronizer with JGroups support ... The JMS change notification framework is known to have problems under load. I have tested the remote synchronizer under VERY heavy load tests (500-1000 insanely clicking updating multiple instances) and verified for accuracy of results (accuracy defined as final state in all instances matches the database). If JMS is invalidating objects, it's going to be screwed -- you CANNOT invalidate and expect a valid graph state ... We take a performance hit, but in the case where an EO is faulted into the snapshot cache and then modified in a remote instance, we force a refresh from the DB. There is really almost no other way to end up in a safe state (I tried LOTS of approaches -- I rewrote this entire process in 3 different ways until I found all the crazy cases that work). We also properly update inverse relationships (which every other change notification system that I've seen does incorrectly). I'm not saying you can't end up in a potentially wonky state, because this ultimately is impossible to get 100% right, but I've tested all the other popular ones and I can tell you for a 100% fact that they get it wrong. You WILL get deadlocks and you WILL get incorrect graph state. Wonder's is the only one that has was able to get through my butt-wuppin' load test and survive, and I've been using this in a multiple instance application in production for almost 2 years without any graph state exceptions. As far as sharding your EO's, I definitely agree ... If your data lends itself to sharding, then this is far more ideal (and dependable ... if you get it right). However, there are some cases where you have globally collaborative data that does not lend itself to sharding, or even if it does, you may want to have a shard on multiple instances, which puts you back where you started. There are also often applications that require users to cross shards, which means you have to have a system for dealing with redirecting the user to another instance mid-flight. All completely possible to do, but none of these solutions are trivial (but then we wouldn't have jobs if they were :) ). ms Mike, Thanks very much for the reply. As always, your knowledge and help is invaluable. We're getting deeper and deeper down the rabbit hole here... deployment date is looming, and we've spent almost two weeks on this one bug that is really really hurting. I have a few very specific questions before we move forward, any help is greatly appreciated. 1. We're thinking that the only way to fix this is to start using Project Wonder and the ERXRemoteSynchronizer for change notifications. I'm still a little concerned that it won't help our specific case. (here is the scary scenario) We get an NSArray of EO's from a relationship, and are enumerating over them. A change notification comes in relating to one of the EO's in the array (it's actually been deleted by another user on another instance, which happens all the time) When the object in question is re-fetched from the database, it turns out to be deleted(!) and is then null, and is then very bad for us. Will ERXRemoteSynchronizer handle this is some nice way, or will the object still resolve to null in the middle of the enumeration? (if it's still null, we need to think of some other solution...) (so if it's still going to get nulled in the middle of the Enumeration - what the heck can we do?) 2. We ditched JGroups in favor of JMS as our message transport so we can use the built-in JMS providers in our J2EE deploys. (which is much nicer) We're thinking that we can write an interface for the constructor of ERXRemoteSynchronizer so that it can use a JGroups or JMS provider transparently. In your reply above you said that JMS will not work, ever. Can you please expand on this? I was under the impression that JMS is used and relied upon heavily in the J2EE world. Will JMS + ERXRemoteSynchronizer be OK? (which I would think is the case, as JMS just delivers messages, the Wonder framework decides what to do about the message). Thanks once again, -Dan Grec
Re: Invalidating objects asynchronously
Hi Dan, On Sep 8, 2008, at 4:47 PM, Dan Grec wrote: On 3-Sep-08, at 4:16 PM, Mike Schrag wrote: We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. Yes, with an asterisk. As Travis said, use Project Wonder and the ERXRemoteSynchronizer with JGroups support ... The JMS change notification framework is known to have problems under load. I have tested the remote synchronizer under VERY heavy load tests (500-1000 insanely clicking updating multiple instances) and verified for accuracy of results (accuracy defined as final state in all instances matches the database). If JMS is invalidating objects, it's going to be screwed -- you CANNOT invalidate and expect a valid graph state ... We take a performance hit, but in the case where an EO is faulted into the snapshot cache and then modified in a remote instance, we force a refresh from the DB. There is really almost no other way to end up in a safe state (I tried LOTS of approaches -- I rewrote this entire process in 3 different ways until I found all the crazy cases that work). We also properly update inverse relationships (which every other change notification system that I've seen does incorrectly). I'm not saying you can't end up in a potentially wonky state, because this ultimately is impossible to get 100% right, but I've tested all the other popular ones and I can tell you for a 100% fact that they get it wrong. You WILL get deadlocks and you WILL get incorrect graph state. Wonder's is the only one that has was able to get through my butt-wuppin' load test and survive, and I've been using this in a multiple instance application in production for almost 2 years without any graph state exceptions. As far as sharding your EO's, I definitely agree ... If your data lends itself to sharding, then this is far more ideal (and dependable ... if you get it right). However, there are some cases where you have globally collaborative data that does not lend itself to sharding, or even if it does, you may want to have a shard on multiple instances, which puts you back where you started. There are also often applications that require users to cross shards, which means you have to have a system for dealing with redirecting the user to another instance mid-flight. All completely possible to do, but none of these solutions are trivial (but then we wouldn't have jobs if they were :) ). ms Mike, Thanks very much for the reply. As always, your knowledge and help is invaluable. We're getting deeper and deeper down the rabbit hole here... deployment date is looming, and we've spent almost two weeks on this one bug that is really really hurting. I have a few very specific questions before we move forward, any help is greatly appreciated. 1. We're thinking that the only way to fix this is to start using Project Wonder and the ERXRemoteSynchronizer for change notifications. I'm still a little concerned that it won't help our specific case. (here is the scary scenario) We get an NSArray of EO's from a relationship, and are enumerating over them. A change notification comes in relating to one of the EO's in the array (it's actually been deleted by another user on another instance, which happens all the time) When the object in question is re-fetched from the database, it turns out to be deleted(!) and is then null, and is then very bad for us. Do you mean that attributes of the EO are suddenly null? I don't see how this is possible _unless_ you are looking at objects that are in an EOEditingContext that is not locked. That won't ever work reliably. Chuck Will ERXRemoteSynchronizer handle this is some nice way, or will the object still resolve to null in the middle of the enumeration? (if it's still null, we need to think of some other solution...) (so if it's still going to get nulled in the middle of the Enumeration - what the heck can we do?) 2. We ditched JGroups in favor of JMS as our message transport so we can use the built-in JMS providers in our J2EE deploys. (which is much nicer) We're thinking that we can write an interface for the constructor of ERXRemoteSynchronizer so that it can use a JGroups or JMS provider transparently. In your reply above you said that JMS will not work, ever. Can you please expand on this? I was under the impression that JMS is used and relied upon heavily in the J2EE world. Will JMS + ERXRemoteSynchronizer be OK? (which I would think is the case, as JMS just delivers messages, the Wonder framework decides what to do about the message). Thanks once again, -Dan Grec ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/dan
Re: Invalidating objects asynchronously
Do you mean that attributes of the EO are suddenly null? I don't see how this is possible _unless_ you are looking at objects that are in an EOEditingContext that is not locked. That won't ever work reliably. I'm assuming the root problem here is the old JMS stuff would invalidate objects, which means you end up in a very unreliable state. But I wasn't digging too much into it. ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
On Sep 8, 2008, at 5:26 PM, Mike Schrag wrote:
Do you mean that attributes of the EO are suddenly null? I don't
see how this is possible _unless_ you are looking at objects that
are in an EOEditingContext that is not locked. That won't ever
work reliably.
I'm assuming the root problem here is the old JMS stuff would
invalidate objects, which means you end up in a very unreliable
state. But I wasn't digging too much into it.
Hmmm, maybe if that happened before willRead() was called, this might
still happen. But after the fault is fired, the attribute values
should be stable if the EC is locked as it queues the notifications
rather than responding to them. So, to be sure you had a real, live
object, you could:
eo.willRead();
if ( ! eo.isDummyFaultEO(eo) ) {
// process
}
/**
* Detecting WO 5 dummy fault EOs. The code below is a
translation of code posted here:
* From http://wodeveloper.com/omniLists/eof/2001/November/msg00023.html
*
* As of WO4.5, EOF resolves faults for objects that don't exist
in the database to a dummy object with mostly empty attributes (except
those
that are set in -init), whereas earlier versions of EOF threw a
_fireFault exception. This method detects whether the receiver is
such a dummy object.
*
* A distinguishing feature of these dummy EOs is that they
don't have a corresponding database context snapshot, so we use this
to detect its dummy-ness. This seems more robust than checking if all
or most attributes are empty, since these can be changed by client code.
*
* Dummy fault EOs should be removed from the editingContext
(using forgetObject) before it is invalidated (invalidateAllObjects or
invalidateObjectWithGlobalID), otherwise an unrecoverable
decrementSnapshotCountForGlobalID - unable to decrement snapshot
count for object with global ID exception is thrown by EOF.brbr
*
* See also: http://www.omnigroup.com/mailman/archive/eof/2001-May/001988.html
andfollowing.
*/
public static boolean isDummyFaultEO(EOEnterpriseObject anObject)
{
/** require [valid_param] anObject != null; **/
JassAdditions.pre(EOObject, isDummyFaultEO
[object_in_ec], anObject.editingContext() != null);
JassAdditions.pre(EOObject, isDummyFaultEO
[object_has_global_id],
anObject.editingContext().globalIDForObject(anObject) != null);
boolean isDummyFaultEO = false;
// This method will fail if the object is still a fault. We
fire the fault so that the correct result is returned.
if (anObject.isFault())
{
anObject.willRead();
}
EOEditingContext ec = anObject.editingContext();
EOGlobalID globalID = ec.globalIDForObject(anObject);
// NB. objects with temporary globalIDs legitimately have no
DB snapshots, since these are by definition not yet saved to the
database.
if (! globalID.isTemporary())
{
// Find the EODatabaseContext instance associated with
anObject, or null if no databaseContext association can be found.
EOObjectStoreCoordinator rootStore =
(EOObjectStoreCoordinator)ec.rootObjectStore();
rootStore.lock();
try
{
EODatabaseContext dbContext =
(EODatabaseContext)rootStore.objectStoreForObject(anObject);
if (dbContext == null)
{
throw new RuntimeException(Could not locate
EODatabaseContext for object ' +
anObject + ' in
editingContext: + ec);
}
isDummyFaultEO =
(dbContext.snapshotForGlobalID(globalID) == null);
}
finally
{
rootStore.unlock();
}
}
return isDummyFaultEO;
}
Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
I think there is a EODatabaseContext Delegate in WOnder (i think
ERXDatabaseContextDelegate) that will return null (or throw an
exception, i can't remember) in such cases instead of a dummy fault.
Regards
Peter
On 09/09/2008, at 10:32 AM, Chuck Hill wrote:
On Sep 8, 2008, at 5:26 PM, Mike Schrag wrote:
Do you mean that attributes of the EO are suddenly null? I don't
see how this is possible _unless_ you are looking at objects that
are in an EOEditingContext that is not locked. That won't ever
work reliably.
I'm assuming the root problem here is the old JMS stuff would
invalidate objects, which means you end up in a very unreliable
state. But I wasn't digging too much into it.
Hmmm, maybe if that happened before willRead() was called, this
might still happen. But after the fault is fired, the attribute
values should be stable if the EC is locked as it queues the
notifications rather than responding to them. So, to be sure you
had a real, live object, you could:
eo.willRead();
if ( ! eo.isDummyFaultEO(eo) ) {
// process
}
/**
* Detecting WO 5 dummy fault EOs. The code below is a
translation of code posted here:
* From http://wodeveloper.com/omniLists/eof/2001/November/msg00023.html
*
* As of WO4.5, EOF resolves faults for objects that don't exist
in the database to a dummy object with mostly empty attributes
(except those
that are set in -init), whereas earlier versions of EOF threw a
_fireFault exception. This method detects whether the receiver is
such a dummy object.
*
* A distinguishing feature of these dummy EOs is that they
don't have a corresponding database context snapshot, so we use this
to detect its dummy-ness. This seems more robust than checking if
all or most attributes are empty, since these can be changed by
client code.
*
* Dummy fault EOs should be removed from the editingContext
(using forgetObject) before it is invalidated (invalidateAllObjects
or invalidateObjectWithGlobalID), otherwise an unrecoverable
decrementSnapshotCountForGlobalID - unable to decrement snapshot
count for object with global ID exception is thrown by EOF.brbr
*
* See also: http://www.omnigroup.com/mailman/archive/eof/2001-May/001988.html
andfollowing.
*/
public static boolean isDummyFaultEO(EOEnterpriseObject anObject)
{
/** require [valid_param] anObject != null; **/
JassAdditions.pre(EOObject, isDummyFaultEO
[object_in_ec], anObject.editingContext() != null);
JassAdditions.pre(EOObject, isDummyFaultEO
[object_has_global_id],
anObject.editingContext().globalIDForObject(anObject) != null);
boolean isDummyFaultEO = false;
// This method will fail if the object is still a fault. We
fire the fault so that the correct result is returned.
if (anObject.isFault())
{
anObject.willRead();
}
EOEditingContext ec = anObject.editingContext();
EOGlobalID globalID = ec.globalIDForObject(anObject);
// NB. objects with temporary globalIDs legitimately have no
DB snapshots, since these are by definition not yet saved to the
database.
if (! globalID.isTemporary())
{
// Find the EODatabaseContext instance associated with
anObject, or null if no databaseContext association can be found.
EOObjectStoreCoordinator rootStore =
(EOObjectStoreCoordinator)ec.rootObjectStore();
rootStore.lock();
try
{
EODatabaseContext dbContext =
(EODatabaseContext)rootStore.objectStoreForObject(anObject);
if (dbContext == null)
{
throw new RuntimeException(Could not locate
EODatabaseContext for object ' +
anObject + ' in
editingContext: + ec);
}
isDummyFaultEO =
(dbContext.snapshotForGlobalID(globalID) == null);
}
finally
{
rootStore.unlock();
}
}
return isDummyFaultEO;
}
Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/peter%40etechgroup.com.au
This email sent to [EMAIL PROTECTED]
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Peter Vandoros
Software Engineer
Etech Group Pty Ltd
Level 3/21 Victoria St
Melbourne VIC 3000
Australia
Ph: +61
Re: Invalidating objects asynchronously
I think there is a EODatabaseContext Delegate in WOnder (i think ERXDatabaseContextDelegate) that will return null (or throw an exception, i can't remember) in such cases instead of a dummy fault. EOObjectNotAvailableException ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
Yes. But he is not using Wonder. :-)
On Sep 8, 2008, at 5:44 PM, Peter Vandoros wrote:
I think there is a EODatabaseContext Delegate in WOnder (i think
ERXDatabaseContextDelegate) that will return null (or throw an
exception, i can't remember) in such cases instead of a dummy fault.
Regards
Peter
On 09/09/2008, at 10:32 AM, Chuck Hill wrote:
On Sep 8, 2008, at 5:26 PM, Mike Schrag wrote:
Do you mean that attributes of the EO are suddenly null? I don't
see how this is possible _unless_ you are looking at objects that
are in an EOEditingContext that is not locked. That won't ever
work reliably.
I'm assuming the root problem here is the old JMS stuff would
invalidate objects, which means you end up in a very unreliable
state. But I wasn't digging too much into it.
Hmmm, maybe if that happened before willRead() was called, this
might still happen. But after the fault is fired, the attribute
values should be stable if the EC is locked as it queues the
notifications rather than responding to them. So, to be sure you
had a real, live object, you could:
eo.willRead();
if ( ! eo.isDummyFaultEO(eo) ) {
// process
}
/**
* Detecting WO 5 dummy fault EOs. The code below is a
translation of code posted here:
* From http://wodeveloper.com/omniLists/eof/2001/November/msg00023.html
*
* As of WO4.5, EOF resolves faults for objects that don't exist
in the database to a dummy object with mostly empty attributes
(except those
that are set in -init), whereas earlier versions of EOF threw a
_fireFault exception. This method detects whether the receiver is
such a dummy object.
*
* A distinguishing feature of these dummy EOs is that they
don't have a corresponding database context snapshot, so we use
this to detect its dummy-ness. This seems more robust than
checking if all or most attributes are empty, since these can be
changed by client code.
*
* Dummy fault EOs should be removed from the editingContext
(using forgetObject) before it is invalidated (invalidateAllObjects
or invalidateObjectWithGlobalID), otherwise an unrecoverable
decrementSnapshotCountForGlobalID - unable to decrement snapshot
count for object with global ID exception is thrown by EOF.brbr
*
* See also: http://www.omnigroup.com/mailman/archive/eof/2001-May/001988.html
andfollowing.
*/
public static boolean isDummyFaultEO(EOEnterpriseObject anObject)
{
/** require [valid_param] anObject != null; **/
JassAdditions.pre(EOObject, isDummyFaultEO
[object_in_ec], anObject.editingContext() != null);
JassAdditions.pre(EOObject, isDummyFaultEO
[object_has_global_id],
anObject.editingContext().globalIDForObject(anObject) != null);
boolean isDummyFaultEO = false;
// This method will fail if the object is still a fault. We
fire the fault so that the correct result is returned.
if (anObject.isFault())
{
anObject.willRead();
}
EOEditingContext ec = anObject.editingContext();
EOGlobalID globalID = ec.globalIDForObject(anObject);
// NB. objects with temporary globalIDs legitimately have no
DB snapshots, since these are by definition not yet saved to the
database.
if (! globalID.isTemporary())
{
// Find the EODatabaseContext instance associated with
anObject, or null if no databaseContext association can be found.
EOObjectStoreCoordinator rootStore =
(EOObjectStoreCoordinator)ec.rootObjectStore();
rootStore.lock();
try
{
EODatabaseContext dbContext =
(EODatabaseContext)rootStore.objectStoreForObject(anObject);
if (dbContext == null)
{
throw new RuntimeException(Could not locate
EODatabaseContext for object ' +
anObject + ' in
editingContext: + ec);
}
isDummyFaultEO =
(dbContext.snapshotForGlobalID(globalID) == null);
}
finally
{
rootStore.unlock();
}
}
return isDummyFaultEO;
}
Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/peter%40etechgroup.com.au
This email sent to [EMAIL PROTECTED]
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Peter Vandoros
Software Engineer
Etech Group Pty Ltd
Level 3/21
Re: Invalidating objects asynchronously
Yes. But he is not using Wonder. :-) I think I found your problem right there ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
Yes, but he could link to WOnder or copy that delegate :-)
On 09/09/2008, at 10:56 AM, Chuck Hill wrote:
Yes. But he is not using Wonder. :-)
On Sep 8, 2008, at 5:44 PM, Peter Vandoros wrote:
I think there is a EODatabaseContext Delegate in WOnder (i think
ERXDatabaseContextDelegate) that will return null (or throw an
exception, i can't remember) in such cases instead of a dummy fault.
Regards
Peter
On 09/09/2008, at 10:32 AM, Chuck Hill wrote:
On Sep 8, 2008, at 5:26 PM, Mike Schrag wrote:
Do you mean that attributes of the EO are suddenly null? I
don't see how this is possible _unless_ you are looking at
objects that are in an EOEditingContext that is not locked.
That won't ever work reliably.
I'm assuming the root problem here is the old JMS stuff would
invalidate objects, which means you end up in a very unreliable
state. But I wasn't digging too much into it.
Hmmm, maybe if that happened before willRead() was called, this
might still happen. But after the fault is fired, the attribute
values should be stable if the EC is locked as it queues the
notifications rather than responding to them. So, to be sure you
had a real, live object, you could:
eo.willRead();
if ( ! eo.isDummyFaultEO(eo) ) {
// process
}
/**
* Detecting WO 5 dummy fault EOs. The code below is a
translation of code posted here:
* From http://wodeveloper.com/omniLists/eof/2001/November/msg00023.html
*
* As of WO4.5, EOF resolves faults for objects that don't exist
in the database to a dummy object with mostly empty attributes
(except those
that are set in -init), whereas earlier versions of EOF threw a
_fireFault exception. This method detects whether the receiver is
such a dummy object.
*
* A distinguishing feature of these dummy EOs is that they
don't have a corresponding database context snapshot, so we use
this to detect its dummy-ness. This seems more robust than
checking if all or most attributes are empty, since these can be
changed by client code.
*
* Dummy fault EOs should be removed from the editingContext
(using forgetObject) before it is invalidated
(invalidateAllObjects or invalidateObjectWithGlobalID), otherwise
an unrecoverable decrementSnapshotCountForGlobalID - unable to
decrement snapshot count for object with global ID exception is
thrown by EOF.brbr
*
* See also: http://www.omnigroup.com/mailman/archive/eof/2001-May/001988.html
andfollowing.
*/
public static boolean isDummyFaultEO(EOEnterpriseObject anObject)
{
/** require [valid_param] anObject != null; **/
JassAdditions.pre(EOObject, isDummyFaultEO
[object_in_ec], anObject.editingContext() != null);
JassAdditions.pre(EOObject, isDummyFaultEO
[object_has_global_id],
anObject.editingContext().globalIDForObject(anObject) != null);
boolean isDummyFaultEO = false;
// This method will fail if the object is still a fault. We
fire the fault so that the correct result is returned.
if (anObject.isFault())
{
anObject.willRead();
}
EOEditingContext ec = anObject.editingContext();
EOGlobalID globalID = ec.globalIDForObject(anObject);
// NB. objects with temporary globalIDs legitimately have no
DB snapshots, since these are by definition not yet saved to the
database.
if (! globalID.isTemporary())
{
// Find the EODatabaseContext instance associated with
anObject, or null if no databaseContext association can be found.
EOObjectStoreCoordinator rootStore =
(EOObjectStoreCoordinator)ec.rootObjectStore();
rootStore.lock();
try
{
EODatabaseContext dbContext =
(EODatabaseContext)rootStore.objectStoreForObject(anObject);
if (dbContext == null)
{
throw new RuntimeException(Could not locate
EODatabaseContext for object ' +
anObject + ' in
editingContext: + ec);
}
isDummyFaultEO =
(dbContext.snapshotForGlobalID(globalID) == null);
}
finally
{
rootStore.unlock();
}
}
return isDummyFaultEO;
}
Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve
specific problems.
http://www.global-village.net/products/practical_webobjects
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/peter%40etechgroup.com.au
This email sent to [EMAIL PROTECTED]
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Peter
Re: Invalidating objects asynchronously
Dan, interesting question and trouble ... I have no clear answer but question that i hope may help you, see below. Le 2 sept. 08 à 18:57, Dan Grec a écrit : Hi everyone, We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. background: We're not using page-based editing contexts. Everything (!) is in the session().defaultEditingContext(). I know, this is bad. We're working hard to fix this for our next release, but for the current release, we need to leave it how it is. We are currently using our own ChangeNotification framework (based on the Wonder implementation from a couple of years ago) primarily because we deploy to J2EE application servers and want to use the internal JMS provider. This implementation listens for objectsChangedInStore notifications from EOF, and then sends out a message over the JMS provider. When the other applications receive the message, a running background thread invalidates any EOs with the global ID that was changed. So, the problem. When we access a relationship on an EO, you take care of fault ? and then Enumerate over the results, how did you acquire the results, the collection ? a change notification can come in from another instance and invalidate some of the objects we're currently Enumerating over. Did you clone the collection before iterating ? Modifying an Iterator while iterating it is not guaranteed to work, your trouble is not a Java Collection issue ? The result is a NullPointerException. Even when we deal with these with something like if (item != null) set.add(item.element()); item may not be null but item.element() may be. What happens if another thread delete/add/refault one of the item, EO of relationship in the same time, in another thread. When reading, acquiring the results, you may have N eos in the collection, but when enumerating any of the EOs in the collection may change and modify the collection you are iterating over ( deleted, added and refaulted ). HTH, Stephane it's still possible (and it happens) that the change notification slips in between the two lines. So, one solution is to make change notification asynchronous, so they only happen when we're not currently serving a request. The basic gist would be to check in application.handleRequest and sleep the thread if we are currently dealing with change notifications. Once all the notifications have been dealt with, we can go on with serving requests. Change notifications would never be processed while we are serving a request. I don't particularly like the above solution, but we need something, and I've got nothing better. (hopefully we can remove it when we move to page based editing contexts) Does anyone see any problems with the above idea? Thanks, -Dan ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/ stephane.guyot11%40wanadoo.fr This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
Interesting problem, my solution to the problem is: don't. Keeping multiple data stores in sync is a pain. It is in WO as is in in any other distributed environment - for example database clusters. The challenges are so numerous that i consider sharding a better solution for load balancing. In the WO case, if we need fresh data we refresh them from the database or use optimistic locking when it becomes vital that state in the DB is the same as the state we operated on in the application. Cheers Tonny On Wed, Sep 3, 2008 at 6:56 AM, Stephane Guyot [EMAIL PROTECTED]wrote: Dan, interesting question and trouble ... I have no clear answer but question that i hope may help you, see below. Le 2 sept. 08 à 18:57, Dan Grec a écrit : Hi everyone, We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. background: We're not using page-based editing contexts. Everything (!) is in the session().defaultEditingContext(). I know, this is bad. We're working hard to fix this for our next release, but for the current release, we need to leave it how it is. We are currently using our own ChangeNotification framework (based on the Wonder implementation from a couple of years ago) primarily because we deploy to J2EE application servers and want to use the internal JMS provider. This implementation listens for objectsChangedInStore notifications from EOF, and then sends out a message over the JMS provider. When the other applications receive the message, a running background thread invalidates any EOs with the global ID that was changed. So, the problem. When we access a relationship on an EO, you take care of fault ? and then Enumerate over the results, how did you acquire the results, the collection ? a change notification can come in from another instance and invalidate some of the objects we're currently Enumerating over. Did you clone the collection before iterating ? Modifying an Iterator while iterating it is not guaranteed to work, your trouble is not a Java Collection issue ? The result is a NullPointerException. Even when we deal with these with something like if (item != null) set.add(item.element()); item may not be null but item.element() may be. What happens if another thread delete/add/refault one of the item, EO of relationship in the same time, in another thread. When reading, acquiring the results, you may have N eos in the collection, but when enumerating any of the EOs in the collection may change and modify the collection you are iterating over ( deleted, added and refaulted ). HTH, Stephane it's still possible (and it happens) that the change notification slips in between the two lines. So, one solution is to make change notification asynchronous, so they only happen when we're not currently serving a request. The basic gist would be to check in application.handleRequest and sleep the thread if we are currently dealing with change notifications. Once all the notifications have been dealt with, we can go on with serving requests. Change notifications would never be processed while we are serving a request. I don't particularly like the above solution, but we need something, and I've got nothing better. (hopefully we can remove it when we move to page based editing contexts) Does anyone see any problems with the above idea? Thanks, -Dan ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/stephane.guyot11%40wanadoo.fr This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/mezz.dk%40gmail.com This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
Tonny, I totaly agree with you, simple question but the answer is not obvious. Other people are trying to solve this issue. http://www.jboss.org/file-access/default/members/jbosscache/freezone/ docs/2.2.0.GA/userguide_en/html_single/index.html#d0e2025 Invalidation : http://www.jboss.org/file-access/default/members/jbosscache/freezone/ docs/2.2.0.GA/userguide_en/html_single/index.html#d0e2206 Stephane Le 3 sept. 08 à 08:24, Tonny Staunsbrink a écrit : Interesting problem, my solution to the problem is: don't. Keeping multiple data stores in sync is a pain. It is in WO as is in in any other distributed environment - for example database clusters. The challenges are so numerous that i consider sharding a better solution for load balancing. In the WO case, if we need fresh data we refresh them from the database or use optimistic locking when it becomes vital that state in the DB is the same as the state we operated on in the application. Cheers Tonny On Wed, Sep 3, 2008 at 6:56 AM, Stephane Guyot [EMAIL PROTECTED] wrote: Dan, interesting question and trouble ... I have no clear answer but question that i hope may help you, see below. Le 2 sept. 08 à 18:57, Dan Grec a écrit : Hi everyone, We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. background: We're not using page-based editing contexts. Everything (!) is in the session().defaultEditingContext(). I know, this is bad. We're working hard to fix this for our next release, but for the current release, we need to leave it how it is. We are currently using our own ChangeNotification framework (based on the Wonder implementation from a couple of years ago) primarily because we deploy to J2EE application servers and want to use the internal JMS provider. This implementation listens for objectsChangedInStore notifications from EOF, and then sends out a message over the JMS provider. When the other applications receive the message, a running background thread invalidates any EOs with the global ID that was changed. So, the problem. When we access a relationship on an EO, you take care of fault ? and then Enumerate over the results, how did you acquire the results, the collection ? a change notification can come in from another instance and invalidate some of the objects we're currently Enumerating over. Did you clone the collection before iterating ? Modifying an Iterator while iterating it is not guaranteed to work, your trouble is not a Java Collection issue ? The result is a NullPointerException. Even when we deal with these with something like if (item != null) set.add(item.element()); item may not be null but item.element() may be. What happens if another thread delete/add/refault one of the item, EO of relationship in the same time, in another thread. When reading, acquiring the results, you may have N eos in the collection, but when enumerating any of the EOs in the collection may change and modify the collection you are iterating over ( deleted, added and refaulted ). HTH, Stephane it's still possible (and it happens) that the change notification slips in between the two lines. So, one solution is to make change notification asynchronous, so they only happen when we're not currently serving a request. The basic gist would be to check in application.handleRequest and sleep the thread if we are currently dealing with change notifications. Once all the notifications have been dealt with, we can go on with serving requests. Change notifications would never be processed while we are serving a request. I don't particularly like the above solution, but we need something, and I've got nothing better. (hopefully we can remove it when we move to page based editing contexts) Does anyone see any problems with the above idea? Thanks, -Dan ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/ stephane.guyot11%40wanadoo.fr This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/mezz.dk% 40gmail.com This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Invalidating objects asynchronously + Terracotta + Groovy
Hi list, i'm a little bit in trouble. I post a mail at 10:16:01 and 5 hours later , nothing on the list ? I decide to re-send it Stephane Tonny, the question is so interesting http://www.terracotta.org/web/display/orgsite/What+Is+Terracotta If anybody has experience with this product , technology ? Imagine if one day WebObjects rely on this kind of technology. :-) Dreaming is not forbidden ! Did you know that in Groovy today , you can use the Objective-C Category, but this time in Java. ? http://docs.codehaus.org/display/GROOVY/Groovy+Categories Many years ago it was a dream, ( and a nigthmare when switching from Objective-C to Java ). It would be nice, if some day we would be able to modify java/ groovy code of a component without the needs to recompile all the application before launching, you can do that with Groovy. ( like the in old webscript day :-) ) cheers, Stephane ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously + Terracotta + Groovy
Have you looked at Wonder's ERXObjectStoreCoordinatorPool and ERJGroupsSynchronizer? I think that's a better approach for the EOF side. tb On Sep 3, 2008, at 9:12 AM, Stephane Guyot wrote: the question is so interesting http://www.terracotta.org/web/display/orgsite/What+Is+Terracotta If anybody has experience with this product , technology ? Imagine if one day WebObjects rely on this kind of technology. :-) ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
On Sep 2, 2008, at 1:55 PM, Sherry Tirko wrote: We are seeing some similar issues and are also using session().defaultEditingContext(). Can someone tell me what the problems are with using the default editing context? Unless you are doing some sort of change notifications or abusing EOF, using the default EC will not cause exceptions. So, what is wrong with using the default editing context for everything? - if the data in the EC gets trashed, the user’s entire session become useless. Unsaved changes and validation problems from one page need to be cleared before something can be saved on a different page. This is possible, but difficult to do consistently. - unless you do a lot of refreshing fetches, the data the user sees can be as old as (or older than) their session - (this is more of a historical problem) large amounts of data fetching into the default editing context will stay around much longer than if each session creates many editing contexts The best practice is to reserve the default editing context for read only data that applies to the scope of the entire session. Chuck On Sep 2, 2008, at 12:57 PM, Dan Grec wrote: Hi everyone, We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. background: We're not using page-based editing contexts. Everything (!) is in the session().defaultEditingContext(). I know, this is bad. We're working hard to fix this for our next release, but for the current release, we need to leave it how it is. We are currently using our own ChangeNotification framework (based on the Wonder implementation from a couple of years ago) primarily because we deploy to J2EE application servers and want to use the internal JMS provider. This implementation listens for objectsChangedInStore notifications from EOF, and then sends out a message over the JMS provider. When the other applications receive the message, a running background thread invalidates any EOs with the global ID that was changed. So, the problem. When we access a relationship on an EO, and then Enumerate over the results, a change notification can come in from another instance and invalidate some of the objects we're currently Enumerating over. The result is a NullPointerException. Even when we deal with these with something like if (item != null) set.add(item.element()); it's still possible (and it happens) that the change notification slips in between the two lines. So, one solution is to make change notification asynchronous, so they only happen when we're not currently serving a request. The basic gist would be to check in application.handleRequest and sleep the thread if we are currently dealing with change notifications. Once all the notifications have been dealt with, we can go on with serving requests. Change notifications would never be processed while we are serving a request. I don't particularly like the above solution, but we need something, and I've got nothing better. (hopefully we can remove it when we move to page based editing contexts) Does anyone see any problems with the above idea? Thanks, -Dan ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/slk24%40outreach.psu.edu This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village.net This email sent to [EMAIL PROTECTED] -- Chuck Hill Senior Consultant / VP Development Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems. http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. Yes, with an asterisk. As Travis said, use Project Wonder and the ERXRemoteSynchronizer with JGroups support ... The JMS change notification framework is known to have problems under load. I have tested the remote synchronizer under VERY heavy load tests (500-1000 insanely clicking updating multiple instances) and verified for accuracy of results (accuracy defined as final state in all instances matches the database). If JMS is invalidating objects, it's going to be screwed -- you CANNOT invalidate and expect a valid graph state ... We take a performance hit, but in the case where an EO is faulted into the snapshot cache and then modified in a remote instance, we force a refresh from the DB. There is really almost no other way to end up in a safe state (I tried LOTS of approaches -- I rewrote this entire process in 3 different ways until I found all the crazy cases that work). We also properly update inverse relationships (which every other change notification system that I've seen does incorrectly). I'm not saying you can't end up in a potentially wonky state, because this ultimately is impossible to get 100% right, but I've tested all the other popular ones and I can tell you for a 100% fact that they get it wrong. You WILL get deadlocks and you WILL get incorrect graph state. Wonder's is the only one that has was able to get through my butt-wuppin' load test and survive, and I've been using this in a multiple instance application in production for almost 2 years without any graph state exceptions. As far as sharding your EO's, I definitely agree ... If your data lends itself to sharding, then this is far more ideal (and dependable ... if you get it right). However, there are some cases where you have globally collaborative data that does not lend itself to sharding, or even if it does, you may want to have a shard on multiple instances, which puts you back where you started. There are also often applications that require users to cross shards, which means you have to have a system for dealing with redirecting the user to another instance mid-flight. All completely possible to do, but none of these solutions are trivial (but then we wouldn't have jobs if they were :) ). ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Invalidating objects asynchronously
Hi everyone, We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. background: We're not using page-based editing contexts. Everything (!) is in the session().defaultEditingContext(). I know, this is bad. We're working hard to fix this for our next release, but for the current release, we need to leave it how it is. We are currently using our own ChangeNotification framework (based on the Wonder implementation from a couple of years ago) primarily because we deploy to J2EE application servers and want to use the internal JMS provider. This implementation listens for objectsChangedInStore notifications from EOF, and then sends out a message over the JMS provider. When the other applications receive the message, a running background thread invalidates any EOs with the global ID that was changed. So, the problem. When we access a relationship on an EO, and then Enumerate over the results, a change notification can come in from another instance and invalidate some of the objects we're currently Enumerating over. The result is a NullPointerException. Even when we deal with these with something like if (item != null) set.add(item.element()); it's still possible (and it happens) that the change notification slips in between the two lines. So, one solution is to make change notification asynchronous, so they only happen when we're not currently serving a request. The basic gist would be to check in application.handleRequest and sleep the thread if we are currently dealing with change notifications. Once all the notifications have been dealt with, we can go on with serving requests. Change notifications would never be processed while we are serving a request. I don't particularly like the above solution, but we need something, and I've got nothing better. (hopefully we can remove it when we move to page based editing contexts) Does anyone see any problems with the above idea? Thanks, -Dan ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Invalidating objects asynchronously
We are seeing some similar issues and are also using session().defaultEditingContext(). Can someone tell me what the problems are with using the default editing context? Thanks, Sherry On Sep 2, 2008, at 12:57 PM, Dan Grec wrote: Hi everyone, We're having an interesting problem in our apps, and I wanted to see if anyone else has solved it. background: We're not using page-based editing contexts. Everything (!) is in the session().defaultEditingContext(). I know, this is bad. We're working hard to fix this for our next release, but for the current release, we need to leave it how it is. We are currently using our own ChangeNotification framework (based on the Wonder implementation from a couple of years ago) primarily because we deploy to J2EE application servers and want to use the internal JMS provider. This implementation listens for objectsChangedInStore notifications from EOF, and then sends out a message over the JMS provider. When the other applications receive the message, a running background thread invalidates any EOs with the global ID that was changed. So, the problem. When we access a relationship on an EO, and then Enumerate over the results, a change notification can come in from another instance and invalidate some of the objects we're currently Enumerating over. The result is a NullPointerException. Even when we deal with these with something like if (item != null) set.add(item.element()); it's still possible (and it happens) that the change notification slips in between the two lines. So, one solution is to make change notification asynchronous, so they only happen when we're not currently serving a request. The basic gist would be to check in application.handleRequest and sleep the thread if we are currently dealing with change notifications. Once all the notifications have been dealt with, we can go on with serving requests. Change notifications would never be processed while we are serving a request. I don't particularly like the above solution, but we need something, and I've got nothing better. (hopefully we can remove it when we move to page based editing contexts) Does anyone see any problems with the above idea? Thanks, -Dan ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/slk24%40outreach.psu.edu This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Invalidating objects in change notification framework
Hi, I'm currently taking a look on Wonder's ERChangeNotificationJMS framework, as it is causing deadlocks sometimes (often). The framework currently sends around snapshots via JMS and replaces them at the EODatabase level in the recieving instance: EODatabase database; EOGlobalID globalID; NSDictionary snapshot; ... database.forgetSnapshotForGlobalID(globalID); database.recordSnapshotForGlobalID(snapshot, globalID); I guess that this is taken from the father of all change notification frameworks by D. Neumann, as almost the same lines can be found in the WHChangeNotification framework. Several people reported, that everything works a lot better, if no snapshots but just globalIDs are used. Now I can see several ways to invalidate the data once the GIDs are received and I'm wondering, what's the best way to do so: 1. database.forgetSnapshotForGlobalID(globalID); or 2. database.recordSnapshotForGlobalID(null, globalID); or 3. ec.invalidateObjectsWithGlobalIDs(globalIDs); or 4. ec.rootObjectStore().invalidateObjectsWithGlobalIDs(globalIDs); Or a combination of the above ones? Or a completely different approach? Am I right, that 1+2 would work across different EOF stacks within one instance but 3+4 would only work for one? Any hints would be very welcome. Timo ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
