I agree on needing to be able to run without a db.
If you think about it, a db is just a highly specialised node in the cluster. It is much simpler, and therefore maintainable, if all nodes in a Geronimo cluster are homogeneous. We can't lose the db your business data lives in, but if we can avoid adding another for session replication it might be of advantage.
To this end, my design should work without a db. You just tune it so that passivation never occurs - i.e. unlimited active sessions. You trade of in-vm space against db space.
Likewise, if you were an advocate of the 'shared store' approach, you should be able to constrain the web container to keep '0' active sessions in memory, but passivate everything immediately to the db.
So, yes, I'm sure we are on the same page.
Jules
Bhagwat, Hrishikesh wrote:
hi Jules,
I am happy that we are converging, in that, the following approach does very well cover some of the main goal that i was trying to achieve when i wrote my first proposal. I have marked those points below as [hb-X] with brief comments.
Thus keeping the "Goal of the Architecture" the same I would like to propose a new
scheme for "replication". While initially, I was keen on having NO DATA EXCHANGE BETWEEN SERVERS but rather on having them all just use the shared store to persist and retrieve session
data, you were interested in the contrary. Your approach (mentioned in section 2 of http://wiki.codehaus.org/geronimo/Architecture/Clustering) is about Session object exchange
between servers which are clustered and about clusters that are partitioned (stat/dynamically).
I think the discussion there stops with some issues that you think that may arise with that
kind of a scheme.
After Vivek Biswas first pointed out to me, I have been feeling increasing uncomfortable with the idea of
Geronimo being DEPENDENT on an EXTERNAL system like a Database for implementing its web clustering.
Some ppl on the mailing list have spoken about performanced issues with the DB approach but i think with techniques like asny-write-to-DB etc such problems can be circumvented. Thus though I still believe that its a solution that can solve the problem, I am not comfortable with the usage of an external system to aid in clustering. I dont know of any GOOD (hi-av) DBs that comes for FREE. This
mean that even if Geronimo is available for FREE it cant be used without purchasing a DB. The solution
as a whole, is then, not truely FREE-of-cost.
I have been since then looking at many alternatives ... like the Jetty implemented solution of having all m/c in one cluster to a 1Primary:1Secondary solution.
Presenly I have something cooking. As I come close to working on details I find it quite similar to your original solution of dynamic patitioning. This is what again convinces me that we seem to converge.
my porposal in to exchange note and to jointly come up with a detailed design.
Do let me know your thoughts on that .... I am tring to complete a document on this new schema ASAP and will mail you the same.
thanks - hb
-----Original Message----- From: Jules Gosnell [mailto:[EMAIL PROTECTED] Sent: Thursday, October 23, 2003 3:44 AM To: [EMAIL PROTECTED] Subject: Re: [Re] Web Clustering : Stick Sessions with Shared Store - current state of play.
Guys,
since this topic has come up again, I thought this would a useful point to braindump my current ideas for comment and as a common point of reference...
Here goes :
Each session has one 'primary' node and n 'replicant' nodes associated with it.
Sticky load-balancing is a hard requirement.
Changes to the session may only occur on the primary node.
[hb-1] i always intended only the OWNER NODE (remember ... i wasn't keen on having Primary/Sec. concept so i am using this term)to make changes to the SESSION. Also
only ON CHANGE the session would be sent for persistance.
Such changes are then replicated (possibly asynchronously, depending on data integrity requirements) to the replicant nodes.
[hb-2] My approach was about NOT USING REPLICATION AT ALL but using a shared data store. Though I never mentioned it explicitly (bocz I regarded this as a finer detail) ... i always intended on having an async sys do the persistence.
If, for any reason, a session is required to 'migrate' to another node (fail-over or clusterwide-state-balancing), this 'target' node makes a request to the cluster for this session, the current 'source' node handshakes and the migration ensues, after which the target node is promoted to primary status.
[hb-3] I am not sure how a target server can initiate a migration process
I would imagine that on a fail-over or on a systematic-removal of a node or any other such action that requires a cluster wide state-balancing, it is
the lb/adminS that would sense this first and initiate actions.
Any inbound request landing on a node that is not primary for the required session results in a forward/redirect of the request to it's current primary, or a migration of the session to the receiving node and it's promotion to primary.
[hb-4] Not sure when this scenario would occur when a "secondary" would receieve a HTTP request even when the primary is functionning well.
A shared store is used to passivate sessions that have been inactive for a given period, or are surplus to constraints on a node's session cache size.
[hb-5] yes this is very much a point that I have been saying since my first proposal. Just to quote from that doc. "With a little intelligence built in an MS can store away, less busy sessions to DB and retrieve them when needed thus offering something that is near to "virtually unlimited amount of sessions (section 1.1.1.1)"
Once in the shared store, a session is disassociated from it's primary and replicant nodes. Any node in the cluster, receiving a relevant request, may load the session, become it's primary and choose replicant nodes for it. [hb-6] this is a good optimization to sit on the scheme mentioned in [hb-5]
Correct tuning of this feature, in a situation where frequent migration is taking place, might cut this dramatically.
The reason for the hard node-level session affinity requirement is to ensure maximum cache hits in e.g. the business tier. If a web session is interacting with cached resources that are not explicitly tied to it (and so could be associated with the same replicant nodes), the only way to ensure that subsequent uses of this session hit resources in these caches is to ensure that these occur on the same node as the cache - i.e. the session's primary node.
By only having one node that can write to a session, we remove the possibility of concurrent writes occurring on different nodes and the subsequent complexity of deciding how to merge them.
[hb-7] I complete agree
The above strategy will work for a 'implicit-affinity' lb (e.g. BigIP), which remembers the last node that a session was successfully accessed on and rolls this value forward as and when it has to fail-over to a new node. We should be able to migrate sessions forward to the next node picked by the lb, underneath it, keeping the two in sync.
With an 'explicit-affinity' lb (e.g. mod_jk), where routing info is
actually encoded into the jsessionid/JSESSIONID value (or maybe an
auxiliary path param or cookie), it should be possible, in the case of
fail-over, to choose a (probably) replicant node to promote to primary and to stick
requests falling elsewhere to this new primary by resetting this
routing info on their jsessionid/JSESSIONID and redirecting/forwarding
them to it.
If, in the future, we write/enhance an lb to be Geronimo-aware, we can
be even smarter in the case of fail-over and just ask the cluster to
choose a (probably) replicant node to promote to primary and then direct requests
directly to this node.
The cluster should dynamically inform the lb about joining/leaving nodes, and sessions should likewise maintain their primary/replicant lists accordingly.
[hb-8] I complete agree
LBs also need to be kept up to date with the locations and access points of the various webapps deployed around the cluster, relevant node and webapp stats (on which to base balancing decisions), etc...
All of this information should be available to any member of the cluster and a Geronimo-aware lb should be a full cluster member.
On shutting down every node in the cluster all session state should end up in the shared store.
These are fairly broad brushstrokes, but they have been placed after some thought and outline the sort of picture that I would like to see.
Your thoughts ?
Jules
Bhagwat, Hrishikesh wrote:
I am also not convinced it reduces the amount of net traffic. After eachI agree. However, this is based on the assumption that only one unicast write is required. In other words, this is a primary/secondary topology. I think that hd did not intended such a topology and hence his statement.
request the MS must write to the shared store, which is the same traffic as
a unicast write to another node or a multicast write to the partition
(discounting the processing power needed to receive the message).
[hb] Yes i was not assuming a Pri/Sec design but a layout where any active server can be request to pick up a client request which is destined to server that has just failed
-----Original Message----- From: gianny DAMOUR [mailto:[EMAIL PROTECTED] Sent: Sunday, October 19, 2003 7:35 AM To: [EMAIL PROTECTED] Subject: [Re] Web Clustering : Stick Sessions with Shared Store
Jeremy Boynes wrote:
However, as Andy says, the cost of storing a serialized object in a BLOB is
significant. Other forms of shared store are available though which may
offer better performance (e.g. a hi-av NFS server).
Do we need a shared repository or a replicated repository?
The issue I have with hb's approach is the reliance on an Admin Server, ofI agree. It seems that in such a design an Admin Server is only used to route incoming requests to the relevant node.
which there would need to be at least two and they would need to co-operate
between themselves and with any load-balancers. I think this can be handled
by the regular servers themselves just as efficiently.
However, I do not believe that regular servers can do this job. I assume that they will implement a standard peer-to-peer cluster topology to provide redundancies, however I do not see how they can handle the dispatch of incoming requests.
This feature seems to be either a client or a proxy one: I mean it should be done prior to reach the nodes.
For instance, this feature is treated on the client-side via a stub aware of the available nodes in WebLogic. It seems that JBoss (correct me if I am wrong) has also followed this design.
I am also not convinced it reduces the amount of net traffic. After eachI agree. However, this is based on the assumption that only one unicast write is required. In other words, this is a primary/secondary topology. I think that hd did not intended such a topology and hence his statement.
request the MS must write to the shared store, which is the same traffic as
a unicast write to another node or a multicast write to the partition
(discounting the processing power needed to receive the message).
Gianny
_________________________________________________________________
MSN Search, le moteur de recherche qui pense comme vous ! http://search.msn.fr/
-- /************************************* * Jules Gosnell * Partner * Core Developers Network (Europe) * http://www.coredevelopers.net *************************************/
