Pier, Tom, cool, the discussion is starting to become interesting. :-)
comments below: ----- Original Message ----- From: "Pier Fumagalli" <[EMAIL PROTECTED]> To: "Tomcat Developers List" <[EMAIL PROTECTED]> Sent: Tuesday, November 13, 2001 3:04 AM Subject: Re: Tomcat: Distributed Session Management revisited > On 13/11/2001 12:54 am, "Tom Drake" <[EMAIL PROTECTED]> wrote: > > > Mika: > > > > Thanks for the reply. Here's some more thoughts on this subject. > > > > The primary problem that I see with the collaborative method > > (e.g. extending the multicast solution) is > > that all sessions will have to be sent to all cluster nodes. The > > number session updates that have to travel 'on the wire' is in > > relation to the number of nodes in the cluster. > > Linear growth, that's the best we can aim for... > > > Further more, when a new tomcat is brought on-line, it must > > somehow retrieve a copy of all active sessions from somewhere. > > There is nothing in place for this currently. Using multicast > > is problematic. If a multicast request is made then all other nodes > > would respond with all sessions. So, some other approach would > > need to be taken which would result in two protocols being used > > to make this feature work. This seems too complicated. > > Not "that" complicated. Most of the work on elective processes has been done > already in the scope of other projects, so, we would only need to adapt it > to our scope... I agree with Pier, in my view that's separating the "application layer" (content) from the transportation control layer (where, how). > > > --------------------------------------- > > Consider this scenario: > > > > A user establishes a session on node 1 (of a 10 node cluster), > > Tomcat would create a new session and transmit it to the > > multicast port, which would then transmit 10 copies of this > > session (1 to each cluster node). > > Now suppose that the next request from this user is sent to > > node 2, which causes an update to the session to occur. Again > > 11 copies of the Session are transferred. > > [...] > > NOTE: remember this is UDP traffic. The more packets that > > fly around, the greater the likely-hood of dropping packets. > > Dropped packets in this case means that some tomcat > > instances may have stale (or no) data for a given session. > > Indeed... Quite huge... Yes, multicast udp should only be used to autoconfigure the cluster (who's there, who's taking sessions etc.), which should also be configurable for non-multicast-environments. In that case lists of adresses are used to select who's the next to take over. In fact, if all node's are holding information about the peers, we don't need to have long lists. An upcoming node would need only one already configured node to ask the cluster's spread via TCP. It's join could be communicated via daisy-chain. (one message per member is linear). > > > ------------------------------------------ > > With a centralized session manager the following traffic would > > occur instead: > > > > node1 sends new session to server manager > > node 2 requests the given (session id) session from the server manager > > manager sends a copy of the session to node 2 > > node 2 updates the session and sends it back to the manager. > > manager sends the 'invalidateSession(sessionId)' method in each of nodes. > > (note: invalidateSession only contains the value of 'SessionId' + any > > additional > > RMI overhead. This is far smaller than a complete Session object) > > > > The number of session copies sent as the result of an update is 2. > > This number does not depend or vary based on the number of nodes. > > > > Now, let's add to the story. Let's say that Tomcat is smart enough to cache > > Session objects in it's memory space. Once Tomcat gets its hands on a > > 'Session' > > it keeps it until it becomes 'too old' or an 'invalidateSession(sessionId)' > > message is > > received from the remote Session Manager. This could cut down the the number > > of transfers of Session data from 2 to somewhere between 1 and 2. > > Yes, but in this case, we don't have redundancy of sessions... So, if the > Tomcat which has the session dies, the whole session dies with him... > > > ----------------------------------------------------- > > On Redundant Session Managers. > > > > There are a couple ways to achieve this. One way is to place two Session > > Managers in the network. One of them is the 'active' one, the other one could > > simply register itself as a client of the 'active' server. As a client, it can > > obtain copies of all new and changed sessions from the active server. If for > > some reason the active server needs to be brought down, it will send a message > > to all of it's clients (including the 'dormant' session manager) indicating > > that it's shutting down. The clients could, on receipt of this message, > > connect to the 'next' session server (in their pre-configured list of > > servers). The clients could simply carry on with the new server. > > Indeed... > > > If the active server simply goes off the air for some mysterious reason. The > > clients would get a RemoteException the next time they tried to talk to the > > server. This would be their clue to 'cut-over' to the other server (as > > described above). > > But how would they know where the sessions ended up???? > > > Last point. Sending Session delta's instead of the entire Session: > > > > This should be doable. The main thing that we care about are Session > > attributes which are changed by the application. It's up to the > > web-application to replace these values into the Session if their contents > > change. This is enough for us to be able to track which attributes have > > actually changed. > > This can actually be done if we consider every operation on a session > (adding/replacing/removing an attribute) and atomic operation.... > > Let's see if I can complicate things a little bit :) (Love doing that). > > Let's imagine to have a pool of session managers (SA, SB, SC...) and a pool > of servlet containers (T1, T2, T3...). > > The first thing we want to do is bring up our session managers. Once we > start them SA, SB, SC and SD are available to accept sessions. > > Then we start our servlet containers T1, T2, T3 and T4. When a request comes > in in any of the servlet containers, the servlet container simply broadcasts > a message saying "who can hold a session for me?"? All four managers will > reply to that request, and the servlet manager can "order" them. For > example, if we want a redundancy level of 2, the container might choose SB > as the "primary" session manager, and SA as the "replica 1" session manager; > if we want a redundancy level of 3, the container might choose SD as > "primary", SA as "replica 1" and SB as "replica 2". > > The information about "who is primary" and "who is replica X" is stored > within the session manager itself. > > When one of the servlet containers needs to read or write from a session, he > will broadcast (again) the message "who holds this session?", of course, all > session managers holding (primary or replica) that session, will reply with > their "status" (primary, or replica #), and then the servlet container will > persist the data in -first of all- the primary session manager, -then- in > all the replicas, and at the end return control to the caller (the thread > which called "setAttribute/getAttribute"). > > What happens if one of the session managers goes down? That simply the > servlet container will notice that something is going wrong, because if > configured with a "replica factor" of 3, he gets only 2 responses to "who's > holding this session?", we know for sure that one of the replicas (or the > master) has gone down, so, simply, we can "elect" one of the replicas as > "primary" (if the primary has gone down), and/or broadcast a message saying > "who can be replica for this session?"... The session is then persisted in > all three places (the two old ones, plus the new one), and the thing goes > on... > > What does it gives us? A lot of flexibility in terms that only a little data > is broadcasted (messages such as "who can hold this session?" or "who has > this session?" or "who can be replica for this session"), so we avoid > problems with UDP, then we have a sub-linear growth in a way that the > traffic over the network is only (N*(sessiondata+overhead)) where N is the > replica factor, and the administrator is free to trade his own data safety > (more replicas, more traffic, more redundancy), with speed (less replicas, > less traffic, less redundancy)... > > We don't have a single point of failure (whohoooo!), we don't need to > replicate sessions with linear growth on N (where N is the number of session > managers), and we get load balancing of sessions for free... > > The only problem is that we need to use multicast, but that shouldn't be a > big issue... > Yeah, ok, consider as follows: We don't differenciate between servers beeing SessionManager or ServletContainer, instead we act as hermaphrodite. When a new session is about to be created (FancyDistributedFailOverSaveSessionManager.createSession()), it asks "Who's holding that stuff". If he doesn't get an answer, he empowers himself as the primary SessionManager and elects n secondary nodes. (Ahhh yes, as well when a container calls findSession()) Only the primary SC is allowed to change the session, secondaries only listen to changes. No lock distribution, no transaction problem so far. If createsession() get's an answer, we need to decide what to do. On one hand, we could get the primary status over from the original owner (how to do that savely is another question). The original owner could become a secondary or just drop the session at all. On the other hand, we could pass over the processing to the other node. Why transfer all that data to us, if there is a guy who has all that stuff. The latter would restrict us somewhat in load-balancing, but would reduce the traffic enormously and get around locks and transaction problems. Is there a way to do that? I.e. tell mod_webapp/mod_jk "Use that node instead"? Or just proxying to the other node (which would result in passing over the response twice in our network). If the createSession() only get's answers from secondaries, we suddenly realise that we lost a beloved member of our group. Then we get the session from a secondary and make ourself the new numbero uno. (Or pass over to a secondary to become the new leader of the pack). Again no locking problems. But we may be in the middle of a transaction. I can't see a way yet, how to prevent that, as we can't know when a node falls over. I think taking over is better because we have to replicate to another node anyway, so why not ourself? Mika :wq -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
