Really cool discussion/solution, did this work May Charles ?
[]s Flavio. ----- Original Message ----- From: "Juan Pablo Lorandi" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, January 28, 2004 12:37 PM Subject: Re: Multithreading batch processes/Session bean calls within cloned Application Server environment > Regarding the lack of use of local interfaces, you're correct. This is > still an EJB 1.1 application being hosted by WebSphere Application > Server 4.0. "Cloning" here certainly isn't Java cloning. What I mean is > that the workload manager relays EJB remote method calls to the various > "clones" (separate boxes running the same collocated EAR) in a > configurable manner -- for instance "Round Robin" style where the first > request is relayed to machine A, then the next request is relayed to > machine B, and requests alternate between A and B. If either A or B goes > down, then the workload manager relays all requests to the remaining > machine. > > JPL> Ahhhh EJB 1.1. I had to pull tricks similar to those you're doing > now. You will have to implement the load balancing on your own. That > also means that for each "task" sent to a different server you'll have > to create a new InitialContext passing in a Hashtable to config it, as > if the daemon servlet was a stand-alone application client. Now, my > question is whether you could batch *some* of the calls to processId(id) > so that the RMI and Transaction spawning costs can be diminished (say, > 10 at a time?). > > Regarding the lack of "one big" transaction wrapping the entire > iteration process: we had to move the iteration outside EJB transaction > scope because the "one big" transaction was timing out on us when there > were many items to process. Altering the transaction timeout value was > not an option at the time when we first encountered the problem (tightly > controlled by another team). So we made the transactions more granular, > one per iterate. I realize this makes the overall iteration take longer > but at least it has a chance to finish. > > JPL> I understand the problem with the time-outs, and I suggest you > leave the value untouched. In fact this is precisely what made me code > the daemon servlet in EJB 1.1 in the first place. Nevertheless, you'll > need to try and run a transaction per a number of tasks. > > > JPL> Here's another take on your problem: use a DB table as a queue: > > 1) Have 1 daemon servlet in one single server which determines WHEN to > run a batch, and collects all those ids. As an initialization parameter, > the "machine-id" list should be passed to the scheduler servlet > 2) When a batch should be run, the "task pool" should be constituted by > inserting records in a DB table. Records will have a field called > "machine" or something, which would be added in a round robin fashion or > a weighted round robin fashion. > 3) Have 1 daemon servlet per container, this one is the worker servlet. > As an init param pass in the machine's id. > 4) Worker servlet read the tasks from a recordset for a certain machine, > get one record, then process it. > 5) Worker servlet eliminates records from the queue. > > This kind of operation insures that if a problem ocurrs (such as 1 > server goes down) the batch will be completed. > > HTH, > > JP > -----Original Message----- > From: Juan Pablo Lorandi [mailto:[EMAIL PROTECTED] > Sent: Tue 1/27/2004 8:00 PM > To: > Cc: > Subject: Re: Multithreading batch processes/Session bean calls within > cloned Application Server environment > > > I have the details mocked up for a code simulation to multithread our > EJB-based batch processing, and I have some questions. See the last two > paragraphs for the context driving this problem. The simulation I've > developed does not use EJBs, but addresses most of the multithreading > issues. Here are my questions: > If a servlet in a cloned Application Server environment spawns threads > (note the security authentication issues are already solved) and the > multiple threads include Session Bean calls, will the Session Bean calls > be workload-managed? For example, in a cloned application server with > two servers - machine A and machine B, if I spawn eight threads in a > servlet call on machine A, can we ensure that some of the Session Bean > calls made inside those threads will be executed on machine B, as well > as some executed on machine A? > If the above can't happen, then how do we accomplish multiple threads > operating in the application server where each thread includes Session > Bean calls and the calls are workload-managed across all application > server clones? Would an application client be able to accomplish the > same thing? > > JPL> This depends greatly on what you understand by cloning. Obviously, > you're leaving Local interfaces out of this. > > Background: Context driving the problem: > We have been considering different ways to speed up our EJB-based batch > processes. The current sequence of events is: > Servlet method calls a Session Bean method, call it batch(), which has > transaction attribute "Not Supported". > batch() makes a read-only JDBC call to assemble a List of ids of items > to process > batch() iterates through the ids singly. For each id, calls another > Session Bean method, call it processId(id), which has transaction > attribute "Required" > processId(id) loads Entity Beans and processes them (each entity bean's > methods have transaction attribute "Mandatory"), committing an EJB > transaction > > JPL> You make a mistake here; the whole iteration should spawn 1 single > transaction. You can have the read-only JDBC call out of the > transaction, but the whole iteration should be run under a single > transaction, if possible. Otherwise you're paying the price of starting > and commiting a transaction for each id. > > JPL> How about this solution roughly based on your approach: > > 1) Servlet method makes a read-only JDBC call to assemble a List of ids > of items to process, and contains these in a "task pool" > 2) Servlet method sends JMS messages to a number of servers in a round > robin fashion from a list loaded by a Properties object (from a file or > a DB or whatever you want), or if the JMS service is clustered, to just > one server. The message is the description of a mini-batch/task job to > be performed. > 3) Each time an MDB receives a task, it calls processId(id) on a Session > Bean > > HTH, > > JP > > PS: Do you mind not using HTML for the messages? It makes them quite > difficult to answer... > > ======================================================================== > === > To unsubscribe, send email to [EMAIL PROTECTED] and include in the > body > of the message "signoff EJB-INTEREST". For general help, send email to > [EMAIL PROTECTED] and include in the body of the message "help". > > The information contained in this e-mail may be confidential and is > intended solely for the use of the named addressee. > Access, copying or re-use of the e-mail or any information contained > therein by any other person is not authorized. > If you are not the intended recipient please notify us immediately by > returning the e-mail to the originator.(B) > ======================================================================== > === To unsubscribe, send email to [EMAIL PROTECTED] and include in > the body of the message "signoff EJB-INTEREST". For general help, send > email to [EMAIL PROTECTED] and include in the body of the message > "help". > > =========================================================================== > To unsubscribe, send email to [EMAIL PROTECTED] and include in the body > of the message "signoff EJB-INTEREST". For general help, send email to > [EMAIL PROTECTED] and include in the body of the message "help". > --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.581 / Virus Database: 368 - Release Date: 09-Feb-04 =========================================================================== To unsubscribe, send email to [EMAIL PROTECTED] and include in the body of the message "signoff EJB-INTEREST". For general help, send email to [EMAIL PROTECTED] and include in the body of the message "help".
