Hello Johannes, Beware, as this post is a rather lengthy one :-)
> Hi Fabrizio, > > it seems that you alread have a very elaborate solution. I do not > understand some points completely yet, so let me ask some further > questions: > >> Therefore I ran various stress-tests using the 'ab' tool ('apache >> benchmark', included in the httpd distribution), in which several >> thousand requests (with configurable concurrency levels) are launched >> against a Cocoon/Hibernate webapp. >> ... > I am aware of ab and use it alot myself, but just for "one-shot" testing > (massive requests on one, or more, single URLs, but each request being > isolated without sessions) Most of my ab runs are indeed such "one-shot" tests, with each request being isolated, ie. the Hibernate session is opened on an incoming request, and closed by the servlet filter after sending the response to the client. The continuations are terminated as well. This kind of test is only meant to test the server's responsiveness under heavy load. > For testing the "long sessions", wouldn't > you need to craft HTTP requests that open and maintain sessions, span > multiple pages etc.? (think 100 peope working with the form at the same > time) That really depends on the purpose of your tests! If you want to estimate worst-case memory requirements, then you actually do not wish to maintain a same session during the ab tests. In fact it is more interesting to analyze the server's behaviour when continuations are being suspended instead of being terminated, for example after displaying a Cocoon form without the user ever submitting it - as this will keep a lot of objects in memory as part of the suspended continuation. For a long session, this additionnaly includes the Hibernate session object as part of the suspended conversation. >> For the long-session approach, I've modified the HibernateFilter so that >> it will always disconnect the underlying JDBC connection from the >> Hibernate session, thus returning it to Cocoon's connection pool, but >> the Hibernate session is left open (except when a 'end-of-conversation' >> flag is being set) >> > OK. But how do you provide Hibernate with a new connection the next time > you need one? Manually, in Flowscript - just after the continuation is resumed! (...I can hear the design philosophers out there yelling 'blasphemy' in chorus now ;-) Written out verbosely, ie. with all transaction demarcations, a Flowscript CForm handler generally takes the form: form_example() { beginConversation(); // create a new Hibernate session ... // fetch data from model, bind to / initialize form rollbackTransaction(); // disconnect JDBC connection from session showForm(); // display form - this suspends the continuation // and the Hibernate long session // the continuation is resumed here beginTransaction(); // re-associate a JDBC connection to the session ... // apply changes to model (form.save()) commitConversation(); // flush the Hibernate session, commit the // underlying JDBC transaction / close session cocoon.sendPage(); // display response } Note that the rollbackTransaction() and commitConversation() methods only set a flag for the 'HibernateFilter' servlet filter. The latter actually performs the transaction rollback/commit, resp. the Hibernate session disconnect/close operations, when the form or page have been completely rendered and sent to the client. Also note that beginConversation() and beginTransaction() will by default set a 'rollbackConversation' flag. Thus, if an error occurs before reaching the end-of-transaction demarcations, the servlet filter will rollback the database transaction and close the Hibernate long session (...adding a touch of paranoia to the pragmatic ;-) In practice however, the code is a lot less verbose, as I hate noisy code! The transaction demarcations are therefore hidden away in a generic dispatch method and showForm/sendPage wrappers. The above Flowscript CForm handler thus becomes: form_example() { ... // fetch data from model, bind to / initialize form this.showForm(); // wrapper around Cocoon's showForm() ... // apply changes to model (form.save()) this.sendPage(); // wrapper around Cocoon's sendPage() } Who is 'this', you may now ask? Well, the very first thing the almighty dispatch() method does, is to instantiate a local 'webapp' Javascript object that will be associated to the current, newly created continuation. The dispatch() method then invokes webapp.init(), which in turn invokes beginConversation(), which in turn sets webapp's 'hs' attribute. This 'hs' attribute holds a reference to the Hibernate session object for the scope of this continuation. The Hibernate session stored in the hs attribute is passed to the constructor of the POJOs implementing the business model, when they are instantiated by Flowscript. Admittedly, this is not the cleanest solution from a designer's perspective, but so far it appears to work fine with Hibernate long sessions, with only minor changes being required to switch from a similar Hibernate session-per-request to a conversation pattern. In fact, those changes are welcome, as you get rid of the noisy, error-prone constructs dealing with detached objects. >> I've never delved into the Java Transaction API ...I probably will when >> I stumble upon a distributed transaction issue :-/ >> >> > Hmm I do think you have used this. I'm referring to something like > > > openHibernateSession( ); // stores session in global var hs > > var tx = hs.openTransaction(); // this is atomary > > form2medium(form,medium); // store CForms user input into medium > > hs.save( medium ); > > tx.commit(); // close transaction, ensuring that changes are written to DB > > > Depending on your setup, openTransaction() could mean JTA or JDBC > transactions. JDBC transactions in my case! I think JTA will be written in a later chapter :-> >> I've somewhat limited the impact of this Flowscript/Hibernate code mix >> by encapsulating the Hibernate session setup in a generic dispatch >> method >> (invoked by the sitemap) and by using transaction defaults which apply >> to most (over 90%) cases. >> Thus, the only flowscript methods where you will see explicit >> transaction >> demarcation statements are those dealing with CForms. >> > Does that mean your dispatch method always calls "openTransaction()"? Yes! (see code snippets below) >> Opening and closing sessions in DAOs? >> Hmm, here I have to agree with some philosophers out there that do not >> quite agree (see http://www.hibernate.org/328.html) >> >> The bit that troubles me is that you lose transaction atomicity when >> changes to multiple DAOs have to be performed in the same transaction... >> > No, I've just moved the task of "ensure that session is open" to the DAO > level. Transactions are demarked in the DAOs as needed. I'll paste some > example code below which I hope explains it better than my words could, > would be nice to hear your comments. > > Regards, > Johannes > > public class DAO { > public boolean assertSession(){ > return User.getHibernateSession() == null > && User.getHibernateSession().isOpen(); > // "User" is the glue between cocoon and DAO level, see > below > } > > public Session session(){ > return User.getHibernateSession(); > } > } > > public class IconsDAO extends DAO { > public Icon getIcon(int id) { > if (!assertSession()) > return null; // could throw an exception here, or print some > warning > try { > return (Icon) session().load(Icon.class, new Long(id)); > } catch (HibernateException e) { > e.printStackTrace(); > return null; > } > } > public List alleIcons() { > if( !assertSession() ) return null; > try { > return session().createQuery("from Icon i order by id > asc").list(); > } catch (HibernateException e) { > e.printStackTrace(); > return null; > } > } > } > > public class User { > /* This object is local only to the current thread. Since a new > thread is opened > for each request, this allows passing the HibernateSession from > flowscript > (where it is opened) to java (where it is used). It would still > be better to create > the session in the Java layer in the first place. Only reason I'm > not doing this yet > is that I still use the cocoon connection pooling. */ > private static InheritableThreadLocal scope = new > InheritableThreadLocal() { > protected Object initialValue() { > return null; > } > }; > > /* This is called in the servlet filter, see below. */ > public static void init(){ > scope.set( new HashMap() ); > } > > public static Session getHibernateSession(){ > return (Session) scope().get("hibernateSession"); > } > > public static void setHibernateSession(Session hs){ > scope().put("hibernateSession",hs); > } > > /* Null-Safe getter for scope. > * > */ > > private static HashMap scope(){ > if( (HashMap) scope.get() == null ) > scope.set( new HashMap() ); > return (HashMap) scope.get(); > } > } > > /** servlet filter: getters / setters */ > > public void doFilter(ServletRequest request, ServletResponse response, > FilterChain chain) throws IOException, > ServletException { > // Create request-local scope in User Object > User.init(); > > // Pass the request on to cocoon > chain.doFilter(request, response); > > // After cocoon has finished processing, close the > // corresponding Hibernate session if it has been opened > if( User.getHibernateSession() != null ) > { > Session hs = User.getHibernateSession(); > try{ > hs.connection().close(); > hs.close(); > //System.out.println("Session closed."); > } > catch( HibernateException e ){ > e.printStackTrace(); > } > catch( SQLException e ){ > e.printStackTrace(); > } > User.setHibernateSession(null); > } > } Using InheritableThreadLocal for storing the request-local scope is a neat idea! (Why didn't I think of that?) That definitely makes for cleaner code, as this elegantly solves the problem of passing around Hibernate session references! (Note that this is from a pragmatic point of view! As a design philosopher, I would rather have pinpointed your use of DAOs as a neat idea ;-) Still, I don't see from your above code samples how you would demark transactions in your DAOs while preserving transaction atomicity. Consider for instance a deletion form that would display all your icons, with a checkbox next to them (as you have already kindly provided the alleIcons() method ;-) On submission, you want to delete the selection in one single transaction, to make sure that either all selected icons are deleted as a whole, or none at all (if an unexpected error should occur midway!) What approach would you take in your DAOs? Best regards, Fabrizio ====================================================================== Here are some code snippets of my long conversation implementation, with an hypothetical icon deletion form example: ========= sitemap.xmap <map:flow language="javascript"> <map:script src="js/flow.js"/> </map:flow> ... <map:match pattern="form_example"> <map:call function="dispatch"> <map:parameter name="method" value="form_example"/> </map:call> </map:match> ... <!-- handle exceptions --> <map:handle-errors> <map:generate type="notifying"/> <map:transform src="common/error.xslt"/> <map:transform type="jx"/> <map:serialize type="html"/> </map:handle-errors> ========= flow.js // Imports cocoon.load ("js/webapp.js"); // Flowscript method dispatcher function dispatch () { // 1. instantiate an application context object for this request var application = new webapp(); // 2. initialize the application context var method = application.init (); // 3. invoke the handler for this request application[method](); } ========= webapp.js // Imports cocoon.load ("js/core.js"); // Constructor function webapp () { // inherit from core this.inheritFromCore = core; this.inheritFromCore(); } webapp.prototype = new core(); // Initialise the application context webapp.prototype.init = function() { // open a new Hibernate session for this unit of work. this.beginTransaction (); // application-specific initialization stuff ... // return the method to invoke return cocoon.parameters.method; } // Our form example method webapp.prototype.form_example = function() { // fetch all icons ;-) var icon = new Packages.lu.chem.example.Icons (this.hs); this.context.icons = icon.alleIcons(); // display the icon deletion form var form = new Form ("form/icons_delete.form"); this.showForm (form); // has the user clicked the "delete" form submission button? if (form.getWidget().getSubmitWidget().getId() == "delete") { // We assume here that a custom icon selection CForms widget // returns a list of the selected icon ids. var iterator = form.lookupWidget("icon_selection").value().iterator(); while (iterator.hasNext()) { // Note that the following statement is going to retrieve an // icon from the Hibernate cache ...fetched during our prior // call to alleIcons() var selected_icon = icon.getIcon (iterator.next()); selected_icon.delete(); } } // commit the conversation and display the main page this.context.page = "main_page"; this.sendPage(); } ... ========= core.js // Imports cocoon.load ("resource://org/apache/cocoon/forms/flow/javascript/Form.js"); // Constructor function core () { this.hs = null; // Hibernate session this.context = new Object(); // context to pass to the view } /* beginTransaction() - starts a new, resp. resumes an existing Hibernate conversation */ core.prototype.beginTransaction = function() { if (this.hs == null) { // --- Start a new conversation // get a new session from the PersistenceFactory var factory = cocoon.getComponent (Packages.lu.chem.cocoon.PersistenceFactory.ROLE); this.hs = factory.createSession ("db"); cocoon.releaseComponent( factory); if (this.hs == null) throw new Packages.org.apache.cocoon.ProcessingException ("Hibernate session is null "); // set this session's FlushMode to MANUAL, to avoid unwanted interactions of // the Hibernate session with the underlying database connection this.hs.setFlushMode (Packages.org.hibernate.FlushMode.MANUAL); } else { // --- Resume an existing conversation // reconnect the Hibernate session var factory = cocoon.getComponent (Packages.lu.chem.cocoon.PersistenceFactory.ROLE); var errcode = factory.reconnectSession ("db", this.hs); cocoon.releaseComponent( factory); if (errcode != 0) throw new Packages.org.apache.cocoon.ProcessingException ( "failed to reconnect Hibernate session (errcode " + errcode + ")"); } // ...pure paranoia ;-) this.hs.connection().rollback(); // start a new transaction by rolling back any pending transactions this.rollbackConversation(); // the default action is to rollback the whole conversation } /* The following transaction & conversation demarcation methods set the 'HibernateAction' request attribute. The 'HibernateFilter' servlet filter reads this attribute and performs the appropriate action after the view has been rendered. The Transaction() methods will commit/rollback the current database transaction and disconnect the database connection from the Hibernate session, which is kept open. The Conversation() methods will commit/rollback the current database transaction, but additionnally they will close the Hibernate session, thus terminating the conversation. */ core.prototype.commitTransaction = function() { cocoon.request.setAttribute ("HibernateSession", this.hs); cocoon.request.setAttribute ("HibernateAction", "commitTransaction"); } core.prototype.rollbackTransaction = function() { cocoon.request.setAttribute ("HibernateSession", this.hs); cocoon.request.setAttribute ("HibernateAction", "rollbackTransaction"); } core.prototype.commitConversation = function() { cocoon.request.setAttribute ("HibernateSession", this.hs); cocoon.request.setAttribute ("HibernateAction", "commitConversation"); } core.prototype.rollbackConversation = function() { cocoon.request.setAttribute ("HibernateSession", this.hs); cocoon.request.setAttribute ("HibernateAction", "rollbackConversation"); } /* Convenience showForm & sendPage methods, which include transaction demarcation statements. */ core.prototype.showForm = function (form) { this.rollbackTransaction(); form.showForm ("generic_form", this.context); this.beginTransaction(); } core.prototype.sendPage = function (form) { this.commitConversation(); cocoon.sendPage ("generic_page", this.context); } ========= HibernateFilter.java ... public void doFilter (ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { Session hs; // conversation's Hibernate session String action; // action to perform (commit/rollback transaction/conversation) java.sql.Connection conn; // current transaction (database connection associated // to the Hibernate session) // Pass the request on to cocoon chain.doFilter (request, response); // commit/rollback the current database transaction/conversation hs = (Session) request.getAttribute("HibernateSession"); if (hs != null) { action = (String) request.getAttribute ("HibernateAction"); try { if ("commitTransaction".equals (action)) { hs.flush(); // flush the Hibernate session conn = hs.disconnect(); // detach the database connection from the Hibernate session conn.commit(); // commit the current transaction } else if ("commitConversation".equals (action)) { hs.flush(); // flush the Hibernate session conn = hs.disconnect(); // detach the database connection from the Hibernate session conn.commit(); // commit the current transaction hs.close(); // close the Hibernate session (terminate the conversation) } else if ("rollbackTransaction".equals (action)) { conn = hs.disconnect(); // detach the database connection from the Hibernate session conn.rollback(); // rollback the current transaction } else { conn = hs.disconnect(); // detach the database connection from the Hibernate session conn.rollback(); // rollback the current transaction hs.close(); // close the Hibernate session (terminate the conversation) } conn.close(); // close the database connection / return it to Cocoon's // database connection pool } catch (HibernateException e) { System.out.println(e.getMessage()); } catch( SQLException e ){ System.out.println(e.getMessage()); } } } ========= PersistenceFactory.java package lu.chem.cocoon; import org.apache.avalon.framework.component.Component; /** * <i>PersistenceFactory</i> extends Avalon's Component interface so * we will be able to create Hibernate sessions from inside Cocoon. */ public interface PersistenceFactory extends Component { String ROLE = PersistenceFactory.class.getName(); public org.hibernate.Session createSession (); public org.hibernate.Session createSession (String datasource_name); public int reconnectSession (String datasource_name, org.hibernate.Session hs); } ========= HibernateFactory.java package lu.chem.cocoon; import org.apache.avalon.excalibur.datasource.DataSourceComponent; import org.apache.avalon.framework.activity.Disposable; import org.apache.avalon.framework.activity.Initializable; import org.apache.avalon.framework.configuration.Configurable; import org.apache.avalon.framework.configuration.Configuration; import org.apache.avalon.framework.configuration.ConfigurationException; import org.apache.avalon.framework.logger.AbstractLogEnabled; import org.apache.avalon.framework.service.ServiceException; import org.apache.avalon.framework.service.ServiceManager; import org.apache.avalon.framework.service.ServiceSelector; import org.apache.avalon.framework.service.Serviceable; import org.apache.avalon.framework.thread.ThreadSafe; import org.hibernate.HibernateException; import java.sql.SQLException; import java.util.Date; import java.util.Hashtable; /** * <i>HibernateFactory</i> implements the <code>PersistenceFactory</code> interface. * Its crucial part is createSession(), where Hibernate is told to use a connection * from the Cocoon pool. * This connection is selected using the Avalon framework API. */ public class HibernateFactory extends AbstractLogEnabled implements PersistenceFactory, Configurable, Serviceable, Initializable, Disposable, ThreadSafe { /* * ------------------------------------------------------------ * Constants * ------------------------------------------------------------ */ // reconnectSession() return codes public static final int RECONNECT_OK = 0; public static final int RECONNECT_STALE_SESSION = 1; public static final int RECONNECT_CLOSED_SESSION = 2; public static final int RECONNECT_CONNECTED_SESSION = 3; public static final int RECONNECT_EXCEPTION = 4; /* * ------------------------------------------------------------ * Instance attributes * ------------------------------------------------------------ */ // Avalon stuff private boolean initialized = false; private boolean disposed = false; private ServiceManager manager = null; // do not confuse with Avalon Configuration, Cocoon Session, etc. org.hibernate.cfg.Configuration cfg; org.hibernate.SessionFactory sf; // for debugging: which instance am I using? long time_start; // attributes for cleanupSessions() private long cleanup_interval; // cleanup interval in milliseconds private long session_timeout; // session timeout in milliseconds private long last_cleanup = new Date().getTime(); // the last time a sessions cleanup has been performed private Hashtable sessions = new Hashtable(); // sessions created by this factory /* * ------------------------------------------------------------ * Getters and setters * ------------------------------------------------------------ */ // getter for the Hibernate session factory public org.hibernate.SessionFactory getSf() { return sf; } /* * ------------------------------------------------------------ * Constructors * ------------------------------------------------------------ */ public HibernateFactory () { System.out.println ("Hibernate factory instance created"); } /* * ------------------------------------------------------------ * Avalon interface methods * ------------------------------------------------------------ */ public final void configure (Configuration conf) throws ConfigurationException { if (initialized || disposed) { throw new IllegalStateException ("Illegal call"); } System.out.println ("Hibernate configure called"); // try to read the 'cleanup-interval' & 'session-timeout' parameters this.cleanup_interval = conf.getChild("cleanup-interval").getValueAsLong (60000); this.session_timeout = conf.getChild("session-timeout").getValueAsLong (600000); System.out.println (" cleanup-interval: " + this.cleanup_interval + "ms"); System.out.println (" session-timeout : " + this.session_timeout + "ms"); } public final void service (ServiceManager smanager) throws ServiceException { if (initialized || disposed) { throw new IllegalStateException ("Illegal call"); } if (null == this.manager) { this.manager = smanager; } System.out.println ("Hibernate service called"); } public final void initialize () throws Exception { if (null == this.manager) { throw new IllegalStateException ("Not Composed"); } if (disposed) { throw new IllegalStateException ("Already disposed"); } try { cfg = new org.hibernate.cfg.Configuration (); sf = cfg.configure().buildSessionFactory(); } catch (Exception e) { getLogger ().error ("Hibernate:" + e.getMessage ()); return; } this.initialized = true; System.out.println ("Hibernate initialize called"); } public final void dispose () { // try { sf.close (); } catch (Exception e) { getLogger ().error ("Hibernate:" + e.getMessage ()); } finally { sf = null; cfg = null; } this.disposed = true; this.manager = null; System.out.println ("Hibernate dispose called"); } /* * ------------------------------------------------------------ * Private methods * ------------------------------------------------------------ */ /** * Cleanup any closed or timed out Hibernate sessions. */ private synchronized void cleanupSessions() { long now = new Date().getTime(); // perform a cleanup now ? if ((now - this.last_cleanup) > this.cleanup_interval) { //System.out.println("running cleanupSessions()"); // loop through the registered sessions java.util.Enumeration keys = sessions.keys(); long timeout_limit = now - this.session_timeout; while (keys.hasMoreElements()) { org.hibernate.Session hs = (org.hibernate.Session) keys.nextElement(); if (hs.isOpen()) { // check whether the session has timed out if (((Date) sessions.get(hs)).getTime() < timeout_limit) { //System.out.println("cleanup: removing timed out session " + hs.hashCode()); // check whether the session is connected (this should not happen) if (hs.isConnected()) { // try to rollback & close the JDBC connection try { java.sql.Connection conn = hs.disconnect(); conn.rollback(); conn.close(); } catch (SQLException e) { getLogger().error ("SQLException: " + e.getMessage()); } } // close the Hibernate session & remove it try { hs.close(); } catch (HibernateException e) { getLogger().error ("HibernateException: " + e.getMessage()); } sessions.remove (hs); } } else { //System.out.println("cleanup: removing closed session " + hs.hashCode()); // remove the closed session sessions.remove (hs); } } // remember the last time we have been called this.last_cleanup = now; } } /* * ------------------------------------------------------------ * Public methods * ------------------------------------------------------------ */ /** * Create a new Hibernate session, using a JDBC connection from the specified * cocoon connection pool. * * @param datasource_name name of the cocoon connection pool (defined in cocoon.xconf) * @return Hibernate session */ public org.hibernate.Session createSession (String datasource_name) { org.hibernate.Session hs; DataSourceComponent datasource = null; // cleanup any closed or timed out sessions this.cleanupSessions(); // try to create a new Hibernate session, using a JDBC connection from // cocoon's connection pool identified by 'datasource_name' try { // lookup the named datasource ServiceSelector dbselector = (ServiceSelector) manager.lookup (DataSourceComponent.ROLE + "Selector"); datasource = (DataSourceComponent) dbselector.select (datasource_name); manager.release (dbselector); // open a new Hibernate session hs = sf.openSession (datasource.getConnection ()); // store a reference to the new session, along with a timestamp sessions.put (hs, new Date()); } catch (Exception e) { getLogger().error (e.getMessage ()); hs = null; } return hs; } public org.hibernate.Session createSession () { return this.createSession ("db"); } /** * In a long session (conversation) context: Reconnect the Hibernate session * to a JDBC connection from the specified cocoon connection pool. * * The reason we do it here is to keep the factory's sessions list updated. * * @param datasource_name name of the cocoon connection pool (defined in cocoon.xconf) * @param hs a disconnected Hibernate session * @return error code (0 if OK, non-zero otherwise) */ public int reconnectSession (String datasource_name, org.hibernate.Session hs) { if (! sessions.containsKey (hs)) { getLogger().error ("stale Hibernate session " + hs.hashCode()); return RECONNECT_STALE_SESSION; } if (! hs.isOpen()) { getLogger().error ("Hibernate session is closed " + hs.hashCode()); return RECONNECT_CLOSED_SESSION; } if (hs.isConnected()) { getLogger().error ("Hibernate session is already connected " + hs.hashCode()); return RECONNECT_CONNECTED_SESSION; } try { // lookup the named datasource ServiceSelector dbselector = (ServiceSelector) manager.lookup (DataSourceComponent.ROLE + "Selector"); DataSourceComponent datasource = (DataSourceComponent) dbselector.select (datasource_name); manager.release (dbselector); // reconnect the Hibernate session to a JDBC connection from cocoon's connection pool hs.reconnect (datasource.getConnection ()); // update the timestamp for this session sessions.put (hs, new Date()); return RECONNECT_OK; } catch (Exception e) { getLogger().error (e.getMessage ()); } return RECONNECT_EXCEPTION; } } ========= cocoon.xconf ... <component class="lu.chem.cocoon.HibernateFactory" role="lu.chem.cocoon.PersistenceFactory"> <!-- Cleanup interval time (in milliseconds) to check for closed or timed out sessions --> <cleanup-interval>300000</cleanup-interval> <!-- Session timeout (in milliseconds) --> <session-timeout>14400000</session-timeout> </component> --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]