From what I see some instances don't log the DB Connection error and still have immortal sessions. The main problems with immortal sessions is that eventually they end up with a memory problem, I recall there is a way to make the instance restart if memory errors happen but I can't remember how.
I'm still in dire straits, no idea how to fix this... no idea what's happening either. Any suggestions? Matteo On Tue, Jan 15, 2013 at 6:14 PM, Chuck Hill <[email protected]> wrote: > > On 2013-01-15, at 3:20 AM, Matteo Centro wrote: > >> Unfortunately I don't have full control of the deployment >> environment... It looks like at some times the DB stops responding >> with no apparent reason. >> Could a simple DB Connection glitch cause a EOObjectStoreCoordinator lock? > > I recall a bug at some level related to this. I could be wrong. > > >> Maybe I can tweak the connection dictionary to enable auto reconnect. >> I'll try > > Chuck > > >> On Mon, Jan 14, 2013 at 8:17 PM, Chuck Hill <[email protected]> wrote: >>> Hi Matteo, >>> >>> Something locked the EOObjectStoreCoordinator and did not unlock it. Does >>> the log for this instance show any exceptions? >>> >>> This exception could perhaps cause this problem: >>> >>>>> ava.io.EOFException >>>>> at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1394) >>>>> at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:1538) >>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1929) >>>>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1167) >>>>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1278) >>>>> at com.mysql.jdbc.Connection.execSQL(Connection.java:2247) >>>>> at >>>>> com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1371) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel._bindInputVariablesWithBindingsAndExecute(JDBCChannel.java:265) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel._evaluateExpression(JDBCChannel.java:337) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel.evaluateExpression(JDBCChannel.java:296) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel.selectAttributes(JDBCChannel.java:220) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseChannel._selectWithFetchSpecificationEditingContext(EODatabaseChannel.java:897) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseChannel.selectObjectsWithFetchSpecification(EODatabaseChannel.java:234) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseContext._objectsWithFetchSpecificationEditingContext(EODatabaseContext.java:3055) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseContext.objectsWithFetchSpecification(EODatabaseContext.java:3195) >>>>> at >>>>> com.webobjects.eocontrol.EOObjectStoreCoordinator.objectsWithFetchSpecification(EOObjectStoreCoordinator.java:488) >>>>> at >>>>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4069) >>>>> at >>>>> er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1305) >>>>> at >>>>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444) >>> >>> >>> Chuck >>> >>> >>> On 2013-01-14, at 10:05 AM, Matteo Centro wrote: >>> >>>> Hi all, suspecting there was some sort of deadlock I ran jstack >>>> against one of the immortal sessions instances, I'm attaching the >>>> output, anything helps... >>>> >>>> Thanks, >>>> >>>> >>>> Matteo >>>> >>>> On Mon, Jan 14, 2013 at 12:35 PM, Matteo Centro <[email protected]> >>>> wrote: >>>>> Sure, here it is: >>>>> Hi Chuck, >>>>> >>>>> I'm posting just to you, I can't have google to index this... >>>>> >>>>> anyway, here is the stack trace: >>>>> java.lang.IllegalMonitorStateException >>>>> at >>>>> java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:127) >>>>> at >>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1239) >>>>> at >>>>> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:431) >>>>> at >>>>> com.webobjects.eocontrol.EOObjectStoreCoordinator.unlock(EOObjectStoreCoordinator.java:448) >>>>> at >>>>> com.webobjects.eocontrol.EOEditingContext.unlockObjectStore(EOEditingContext.java:4684) >>>>> at er.extensions.eof.ERXEC.unlockObjectStore(ERXEC.java:805) >>>>> at >>>>> com.webobjects.eocontrol.EOCustomObject.willReadRelationship(EOCustomObject.java:1281) >>>>> at >>>>> er.extensions.eof.ERXGenericRecord.willReadRelationship(ERXGenericRecord.java:378) >>>>> at >>>>> com.webobjects.eocontrol._EOMutableKnownKeyDictionary$Initializer$_LazyGenericRecordBinding.valueInObject(_EOMutableKnownKeyDictionary.java:614) >>>>> at >>>>> com.webobjects.eocontrol.EOCustomObject.storedValueForKey(EOCustomObject.java:1634) >>>>> at >>>>> com.tla.logic._RigaCarrello.clientiPerTicket(_RigaCarrello.java:275) >>>>> at com.tla.logic.RigaCarrello.setPosti(RigaCarrello.java:94) >>>>> at com.tla.logic.Carrello.cancellaTutto(Carrello.java:164) >>>>> at com.tla.calendar.Session.terminate(Session.java:81) >>>>> at >>>>> com.webobjects.appserver.WOSession._terminateByTimeout(WOSession.java:610) >>>>> at >>>>> com.webobjects.appserver.WOSessionStore$_SessionTimeoutManager.run(WOSessionStore.java:115) >>>>> at java.lang.Thread.run(Thread.java:662) >>>>> >>>>> Sorry for the italian Class Names and Methods, as I said I inherited >>>>> that... >>>>> >>>>> The exception is caught, that printout is the printStackTrace of the >>>>> exception... >>>>> But anyway, I'm not sure that's the real problem, I found this morning >>>>> an instance with 295 sessions (still alive) and the only exception >>>>> logged was a weird >>>>> >>>>> er.transaction.adaptor.Exceptions - Database Exception occured: No >>>>> operations allowed after connection closed. >>>>> >>>>> Connection was closed due to the following exception: >>>>> >>>>> ** BEGIN NESTED EXCEPTION ** >>>>> >>>>> java.sql.SQLException >>>>> MESSAGE: Communication link failure: java.io.EOFException, underlying >>>>> cause: null >>>>> >>>>> ** BEGIN NESTED EXCEPTION ** >>>>> >>>>> java.io.EOFException >>>>> >>>>> STACKTRACE: >>>>> >>>>> java.io.EOFException >>>>> at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1394) >>>>> at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:1538) >>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1929) >>>>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1167) >>>>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1278) >>>>> at com.mysql.jdbc.Connection.execSQL(Connection.java:2247) >>>>> at >>>>> com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1371) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel._bindInputVariablesWithBindingsAndExecute(JDBCChannel.java:265) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel._evaluateExpression(JDBCChannel.java:337) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel.evaluateExpression(JDBCChannel.java:296) >>>>> at >>>>> com.webobjects.jdbcadaptor.JDBCChannel.selectAttributes(JDBCChannel.java:220) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseChannel._selectWithFetchSpecificationEditingContext(EODatabaseChannel.java:897) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseChannel.selectObjectsWithFetchSpecification(EODatabaseChannel.java:234) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseContext._objectsWithFetchSpecificationEditingContext(EODatabaseContext.java:3055) >>>>> at >>>>> com.webobjects.eoaccess.EODatabaseContext.objectsWithFetchSpecification(EODatabaseContext.java:3195) >>>>> at >>>>> com.webobjects.eocontrol.EOObjectStoreCoordinator.objectsWithFetchSpecification(EOObjectStoreCoordinator.java:488) >>>>> at >>>>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4069) >>>>> at >>>>> er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1305) >>>>> at >>>>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444) >>>>> at >>>>> com.tla.logic.Performance.fetchFreshPerformancesWithQualifier(Performance.java:414) >>>>> at com.tla.logic.Carrello.cancellaTutto(Carrello.java:154) >>>>> at >>>>> com.tla.calendar.components.ALTComponent.invokeAction(ALTComponent.java:167) >>>>> at >>>>> com.webobjects.appserver.WOSession.invokeAction(WOSession.java:1357) >>>>> at >>>>> com.webobjects.appserver.WOApplication.invokeAction(WOApplication.java:1745) >>>>> at >>>>> er.extensions.appserver.ajax.ERXAjaxApplication.invokeAction(ERXAjaxApplication.java:117) >>>>> at >>>>> er.extensions.appserver.ERXApplication.invokeAction(ERXApplication.java:2018) >>>>> at >>>>> er.extensions.appserver.ERXComponentRequestHandler._dispatchWithPreparedPage(ERXComponentRequestHandler.java:157) >>>>> at >>>>> er.extensions.appserver.ERXComponentRequestHandler._dispatchWithPreparedSession(ERXComponentRequestHandler.java:235) >>>>> at >>>>> er.extensions.appserver.ERXComponentRequestHandler._dispatchWithPreparedApplication(ERXComponentRequestHandler.java:268) >>>>> at >>>>> er.extensions.appserver.ERXComponentRequestHandler._handleRequest(ERXComponentRequestHandler.java:302) >>>>> at >>>>> er.extensions.appserver.ERXComponentRequestHandler.handleRequest(ERXComponentRequestHandler.java:377) >>>>> at >>>>> com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687) >>>>> at >>>>> er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:2139) >>>>> at >>>>> er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:2104) >>>>> at >>>>> com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144) >>>>> at >>>>> com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226) >>>>> at java.lang.Thread.run(Thread.java:662) >>>>> >>>>> So I'm not that confident that I am on the right track... >>>>> Looks like if anything goes wrong in the life of the instance sessions >>>>> won't die... Which is kind of disturbing. >>>>> >>>>> >>>>> Matteo >>>>> >>>>> On Sun, Jan 13, 2013 at 7:01 PM, Chuck Hill <[email protected]> >>>>> wrote: >>>>>> Can you post the entire stack trace? >>>>>> >>>>>> At the very least, I would catch that exception and now allow it out of >>>>>> your finalize method. >>>>> >>>>> >>>>> On Sun, Jan 13, 2013 at 7:01 PM, Chuck Hill <[email protected]> >>>>> wrote: >>>>>> Can you post the entire stack trace? >>>>>> >>>>>> At the very least, I would catch that exception and now allow it out of >>>>>> your finalize method. >>>>>> >>>>>> >>>>>> On 2013-01-12, at 2:01 PM, Matteo Centro wrote: >>>>>> >>>>>>> Thanks, I'll try that. >>>>>>> Anyway (I can't say it's that but this happens often in instances with >>>>>>> immortal sessions): >>>>>>> I get this in my terminate() method >>>>>>> java.lang.IllegalMonitorStateException >>>>>>> at >>>>>>> java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:127) >>>>>>> at >>>>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1239) >>>>>>> at >>>>>>> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:431) >>>>>>> at >>>>>>> com.webobjects.eocontrol.EOObjectStoreCoordinator.unlock(EOObjectStoreCoordinator.java:460) >>>>>>> at >>>>>>> com.webobjects.eocontrol.EOEditingContext.unlockObjectStore(EOEditingContext.java:4684) >>>>>>> at er.extensions.eof.ERXEC.unlockObjectStore(ERXEC.java:805) >>>>>>> at >>>>>>> com.webobjects.eocontrol.EOCustomObject.willReadRelationship(EOCustomObject.java:1281) >>>>>>> at >>>>>>> er.extensions.eof.ERXGenericRecord.willReadRelationship(ERXGenericRecord.java:378) >>>>>>> at >>>>>>> com.webobjects.eocontrol._EOMutableKnownKeyDictionary$Initializer$_LazyGenericRecordBinding.valueInObject(_EOMutableKnownKeyDictionary.java:614) >>>>>>> at >>>>>>> com.webobjects.eocontrol.EOCustomObject.storedValueForKey(EOCustomObject.java:1634) >>>>>>> >>>>>>> I'm using ERXEC everywhere and I'm not locking explicitly, should I >>>>>>> lock the ec in the terminate method? >>>>>>> >>>>>>> >>>>>>> Matteo >>>>>>> >>>>>>> On Sat, Jan 12, 2013 at 8:39 PM, Simon <[email protected]> wrote: >>>>>>>> stick this in your session constructor, it will log out whenever a >>>>>>>> session is created. then you can start figuring out where you are >>>>>>>> creating a session outside the RR loop which is the normal culprit... >>>>>>>> >>>>>>>> StringWriter stringWriter = new StringWriter(); >>>>>>>> PrintWriter printWriter = new >>>>>>>> PrintWriter(stringWriter); >>>>>>>> (new Throwable()).printStackTrace(printWriter); >>>>>>>> String trace = stringWriter.toString(); >>>>>>>> log.debug("Session count = " + >>>>>>>> application().activeSessionsCount()); >>>>>>>> >>>>>>>> ClickEventLogger2.getLogger().fatal(ClickEventCode.E00129, "Session >>>>>>>> Created:\n\n " + trace, >>>>>>>> this.getClass().getSimpleName()); >>>>>>>> >>>>>>>> hmmm, you'll have to replace the "ClickEventLogger2" line because >>>>>>>> that's our logging mechanism. you could log.fatal it and set up an >>>>>>>> email appender in log4j. >>>>>>>> >>>>>>>> simon >>>>>>>> >>>>>>>> >>>>>>>> On 12 January 2013 17:43, Matteo Centro <[email protected]> >>>>>>>> wrote: >>>>>>>>> Sorry to resuscitate such an old discussion but I'm having the exact >>>>>>>>> same issue... >>>>>>>>> It's an old application that we "inherited", we wonderized it as much >>>>>>>>> as it's possible but something weird happens in production, sessions >>>>>>>>> on some instances simply won't die. >>>>>>>>> Some instances go out of memory and they hang there. >>>>>>>>> I'm in trouble and I needs some hints, both to fix the issue >>>>>>>>> temporarily and to fix it for good (of course in that case I assume >>>>>>>>> I'll have to rewrite the app, if only I could find the budget). >>>>>>>>> What are the most common causes of sessions not dying? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> >>>>>>>>> Matteo >>>>>>>>> >>>>>>>>> On Thu, Aug 21, 2008 at 5:35 AM, Joe Little <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> I had something similar with sessions going bonkers on a public WO >>>>>>>>>> page that our internal google search engine completely trashed. In >>>>>>>>>> the >>>>>>>>>> end, robots.txt and explicit rules to deny certain patterns were >>>>>>>>>> added >>>>>>>>>> to prevent this. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Aug 20, 2008 at 8:17 PM, D Tim Cummings <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> We have a couple of sessionless apps that have started showing this >>>>>>>>>>> problem >>>>>>>>>>> with sessions that don't terminate. It turned out the sessions >>>>>>>>>>> were being >>>>>>>>>>> created by malformed urls coming from malicious robot web crawlers. >>>>>>>>>>> The >>>>>>>>>>> urls were of the form >>>>>>>>>>> http://www.courses.qut.edu.au/cgi-bin/WebObjects/Courses.woa/wa/cgi-bin/WebObjects/Courses.woa >>>>>>>>>>> Maybe see if you are getting incorrect links to your sessionless >>>>>>>>>>> login page. >>>>>>>>>>> We solved the problem by catching unknown direct actions in >>>>>>>>>>> DirectAction.java >>>>>>>>>>> @Override >>>>>>>>>>> public WOActionResults performActionNamed(String actionName) { >>>>>>>>>>> try { >>>>>>>>>>> return super.performActionNamed(actionName); >>>>>>>>>>> } catch (NSForwardException nsfe) { >>>>>>>>>>> log.info("ns forward exception - prbalby no such method for " + >>>>>>>>>>> actionName); >>>>>>>>>>> } >>>>>>>>>>> return defaultAction(); >>>>>>>>>>> } >>>>>>>>>>> and in Application.java directing exceptions back to the Main page >>>>>>>>>>> (for URLs >>>>>>>>>>> with more than one / after wa). >>>>>>>>>>> @Override >>>>>>>>>>> public WOComponent pageWithName(String namePage, WOContext context) >>>>>>>>>>> { >>>>>>>>>>> if ( "WOExceptionPage".equals(namePage) ) { >>>>>>>>>>> namePage = "Main"; >>>>>>>>>>> } >>>>>>>>>>> if ( "WOSessionRestorationError".equals(namePage) ) { >>>>>>>>>>> namePage = "Main"; >>>>>>>>>>> } >>>>>>>>>>> return super.pageWithName(namePage, context); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> and in Main.java >>>>>>>>>>> public void setException ( Exception e ) { >>>>>>>>>>> log.error("an exception occurred " + e); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> We are running apps with embedded Wonder 4 and WebObjects 5.3.3 on >>>>>>>>>>> Mac OS X >>>>>>>>>>> Server 10.5.4 with WebObjects 5.4.2 deployment. We didn't have the >>>>>>>>>>> problem >>>>>>>>>>> before we went to this setup, but maybe we weren't getting hit with >>>>>>>>>>> the same >>>>>>>>>>> url format then. >>>>>>>>>>> Tim >>>>>>>>>>> On 21/08/2008, at 4:02 AM, Chuck Hill wrote: >>>>>>>>>>> >>>>>>>>>>> On Aug 20, 2008, at 9:54 AM, Simon McLean wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All - >>>>>>>>>>> Wondering if someone can throw me some ideas on solving what looks >>>>>>>>>>> like an >>>>>>>>>>> immortal session problem. >>>>>>>>>>> - Start up 1 instance in java monitor. >>>>>>>>>>> - User lands on sessionless login page. No sessions. >>>>>>>>>>> - User logs in. 1 session. >>>>>>>>>>> - User logs out. 0 sessions. >>>>>>>>>>> - User logs in. 1 session. >>>>>>>>>>> - User does nothing. Session times out. 0 sessions. >>>>>>>>>>> All looks good. However, we get to the end of the day and the >>>>>>>>>>> instance has >>>>>>>>>>> 376 active sessions. But I know for this particular app there are >>>>>>>>>>> only 6 >>>>>>>>>>> users, because they are all sitting next door :-) Also, If i now >>>>>>>>>>> leave this >>>>>>>>>>> instance overnight none of those active sessions will go away. >>>>>>>>>>> So what could be going on ? Something appears to be creating >>>>>>>>>>> immortal >>>>>>>>>>> sessions, but forced session termination (by the user logging out) >>>>>>>>>>> and >>>>>>>>>>> session expiration seem to be behaving themselves. >>>>>>>>>>> >>>>>>>>>>> If the users don't notice any problems, my money would be on an >>>>>>>>>>> exception >>>>>>>>>>> thrown from your Session.terminate(). Also check the sleep() >>>>>>>>>>> method to >>>>>>>>>>> ensure it can never throw. >>>>>>>>>>> I have seen a related problem where two requests come in for the >>>>>>>>>>> same >>>>>>>>>>> session (user double clicks, Ajax etc), and the first calls >>>>>>>>>>> terminate() on >>>>>>>>>>> the session. The second thread is stuck and the app can't >>>>>>>>>>> gracefully shut >>>>>>>>>>> down. IIRC, you would see zero sessions in this case. The "fix" >>>>>>>>>>> for this >>>>>>>>>>> is to not call terminate() and instead set the session timeout to a >>>>>>>>>>> small >>>>>>>>>>> number of seconds. >>>>>>>>>>> Chuck >>>>>>>>>>> -- >>>>>>>>>>> Chuck Hill Senior Consultant / VP Development >>>>>>>>>>> Practical WebObjects - for developers who want to increase their >>>>>>>>>>> overall >>>>>>>>>>> knowledge of WebObjects or who are trying to solve specific >>>>>>>>>>> problems. >>>>>>>>>>> http://www.global-village.net/products/practical_webobjects >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Do not post admin requests to the list. They will be ignored. >>>>>>>>>>> Webobjects-dev mailing list ([email protected]) >>>>>>>>>>> Help/Unsubscribe/Update your Subscription: >>>>>>>>>>> http://lists.apple.com/mailman/options/webobjects-dev/timcu%40tpg.com.au >>>>>>>>>>> This email sent to [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Do not post admin requests to the list. They will be ignored. >>>>>>>>>>> Webobjects-dev mailing list ([email protected]) >>>>>>>>>>> Help/Unsubscribe/Update your Subscription: >>>>>>>>>>> http://lists.apple.com/mailman/options/webobjects-dev/jmlittle%40gmail.com >>>>>>>>>>> >>>>>>>>>>> This email sent to [email protected] >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Do not post admin requests to the list. They will be ignored. >>>>>>>>>> Webobjects-dev mailing list ([email protected]) >>>>>>>>>> Help/Unsubscribe/Update your Subscription: >>>>>>>>>> http://lists.apple.com/mailman/options/webobjects-dev/wolists%40matteocentro.it >>>>>>>>>> >>>>>>>>>> This email sent to [email protected] >>>>>>>>> _______________________________________________ >>>>>>>>> Do not post admin requests to the list. They will be ignored. >>>>>>>>> Webobjects-dev mailing list ([email protected]) >>>>>>>>> Help/Unsubscribe/Update your Subscription: >>>>>>>>> https://lists.apple.com/mailman/options/webobjects-dev/simon%40potwells.co.uk >>>>>>>>> >>>>>>>>> This email sent to [email protected] >>>>>>> _______________________________________________ >>>>>>> Do not post admin requests to the list. They will be ignored. >>>>>>> Webobjects-dev mailing list ([email protected]) >>>>>>> Help/Unsubscribe/Update your Subscription: >>>>>>> https://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village.net >>>>>>> >>>>>>> This email sent to [email protected] >>>>>> >>>>>> -- >>>>>> Chuck Hill Senior Consultant / VP Development >>>>>> >>>>>> Practical WebObjects - for developers who want to increase their overall >>>>>> knowledge of WebObjects or who are trying to solve specific problems. >>>>>> http://www.global-village.net/gvc/practical_webobjects >>>>>> >>>>>> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest >>>>>> Growing Companies in B.C! >>>>>> Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking >>>>>> of Canada’s Fastest-Growing Companies by PROFIT Magazine! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>> <jstackoutput.txt> >>> >>> -- >>> Chuck Hill Senior Consultant / VP Development >>> >>> Practical WebObjects - for developers who want to increase their overall >>> knowledge of WebObjects or who are trying to solve specific problems. >>> http://www.global-village.net/gvc/practical_webobjects >>> >>> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest >>> Growing Companies in B.C! >>> Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking of >>> Canada’s Fastest-Growing Companies by PROFIT Magazine! >>> >>> >>> >>> >>> >>> >>> >>> > > -- > Chuck Hill Senior Consultant / VP Development > > Practical WebObjects - for developers who want to increase their overall > knowledge of WebObjects or who are trying to solve specific problems. > http://www.global-village.net/gvc/practical_webobjects > > Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing > Companies in B.C! > Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking of > Canada’s Fastest-Growing Companies by PROFIT Magazine! > > > > > > > > _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [email protected]
