On Fri, Mar 25, 2011 at 4:47 AM, Afkham Azeez <az...@wso2.com> wrote:
> Amila,
The OOM issue is something different. This issue originally reported by
Ishani is related to the ESB hanging on startup, after it is restarted.
I believe this server hanging is also related to the memory issue.
Cos, while testing yesterday i did experience that server dies without
giving any messages to the console. i.e. server does not through a
"PermGen exception" rather it hangs when "Graceful Shutdown is
invoked". Then if i try to do any operations in management console,
server dies. Above test is also carried out without installing LDAP
features. But i didnt try this scenario with ESB. I will do more
testing with ESB and will confirm whether server hanging is due to a
different cause.
Thanks
AmilaJ
> The following logs are printed and then the server hangs.
> osgi> [2011-03-25 17:08:28,736] INFO
> {org.wso2.carbon.core.init.CarbonServerManager} - Gracefully restarting
> WSO2 ESB...
> [2011-03-25 17:08:28,738] INFO {org.wso2.carbon.core.ServerManagement} -
> Starting to switch to maintenance mode...
> [2011-03-25 17:08:28,739] INFO
> {org.apache.synapse.transport.nhttp.HttpCoreNIOListener} - HTTPS Listener
> Shutdown
> [2011-03-25 17:08:28,742] INFO
> {org.apache.synapse.transport.nhttp.HttpCoreNIOListener} - HTTP Listener
> Shutdown
> [2011-03-25 17:08:28,742] INFO {org.wso2.carbon.core.ServerManagement} -
> Stopped all transport listeners
> [2011-03-25 17:08:28,743] INFO {org.wso2.carbon.core.ServerManagement} -
> Waiting for deployment completion...
> [2011-03-25 17:08:28,744] INFO {org.wso2.carbon.core.ServerManagement} -
> All deployment tasks have been completed.
> [2011-03-25 17:08:28,744] INFO {org.wso2.carbon.core.ServerManagement} -
> Waiting for request service completion...
> [2011-03-25 17:08:28,746] INFO {org.wso2.carbon.core.ServerManagement} -
> All requests have been served.
> [2011-03-25 17:08:28,747] INFO
> {org.wso2.carbon.core.init.CarbonServerManager} - Waiting for 5 sec(s)
> before initiating restart
> [2011-03-25 17:08:33,894] INFO {org.apache.synapse.task.SynapseTaskManager}
> - Shutting down the task manager
> [2011-03-25 17:08:33,897] INFO {org.apache.synapse.Axis2SynapseController}
> - Undeploying the Synapse service...
> [2011-03-25 17:08:33,900] INFO {org.apache.synapse.Axis2SynapseController}
> - Undeploying Proxy services...
> [2011-03-25 17:08:33,900] INFO {org.apache.synapse.Axis2SynapseController}
> - Undeploying EventSources...
> [2011-03-25 17:08:33,971] INFO
> {org.apache.directory.server.ldap.LdapServer} - Unbind of an LDAP service
> (10389) is complete.
> [2011-03-25 17:08:33,971] INFO
> {org.apache.directory.server.ldap.LdapServer} - Sending notice of
> disconnect to existing clients sessions.
> [2011-03-25 17:08:33,971] INFO
> {org.apache.directory.server.ldap.LdapServer} - Ldap service stopped.
> [2011-03-25 17:08:34,027] INFO
> {org.apache.directory.server.core.DefaultDirectoryService} - Waiting for
> SynchWorkerThread to die.
> [2011-03-25 17:08:34,027] INFO
> {org.wso2.carbon.apacheds.impl.ApacheLDAPServer} - LDAP server stopped.
>
>
> On Fri, Mar 25, 2011 at 11:28 AM, Amila Jayasekara <ami...@wso2.com> wrote:
>>
>> Hi All,
>> I was able to reproduce this issue with carbon core server. I am
>> getting "java.lang.OutOfMemoryError: PermGen space" as mentioned in
>> https://wso2.org/jira/browse/CARBON-9145. In carbon core server if we
>> invoke "Graceful Restart" several times we can reproduce this issue
>> (In my machine i was able to reproduce the issue by invoking 3 times).
>> In-order to narrow down the issue i un-installed LDAP specific
>> (apacheds and org.wso2.carbon.ldap) features and connected user
>> manager to a h2 database. Still i was able to reproduce above error
>> after 3 attempts. Therefore i doubt, whether this issue has an impact
>> from LDAP server.
>> When monitoring memory usage in each restart, i observed that memory
>> usage increases about 1-2% for each attempt. (I did the same setof
>> actions in each attempt). I am not quite sure what could be the root
>> cause for the memory increase. We need to further investigate this
>> issue.
>>
>> Thanks
>> AmilaJ
>>
>> On Thu, Mar 24, 2011 at 8:59 PM, Kasun Indrasiri <ka...@wso2.com> wrote:
>> >
>> >
>> > On Thu, Mar 24, 2011 at 9:33 PM, Amila Jayasekara <ami...@wso2.com>
>> > wrote:
>> >>
>> >> On Thu, Mar 24, 2011 at 12:07 AM, Kasun Indrasiri <ka...@wso2.com>
>> >> wrote:
>> >> > Hi Azeez,
>> >> Yeah. The graceful restarting is still broken.
>> >>
>> >> I am still trying to find the exact cause for the error. Kasun, are
>> >> you getting the PermGen error when you execute "Graceful Shutdown" ?
>> >>
>> > No. I'm not getting that error in esb logs.
>> >>
>> >> Thanks
>> >> AmilaJ
>> >>
>> >> > Thanks.
>> >> >
>> >> > On Thu, Mar 24, 2011 at 12:15 PM, Afkham Azeez <az...@wso2.com>
>> >> > wrote:
>> >> >>
>> >> >> Kasun,
>> >> >> Are you seeing the restart issue?
>> >> >>
>> >> >> On Thu, Mar 24, 2011 at 12:02 PM, Kasun Indrasiri <ka...@wso2.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> Adding following entries to esb log4j.properties solved the issues.
>> >> >>> Thanks a lot Amila and Hasini.
>> >> >>> log4j.logger.org.apache.directory.shared.ldap=WARN, CARBON_CONSOLE
>> >> >>> log4j.logger.org.apache.directory.server.ldap.handlers=WARN,
>> >> >>> CARBON_CONSOLE
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> log4j.logger.org.apache.directory.shared.ldap.entry.DefaultServerAttribute=FATAL,
>> >> >>> CARBON_CONSOLE
>> >> >>>
>> >> >>> log4j.logger.org.apache.directory.shared.ldap.ldif.LdifReader=ERROR,
>> >> >>> CARBON_CONSOLE
>> >> >>>
>> >> >>>
>> >> >>> log4j.logger.org.apache.directory.server.ldap.LdapProtocolHandler=ERROR,
>> >> >>> CARBON_CONSOLE
>> >> >>> On Thu, Mar 24, 2011 at 10:35 AM, Amila Jayasekara
>> >> >>> <ami...@wso2.com>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> Hi All,
>> >> >>>>
>> >> >>>> According to apacheds mailing lists and jiras the "null ldap
>> >> >>>> session"
>> >> >>>> warning is a harmless message.
>> >> >>>> You can find more information about this in other mail discussion
>> >> >>>> [1]
>> >> >>>> we had about this.
>> >> >>>>
>> >> >>>> Thus this warning is suppressed at log4j.properties file in core
>> >> >>>> and
>> >> >>>> therefore we dont see this message in carbon and other products
>> >> >>>> which
>> >> >>>> uses log4j.properties file in core.
>> >> >>>> But ESB, MB and few other products are using log4j.properties
>> >> >>>> files
>> >> >>>> specific to them. On those files the warning is not suppressed. I
>> >> >>>> will
>> >> >>>> work on suppressing these warning in other product specific
>> >> >>>> log4j.properties files.
>> >> >>>>
>> >> >>>> Thanks
>> >> >>>> AmilaJ
>> >> >>>>
>> >> >>>> [1] http://wso2.org/mailarchive/carbon-dev/2011-March/049465.html
>> >> >>>>
>> >> >>>> On Wed, Mar 23, 2011 at 1:47 AM, Hasini Gunasinghe
>> >> >>>> <has...@wso2.com>
>> >> >>>> wrote:
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Wed, Mar 23, 2011 at 2:09 PM, Ishani Ratnayake
>> >> >>>> > <ish...@wso2.com>
>> >> >>>> > wrote:
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> On Wed, Mar 23, 2011 at 1:56 PM, Hasini Gunasinghe
>> >> >>>> >> <has...@wso2.com>
>> >> >>>> >> wrote:
>> >> >>>> >>>
>> >> >>>> >>>
>> >> >>>> >>> On Wed, Mar 23, 2011 at 1:43 PM, Kasun Indrasiri
>> >> >>>> >>> <ka...@wso2.com>
>> >> >>>> >>> wrote:
>> >> >>>> >>>>
>> >> >>>> >>>> https://wso2.org/jira/browse/CARBON-9163
>> >> >>>> >
>> >> >>>> > Hi,
>> >> >>>> >
>> >> >>>> >>
>> >> >>>> >> We have already reported two similar issue with related to the
>> >> >>>> >> same.
>> >> >>>> >> [1] https://wso2.org/jira/browse/CARBON-8853
>> >> >>>> >> [2] https://wso2.org/jira/browse/CARBON-8965
>> >> >>>> >
>> >> >>>> > Above two issues are same as the
>> >> >>>> > issue: https://wso2.org/jira/browse/CARBON-8919
>> >> >>>> > And since that issue has been fixed now, two issues you have
>> >> >>>> > mentioned
>> >> >>>> > should also have been fixed.
>> >> >>>> > Thanks for reporting.. can you please verify and close the
>> >> >>>> > issues,
>> >> >>>> > if
>> >> >>>> > resolved?
>> >> >>>> > Thanks,
>> >> >>>> > Hasini.
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> Thanks,
>> >> >>>> >> Ishani
>> >> >>>> >>>
>> >> >>>> >>> We have discussed and avoided a similar continuous log (in
>> >> >>>> >>> that
>> >> >>>> >>> case
>> >> >>>> >>> it
>> >> >>>> >>> is an INFO log) here [1].
>> >> >>>> >>> Will look into the issue that Kasun has reported as well when
>> >> >>>> >>> we
>> >> >>>> >>> come to
>> >> >>>> >>> fixing issues.
>> >> >>>> >>> Azeez, would it be sufficient if we look into the issue you
>> >> >>>> >>> reported
>> >> >>>> >>> as
>> >> >>>> >>> well, when we come to fixing issues?
>> >> >>>> >>>
>> >> >>>> >>> [1]http://wso2.org/mailarchive/carbon-dev/2011-March/049465.html
>> >> >>>> >>>>
>> >> >>>> >>>>
>> >> >>>> >>>>
>> >> >>>> >>>> On Tue, Mar 22, 2011 at 10:27 AM, Kasun Indrasiri
>> >> >>>> >>>> <ka...@wso2.com>
>> >> >>>> >>>> wrote:
>> >> >>>> >>>>>
>> >> >>>> >>>>> Hi,
>> >> >>>> >>>>> Getting this warning continuously in the latest ESB builds.
>> >> >>>> >>>>> Any
>> >> >>>> >>>>> thoughts ?
>> >> >>>> >>>>> [2011-03-22 10:23:14,236] INFO
>> >> >>>> >>>>>
>> >> >>>> >>>>> {org.apache.directory.server.ldap.handlers.LdapRequestHandler}
>> >> >>>> >>>>> -
>> >> >>>> >>>>> ignoring
>> >> >>>> >>>>> the message
>> >> >>>> >>>>>
>> >> >>>> >>>>>
>> >> >>>> >>>>>
>> >> >>>> >>>>> org.apache.directory.shared.ldap.message.UnbindRequestImpl@3f8efd86
>> >> >>>> >>>>> received
>> >> >>>> >>>>> from null session
>> >> >>>> >>>>> [2011-03-22 10:23:14,243] WARN
>> >> >>>> >>>>> {org.apache.directory.server.ldap.LdapProtocolHandler} -
>> >> >>>> >>>>> Null
>> >> >>>> >>>>> LdapSession
>> >> >>>> >>>>> given to cleanUpSession.
>> >> >>>> >>>>> [2011-03-22 10:23:18,656] WARN
>> >> >>>> >>>>> {org.apache.directory.server.ldap.LdapProtocolHandler} -
>> >> >>>> >>>>> Null
>> >> >>>> >>>>> LdapSession
>> >> >>>> >>>>> given to cleanUpSession.
>> >> >>>> >>>>> [2011-03-22 10:23:18,662] WARN
>> >> >>>> >>>>> {org.apache.directory.server.ldap.LdapProtocolHandler} -
>> >> >>>> >>>>> Null
>> >> >>>> >>>>> LdapSession
>> >> >>>> >>>>> given to cleanUpSession.
>> >> >>>> >>>>>
>> >> >>>> >>>>> --
>> >> >>>> >>>>> Kasun Indrasiri
>> >> >>>> >>>>> Senior Software Engineer
>> >> >>>> >>>>> WSO2, Inc.; http://wso2.com
>> >> >>>> >>>>> lean.enterprise.middleware
>> >> >>>> >>>>>
>> >> >>>> >>>>> cell: +94 71 536 4128
>> >> >>>> >>>>> Blog : http://kasunpanorama.blogspot.com/
>> >> >>>> >>>>
>> >> >>>> >>>>
>> >> >>>> >>>>
>> >> >>>> >>>> --
>> >> >>>> >>>> Kasun Indrasiri
>> >> >>>> >>>> Senior Software Engineer
>> >> >>>> >>>> WSO2, Inc.; http://wso2.com
>> >> >>>> >>>> lean.enterprise.middleware
>> >> >>>> >>>>
>> >> >>>> >>>> cell: +94 71 536 4128
>> >> >>>> >>>> Blog : http://kasunpanorama.blogspot.com/
>> >> >>>> >>>>
>> >> >>>> >>>> _______________________________________________
>> >> >>>> >>>> Carbon-dev mailing list
>> >> >>>> >>>> Carbon-dev@wso2.org
>> >> >>>> >>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>> >> >>>> >>>>
>> >> >>>> >>>
>> >> >>>> >>>
>> >> >>>> >>> _______________________________________________
>> >> >>>> >>> Carbon-dev mailing list
>> >> >>>> >>> Carbon-dev@wso2.org
>> >> >>>> >>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>> >> >>>> >>>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> --
>> >> >>>> >> Ishani
>> >> >>>> >> Mobile : 0727136198
>> >> >>>> >>
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > _______________________________________________
>> >> >>>> > Carbon-dev mailing list
>> >> >>>> > Carbon-dev@wso2.org
>> >> >>>> > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>> >> >>>> >
>> >> >>>> >
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Kasun Indrasiri
>> >> >>> Senior Software Engineer
>> >> >>> WSO2, Inc.; http://wso2.com
>> >> >>> lean.enterprise.middleware
>> >> >>>
>> >> >>> cell: +94 71 536 4128
>> >> >>> Blog : http://kasunpanorama.blogspot.com/
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Afkham Azeez
>> >> >> Senior Software Architect & Senior Manager; WSO2, Inc.;
>> >> >> http://wso2.com,
>> >> >>
>> >> >> Member; Apache Software Foundation; http://www.apache.org/
>> >> >> email: az...@wso2.com cell: +94 77 3320919
>> >> >> blog: http://blog.afkham.org
>> >> >> twitter: http://twitter.com/afkham_azeez
>> >> >> linked-in: http://lk.linkedin.com/in/afkhamazeez
>> >> >>
>> >> >> Lean . Enterprise . Middleware
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Kasun Indrasiri
>> >> > Senior Software Engineer
>> >> > WSO2, Inc.; http://wso2.com
>> >> > lean.enterprise.middleware
>> >> >
>> >> > cell: +94 71 536 4128
>> >> > Blog : http://kasunpanorama.blogspot.com/
>> >> >
>> >
>> >
>> >
>> > --
>> > Kasun Indrasiri
>> > Senior Software Engineer
>> > WSO2, Inc.; http://wso2.com
>> > lean.enterprise.middleware
>> >
>> > cell: +94 71 536 4128
>> > Blog : http://kasunpanorama.blogspot.com/
>> >
>
>
>
> --
> Afkham Azeez
> Senior Software Architect & Senior Manager; WSO2, Inc.; http://wso2.com,
>
> Member; Apache Software Foundation; http://www.apache.org/
> email: az...@wso2.com cell: +94 77 3320919
> blog: http://blog.afkham.org
> twitter: http://twitter.com/afkham_azeez
> linked-in: http://lk.linkedin.com/in/afkhamazeez
>
> Lean . Enterprise . Middleware
>
_______________________________________________
Carbon-dev mailing list
Carbon-dev@wso2.org
http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
