Bugs item #804491, was opened at 2003-09-11 14:56
Message generated for change (Comment added) made by bcotton969
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=376685&aid=804491&group_id=22866

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Bob Cotton (bcotton969)
Assigned to: Nobody/Anonymous (nobody)
Summary: Unable to add new nodes to cluster

Initial Comment:
OS: Solaris 8

JDK: 1.3.1

Logs as attachment



We run a cluster of EJB vms with JNDI and EJBs as

seperate cluster services.



Lately, in production, the cluster gets into a state

where each node thinks there are no other nodes in the

cluster.



The clients are ok with this because the client-side

stubs remeber the last cluster configuration and

continue to RR over each vm.



However, when starting new VMs that want to join the

cluster, they do not join. If multiple VMs are started,

none of the new VMs cluster with each other.



To correct this problem we have to bring down the whole

system.



Cluster logs with full TRACE debugging are attached.

----------------------------------------------------------------------

>Comment By: Bob Cotton (bcotton969)
Date: 2003-09-15 20:38

Message:
Logged In: YES 
user_id=424902

Reading the docs on jnp.disableDiscovery it seems that

autoDiscovery is used if there is no Context.PROVIDER_URL

specified or if there are no naming services found.



>From the EJB clients we do set java.naming.provider.url to

the list of ejb servers that are ALWAYS running. So the URL

is set and the naming servers are always there.



The EJB servers (which are the only vms that clustered) do

not have  java.naming.provider.url set, but all EJB->EJB

calls should be in-vm.



Also, the "AutoDiscovery" in the logs are just :

clusterprod-ejb1.log:2003-09-11 00:38:30,445 TRACE

[org.jboss.ha.jndi.HANamingService$AutomaticDiscovery]

HA-JNDI AutomaticDiscovery waiting for queries...



How is this caused by the client, which shold not be doing

autoDiscovery?



The CPU usage on the system is only 50% (this is an 8-way

machine)

So if the machine gets busy does this mean that the clusters

will break?







----------------------------------------------------------------------

Comment By: Sacha Labourey (slaboure)
Date: 2003-09-12 18:36

Message:
Logged In: YES 
user_id=95900

It is not possible to disable it on the server (only on the 

client).



Cheers,





sacha

----------------------------------------------------------------------

Comment By: Bob Cotton (bcotton969)
Date: 2003-09-12 15:57

Message:
Logged In: YES 
user_id=424902

How do you disable automatic discovery?

----------------------------------------------------------------------

Comment By: Sacha Labourey (slaboure)
Date: 2003-09-12 06:43

Message:
Logged In: YES 
user_id=95900

3.2.1 is old, it is from May, I am pretty sure this has been 

fixed after, in some of the 3.2.2RCx

----------------------------------------------------------------------

Comment By: Bela Ban (belaban)
Date: 2003-09-12 01:14

Message:
Logged In: YES 
user_id=34890

Looks like the AutomaticDiscovery endless loop on

HANamingService.stop() is still there. Grep for

'AutomaticDiscovery' and you'll see that 35% of *all* logs

statements are these.

This sucks a lot of CPU out of the system, maybe the JOINs

run into timeouts all the time, that's why we never see each

other.



Sacha: I thought this was fixed in 3.2.1 ?

Bob: can you disable automaticDiscovery for now and test again ?

Bela

----------------------------------------------------------------------

Comment By: Bob Cotton (bcotton969)
Date: 2003-09-11 18:26

Message:
Logged In: YES 
user_id=424902

Here is jndi-cluster-service.xml



<?xml version="1.0" encoding="UTF-8"?>



<server>



  <classpath codebase="lib" archives="jbossha.jar"/>



  <!--

====================================================================

-->

  <!-- Cluster Partition: defines cluster                  

                -->

  <!-- Based on cluser-services.xml from

jboss/server/all/deploy/           -->          

  <!--

====================================================================

-->



  <mbean code="org.jboss.ha.framework.server.ClusterPartition"

         name="jboss:service=JndiPartition">

         

    <!-- Name of the partition being built -->

    <attribute name="PartitionName">DefaultPartition</attribute>

    <!-- Determine if deadlock detection is enabled -->

    <attribute name="DeadlockDetection">False</attribute>

    <!-- The JavaGroups protocol configuration -->

    <attribute name="PartitionConfig">

      <Config>

        <!-- UDP: if you have a multihomed machine, 

             set the bind_addr attribute to the appropriate

NIC IP address -->

        <!-- UDP: On Windows machines, because of the media

sense feature

             being broken with multicast (even after

disabling media sense)

             set the loopback attribute to true -->

        <UDP mcast_addr="228.1.2.3" mcast_port="45566" 

             ip_ttl="64" ip_mcast="true"

             mcast_send_buf_size="150000"

mcast_recv_buf_size="80000" 

             ucast_send_buf_size="150000"

ucast_recv_buf_size="80000" 

             loopback="false" />

        <PING timeout="2000" num_initial_members="3" 

              up_thread="true" down_thread="true" />

        <MERGE2 min_interval="5000" max_interval="10000" />

        <FD up_thread="true" down_thread="true"

timeout="2500" max_tries="5" shun="true"/>

        <VERIFY_SUSPECT timeout="3000" num_msgs="3"

                        up_thread="true" down_thread="true" />

        <pbcast.STABLE desired_avg_gossip="20000"

                       up_thread="true" down_thread="true" />

        <pbcast.NAKACK gc_lag="50"

retransmit_timeout="300,600,1200,2400,4800"

                       up_thread="true" down_thread="true" />

        <UNICAST timeout="5000" window_size="100"

min_threshold="10"

                 down_thread="true" />

        <FRAG frag_size="8192"

              down_thread="true" up_thread="true" />

        <pbcast.GMS join_timeout="5000"

join_retry_timeout="2000"

                    shun="true" print_local_addr="true" />

        <pbcast.STATE_TRANSFER up_thread="true"

down_thread="true" />

      </Config>

    </attribute>

    <attribute name="PartitionName">Jndi</attribute>

  </mbean>



  <!--

====================================================================

-->

  <!-- HA JNDI                                             

                -->

  <!--

====================================================================

-->



  <mbean code="org.jboss.ha.jndi.HANamingService" 

         name="jboss:service=HAJNDI">

     <depends>jboss:service=JndiPartition</depends>

    <!-- Name of the partition to which the service is

linked -->

    <attribute name="PartitionName">Jndi</attribute>

    <!-- RmiPort to be used by the HA-JNDI service 

         once bound. 0 => auto. -->

    <attribute name="RmiPort">0</attribute>

    <!-- Port on which the HA-JNDI stub is made available -->

    <attribute name="Port">1098</attribute>

    <!-- Backlog to be used for client-server RMI 

         invocations during JNDI queries -->

    <attribute name="Backlog">50</attribute>

    

    <!-- Client socket factory to be used for client-server 

         RMI invocations during JNDI queries -->

    <!--attribute

name="ClientSocketFactory">custom</attribute-->

    <!-- Server socket factory to be used for client-server

         RMI invocations during JNDI queries -->

    <!--attribute

name="ServerSocketFactory">custom</attribute-->

  </mbean>



  <mbean code="org.jboss.invocation.jrmp.server.JRMPInvokerHA"

         name="jboss:service=invoker,type=jrmpha">

    <!--

    <attribute name="RMIObjectPort">0</attribute>

    <attribute name="RMIClientSocketFactory">custom</attribute>

    <attribute name="RMIServerSocketFactory">custom</attribute>

    <attribute name="RMIServerSocketAddr">custom</attribute>

    -->

  </mbean>



</server>





>From the service-bindings.xml

    <service-config name="jboss:service=JndiPartition"

     

delegateClass="org.jboss.services.binding.AttributeMappingDelegate">

      <delegate-config>

        <attribute name="PartitionName"

          >ProdJndi</attribute>

      </delegate-config>

      <binding/>

    </service-config>









----------------------------------------------------------------------

Comment By: Bob Cotton (bcotton969)
Date: 2003-09-11 18:17

Message:
Logged In: YES 
user_id=424902

This is jboss 3.2.1 with a new javagroups.jar that I

received from you.



Decompile of org.javagroups.Version yields:



    public static String version = "2.0.8";

    public static byte[] version_id = { 48, 50, 48, 56 };

    public static String cvs

        = "$Id: Version.java,v 1.19 2003/04/04 22:54:12

belaban Exp $";



----------------------------------------------------------------------

Comment By: Sacha Labourey (slaboure)
Date: 2003-09-11 15:20

Message:
Logged In: YES 
user_id=95900

Which specific JBoss release? JavaGroups bugs have been 

fixed in recent jboss release that fix that kind of merge 

issues, you may want to try it.

----------------------------------------------------------------------

Comment By: Bob Cotton (bcotton969)
Date: 2003-09-11 15:02

Message:
Logged In: YES 
user_id=424902

Logs too big to attach. Logs can be accessed at:

 http://clubcotton.homeip.net/jboss/synxis-cluster-logs.zip







----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=376685&aid=804491&group_id=22866


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
JBoss-Development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development

Reply via email to