Hello All
I have added a test suite in csit, which tests following functions in a 3 node cluster. "Initial inventory shard status test among cluster nodes. Switch connected to leader or follower node and verify flow add, modify, delete in leader or follower nodes. Similarly verify RPC flow add, modify, delete in leader as well as follower nodes. Reiterate the same functional test after leader restart, follower restart and cluster restart." /test/csit/suites/openflowplugin/Clustering# 020__Cluster_HA_Leader_Follower_Failover.robot Please give your feedback. Sample test run result: root@mininet-vm:/home/mininet/integration/test/csit/suites/openflowplugin/Clustering# pybot -L TRACE -v MININET_USER:mininet -v USER_HOME:/home/mininet -v CONTROLLER:10.183.181.51 -v CONTROLLER1:10.183.181.52 -v CONTROLLER2:10.183.181.53 -v USER:root -v PASSWORD:rootroot -v DEFAULT_LINUX_PROMPT:\# -v NUM_ODL_SYSTEM:3 -v MININET_PASSWORD:rootroot -v ODL_OF_PLUGIN:helium -v KARAF_HOME:/home/mininet/controller-Be/deploy/current/odl -v WORKSPACE:/home/mininet -v BUNDLEFOLDER:controller-Be/deploy/current/odl 020__Cluster_HA_Leader_Follower_Failover.robot ============================================================================== Cluster HA Leader Follower Failover :: Test suite for Cluster HA - Device L... ============================================================================== Create Original Cluster List :: Create original cluster list. | PASS | ------------------------------------------------------------------------------ Check Shards Status Before Leader Restart :: Check Status for all ... | PASS | ------------------------------------------------------------------------------ Get inventory Leader Before Leader Restart :: Find leader in the i... | PASS | ------------------------------------------------------------------------------ Start Mininet Connect To Leader :: Start mininet with connection t... | PASS | ------------------------------------------------------------------------------ Add Flows In Leader and Verify Before Leader Restart :: Add Flow v... | PASS | ------------------------------------------------------------------------------ Modify Flows In Leader and Verify Before Leader Restart :: Modify ... | PASS | ------------------------------------------------------------------------------ Delete Flows In Leader and Verify Before Leader Restart :: Delete ... | PASS | ------------------------------------------------------------------------------ Send RPC Add to Leader and Verify Before Leader Restart :: Add Flo... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Leader and Verify Before Leader Restart :: Dele... | PASS | ------------------------------------------------------------------------------ Send RPC Add to Follower Node1 and Verify Before Leader Restart ::... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Follower Node2 and Verify Before Leader Restart... | PASS | ------------------------------------------------------------------------------ Stop Mininet Connected To Leader and Exit :: Stop mininet and exit... | PASS | ------------------------------------------------------------------------------ Restart Leader From Cluster Node :: Kill Leader Node and Start it ... | PASS | ------------------------------------------------------------------------------ Get inventory Follower After Leader Restart :: Find new Followers ... | PASS | ------------------------------------------------------------------------------ Start Mininet Connect To Follower Node1 :: Start mininet with conn... | PASS | ------------------------------------------------------------------------------ Add Flows In Follower Node2 and Verify Before Follower Restart :: ... | PASS | ------------------------------------------------------------------------------ Modify Flows In Follower Node2 and Verify Before Follower Restart ... | PASS | ------------------------------------------------------------------------------ Delete Flows In Follower Node2 and Verify Follower Restart :: Dele... | PASS | ------------------------------------------------------------------------------ Send RPC Add to Leader and Verify Before Follower Restart :: Add F... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Leader and Verify Before Follower Restart :: De... | PASS | ------------------------------------------------------------------------------ Send RPC Add to First Follower Node1 and Verify Before Follower Re... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Follower Node2 and Verify Before Follower Resta... | PASS | ------------------------------------------------------------------------------ Stop Mininet Connected To Follower and Exit :: Stop mininet and ex... | PASS | ------------------------------------------------------------------------------ Restart Follower Node2 :: Kill Follower Node2 and Start it Up, Ver... | PASS | ------------------------------------------------------------------------------ Get inventory Follower After Follower Restart :: Find Followers an... | PASS | ------------------------------------------------------------------------------ Start Mininet Connect To Follower Node2 :: Start mininet with conn... | PASS | ------------------------------------------------------------------------------ Add Flows In Follower Node1 and Verify Before Cluster Restart :: A... | PASS | ------------------------------------------------------------------------------ Modify Flows In Follower Node1 and Verify Before Cluster Restart :... | PASS | ------------------------------------------------------------------------------ Delete Flows In Follower Node1 and Verify Before Cluster Restart :... | PASS | ------------------------------------------------------------------------------ Send RPC Add to Leader and Verify Before Cluster Restart :: Add Fl... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Leader and Verify Before Cluster Restart :: Del... | FAIL | ------------------------------------------------------------------------------ Send RPC Add to Follower Node2 and Verify Before Cluster Restart :... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Follower Node1 and Verify Before Cluster Restar... | PASS | ------------------------------------------------------------------------------ Stop Mininet Connected To Other Follower and Exit :: Stop mininet ... | PASS | ------------------------------------------------------------------------------ Restart Full Cluster :: Kill all Cluster Nodes and Start it Up All. | PASS | ------------------------------------------------------------------------------ Get inventory Status After Cluster Restart :: Find New Followers a... | PASS | ------------------------------------------------------------------------------ Start Mininet Connect To Follower Node2 After Cluster Restart :: S... | PASS | ------------------------------------------------------------------------------ Add Flows In Follower Node1 and Verify After Cluster Restart :: Ad... | PASS | ------------------------------------------------------------------------------ Modify Flows In Follower Node1 and Verify After Cluster Restart ::... | PASS | ------------------------------------------------------------------------------ Delete Flows In Follower Node1 and Verify After Cluster Restart ::... | PASS | ------------------------------------------------------------------------------ Send RPC Add to Leader and Verify After Cluster Restart :: Add Flo... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Leader and Verify After Cluster Restart :: Dele... | FAIL | ------------------------------------------------------------------------------ Send RPC Add to Follower Node2 and Verify After Cluster Restart ::... | PASS | ------------------------------------------------------------------------------ Send RPC Delete to Follower Node2 and Verify After Cluster Restart... | PASS | ------------------------------------------------------------------------------ Stop Mininet Connected To Follower Node2 and Exit After Cluster Re... | PASS | ------------------------------------------------------------------------------ Cluster HA Leader Follower Failover :: Test suite for Cluster HA -... | FAIL | 45 critical tests, 43 passed, 2 failed 45 tests total, 43 passed, 2 failed ============================================================================== Output: /home/mininet/integration/test/csit/suites/openflowplugin/Clustering/output.xml Log: /home/mininet/integration/test/csit/suites/openflowplugin/Clustering/log.html Report: /home/mininet/integration/test/csit/suites/openflowplugin/Clustering/report.html root@mininet-vm:/home/mininet/integration/test/csit/suites/openflowplugin/Clustering# PS: The two failed test cases passed in subsequent run. However there are various BUG raised in https://bugs.opendaylight.org/ for OF cluster sporadic failures. Thanks Sanjib
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
