unacked persistent messages don't get to messageStore of newbie cluster broker.
-------------------------------------------------------------------------------
Key: QPID-1618
URL: https://issues.apache.org/jira/browse/QPID-1618
Project: Qpid
Issue Type: Bug
Components: C++ Broker
Affects Versions: M4
Reporter: michael j. goulish
Priority: Critical
Attachments: unack_newbie.diff
When a new broker is added to a cluster, persistent messages that have not yet
been ACKed do not get stored in the new brokers messageStore.
How to reproduce: ( this is overview -- actual scripts follow )
========================================================
1. kill any MY_CLUSTER brokers from previous test
2. remove data dirs from previous test, and recreate
3. start node 1 as first member of MY_CLUSTER
4 declare the MY_CLUSTER queue -- durable
5. connect to it with receiver -- use ack frequency of 10;
6. connect with "publish" send only 5 persistent msgs, receiver
will get them but not ack.
7. start second node
8. when the state transfer completes kill both nodes.
(receiver should also perish)
9. start up the second node only, as new first member of MY_CLUSTER.
( I.e. use 2nd broker's data dir path. )
10. start and attach a new receiver
PREVIOUS RESULT -- nothing. messages were never stored in broker2's
messageStore.
RESULT -- new receiver now gets 5 messages.
durable queue says that "sender confirmed point moved to (5+0)"
==============================================
Scripts for reproducing problem.
==============================================
###################################################################
############# step 1 ##############
###################################################################
#! /bin/bash
echo "------------------------------------------------"
echo "Step 1: Kill any brokers already running in the MY_CLUSTER cluster"
echo "------------------------------------------------"
kill -9 `pgrep -f MY_STORE`
echo "There should be no remaining brokers. Here they are: "
ps -aef | grep qpidd | grep -v grep
###################################################################
############# step 2 ##############
###################################################################
#! /bin/bash
echo "------------------------------------------------"
echo "Step 2: remove and rebuild the store data dirs."
echo "------------------------------------------------"
rm -rf ./data
mkdir -p ./data/1 ./data/2
###################################################################
############# step 3 ##############
###################################################################
#! /bin/bash
echo "------------------------------------------------"
echo "Step 3: Start the first broker in a cluster, using Store in ./data/1 ."
echo "------------------------------------------------"
rm broker_1.log
$QPID_ROOT/cpp/src/qpidd --no-module-dir \
--load-module $QPID_ROOT/cpp/src/.libs/cluster.so \
--load-module $STORE_ROOT/cpp/lib/.libs/msgstore.so \
--cluster-name MY_CLUSTER -p 5813 \
--auth=no --mgmt-enable=no \
--log-enable debug --log-to-file ./broker_1.log \
--data-dir ./data/1
###################################################################
############# step 4 ##############
###################################################################
#! /bin/bash
echo "Step 4: Declaring queue."
$QPID_ROOT/cpp/examples/direct/declare_queues
# !! NOTE !!
# edit declare_queues.cpp to do port 5813
# in call to session.queueDeclare use arg::durable=true
# and arg::queue="MY_QUEUE",
# edit exchangeBind call to use:
# arg::queue = "MY_QUEUE",
###################################################################
############# step 5 ##############
###################################################################
#! /bin/bash
echo "Step 5: Starting receiver..."
$QPID_ROOT/cpp/src/tests/receiver \
-p 5813 \
--queue MY_QUEUE \
--messages 10 \
--ack-frequency 10
###################################################################
############# step 6 ##############
###################################################################
#! /bin/bash
echo "Publish only 5 messages, so the receiver will not yet ack."
$QPID_ROOT/cpp/src/tests/publish \
-p 5813 \
--count 5 \
--durable yes \
--destination amq.direct \
--routing-key routing_key \
--log-enable debug
#! /bin/bash
###################################################################
############# step 7 ##############
###################################################################
rm ./broker_2.log
echo "------------------------------------------------"
echo "Step 7: Start second node, using Store in ./data/2 ."
echo " note! you should see correct store dirs replicated to this broker."
echo "------------------------------------------------"
$QPID_ROOT/cpp/src/qpidd --no-module-dir \
--load-module $QPID_ROOT/cpp/src/.libs/cluster.so \
--load-module $STORE_ROOT/cpp/lib/.libs/msgstore.so \
--cluster-name MY_CLUSTER -p 5814 \
--auth=no --mgmt-enable=no \
--log-enable debug --log-to-file ./broker_2.log \
--data-dir ./data/2
###################################################################
############# step 8 ##############
###################################################################
#! /bin/bash
echo "When the second node is up and synced, kill both nodes."
echo " note: receiver should now die also."
kill -9 `pgrep -f MY_CLUSTER`
ps -aef | grep qpidd | grep -v grep
###################################################################
############# step 9 ##############
###################################################################
#! /bin/bash
echo "------------------------------------------------"
echo "Step 9: Start second node, using Store in ./data/2 ."
echo "------------------------------------------------"
$QPID_ROOT/cpp/src/qpidd --no-module-dir \
--load-module $QPID_ROOT/cpp/src/.libs/cluster.so \
--load-module $STORE_ROOT/cpp/lib/.libs/msgstore.so \
--cluster-name MY_CLUSTER -p 5813 \
--auth=no --mgmt-enable=yes \
--log-enable debug --log-to-file ./broker_3.log \
--data-dir ./data/2
###################################################################
############# step 10 ##############
###################################################################
#! /bin/bash
echo " "
echo "Start final receiver -- it should get unacked messages."
echo " "
$QPID_ROOT/cpp/src/tests/receiver \
-p 5813 \
--queue MY_QUEUE \
--messages 10 \
--ack-frequency 5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
