Reappearing messages with qpidc and bdb persistency store

[Aleš Trček]

Hi all,

Every now and then, when I restart my qpidc broker (version 0.22) some messages 
reappear in the queues, although they were long since taken from the queues 
(and committed).
Something like that (qpid-stat output):
  queue  dur  autoDel  excl  msg   
msgIn  msgOut  bytes  bytesIn  bytesOut  cons  bind
  
==
  some_queue_or_another  Y0   125
125   0   1.31k1.31k1 5

I then restart (qpidd --quit; qpidd --daemon --load-module ) the broker, 
and I get something like this:
  queue  dur  autoDel  excl  msg   
msgIn  msgOut  bytes  bytesIn  bytesOut  cons  bind
  
==
  some_queue_or_another  Y   125   125   0  
  1.31k   1.31k   01 5

Queues are created like this:
qpid-config add queue some_queue_or_another --durable 
--argument=qpid.priorities=10 --file-count=16 --file-size=512

As said, sometimes the messages magically reappear, sometimes they don't. I 
didn't yet get the pattern when or why.
I use transacted sessions, and I am sure, that all messages get committed OK.

Did anybody else have some similar problem?


Regards,
Ales

RE: Possibility Running multiple instances of broker on the same machine.

[Steve Huston]

Yes, they are independent.
Good luck,
-Steve

> -Original Message-
> From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> Sent: Thursday, August 29, 2013 4:51 PM
> To: users@qpid.apache.org
> Subject: Re: Possibility Running multiple instances of broker on the same
> machine.
> 
> Yeah I tried this
> 
> C:\qpid-0.18\qpid-0.18\cpp\src\Release>qpidd --tcp-nodelay --auth=no
> --port=5677
>  --data-dir=QPIDDEVENTORDER.DATA
> 
> and it works. So both of them are totally independant of each other now am I
> correct ?
> 
> 
> On Thu, Aug 29, 2013 at 4:48 PM, Steve Huston 
> wrote:
> 
> > Yes, you'll need a separate data directory. I apologize for leaving
> > that out.
> >
> > > -Original Message-
> > > From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> > > Sent: Thursday, August 29, 2013 4:45 PM
> > > To: users@qpid.apache.org
> > > Subject: Re: Possibility Running multiple instances of broker on the
> > > same machine.
> > >
> > > Thank you for your reply Steve. I tried doing that and I get the
> > following
> > > response:
> > >
> > > C:\qpid-0.18\qpid-0.18\cpp\src\Release>qpidd --tcp-nodelay --auth=no
> > > --port=5677
> > >
> > > 2013-08-29 16:44:18 [Broker] critical Unexpected error:
> > > \TEMP\QPIDD.DATA/lock: T
> > > he process cannot access the file because another process has locked
> > > a portion o f the file.
> > >
> > > any way I could tell it to create or access another .DATA file ?
> > >
> > >
> > > On Thu, Aug 29, 2013 at 4:40 PM, Steve Huston 
> > > wrote:
> > >
> > > > Add the --port= option to specify a new port.
> > > >
> > > > > -Original Message-
> > > > > From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> > > > > Sent: Thursday, August 29, 2013 4:37 PM
> > > > > To: users@qpid.apache.org
> > > > > Subject: Possibility Running multiple instances of broker on the
> > > > > same machine.
> > > > >
> > > > > We are currently running a C++ broker on our windows 7 machine
> > > > > using the following command:
> > > > >
> > > > > *qpidd --tcp-nodelay --auth=no*
> > > > >
> > > > > I wanted to know if it will be possible to launch another
> > > > > instance of the broker on the same machine. I believe we will
> > > > > have to give it a
> > > > different port
> > > > > to bind on but then again I am not sure. Any suggestions on how
> > > > > I could accomplish this?
> > > > >
> > > > > Thanks
> > > >
> > > > --
> > > > --- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For
> > > > additional commands, e-mail: users-h...@qpid.apache.org
> > > >
> > > >
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For
> > additional commands, e-mail: users-h...@qpid.apache.org
> >
> >

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Possibility Running multiple instances of broker on the same machine.

[Rajesh Khan]

Yeah I tried this

C:\qpid-0.18\qpid-0.18\cpp\src\Release>qpidd --tcp-nodelay --auth=no
--port=5677
 --data-dir=QPIDDEVENTORDER.DATA

and it works. So both of them are totally independant of each other now am
I correct ?


On Thu, Aug 29, 2013 at 4:48 PM, Steve Huston  wrote:

> Yes, you'll need a separate data directory. I apologize for leaving that
> out.
>
> > -Original Message-
> > From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> > Sent: Thursday, August 29, 2013 4:45 PM
> > To: users@qpid.apache.org
> > Subject: Re: Possibility Running multiple instances of broker on the same
> > machine.
> >
> > Thank you for your reply Steve. I tried doing that and I get the
> following
> > response:
> >
> > C:\qpid-0.18\qpid-0.18\cpp\src\Release>qpidd --tcp-nodelay --auth=no
> > --port=5677
> >
> > 2013-08-29 16:44:18 [Broker] critical Unexpected error:
> > \TEMP\QPIDD.DATA/lock: T
> > he process cannot access the file because another process has locked a
> > portion o f the file.
> >
> > any way I could tell it to create or access another .DATA file ?
> >
> >
> > On Thu, Aug 29, 2013 at 4:40 PM, Steve Huston 
> > wrote:
> >
> > > Add the --port= option to specify a new port.
> > >
> > > > -Original Message-
> > > > From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> > > > Sent: Thursday, August 29, 2013 4:37 PM
> > > > To: users@qpid.apache.org
> > > > Subject: Possibility Running multiple instances of broker on the
> > > > same machine.
> > > >
> > > > We are currently running a C++ broker on our windows 7 machine using
> > > > the following command:
> > > >
> > > > *qpidd --tcp-nodelay --auth=no*
> > > >
> > > > I wanted to know if it will be possible to launch another instance
> > > > of the broker on the same machine. I believe we will have to give it
> > > > a
> > > different port
> > > > to bind on but then again I am not sure. Any suggestions on how I
> > > > could accomplish this?
> > > >
> > > > Thanks
> > >
> > > -
> > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For
> > > additional commands, e-mail: users-h...@qpid.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

RE: Possibility Running multiple instances of broker on the same machine.

[Steve Huston]

Yes, you'll need a separate data directory. I apologize for leaving that out.

> -Original Message-
> From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> Sent: Thursday, August 29, 2013 4:45 PM
> To: users@qpid.apache.org
> Subject: Re: Possibility Running multiple instances of broker on the same
> machine.
> 
> Thank you for your reply Steve. I tried doing that and I get the following
> response:
> 
> C:\qpid-0.18\qpid-0.18\cpp\src\Release>qpidd --tcp-nodelay --auth=no
> --port=5677
> 
> 2013-08-29 16:44:18 [Broker] critical Unexpected error:
> \TEMP\QPIDD.DATA/lock: T
> he process cannot access the file because another process has locked a
> portion o f the file.
> 
> any way I could tell it to create or access another .DATA file ?
> 
> 
> On Thu, Aug 29, 2013 at 4:40 PM, Steve Huston 
> wrote:
> 
> > Add the --port= option to specify a new port.
> >
> > > -Original Message-
> > > From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> > > Sent: Thursday, August 29, 2013 4:37 PM
> > > To: users@qpid.apache.org
> > > Subject: Possibility Running multiple instances of broker on the
> > > same machine.
> > >
> > > We are currently running a C++ broker on our windows 7 machine using
> > > the following command:
> > >
> > > *qpidd --tcp-nodelay --auth=no*
> > >
> > > I wanted to know if it will be possible to launch another instance
> > > of the broker on the same machine. I believe we will have to give it
> > > a
> > different port
> > > to bind on but then again I am not sure. Any suggestions on how I
> > > could accomplish this?
> > >
> > > Thanks
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For
> > additional commands, e-mail: users-h...@qpid.apache.org
> >
> >

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Possibility Running multiple instances of broker on the same machine.

[Darryl L. Pierce]

On Thu, Aug 29, 2013 at 04:37:25PM -0400, Rajesh Khan wrote:
> We are currently running a C++ broker on our windows 7 machine using the
> following command:
> 
> *qpidd --tcp-nodelay --auth=no*
> 
> I wanted to know if it will be possible to launch another instance of the
> broker on the same machine. I believe we will have to give it a different
> port to bind on but then again I am not sure. Any suggestions on how I
> could accomplish this?

To specify a different port using --port  to use. However, you'll
also have issues with the lock file created by the broker process. In my
tests for running two parallel processes, even after telling the second
process to use a different location, they both tried to use the same
directory, which may be a bug.

-- 
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/



pgpkLXhop0TA9.pgp
Description: PGP signature

Re: Possibility Running multiple instances of broker on the same machine.

[Rajesh Khan]

Thank you for your reply Steve. I tried doing that and I get the following
response:

C:\qpid-0.18\qpid-0.18\cpp\src\Release>qpidd --tcp-nodelay --auth=no
--port=5677

2013-08-29 16:44:18 [Broker] critical Unexpected error:
\TEMP\QPIDD.DATA/lock: T
he process cannot access the file because another process has locked a
portion o
f the file.

any way I could tell it to create or access another .DATA file ?


On Thu, Aug 29, 2013 at 4:40 PM, Steve Huston  wrote:

> Add the --port= option to specify a new port.
>
> > -Original Message-
> > From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> > Sent: Thursday, August 29, 2013 4:37 PM
> > To: users@qpid.apache.org
> > Subject: Possibility Running multiple instances of broker on the same
> > machine.
> >
> > We are currently running a C++ broker on our windows 7 machine using the
> > following command:
> >
> > *qpidd --tcp-nodelay --auth=no*
> >
> > I wanted to know if it will be possible to launch another instance of the
> > broker on the same machine. I believe we will have to give it a
> different port
> > to bind on but then again I am not sure. Any suggestions on how I could
> > accomplish this?
> >
> > Thanks
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

RE: Possibility Running multiple instances of broker on the same machine.

[Steve Huston]

Add the --port= option to specify a new port.

> -Original Message-
> From: Rajesh Khan [mailto:rajeshkhan...@gmail.com]
> Sent: Thursday, August 29, 2013 4:37 PM
> To: users@qpid.apache.org
> Subject: Possibility Running multiple instances of broker on the same
> machine.
> 
> We are currently running a C++ broker on our windows 7 machine using the
> following command:
> 
> *qpidd --tcp-nodelay --auth=no*
> 
> I wanted to know if it will be possible to launch another instance of the
> broker on the same machine. I believe we will have to give it a different port
> to bind on but then again I am not sure. Any suggestions on how I could
> accomplish this?
> 
> Thanks

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Possibility Running multiple instances of broker on the same machine.

[Rajesh Khan]

We are currently running a C++ broker on our windows 7 machine using the
following command:

*qpidd --tcp-nodelay --auth=no*

I wanted to know if it will be possible to launch another instance of the
broker on the same machine. I believe we will have to give it a different
port to bind on but then again I am not sure. Any suggestions on how I
could accomplish this?

Thanks

Re: [c++] give application more direct control over reconnect & replay

[Gordon Sim]

On 08/29/2013 03:00 PM, Jakub Scholz wrote:

Right now, it seems that the reconnect(...) method always requires the URL
of the broker to reconnect to. Maybe it would be useful to also add a
reconnect() method without the URL parameter, to trigger the reconnect
based on the URL which was specified when creating the connection.


Yes, that sounds like a good idea. I'll update the patch to do that. Thanks!


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: [c++] give application more direct control over reconnect & replay

[Jakub Scholz]

Hi Gordon,

That definitely looks like a very nice feature. There are many situations
when you want to trigger the reconnect manually and not leave it fully on
the client library.

Right now, it seems that the reconnect(...) method always requires the URL
of the broker to reconnect to. Maybe it would be useful to also add a
reconnect() method without the URL parameter, to trigger the reconnect
based on the URL which was specified when creating the connection.

Thanks & Regards
Jakub


On Wed, Aug 28, 2013 at 5:30 PM, Gordon Sim  wrote:

> I have a proposal[1] for a small addition to the qpid::messaging API that
> makes the reconnect feature less of an 'all or nothing' affair[2]. For
> background context, this limitation was brought up most recently on the dev
> list back in June[3].
>
> Basically it exposes a new reconnect() method allowing the application to
> catch the TransportFailure and decide if and when and where to reconnect.
> Invoking that method then re-establishes all the sessions/senders/receivers
> and replays indoubt messages.
>
> Any feedback welcome as usual, either here or on reviewboard.
>
> --Gordon.
>
> [1] https://reviews.apache.org/r/**13885/
> [2] 
> https://issues.apache.org/**jira/browse/QPID-4932
> [3] http://qpid.2158936.n2.nabble.**com/Qpid-post-mortem-and-**
> request-for-suggestions-for-**my-next-release-challenge-10M-**
> msgs-sec-on-Windows-**tp7594096p7594258.html
>
>
> --**--**-
> To unsubscribe, e-mail: 
> users-unsubscribe@qpid.apache.**org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: Clearing up queues in C++

[Gordon Sim]

On 08/28/2013 11:50 PM, Rajesh Khan wrote:

Thanks that seems to do the trick. Any idea how I could do it
programatically .



Attached is a simple example program. However I would advise 
investigating the autodelete issue a little more as I'm fairly confident 
it will do as expected if used in the right manner.


/*
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 * 
 *   http://www.apache.org/licenses/LICENSE-2.0
 * 
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied.  See the License for the
 * specific language governing permissions and limitations
 * under the License.
 *
 */


#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

using namespace qpid::messaging;
using namespace qpid::types;

using std::string;

int main(int argc, char** argv)
{
Connection c(argc > 1 ? argv[1] : "localhost");
std::string queue(argc > 2 ? argv[2] : "my-queue");
try {
c.open();
Session session = c.createSession();
Address responses("qmf.default.direct/my-name; {node: {type: topic}}");
Receiver r = session.createReceiver(responses);
Sender s = session.createSender("qmf.default.direct/broker");
	Message request;
request.setReplyTo(responses);
request.setContentType("amqp/map");
request.setProperty("x-amqp-0-10.app-id", "qmf2");
request.setProperty("qmf.opcode", "_method_request");
Variant::Map oid;
std::stringstream name;
name << "org.apache.qpid.broker:queue:" << queue;
oid["_object_name"] = name.str();
Variant::Map content;
content["_method_name"] = "purge";
content["_object_id"] = oid;
Variant::Map arguments;
arguments["request"] = 0;//set this to non-zero value to purge 'N' messages; 0 will purge all
content["_arguments"] = arguments;

encode(content, request);
s.send(request);
Message response = r.fetch();
Variant::Map contentIn;
decode(response, contentIn);
std::cout << response.getProperties() << ": "  << contentIn << std::endl;
session.acknowledge();

} catch(const std::exception& error) {
std::cout << "ERROR: " << error.what() << std::endl;
}
c.close();
return 0;
}




-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Clearing up queues in C++

[Gordon Sim]

On 08/28/2013 09:53 PM, Rajesh Khan wrote:

I wanted to know what my options are when I need to clear up queues.
My first preference would be to clear them up programatically. I tried
setting a queue to auto-delete however that does not seem to work.I have a
sender and a receiver. The receiver creates a queue and sets the auto
delete parameter. However even after the receiver disconnects there still
is content in the queue.


Can you give more detail on the exact steps you took here? The 
auto-delete setting should certainly work. Note that if the queue is 
also exclusive then it will be auto-deleted when the session that 
created it ends (not just on cancelling the receiver. Note also that 
trying to set the auto-delete property on a queue already created won't 
work.


If you can provide some steps to reproduce the non-functioning of 
auto-delete, I can investigate some more.



-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Connection questions

[Gordon Sim]

On 08/29/2013 06:19 AM, Jeremy Wagner wrote:

Does a copy of a connection still share the same underlying pointer? How about 
through the assignment operator? Thanks!


Yes, Connections, Sessions, Senders and Receivers are all 'Handles', 
which essentially has a smart pointer to the actual state. So assignment 
or copy constructors give new handles pointing to the same logical entity.


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org