Re: High Memory consumption with Qpid Dispatch 1.19.0

[Virgilio Fornazin]

0
>
>   qdr_delivery_ref_t  24 64 128 512
> 384 1,671,475   1,671,477
>
>   qdr_delivery_t  31264 128 2,496
> 192 1,709,088   1,709,124
>
>   qdr_field_t 40 64 128 3,136
> 256 79,489  79,534
>
>   qdr_forward_deliver_info_t  32 64 128 64
> 64  0   0
>
>   qdr_general_work_t  13664 128 576
> 384 3,075,379   3,075,382
>
>   qdr_link_ref_t  24 64 128 17,088
> 13,888  32,905,297  32,905,347
>
>   qdr_link_t  55264 128 7,936
> 7,488   104,286 104,293
>
>   qdr_link_work_t 48 64 128 3,456
> 512 32,470,813  32,470,859
>
>   qdr_node_t  88 64 128 64
>64  0   0
>
>   qdr_query_t 34464 128 192
> 192 107 107
>
>   qdr_terminus_t  64 64 128 6,016
> 320 1,553   1,642
>
>   qdtm_router_t   16 64 128 128
> 128 0   0
>
>
>
> Memory Summary
>
>   VmSizePooled
>
>   
>
>   12.1 GiB  25.5 MiB
>
>
> *Below is the current statistics for qpid1 server -*
>
> Router Statistics
>
>   attr value
>
>   ===
>
>   Version  1.19.0
>
>   Mode interior
>
>   Router Idod-router-1-prod
>
>   Worker Threads   4
>
>   Uptime   006:21:31:41
>
>   VmSize   12.1 GiB
>
>   Area 0
>
>   Link Routes  0
>
>   Auto Links   6418
>
>   Links7246
>
>   Nodes1
>
>   Addresses1088
>
>   Connections  1690
>
>   Presettled Count 0
>
>   Dropped Presettled Count 0
>
>   Accepted Count   106722455
>
>   Rejected Count   0
>
>   Released Count   273624
>
>   Modified Count   962
>
>   Deliveries Delayed > 1sec4901525
>
>   Deliveries Delayed > 10sec   775262
>
>   Deliveries Stuck > 10sec 0
>
>   Deliveries to Fallback   0
>
>   Links Blocked1002
>
>   Ingress Count84870446
>
>   Egress Count 106011425
>
>   Transit Count712114
>
>   Deliveries from Route Container  52221048
>
>   Deliveries to Route Container32649333
>
>
>
> *Thanks,*
>
> *Nilesh Khokale*
>
>
>
> *From:* Virgilio Fornazin 
> *Sent:* Friday, December 15, 2023 4:23 PM
> *To:* users@qpid.apache.org
> *Cc:* Ajit Tathawade (Contractor) ; Nilesh
> Khokale (Contractor) ; Ted Ross <
> tr...@redhat.com>
> *Subject:* Re: High Memory consumption with Qpid Dispatch 1.19.0
>
>
>
> [CAUTION: EXTERNAL SENDER]
>
> We had this issue with qpid ++ broker and was related to vectors used in
> long running objects (queues etc) c++11 introduced shrink_to_fit() and we
> patched the broker at that time to avoid those memory issues.
>
>
>
> On Fri, 15 Dec 2023 at 18:21 Ekta Awasthi <
> ekta.awas...@theodpcorp.com.invalid> wrote:
>
> Hello Tod,
>
>
>
> while running the qdstat -m command, I get below error.
>
>
>
> -sh-4.2$ qdstat -a
>
> ConnectionException: Connection amqp://0.0.0.0:amqp disconnected:
> Condition('proton.pythonio', 'Connection refused to all addresses')
>
>
>
> *Ekta Awasthi*,
>
> Engineer, EAI Operations & Support | Office Depot, Inc.
> 6600 North Military Trail | Boca Raton, FL 33496-2434
> <https://www.google.com/maps/search/6600+North+Military+Trail+%7C+Boca+Raton,+FL+33496-2434+%0D%0AOffice:+561?entry=gmail=g>
> Office: 561
> <https://www.google.com/maps/search/6600+North+Military+Trail+%7C+Boca+Raton,+FL+33496-2434+%0D%0AOffice:+561?entry=gmail=g>-438-3552
> | Mobile: 206-966-5577 | ekta.awas...@officedepot.com
>
>
>
> *-- Tips for EAI Support Engagement --*
>
> -EAI Pre-Prod Support: Create requests on the following JIRA board EAI
> Operations Support
> <https://officedepot.atlassian.net/secure/RapidBoard.jspa?rapidView=823=EOS>
>
> -EAI Production Support: Create requests via IT Servi

Re: High Memory consumption with Qpid Dispatch 1.19.0

[Virgilio Fornazin]

We had this issue with qpid ++ broker and was related to vectors used in
long running objects (queues etc) c++11 introduced shrink_to_fit() and we
patched the broker at that time to avoid those memory issues.

On Fri, 15 Dec 2023 at 18:21 Ekta Awasthi
 wrote:

> Hello Tod,
>
> while running the qdstat -m command, I get below error.
>
> -sh-4.2$ qdstat -a
>
> ConnectionException: Connection amqp://0.0.0.0:amqp disconnected:
> Condition('proton.pythonio', 'Connection refused to all addresses')
>
> *Ekta Awasthi*,
>
> Engineer, EAI Operations & Support | Office Depot, Inc.
> 6600 North Military Trail | Boca Raton, FL 33496-2434
> 
> Office: 561
> -438-3552
> | Mobile: 206-966-5577 | ekta.awas...@officedepot.com
>
>
>
> *-- Tips for EAI Support Engagement --*
>
> -EAI Pre-Prod Support: Create requests on the following JIRA board EAI
> Operations Support
> 
>
> -EAI Production Support: Create requests via IT Service Desk
>  self-service
> portal, instructions click here
> :
> EAI Support queue --> ODP - Enterprise Apps Integration Support
>
> -As a reminder, the Service Availability Managers should be engaged for
> any service impacting issues, with a ***Page*** to
> naitavailabilitym...@officedepot.com or by initiating a MIRT
> --
> *From:* Ted Ross 
> *Sent:* Friday, December 15, 2023 2:48 PM
> *To:* Ekta Awasthi 
> *Cc:* users@qpid.apache.org ; Ajit Tathawade
> (Contractor) ; Nilesh Khokale (Contractor)
> 
> *Subject:* Re: High Memory consumption with Qpid Dispatch 1.19.0
>
> [CAUTION: EXTERNAL SENDER]
>
> Ekta,
>
> You can get more granular memory-use data by using the "qdstat -m" command
> against the router when its memory footprint is larger than you think it
> should be.
>
> I assume you've been using this version for some time.  It might be
> helpful to look into what other things changed right before the memory
> consumption problem started.
>
> -Ted
>
> On Fri, Dec 15, 2023 at 12:08 PM Ekta Awasthi 
> wrote:
>
> Hi All & Tod,
>
> We are currently encountering elevated memory consumption with qpid
> dispatch version 1.19.0. Although the memory is released upon restarting
> qpid, it gradually accumulates again, surpassing 80% memory usage. As QPID
> in our case servers as a routing mechanism, handling traffic from NLB to
> QPID and then to the broker. While investigating the cause of this behavior
> and examining memory usage from New Relic (NR) graph indicates that the
> qdrouterd process is responsible for the memory consumption. We are seeking
> insights into the root cause of this issue and whether it may be related to
> the version (1.19.0). Please find additional information below.
>
> *Architecture*:
> NLB --> QPID(2 qpids acting as consumers) --> BROKER (Total of 3 pairs.
> Master/Slave configuration for HA)
>
> *Qpids* were restarted on 12-10-23 as you can see below the gradual
> increase has been happening ever since.
>
> *Ekta Awasthi*
>
> CONFIDENTIALITY NOTICE: The information contained in this email and
> attached document(s) may contain confidential information that is intended
> only for the addressee(s). If you are not the intended recipient, you are
> hereby advised that any disclosure, copying, distribution or the taking of
> any action in reliance upon the information is prohibited. If you have
> received this email in error, please immediately notify the sender and
> delete it from your system.
>
>

Re: [EXTERNAL]Re: Installing qpid-cpp on RHEL9 with python3 fails

[Virgilio Fornazin]

I've managed to build and get all those working on RHEL8 & RHEL9 some time
ago.
Let me search for these files here to check what I did and I can bring you
some info on how to proceed.

On Sat, Apr 15, 2023 at 3:41 PM Jiri Daněk  wrote:

> I created a Dockerfile/Containerfile that compiles qpid-cpp on CentOS 9.
> I'm linking to it below, after a few more progress updates. Try it out (at
> https://github.com/apache/qpid-cpp/pull/40) if you want.
>
> On Fri, Apr 14, 2023 at 10:51 PM Do, Eling 
> wrote:
>
>> Hi Jiri,
>>
>> Thank you very much for the suggestions.  I will discuss these different
>> options with our development team.  I'm not sure that there are specific
>> qpid features that we are attached to but more concerned about the
>> effort/time needed to make any significant changes.
>>
>
> It turned out that getting the basic build to work on RHEL 9 was not that
> hard. It required only minimal changes on the qpid-cpp side [1] and
> relatively modest changes on the qpid-python [2] side after all
>
> [1] https://issues.apache.org/jira/browse/QPID-8635
> [2] https://issues.apache.org/jira/browse/QPID-8631 (up to commit
> 6d96be6 QPID-8631: make package version string in setup.py compliant with
> PEP-440)
>
> As you can see in the CI, the current main of qpid-cpp is running through
> the install and only fails on running the tests [3]. In the job, the
> current main of qpid-python is installed before trying to install qpid-cpp.
> To do this, the GitHub workflow does, in essence
>
> ```
> git clone https://github.com/apache/qpid-python.git
> cd qpid-python
> python3 setup.py install --user
> ```
>
> which is what I am putting into the Containerfile.
>
> [3]
> https://github.com/apache/qpid-cpp/actions/runs/4699600134/jobs/8341756181?pr=39#step:18:1283
>
> Here is a GitHub pull request that adds a Dockerfile/Containerfile based
> on CentOS 9 that builds qpid-cpp, similarly to how the build from Irina (
> https://github.com/irinabov/docker-qpid-cpp-broker) worked.
>
> Please try it out, whether what is built there seems sufficient for your
> use: https://github.com/apache/qpid-cpp/pull/40
>
> docker build -f Containerfile -t jdanekrh/jd_qpid-cpp .
> docker run --name qpidd --rm -it -p 5672:5672 jdanekrh/jd_qpid-cpp
>
> Regarding what's missing, and the issues I am aware of:
> 1) bindings (perl, python, ruby client) don't work. the binding code
> requires swig3 but rhel9 only has swig4, leading to runtime crashes due to
> incompatibilities when trying to use the bindings
> 2) qpid-tools (for managing the broker, such as `docker exec qpidd
> qpid-stat`) don't work, the broker has to be managed by qpid-tools running
> somewhere else, where python2 is available
>
> Does this help you move forward? What is the next issue that you are
> hitting?
> --
> Mit freundlichen Grüßen / Kind regards
> Jiri Daněk
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org

Re: Building Qpid C++ Broker for macOS

[Virgilio Fornazin]

MacOS does not have pthread_condattr_setclock, Mach kernel has monotonic
clock with different api sets, look at

https://stackoverflow.com/questions/11680461/monotonic-clock-on-osx
https://chromium.googlesource.com/chromium/src/base/+/master/synchronization/condition_variable_posix.cc

The easy way is to run broker on a docker container now

On Wed, Jan 25, 2023 at 11:35 AM Robbie Gemmell 
wrote:

> Even when it was at its most active, there was apparently not much
> developer interest in supporting the qpid-cpp broker running on MacOS
> (or accompanying Qpid::Messaging cpp client), with a concentration of
> interest on Linux. A look at the qpid-cpp repository would show that
> qpid-cpp is largely stagnant these days, so I doubt there will be much
> traction on the MacOS subject now either. The old mailing list posts
> would be your main avenue to any specifics.
>
> C++ client wise specifically, attention also shifted away from the old
> Qpid::Messaging C++ client to the Qpid Proton C++ binding many years
> ago. Proton does compile on MacOS, I believe it possibly uses libuv
> there (in contrast to typical Linux usage), which might be how it
> works at all...though again I think it would be fair to say the bulk
> of dev interest is around Linux.
>
> Robbie
>
> On Thu, 19 Jan 2023 at 11:42, Emmett Brown  wrote:
> >
> > Hello,
> >
> > I was exploring the Qpid project yesterday and noticed that while it
> doesn’t officially support macOS, there have been some online discussions
> in the past related to building Proton C++ and, the Dispatch Router for
> macOS. I think some of the issues had been test compilation failures.
> >
> > I haven’t noticed any such similar discussions related to building the
> C++ Broker on macOS. I thought I’d run a quick build myself, to see how far
> I got.
> >
> > Unsurprisingly, the build failed (I’m on Monterey 12.6.2). It was a
> pthread API issue. I’ll past the output below if anyone has any idea how to
> get past this, but my question is more generally, does anyone have any past
> experience trying to get the C++ Broker to compile on macOS, or does anyone
> know what the past sticking points were that prevented the C++ Broker (feel
> free to include any of the other C++ products in your answer if you wish)
> from being built on macOS?
> >
> > [ 20%] Building CXX object
> src/CMakeFiles/qpidcommon.dir/qpid/sys/posix/Condition.cpp.o
> >
> /Users/ray/Developer/User/qpid-cpp-1.39.0/src/qpid/sys/posix/Condition.cpp:34:36:
> error: use of undeclared identifier 'pthread_condattr_setclock'; did you
> mean 'pthread_condattr_setpshared'?
> > QPID_POSIX_ASSERT_THROW_IF(pthread_condattr_setclock(,
> CLOCK_MONOTONIC));
> >^
> >pthread_condattr_setpshared
> >
> /Users/ray/Developer/User/qpid-cpp-1.39.0/src/qpid/sys/posix/check.h:45:63:
> note: expanded from macro 'QPID_POSIX_ASSERT_THROW_IF'
> > #define QPID_POSIX_ASSERT_THROW_IF(ERRNO) QPID_POSIX_THROW_IF(ERRNO)
> >   ^
> >
> /Users/ray/Developer/User/qpid-cpp-1.39.0/src/qpid/sys/posix/check.h:41:17:
> note: expanded from macro 'QPID_POSIX_THROW_IF'
> > do { int e=(ERRNO); if (e) throw QPID_POSIX_ERROR(e); } while(0)
> > ^
> >
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include/pthread.h:325:5:
> note: 'pthread_condattr_setpshared' declared here
> > int pthread_condattr_setpshared(pthread_condattr_t *, int);
> > ^
> > 1 error generated.
> > make[2]: ***
> [src/CMakeFiles/qpidcommon.dir/qpid/sys/posix/Condition.cpp.o] Error 1
> > make[1]: *** [src/CMakeFiles/qpidcommon.dir/all] Error 2
> > make: *** [all] Error 2
> >
> >
> > Kind regards,
> > Emmett Brown
> >
> > emmett@moda.tools
> > PGP: 2EB563D958F5CB8153AAED477867D39BF7356F59
> >
> >
> >
> >
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: qpid c++ broker status

[Virgilio Fornazin]

I used C++ broker for about a decade, it was able to send/receive 85
messages / second (average 500 bytes, with lowest latency possible, near
few milliseconds on high load) on a 8-core (16 threads) 64gb RAM SCSI 15k
rpm disks, using non-persistent queues and AMQP 0-10. These performance we
are talking are of machines a decade ago (2012 XEON E5690, 3rd CORE2
generation).

It supports clustering out of box, but it doesn't have a good UI to manage
/ administer, but QMF protocol is able to perform lots of tasks.

If you need raw pérformance, there's nothing that compares to.


On Mon, Oct 10, 2022 at 3:00 PM Tom Jordahl 
wrote:

> I will say that my team tried to migrate from ActiveMQ (classic) to
> Artemis a few years ago and it did not go well.
>
> We then migrated to Qpid Broker-J (and the AMQP/JMS client library) and it
> has been solid – more than ActiveMQ ever was for our use case (~100 queues,
> some with GroupIDs, fairly “low” volume – max 100’s of messages per second).
>
> I am a big fan of Broker-J and the team that supports it here – the stuff
> just works and works well.  We have brought problems to the list and they
> have been resolved, either by pointing us to the solution or adding
> functionality that we needed.
>
> --
> Tom
>
> From: Daniil Kirilyuk 
> Date: Monday, October 10, 2022 at 11:02 AM
> To: users@qpid.apache.org 
> Subject: Re: qpid c++ broker status
>
> Broker-J is maintained, although not many people are currently working on
> it. Regarding the Java 17 compatibility, the issues you pointed out were
> fixed with QPID-8586  and
> QPID-8587 . Currently the
> main Broker-J branch should be able to be build both under Java 11 and Java
> 17.
>
> Kind regards,
> Daniil Kirilyuk
>
> On Mon, 10 Oct 2022 at 17:35, Paul  wrote:
>
> > How about Broker-j, is it still being maintained? I posted in the dev
> > mailing list regarding a unit test issues last week and have received no
> > response. I am currently working on updating it (8.0.6 is the code base
> > I am working with) to Java 17 and building a native image (which
> > partially worked with Java 11).
> >
> > Thanks,
> >
> > PGA
> >
> > On 10/10/2022 2:32 AM, Gordon Sim wrote:
> > > ActiveMQ Artemis is probably the most obvious choice as it has the
> > > most ongoing activity.
> > >
> > > The fact is, as you point out, the c++ broker is not being actively
> > > maintained. Unless there are people willing to put in some time to do
> > > that, I think it is better to be clear about how things stand.
> > >
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> > For additional commands, e-mail: users-h...@qpid.apache.org
> >
> >
>

Re: Epel 8 Qpid cpp

[Virgilio Fornazin]

I've built it. Can share you tou.

On Thu, Apr 1, 2021 at 11:43 AM fostercm2  wrote:

> Hello,I just had a question, hopefully someone could answer.Are there any
> plans to publish rpms to Epel 8 for qpid-cpp?  It looks like the
> qpid-proton packages are available, but not qpid-cpp.Thanks!Sent from my
> Verizon, Samsung Galaxy smartphone

Re: [qdr] two-router TCP test passes 220 million HTTP requests.

[Virgilio Fornazin]

Saturate a ethernet card is easy.

The kernel translation to user mode, packet handling, sending back to
kernel stack, network layer
tooks a big path and involve a lots of context / cpu execution mode (ring0
- kernel, ring3 - user) switches,
and this affects directly your latency, stability and throughput.

handling 1000 reqs /s is pretty vanilla since 90's ...


On Mon, Mar 29, 2021 at 1:43 PM Michael Goulish  wrote:

> Oh I know that's not very fast, but this is an endurance test, not a
> throughput test. I want to see that latency does not rise over a long
> period run.  If you try for maximum throughput, you mess up latency and
> then you can't see if something is changing slowly over time.
>
> A while ago I used iperf3 for a throughput test for the TCP adapter in
> which we were able to saturate a 40 Gbit/sec interface.(I was only able
> to do that by using two separate iperf3 sender/receiver pairs, pointing in
> opposite directions.)
>
> I think we were not able to saturate the 40 Gbit link with just one iperf3
> sender/receiver pair because the receiver went to 100% CPU.
>
>
>
>
>
> On Mon, Mar 29, 2021 at 12:11 PM Virgilio Fornazin <
> virgilioforna...@gmail.com> wrote:
>
> > 1000 req/s is SOO
> > sLLLlloooW w w   ww w . . .
> >
> > qpidd c++ broker was able to 800.000k msg in / 800.000k msg out on a
> > 12-core xeon e5690 32gb ram , 2x 10gbe lan, rhel 6.x.
> > Test ran was on 2011, current HW should be at least 2 / 3 times better...
> >
> > On Mon, Mar 29, 2021 at 3:59 AM Michael Goulish 
> > wrote:
> >
> > > * The test has now passed 220,000 seconds (2.5 days) with no failure.
> > 1000
> > > requests per second, and a new batch of 100 Hey workers every 60
> seconds.
> > >
> > > * Average response time is not changing. It has been between 1 and 2
> msec
> > > the whole test.
> > >
> > > * Router memory does *not* appear to be growing without bound. It is
> > larger
> > > than it was at the start, but the intervals between little upticks are
> > > becoming longer and longer. Last uptick (of 264K) was 20 hours ago.
> > > (Looking at router on receiving side.)
> > >
> > >
> > >
> > >
> > > Using the Hey load generator against Nginx server, with two routers in
> > the
> > > middle -- either router on its own box, fast link between them.
> > >
> > > Hey is using 100 parallel workers, each doing 10 HTTP requests per
> > second.
> > >
> > > Hey is doing repeated 60-second tests, and reporting statistics on each
> > > one.
> > >
> > > Unfortunately I still cannot run qdstat, although I have both pythons
> > > installed and have done standard builds of both proton and dispatch.
> > Can't
> > > find python module named 'proton'.
> > >
> > > Test is continuing until I need the machines for something else.
> > >
> >
>

Re: [qdr] two-router TCP test passes 220 million HTTP requests.

[Virgilio Fornazin]

1000 req/s is SOO
sLLLlloooW w w   ww w . . .

qpidd c++ broker was able to 800.000k msg in / 800.000k msg out on a
12-core xeon e5690 32gb ram , 2x 10gbe lan, rhel 6.x.
Test ran was on 2011, current HW should be at least 2 / 3 times better...

On Mon, Mar 29, 2021 at 3:59 AM Michael Goulish  wrote:

> * The test has now passed 220,000 seconds (2.5 days) with no failure. 1000
> requests per second, and a new batch of 100 Hey workers every 60 seconds.
>
> * Average response time is not changing. It has been between 1 and 2 msec
> the whole test.
>
> * Router memory does *not* appear to be growing without bound. It is larger
> than it was at the start, but the intervals between little upticks are
> becoming longer and longer. Last uptick (of 264K) was 20 hours ago.
> (Looking at router on receiving side.)
>
>
>
>
> Using the Hey load generator against Nginx server, with two routers in the
> middle -- either router on its own box, fast link between them.
>
> Hey is using 100 parallel workers, each doing 10 HTTP requests per second.
>
> Hey is doing repeated 60-second tests, and reporting statistics on each
> one.
>
> Unfortunately I still cannot run qdstat, although I have both pythons
> installed and have done standard builds of both proton and dispatch. Can't
> find python module named 'proton'.
>
> Test is continuing until I need the machines for something else.
>

Re: Qpid broker memory increases when a receiver is paused with message pending acks

[Virgilio Fornazin]

this is a well known issue... we had running for almost 8 years with
broker, it never releases cache back to S.O.

it´s related to the vectors<> (C++) used in queue management structures,
they never release back ram to S.O.

c++11 and beyond implemented shrink_to_fit() call to release memory back,
but I don't know if broker is built using C++11 nowadays and call those
metods on queue empty / purges / etc.

On Thu, Sep 10, 2020 at 12:23 PM Kalyanaraman Sivaraman 
wrote:

> Given the situation where we have a receiver "Receiver1" acquire 1 message
> from a queue "ReceiverQueue" but if the message is not acknowledged and
> Receiver1 process is paused using kill -STOP .
>
> Any further messages sent to ReceiverQueue is marked as DELETED even though
> we have another receiver "Receiver2" properly acquire messages from the
> queue and also sends acknowledgements, the broker memory linearly increases
> until all the memory in the box is used. Purger cleaning up messages does
> not help. Once we kill Receiver1 the broker memory stabilizes.
>
> I have opened up a JIRA issue on this QPID-8462
>
> https://issues.apache.org/jira/browse/QPID-8462
>
> There has been no response to it as of now.
>
> Has Anyone Seen this behavior?
>
>
>
> --
> Sent from:
> http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: default limits of qpid-c++ broker, dispatch router, or proton?

[Virgilio Fornazin]

we've managed to do ~78 msg /s on qpid-1.36 on 3 qpid broker instances
on a intel X5670, 64GB ram, 4x 1gb lan, 16 clients flooding, 16 clients
receiving (~48000msg /s on each direction).
on a test running on 2011.

I'm pretty sure C++ broker is able to do a bit more than your numbers in
AMQP-0-10


On Thu, Jul 9, 2020 at 11:43 AM tomt  wrote:

> Hello,
>
> I have been trying to determine the highest rate of messages that can be
> sent at any given time through my environment that includes the qpid-cpp
> broker and the dispatch router all using AMQP 1.0 (via Proton).
>
> There seems to be some maximum cap that only lets me send up to 2500
> messages/sec and I'm trying to determine if that's a limitation of my
> environment or perhaps a default in any of the mentioned packages.  I have
> 2
> senders blasting messages and 10 receivers subscribing to all of them.
>
> I have searched around and it looks like any sort of limiting is off by
> default in the broker and router, but that 2500 number is consistent and
> makes me think that there could be a limiter somewhere.  Any sort of
> pointers or suggestions would be greatly appreciated.
>
> Thank you.
>
>
>
> --
> Sent from:
> http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: QPID C++ API : How to check whether Queue/topic has any consumers

[Virgilio Fornazin]

Ah it's using Artemis Broker... should look at Artemis list instead

On Mon, Jun 22, 2020 at 3:01 PM Gordon Sim  wrote:

> On 22/06/2020 6:49 pm, Virgilio Fornazin wrote:
> > show source code of qpid-stat tools, it does what you want
>
> Not really. That uses QMF which ActiveMQ Artemis will not understand.
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: QPID C++ API : How to check whether Queue/topic has any consumers

[Virgilio Fornazin]

show source code of qpid-stat tools, it does what you want

On Mon, Jun 22, 2020 at 8:59 AM Gordon Sim  wrote:

> On 22/06/2020 11:59 am, mohank wrote:
> > Client : QPID C++
> > Broker : ActiveMQ Artemis
> >
> > Is there any possibility to check whether *topic address*/*queue* has any
> > consumer?
> >
> > Using Artemis.cmd, JMS and rest API provided by ActiveMQ we can get this
> > data.
> > But is it possible to get the same using QPID C++ APIS(i.e. QMF, is it
> only
> > restricted to manage QPID broker or we can manage other brokers like
> > ActiveMQ Artemis)
>
> QMF is a message based protocol for management of the qpid c++ broker
> and will not help with artemis.
>
> However artemis has its own message based protocol for invoking
> management operations, which you can use over AMQP with any client API.
> I'm not sure if that protocol is actually documented anywhere (you could
> ask on activemq user list).
>
> The basics are that you send requests to 'activemq.management' with
> application properties '_AMQ_ResourceName' and '_AMQ_OperationName' used
> to identify the particular call you are making. The body of the message
> is a JSON string representign the parameters to the call. You should set
> up a receiver for replies and set the reply-to on the request in order
> to get them.
>
> Responses will have application property '_AMQ_OperationSucceeded' set
> to indicate success. If it succeeds, any response values will be encoded
> as JSON string in the body. If not you may get an error message in the
> body I think.
>
> E.g. I believe to get the number of consumers on a queue named foo, you
> would send a request with _AMQ_ResourceName set to queue.foo and
> _AMQ_OperationName set to getConsumerCount and the body set to '[]'
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: qpid-cpp-server 1.36 for CentOS6/RHEL6

[Virgilio Fornazin]

https://www.dropbox.com/s/by03lcmhbirqx9m/qpid136rhel7.tar?dl=0

it expires on 24hours

On Mon, Jun 1, 2020 at 2:08 PM rammohan ganapavarapu <
rammohanga...@gmail.com> wrote:

> Virrgilio,
>
> The link you have provided is still exist? i am getting 404, can you please
> share again?
>
> Ram
>
> On Sat, May 30, 2020 at 3:50 AM Virgilio Fornazin <
> virgilioforna...@gmail.com> wrote:
>
> > I've built by myself for rhel6, 7, 8 ...
> >
> > https://www.dropbox.com/s/kawv8dfrf3ez4x8/qpid136rhel6.tar?dl=0
> >
> >
> > On Sat, May 30, 2020 at 1:18 AM rammohan ganapavarapu <
> > rammohanga...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I am looking for the qpid-cpp-server 1.36 and dependent rpms  for
> > > CentOS6/RHEL6, can some one suggest where can i download them?
> > >
> > > Thanks,
> > > Ram
> > >
> >
>

Re: qpid-cpp-server 1.36 for CentOS6/RHEL6

[Virgilio Fornazin]

I've built by myself for rhel6, 7, 8 ...

https://www.dropbox.com/s/kawv8dfrf3ez4x8/qpid136rhel6.tar?dl=0


On Sat, May 30, 2020 at 1:18 AM rammohan ganapavarapu <
rammohanga...@gmail.com> wrote:

> Hi,
>
> I am looking for the qpid-cpp-server 1.36 and dependent rpms  for
> CentOS6/RHEL6, can some one suggest where can i download them?
>
> Thanks,
> Ram
>

Re: QPID C++ AMQP 1.0 (create Queue and routing key)

[Virgilio Fornazin]

sure it has, I have this code somewhere, let me take a look I'll post it
here

On Wed, May 27, 2020 at 2:17 AM mohank 
wrote:

> >> 3) is there any work around to delete the temp queue programmatically?
>
> >Not sure I understand the question. The temp queue would be deleted by
> >closing the receiver.
>
> I have tried with receiver close API, it is blocking call and deleting the
> temp queue.
>
> Message response = receiver.fetch();
> std::cout << request.getContent() << " -> " <<
> response.getContent() <<
> std::endl;
>
> receiver.close();
>
> thanks,
> Mohan
>
>
>
> --
> Sent from:
> http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: C++ Broker

[Virgilio Fornazin]

RHEL is pushing Dispatcher + Broker-J (Artemis based) as the new A-MQ
solution.
With container and another new things happening, could be the new way.
But C++ broker still the fastest and best broker around for me

On Fri, Mar 6, 2020 at 1:33 PM Robbie Gemmell 
wrote:

> The C++ broker sees occasional bug fixes (some are probably overdue a
> release currently, its been a while), but there isnt active feature
> development around it.
>
> On Fri, 6 Mar 2020 at 15:39, Bruce Parr  wrote:
> >
> > Hello,
> >
> > Is the C++ broker still under development/maintenance, or should we
> start thinking about switching to Broker-J?
> >
> > Thanks,
> > Bruce
> >
> >
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> > For additional commands, e-mail: users-h...@qpid.apache.org
> >
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: Message catalog creation

[Virgilio Fornazin]

Use broker exchanges for that.

amq.topic is fair enough for most situations

There were a xml exchange if I remember correctly. Not sure about that.

On Fri, 10 Jan 2020 at 16:01 tomt  wrote:

> Fair enough.  I was hoping to take advantage of the AMQP type system and
> message property filtering if I could have managed it.
>
>
>
> --
> Sent from:
> http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: Message catalog creation

[Virgilio Fornazin]

You probably need msgpack, protobuf (and others) are what you need to look
at.

Also, think in amqp as a 'low-level message transport layer', msgpack (or
another) as 'low-level message codec layer' and develop your app without
knowing how the things are done 'under the hood'.

On Fri, Jan 10, 2020 at 3:42 PM tomt  wrote:

> Hello all,
>
> This is more of a usage question after a successful configuration of broker
> and other components.
>
> Is there standard way or existing library to define fields and body
> structure for particular message for either proton implementation so that a
> sender and receiver each know what the other side is talking about?  I know
> that a message itself can hold the definition of the message inside of it,
> but I don't think that information will be used on the fly.  It seems like
> a
> more common case would be to look at the sender is setting in the message
> and do the reverse.
>
> A single message back and for could be done in a manual fashion like I have
> described, but I would think that things can get to be a headache if there
> are many different message definitions that must be understood for a given
> application to function, especially if you are trying to support multiple
> languages.   Perhaps you define some catalog of messages in some meta
> language and then generate message classes out of that?
>
>
>
>
> --
> Sent from:
> http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: qpidd C++ broker memory leak on Redhat 6.8?

[Virgilio Fornazin]

I've used RHEL MRG (Qpid) in the last 8 years, and what I could say is that
qpidd C++ broker never releases back memory to SO.

I've found that the queue code use std::vector in some paths, and need to
call vector.shrink_to_fit() because std::vector (and some other C++ STL
container) never releases back memory, only when destructed. I didn't
verified if applying a patch that call shrink_to_fit when the queue is
empty could help, but is a good point to start checking.

On Tue, Feb 27, 2018 at 7:12 PM, CLIVE  wrote:

> Hi,
>
> I have had a similar problem with a production QPID system, running Cento
> 6.8 for the past 14 months.
>
> I tried the good advice that Ted gave, using the malloc tuning environment
> variables, but unfortunately
> this made no difference to our environment. We would get about 5 days out
> of the system until the broker
> used up all 64G of memory and got killed by the kernels OOM process.
>
> I even tried using the gperftools tcmalloc library, but again this has
> made no difference to the memory consumption.
>
> Even moving up from version 0.34-1.36 didn't seem to make any difference.
> Then about 8 weeks ago, out of the blue, the
> problem just went away. The QPID broker is now staying solid at 5% memory
> usage. We have not changed our usage pattern
> in any way, still the same number of consumers and producers, but our
> message flow rate has increased.
>
> I have tried everything to replicate this issue, but to no avail.
> Interestingly the problem only affects our production
> system, all or other environments are fine. This leads me to believe it is
> somehow flow related, which I know sounds
> a bit stupid, but we had been running in production for about a year
> before the memory problem arose (at a point in time when the
> flow rates increased).
>
> Then again 8 weeks ago the system encountered another increased step
> change in message rates and the memory problem has now gone away (for the
> time being!)
>
> I don't know how flow rate would affect the memory in this way, hopefully
> someone else might have some ideas.
>
> The other area I looked at was message size (we have a wide range of
> message sizes in our system (1k-10M), but I could never replicate the issue.
>
> Sorry I cannot to be of more help, but if you do uncover the problem I
> would be very interested to hear about it.
>
> Clive
>
>
> On 27/02/2018 20:36, jbelch wrote:
>
>> Thanks.  I will try it and let you know if it works.
>>
>>
>>
>> --
>> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936
>> .html
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
>> For additional commands, e-mail: users-h...@qpid.apache.org
>>
>>
>>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: How to unbind queue and topic with C++ Message API

[Virgilio Fornazin]

you must use QMF framework to dinamically bind/unbind topics to exchanges /
queues

On Fri, Aug 26, 2016 at 9:49 AM, lucas  wrote:

> I have bound a queue and topic with code:
>  "family ; {create:always,node: {type: topic,durable:True,
> x-bindings:[{queue:lucas}]}}";
> Now I want to unbind this connection,but i dont konw how to do;
> Please help!
> I try to use "delete:always" but the whole queue or topic will be delete at
> last.
>
> Any help will be appreciated.
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.
> com/How-to-unbind-queue-and-topic-with-C-Message-API-tp7649664.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: C++ API - Check whether queue exists

[Virgilio Fornazin]

You should use QMF calls to check it.
But, you just can catch the exception, it`s simpler and fastest than a
round-trip to the broker.

On Wed, Jan 13, 2016 at 2:07 PM, rat...@web.de  wrote:

> Hi,
> I am using the C++ messaging API for QPID to realize an
> remote-procedure-call (RPC) via a request-reply pattern (see
> https://qpid.apache.org/releases/qpid-0.24/programming/book/ch02s12.html)
>
> The client pushes to a service-queue and generates an reply-queue.
> The server reads from the service-queue and pushes to the reply-queue.
>
> The reply-queues are automatically deleted if the client disconnects. It
> may
> thus happen, that the reply-to flag of a message (received by the server)
> contains a queue which does not exist anymore.
>
> The code
> const Address& address = request.getReplyTo();
> Sender sender = session.createSender(address);
> will in this case create an exception and invalidates the current session.
>
> I would like to check in advance, whether a queue exists before creating
> the
> sender. Is this anyhow possible with the C++ messaging API?
>
> Thx in advance...
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/C-API-Check-whether-queue-exists-tp7636548.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: regarding qpid C++ broker

[Virgilio Fornazin]

We use it here in our business.

Is the AMQP of choice for throughtput & latency (we use 0.18-20 version,
planning to migrate to 0.34-5, we're testing to see if any regressions on
this side.
Memory footprint depends on how much messages will stay on queue if the
queue doesn't suport paging, you must test by yourself.
Persistent works good for us, scalability / failover worked so good with
0.18 series, in our 3-node cluster).
The communication patterns req-reply, pub-sub we use here, works like a
charm.
By the means 'routing', í'm assuming you're talking about the broker
exchanges (amq.topic). It's very efficient and fast.


On Wed, Oct 28, 2015 at 2:21 AM, Suman.Patro-TRN  wrote:

> Hello Sir,
>I am currently evaluating qpid broker for its
> performance against other available brokers. So I would like to clear
> certain queries regarding the same.
>   Can anyone suggest the way to test the qpid  broker against the
> following parameters:
> 1. Throughput
> 2. Latency
> 3.Memory Footprint
> 4.Persistence
> 5.Scalability
> 6.Failover mechanism
> 7.Communication patterns support
> 8.Security or cybersecurity involvement
> 9.Routing algorithm used and its complexity
> I would like to find statistics on the same by practical
> evaluation , hence such a question.
>Thanks and regards,
>Suman
>
> Larsen & Toubro Limited
>
> www.larsentoubro.com
>
> This Email may contain confidential or privileged information for the
> intended recipient (s). If you are not the intended recipient, please do
> not use or disseminate the information, notify the sender and delete it
> from your system.
>

Re: Go on windows: Visual Studio, gcc, cgo Oh My!

[Virgilio Fornazin]

I think MinGW should *do the work*.
You can cross-compile the code on linux or build on Windows host.

On Wed, Oct 14, 2015 at 7:09 PM, aconway  wrote:

> I have spent a fruitless day trying to get the go binding to work on
> windows. Here's the scoop.
>
> cgo (the Go/C integration) requires gcc to work on windows.
> gcc will not link with libraries produced by Visual Studio C++
> compiler.
> most proton users will be using the Visual Studio compiler.
>
> So I think the only way to make it work on windows is to build a proton
> library with gcc specially for the go binding. This needs to be kept
> separate from any VS generated proton libraries installed on the same
> host. As I understand it, the "idiomatic" strategy for doing this on
> windows is to install them in different, randomly-named directories.
> Indeed as I understand it "install" on windows means "copy to a
> randomly named directory" so I guess this is not a big deal.
>
> I'd love to hear better solutions from developers who are less windows
> -hostile :)
>
> Cheers,
> Alan.
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: Configuring large queues

[Virgilio Fornazin]

Gordon

What's the qpidd broker version that started support paging on queues ?


On Tue, Aug 26, 2014 at 5:53 AM, Gordon Sim g...@redhat.com wrote:

 On 08/24/2014 10:23 PM, Graham Leggett wrote:

 I have a need to configure a very large queue that is able to store about
 1 million messages of around 10kb each. The queue will be too big to fit
 into RAM, and will need to spool to disk.

 I have been struggling to configure the queue, as the queue seems to
 become full after a few thousand messages. Is there a definitive set of
 command line options to qpid-config that will create a queue bigger than
 RAM but that will fit on disk?


 Assuming you are using the c++ broker (since you mention qpid-config), if
 you are using a relatively recent version then the best option would be a
 paged queue.

 A baisc paged queue can be created with:

 qpid-config add queue my-paged-queue --argument qpid.paging=True

 You can control the number of pages held in memory with the
 qpid.max_pages_loaded option and the size of the pages with the
 qpid.page_factor option (which indicates how many multiples of the
 platforms page size each qpidd page size is)

 If you are using the default value for default-queue-limit then you may
 also want to explicitly make the queue infinite in size and turn off any
 producer flow control which you could do by adding:

 --argument qpid.max_size=0 --argument qpid.max_count=0 --argument
 qpid.flow_stop_size= 0 --argument qpid.flow_resume_size=0 --argument
 qpid.flow_stop_count=0 --argument qpid.flow_resume_count=0



  The errors we have received are related to qpid complaining that failover
 has failed - which is strange as there is just a single broker with no
 alternative brokers to fail over to. Does this ring a bell?


 What client are you using? Do you have the error in more detail?



 -
 To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Configuring large queues

[Virgilio Fornazin]

Thanks. I'll take a look on that.


On Thu, Aug 28, 2014 at 1:40 PM, Gordon Sim g...@redhat.com wrote:

 On 08/28/2014 05:35 PM, Virgilio Fornazin wrote:

 What's the qpidd broker version that started support paging on queues ?


 It was first included in the 0.24 release.



 -
 To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Any Reason For : Could not accept socket: Too many open files

[Virgilio Fornazin]

Also remember to change /etc/security/limits.conf and put a entry like that

qpidd   -   nofile  65535



On Fri, Jul 19, 2013 at 2:20 PM, Chuck Rolke cro...@redhat.com wrote:

 You may have too many connections to the broker. Try:
   qpid-stat -b localhost:5672 -c
 To see your connections. When developing an application it's easy to
 forget to call connection.close() in a loop and have lots of outstanding
 connections. Actaully this is a catch since qpid-stat won't be able to
 connect to read the value. Maybe start qpid-tool before you run your tests
 and you can see the connections with list connection as the application
 proceeds.

 By default the connection limit is 500. You can change that by having
   --max-connections N
 command line option.

 Is there an actual broker log entry? That would show the source code line
 from which the error occurred.

 -Chuck

 - Original Message -
  From: Rajesh Khan rajeshkhan...@gmail.com
  To: users@qpid.apache.org
  Sent: Friday, July 19, 2013 12:53:33 PM
  Subject: Any Reason For : Could not accept socket: Too many open files
 
  I started getting a message from qpidd stating Could not accept socket:
  Too many open files any reason for that message. Are there too many
 users
  connected to qpid ?
 

 -
 To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: C++ client library Send() timeout

[Virgilio Fornazin]

You raised a interesting question here Connor: aren't Sender objects
'thread-safe' ?

On Thu, Mar 14, 2013 at 3:17 PM, Gordon Sim g...@redhat.com wrote:

 On 03/14/2013 05:15 PM, Connor Poske wrote:

 Thanks Gordon, I created QPID-4648

 https://issues.apache.org/**jira/browse/QPID-4648https://issues.apache.org/jira/browse/QPID-4648

 I didn't see how to assign it to someone, maybe I don't have those
 privileges?


 Thanks! I've assigned it to myself.



 --**--**-
 To unsubscribe, e-mail: 
 users-unsubscribe@qpid.apache.**orgusers-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Question for Client Libraries to declare Queues, add bindings,...

[Virgilio Fornazin]

We do this using QMF calls... they achieve the same effect as old C++
client libraries

On Tue, Dec 11, 2012 at 1:40 PM, Chuck Rolke cro...@redhat.com wrote:

 Hi,

 In the source tree there is a picture of how the .NET binding fits into
 the native C++ messaging world:
 qpid/cpp/doc/dev-readme/QPID-Component-README.pdf

 The .NET Binding gives you access to the messaging library but no direct
 access to the client library. Going forward the messaging interface will
 give you a stable API to use when migrating to future versions Qpid
 distributions and AMQP 1.0.

 Some tasks like declaring queues and topics may be done programatically by
 using features contained in the addressing syntax. Other management tasks
 are done simply and directly using the python script qpid-config.
 See http://qpid.apache.org/books/0.18/Programming-In-Apache-Qpid/html/

 -Chuck

 - Original Message -
  From: Wurzinger Peter peter.wurzin...@noel.gv.at
  To: users@qpid.apache.org
  Sent: Tuesday, December 11, 2012 6:57:15 AM
  Subject: Question for Client Libraries to declare Queues, add
 bindings,...
 
  Hi!
 
  In our company we're using Apache QPID in some of our
  company-internal web-applications (ASP.NET, C#).
  One of them has the job to manage the QPID-Broker running on one of
  our internal servers.
  This is a quite old application and my job is to get it working for
  the newest QPID-Version (0.16), currently we're using QPID 0.6.
  I've managed to run the 0.16 Broker and compile the
  org.apache.qpid.messaging - Assemblies for the applications
  exchanging data  using QPID.
 
  My problem now is, that the QPID-managing application uses an
  Assembly called qpid.client.dll, which provides functionality to
  declare queues, add bindings and so on, but in the newest version of
  QPID I cannot find this Assembly. Is/Are there any replacement/s for
  this Assembly?
 
  Best Regards
  Peter
 
 

 -
 To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: proposal to remove certain features from qpidd

[Virgilio Fornazin]

We use MRG-M here too, and we are running in trouble sometimes with this
confuse flow-to-disk implementation.

What we expect to have, to replace it, it's something like a real
'queue-on-disk' with parameters like current
implementation of flow-to-disk have (max messages/bytes on memory, max
messages/bytes on disk file, etc).


On Sun, Jul 22, 2012 at 5:31 PM, Jakub Scholz ja...@scholz.cz wrote:

 We use Qpid (well, MRG-M) as an interface to our costumers. As such,
 we have quite limited control over
 - the amount of messages
 - the time when the messages are consumed (i.e. does the client
 connect to consume the messages or not)
 We expect the brokers to deliver approximately hundreds of GB of
 messages per day. Under normal circumstances, most of the messages
 will be consumed by the clients almost immediately, but in some
 exceptional situations, they may need to be stored on the broker. And
 since the performance isn't the biggest issue for us (while on the
 other hand reliability of the broker is), the flow-to-disk queues kind
 of help (yes, the headers are still kept in memory).

 Although you can always say that we can get HW with more memory or
 split the load between multiple brokers if necessary, it would still
 be better if the flow-to-disk queues are replaced by some better
 alternative.

 Regards
 Jakub


 On Fri, Jul 20, 2012 at 11:54 AM, Gordon Sim g...@redhat.com wrote:
  On 07/20/2012 10:22 AM, Jakub Scholz wrote:
 
  While I agree with you that the flow-to-disk queues have a lot of
  problems, I do not think they are totally useless. If you remove them
  without any real alternative, you may block the upgrade path for many
  people using them. At least speaking for my self, it probably would be
  a problem for some of our brokers.
 
 
  Understood. Can you give any more detail of how you use them? i.e. what
 sort
  of scenarios the feature is triggered in?
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
  For additional commands, e-mail: users-h...@qpid.apache.org
 

 -
 To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Overhead of creating (private reply) queue?

[Virgilio Fornazin]

We used here to create a private reply queue for each request/reply (and
also, we discovered leaks with Ted Ross in 0.10 version that time).
After that, we changed our code to pre-create a private reply queue for
each connection that perform request / reply operations, using message.

message.ReplyTo = ...
message.CorrelationId = correlationId;

sender.Send(message);

In the reply side, we put:

replyMessage.Subject = _message.ReplyTo.Name;
replyMessage.CorrelationId = _message.CorrelationId;

_replySender.Send(replyMessage);

To reply back the message with previous sent correlation Id. This make
things faster and also, lowered the latency since we don't need to setup a
queue in the broker,
avoid resource usage and network round-trips (this helps a lot in
high-latency connections).


On Mon, Jul 16, 2012 at 10:45 AM, Toralf Lund toralf.l...@pgs.com wrote:

 On 16/07/12 15:01, Rajith Attapattu wrote:

 I'd agree with Gordon.
 If at all possible I will pre-create my private queues, rather than
 creating them on demand.
 Writing a bit of extra code for working with a fix number of queues is
 worth from a performance standpoint.

 It's not just about handling the queues, though. If I reuse the response
 queue, I also have to worry about responses that actually belong to
 requests sent in the past. A simplistic way of handling those would be to
 just clear the queue before a new request is sent, but there is a risk that
 the remote end sends a reply that I've given up on after the queue is
 cleared, but before the new request is processed. So, to make sure I always
 read the correct reply, I probably have to add/match extra id fields -
 possibly messageId/correlationId. This also represents *some*
 overhead...

 - Toralf


 Regards,

 Rajith

 On Mon, Jul 16, 2012 at 8:39 AM, Gordon Simg...@redhat.com  wrote:

 On 07/16/2012 12:34 PM, Toralf Lund wrote:

 Hi.

 What kind of overhead to you expect from having to create the
 (private) queue when initialising a qpid::messaging::receiver?


 If it is not a durable queue then the overhead is not that high,
 however...


  I'm implementing request-response type communication over a direct
 exchange, with a private auto-delete queue for responses (whose
 address is specified in replyTo on the request message.) So far, I've
 just created a new queue on every call rather than trying to keep the
 same one throughout the session because it's simpler in many ways - it
 saves me from some queue management, and I don't have to worry about
 reading the wrong response e.g. because of a time-out waiting for a
 reply to a previous request.

 So, do you think that this sounds like a bad idea, or is it quite all
 right because the overhead imposed by creating and deleting queues all
 the time is negligible?


 ... this probably depends on the expected/required rate of requests.

 For low rates (say 100s or perhaps 1000s per second) it probably wouldn't
 have too much impact, but as you start getting above that then you would
 be
 spending time on queue creation and deletion that would be better spent
 processing messages and requests.



 --**--**
 -
 To unsubscribe, e-mail: 
 users-unsubscribe@qpid.apache.**orgusers-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

  --**--**
 -
 To unsubscribe, e-mail: 
 users-unsubscribe@qpid.apache.**orgusers-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org



 This e-mail, including any attachments and response string, may contain
 proprietary information which is confidential and may be legally
 privileged. It is for the intended recipient only. If you are not the
 intended recipient or transmission error has misdirected this e-mail,
 please notify the author by return e-mail and delete this message and any
 attachment immediately. If you are not the intended recipient you must not
 use, disclose, distribute, forward, copy, print or rely on this e-mail in
 any way except as permitted by the author.

 --**--**-
 To unsubscribe, e-mail: 
 users-unsubscribe@qpid.apache.**orgusers-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Issue executing code on different machine

[Virgilio Fornazin]

This issue happens because Microsoft doesn't allow you to distribute debug
libraries from C runtime.
See
http://social.technet.microsoft.com/Forums/zh/itproxpsp/thread/dbb413ef-3782-4a26-b540-8f3b3269cbe5

Algo, VC90.DebugCRT is related to a VS 2008 compiled file, not VS 2010 one.


On Mon, Apr 16, 2012 at 11:52, Chuck Rolke cro...@redhat.com wrote:

 Please check again which files you have copied to your target machine.
 From the Application Context error message the file is
 bin\Debug\org.apache.qpid.messaging.dll which is Debug and not Release.

 Also check which assemblies your application has referenced. This is made
 more difficult by the SxS services so you can't necessarily tell what's
 going on just by static analysis. Further tools to use are from a Visual
 Studio Command Prompt :
 * depends dll or exe file. This shows some of the static linkage of DLL
 files.
 * sxstrace understands dynamic bindings and the SxS stuff. See
 http://blogs.msdn.com/b/junfeng/archive/2006/04/14/576314.aspx

 -Chuck

 - Original Message -

  From: Todd Herman t...@apx-labs.com
  To: users@qpid.apache.org
  Sent: Friday, April 13, 2012 10:54:36 AM
  Subject: Issue executing code on different machine

  I compiled the latest version of the c++ qpid client/broker (from the
  repository). I compiled it as a release version, not debug. I then
  compiled the dotnet binding (org.apache.qpid.messaging.dll) also as
  release. Everything works great and my code, using that dll, runs
  fine on the development machine. For the record, I am compiling
  everything in Visual Studio 2010.

  Now, I move my code (and all required files) to a different machine.
  When the code tries to use the org.apache.qpid.messaging.dll my
  application crashes and I see this in the event log:

  Activation context generation failed for C:\Programming\APX-LABS
 
 SVN\Pvision_Server\com.apx-labs.components.Windshear\bin\Debug\org.apache.qpid.messaging.dll.
  Dependent Assembly
 
 Microsoft.VC90.DebugCRT,processorArchitecture=x86,publicKeyToken=1fc8b3b9a1e18e3b,type=win32,version=9.0.21022.8
  could not be found. Please use sxstrace.exe for detailed diagnosis.

  Why is code compiled in VS 2010 as a release (not debug) using an
  older debug MSVC file? Any suggestions on what I can do to overcome
  this?

 
  Description: Description: Description:
  C099A092-D672-44AE-BE50-B9C847BC176F
  Todd Herman
  Senior Software Engineer

  APX Labs
  2350 Corporate Park Drive, Ste. 510 | Herndon, VA 20171
  m: 703-489-8761 | www.apx-labs.com | @APXLabs

Re: Qpid destroys session after max queue size reached

[Virgilio Fornazin]

Gordon

If I have the Sender Capacity to 1000 Messages per example, and the queue
has a limit of 800 Messages, and after I send 801 messages, the session is
closed after the
exception while trying to delivery the message to the queue.

But the 800 messages sent previously are guaranteed to be delivered and
stay in queue, since they couldn't be acknowledged by the broker after the
session is terminated?

How the API deals with this situation ?

On Thu, Mar 22, 2012 at 11:27, Gordon Sim g...@redhat.com wrote:

 On 03/22/2012 02:23 PM, Alan Conway wrote:

 On 03/21/2012 09:22 AM, Gordon Sim wrote:

 On 03/20/2012 09:43 PM, Jeff Armstrong wrote:

 I have a queue with REJECT policy and a max count. If I connect a
 sender client and fill the queue to the max size, the broker seems to
 destroy the session. Using qpid-tool I can see that the session is
 gone, though the connection is still there. In the sender client
 session.isValid() still returns true. If I then connect a receiving
 client to drain the queues, the sender still fails to send messages
 to the broker unless I close the session and re-open it. This seems
 like really weird behaviour to have your session deleted because you
 hit a max count and then not being able to tell from the sender that
 this has happened.


 Yes, it is annoying. The AMQP 0-10 (and earlier) specification(s)
 state that
 when an 'exception' indication is sent on a session, the session must
 then be
 terminated. What you are seeing is that fact bubbling up to the API.

 The qpid::client::Session::**isValid() at present only tests whether the
 'handle'
 refers to an actual session object and doesn't take into account the
 state of
 that session. Again, I can see that is not intuitive. There isn't an
 ideal way
 to workaround that either. You can call flush() on the session which
 has minimal
 effect but would act as a test that it is active (you would need to
 catch an
 exception in the event that it was not).


 There are 2 methods to check if a session was terminated by an error:
 /**
 * @returns true if the session has been rendered invalid by some
 * exception, false if it is valid for use.
 */
 QPID_MESSAGING_EXTERN bool hasError();
 /**
 * If the session has been rendered invalid by some exception,
 * this method will result in that exception being thrown on
 * calling this method.
 */
 QPID_MESSAGING_EXTERN void checkError();


 Unfortunately those are only available on qpid::messaging::Session, not
 the older qpid::client::Session.


  I would like to revise the general lifecycle of sessions in the case of
 exceptional conditions, but any change would almost certainly be in the
 messaging API only as the session abstraction there is not tied so
 directly to
 an AMQP 0-10 session.

 I could modify the qpid::client::Session::**isValid() method or expose
 an
 additional method, in order to make testing full 'validity' simpler.
 Not sure
 how much value that would be to you however.


 isValid() is inherited from class Handle so I don't think we should put
 additional semantics into it.
 It's actually redundant since isNull() does the same test inverted and
 is IMO more clearly named. Maybe we should deprecate isValid().


 Again I think that is true for the messaging API, not the older 'client'
 API.


 --**--**-
 To unsubscribe, e-mail: 
 users-unsubscribe@qpid.apache.**orgusers-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Qpid destroys session after max queue size reached

[Virgilio Fornazin]

For my safe, since my app is multithreaded, the best I can do is to
serialize access to the same Sender/Session object on each send() call and
check for errors
to try to handle such situation.

On Thu, Mar 22, 2012 at 14:32, Gordon Sim g...@redhat.com wrote:

 On 03/22/2012 04:58 PM, Virgilio Fornazin wrote:

 Gordon

 If I have the Sender Capacity to 1000 Messages per example, and the queue
 has a limit of 800 Messages, and after I send 801 messages, the session is
 closed after the
 exception while trying to delivery the message to the queue.

 But the 800 messages sent previously are guaranteed to be delivered and
 stay in queue, since they couldn't be acknowledged by the broker after the
 session is terminated?

 How the API deals with this situation ?


 Not terribly well at present is the honest answer. This is something I
 would like to revisit. In the messaging API we could probably shield the
 application from the underlying loss of AMQP session, or at least ensure
 that their session object can continue to be used.

 All messages that the broker received before hitting the limit will be on
 the queue. If they were durable messages, they may not all have been
 written to disk, but the loss of session won't interfere with that. The
 problem is how the application can infer this from the messaging API.

 Generally you would use the unsettled count on the sender to determine how
 many of your sent messages remained in doubt. However that is only
 periodically updated so it often won't be up to date when a session error
 occurs. Again that is something we could change, and have the broker issue
 a completion before sending any exception and ending the session.

 I do have a JIRA open for this: https://issues.apache.org/**
 jira/browse/QPID-3179 https://issues.apache.org/jira/browse/QPID-3179,
 just need to get some time to work on it.


 --**--**-
 To unsubscribe, e-mail: 
 users-unsubscribe@qpid.apache.**orgusers-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Setting RoutingKey, QPID 0.6

[Virgilio Fornazin]

A little snippet that may helps

void qpid_connection::send(const destination  destination,
const message_interface  message_to_send, const bool durable)
{
context * qpid_context = reinterpret_castcontext
*(m_qpid_context);

qpid::messaging::Sender  =
qpid_context-get_sender(destination);

try
{
qpid::messaging::Message message(reinterpret_castconst
char *(message_to_send.ptr()), message_to_send.size());

if (destination.type() != destination_type::broadcast)
{
message.setSubject(destination.name());
}

message.setDurable(durable);

sender.send(message,
qpid_context-m_send_messages_sync);
}
catch (std::exception  e)
{
throw
qpid_operation_exception(valor::core::runtime::string_format::format(Send
operation failed: %s, e.what()));
}
}

On Wed, Mar 14, 2012 at 17:39, Joe Drumm joe.dr...@gmail.com wrote:

 Hi,

 We are new to QPID and using Apache QPID 0.6.  We are able to send
 messages to a queue, but we need to set a 'routing key' and can't
 figure out how to do this in 0.6.

 We see in and older version of QPID that you are able to do:

   qpid::client::Message message;
   message.getDeliveryProperties().setRoutingKey(routingKey);

 ...but the API in 0.6 seems to have changed.

 Does anyone have any info here?

 Thanks in advance

 -
 To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
 For additional commands, e-mail: users-h...@qpid.apache.org

Re: Compiling qpid in Windows

[Virgilio Fornazin]

We built our own qpid client libraries, since we need to use the sample
boost version in project.
That´s a mess, but is the only option you have.

On Thu, Sep 22, 2011 at 17:47, yuriygeorge yuriygeo...@gmail.com wrote:

 I had the same problem, and then I realized I wasn't linking against to
 necessary boost libraries. Remember, qpid relies on boost libraries, so
 every time you compile a Qpid program, it's likely that you'll also link it
 against the boost libraries, not just qpid libraries. Hopefully that's the
 issue for many people with this issue.

 --
 View this message in context:
 http://apache-qpid-users.2158936.n2.nabble.com/Compiling-qpid-in-Windows-tp2244367p6821872.html
 Sent from the Apache Qpid users mailing list archive at Nabble.com.

 -
 Apache Qpid - AMQP Messaging Implementation
 Project:  http://qpid.apache.org
 Use/Interact: mailto:users-subscr...@qpid.apache.org