Re: Unexpected error from qpidd-cpp 1.39 when queue is faulty
Not being such an expert to linearstore as Kim, I have two ideas: 1) in case you have thousands of durable queues, you can hit kernel's limit on AIO operations and need to increase fs.aio-max-nr parameter. For calculation: I recall on some systems (rhel6?) one durable queue required 33 AIO handlers, on rhel7 it seems less (half?), but take this as a rule of thumb only. 2) It seems the journal file handler has not been initialized as it is null pointer. That could be consequence of the improper shutdown (though a buggy one). If you don't care about the data in the queue, you can replace the jrnl file(s) by empty one (I can share the file). But I expect you would like to get the data - then I would start with enabling trace logs via adding log-enable=trace+:linearstore log-to-file=/path/to/file.log # if not already logging somewhere, i.e. syslog (with trace logs not dropped) and observing how journal recovery happened on all jrnl files (or symlinks to them) under /var/lib/qpidd/.qpidd/qls/jrnl2/440d04db-7fb6-3424-a83c-b70014fa32a0 directory (here I deduce the uuid is a real queue name, per your error logs). I expect one jrnl file (the most current) recovery would fail in some manner. Kind regards, Pavel On Thu, May 30, 2019 at 11:57 PM Justin Ross wrote: > Kim? > > On Tue, May 14, 2019, 14:01 Gordon Sim wrote: > > > On 14/05/2019 10:46 am, Pål Skjager Løberg wrote: > > > For a client, just getting "illegal-argument: Value for replyText is > too > > > large" back as an error when sending is not the most useful info and I > > > suspect, especially after reading the mentioned thread from November, > > there > > > might be a bug in how the error responses to the client is generated > > > causing the actual error to be masked by another error. > > > > > > Also, there seems to be a possibility that the Qpid broker will start > wth > > > broken queues, causing it to fail only when messages are written to > that > > > queue, including some null pointer problems. > > > > > > Are any of these known issues or is it the expected behavior? > > > > No, neither of these is the correct behaviour. > > > > I have committed a fix for the first issue: > > https://issues.apache.org/jira/browse/QPID-8313 > > > > For the issue with the journal recovery, I'd need to defer to the > > expert. Kim, can you recommend any diagnostics to figure out what would > > cause the problems in the queues on recovery? i.e. errors such as: > > > > > jexception 0x010b LinearFileController::getCurrentSerial() threw > > > JERR__NULL: Operation on null pointer > > > > > > > > > > - > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > >
Re: Misleading messages when starting up Qpid c++ 1.36.0 ?
Hello, On Sat, May 20, 2017 at 1:14 PM, kristianrauewrote: > Hi, > > I have successfully compiled qpid-cpp-1.36.0, qpid-proton-0.17.0 and > qpid-python-1.36.0 on CentOS 7.1 x86_64. > > I want to run Qpid with AMQP 1.0 and Linear Store enabled. And I should be > happy, because both seem to work when I start qpid with "$ qpidd > --load-module amqp.so --auth no" > For Linear Store, I would expect adding --load-module linearstore.so (but I dont see the warning message without this option so I might be wrong). > (both also works even when I just use “$ qpidd --auth no”). > > I can connect to the broker with AMQP 1.0 only clients like > - Proton, > - node-amqp10 (https://github.com/noodlefrenzy/node-amqp10) or > - node-red-contrib-rhea > (https://flows.nodered.org/node/node-red-contrib-rhea). > > So AMQP 1.0 seems to work. > > Also I tested persistence by stopping and restarting the broker with > durable > queues and durable messages. So Linear Store seems to work perfectly too. > > But there are 2 problems that irritate me (and took a few hours of maybe > unnecessarily spend time) > > Problem 1: > > In other threads I read that with AMQP 1.0 enabled I should be seeing: > > "[Broker] info Loaded protocol AMQP 1.0" > > when starting Qpid. > That message is on info level, while default qpidd logging is notice+ (notice verbosity and anything less verbose). All logging verbosity levels are: trace, debug, info, notice, warning, error, fatal If you add --log-enable=info+ you should start to see the above log. Kind regards, Pavel
Re: Modification of queue arguments after declaration
Hello, I guess you need that for C++ broker. There is no way of reconfiguring a queue like that, you would have to delete the queue and re-create again with the extra parameters added. Kind regards, Pavel - Original Message - > From: "Jerome O'Neil"> To: users@qpid.apache.org > Sent: Thursday, 8 October, 2015 8:57:39 PM > Subject: Modification of queue arguments after declaration > > I need to be able to modify the configuration of specific queues after they > have been declared. The schema tells me that the arguments element is a > ReadOnly property. > > Specifically, I'd like to be able to add "qpid.alert_count" and > "qpid.alert_repeat_gap" to specific queues. I can access the new queues as > QMF sends me events when they are declared, but I'm unsure how I can modify > these two arguments. > Is there some other way to achieve this? > > Best, > > -Jerome > > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: 'Error reading socket: Success(0)'
- Original Message - From: Gordon Sim g...@redhat.com To: Andrew Stitcher astitc...@redhat.com, users@qpid.apache.org Sent: Friday, 6 March, 2015 1:56:23 PM Subject: 'Error reading socket: Success(0)' Has anyone ever seen the error message 'Error reading socket: Success(0)' from the q++ broker? If it helps, here is a trivial reproducer (not all configuration might be necessary): qpidd.conf: auth=no require-encryption=yes ssl-require-client-authentication=yes ssl-port=5671 ssl-cert-db=.. ssl-cert-password-file=.. ssl-cert-name=.. and then run qpid-stat -q -b localhost:5671 and terminate the command after a while. (I recall some other scenario with goferd/katello/.. involvment but cant find the katello(?) redmine issue atm) From what I understand this implies ::read() returning -1 without setting errno, which shouldn't happen. Can anyone suggest what might be happening here? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Check Exchange present or not in Qpid C++ broker ?
Use exchange instead. For understanding what QMF classes (and what stats and methods within each) qpid broker offers, let find management-schema.xml files (esp. cpp/src/qpid/broker/management-schema.xml) in source code. Kind regards, Pavel - Original Message - From: RajendraNPTL prr...@gmail.com To: users@qpid.apache.org Sent: Monday, 24 November, 2014 1:46:12 PM Subject: Re: Check Exchange present or not in Qpid C++ broker ? Thank you * Pavel Moravec *With your program i am getting the total details of the queue.But, i am not able to get Exchange details. I tried changing type * queue* to * topic*. But, no results.Do you have any idea about that ? - Time and tide waits for NONE ..!! -- View this message in context: http://qpid.2158936.n2.nabble.com/Check-Exchange-present-or-not-in-Qpid-C-broker-tp7616713p7616719.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: qpid-stat/qpid-config and SSL connections
Hello, you have to use qpid-stat options: --ssl-certificate=cert Client SSL certificate (PEM Format) --ssl-key=key Client SSL private key (PEM Format) to specify client SSL certificate and key. Self-signed certificate should not be a problem, afaik. Kind regards, Pavel - Original Message - From: Tim Wojtulewicz tim...@ieee.org To: users@qpid.apache.org Sent: Wednesday, November 5, 2014 9:01:53 PM Subject: qpid-stat/qpid-config and SSL connections I recently configured our qpid C++ brokers to use SSL encryption via a self-signed certificate for all connections. We have it working with both C++ and Java clients without any problems, but all of the qpid-tools applications fail. Here's how everything is configured: /etc/qpid/qpidd.conf: require-encryption=yes ssl-cert-db=/etc/pki/TGS ssl-cert-password-file=/etc/pki/TGS/password ssl-cert-name=infra ssl-port=5672 ssl-require-client-authentication=yes /etc/pki/TGScontains cert8.db and key3.db holding the certs and keys, as well as the java truststore and keystore. certutil -L -d . in /etc/pki/TEST returns: Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI TGS_RootCA CT,, infrau,u,u client u,u,u That is correct for the CA certificate and the server/client certificates that I generated. For the C++ clients some environment variables are set to point to the certificates: QPID_SSL_CERT_DB=/etc/pki/TGS/ QPID_SSL_CERT_NAME=client QPID_SSL_CERT_PASSWORD_FILE=/etc/pki/TGS/password openssl s_client -connect localhost:5672 finds the certificate correctly and complains about it being a self-signed cert, which I'm pretty sure is normal because it is one after all. CONNECTED(0003) depth=1 CN = TGS_RootCA verify error:num=19:self signed certificate in certificate chain verify return:0 139681568163656:error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate:s3_pkt.c:1257:SSL alert number 42 139681568163656:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177: --- Certificate chain 0 s:/CN=infra i:/CN=TGS_RootCA 1 s:/CN=TGS_RootCA i:/CN=TGS_RootCA qpid-stat -q -b amqps://localhost:5672 returns this: Failed: ConnectError - [Errno 1] _ssl.c:492: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate which matches the error that openssl threw too. I also tried extracting the cert and the private key and passing those as arguments to qpid-stat, but it gave me the same errors. Is this an issue with python not accepting the certificate because it's self-signed? Does everything look like I have it set up right? Tim - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Question regarding durable exchanges
- Original Message - From: Raphael Fuchs raphael.fuc...@gmail.com To: users@qpid.apache.org Sent: Friday, October 31, 2014 5:07:52 PM Subject: Question regarding durable exchanges Hello Qpid users, I am trying to create an exchange on the broker which is available after stopping and restarting the broker. I have seen posts about durable queues: http://qpid.2158936.n2.nabble.com/Qpid-dont-save-durable-queues-td4182044.html http://qpid.2158936.n2.nabble.com/Durable-queues-are-lost-after-restart-td6764543.html Is it really neccessary to load the store module to get an exchange which survives restarting the demon? Or would it be a better approach to recreate the exchange when starting the qpid broker? Both approaches are possible. In the first, you have to load one extra module. In the second, you have to run some script just after broker startup (while some clients wanting to use the exchanges might already be trying to connect). Kind regards, Pavel Moravec Thanks Raphael - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Duplicate Messages
Hi, you can get more duplicated messages, depending on your client's capacity/prefetch. The higher capacity is, the more messages the broker can send to the client without getting acknowledgement to prior messages. I.e. the bigger message throughput you get (until some threshold). On the other side, higher capacity means more messages in-flight/in-doubt. Every message that the broker is re-delivering should have message header redelivered:true. Kind regards, Pavel - Original Message - From: Andreas Welchlin andr...@welchlin.de To: users@qpid.apache.org Sent: Tuesday, September 30, 2014 10:20:18 AM Subject: Re: Duplicate Messages Hi Steve, I wonder how to check for duplicates, as well. I create topics using --sequence like: qpid-config -b localhost exchange topic test_topic --durable --sequence I understood that the the order of messages is guaranteed, then. This means a duplicate message can only be the last message, I received. Otherwise the sequence order would have been broken. Do you think this assumption is right? Kind Regards, Andreas Am 29.09.2014 um 22:28 schrieb Steve Huston: Hi Raphael, I am new to Qpid, starting to work on an extensive projects code base. Welcome! I am trying to find the spot in the documentation where guarantees of the messaging system are discussed. For example: in case of a network problem, could it happen that the broker sends the same message twice? Yes: http://qpid.apache.org/releases/qpid-0.30/programming/book/acknowledgements.html There is some duplicate checking and I would like to confirm it is necessary. Yes, it is. -Steve Huston - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ - Dynamically Managing Broker]
Kind regards, Pavel - Original Message - From: Alan Conway acon...@redhat.com To: users@qpid.apache.org Cc: Chuck Rolke cro...@redhat.com, Gordon Sim g...@redhat.com Sent: Monday, September 22, 2014 10:10:12 PM Subject: Add qmf shutdown command to the broker [was Re: QPID C++ - Dynamically Managing Broker] On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote: Hey Gordon, Thank you very much! That should give me a great start on this task. As for the 'shutdown' command, that's actually exactly what I was thinking too. I'm thinking about running a receiver process on the broker machine. When it receives a message that says shutdown from an authenticated user, it will perform 'system(/sbin/service qpid-stop);' or whatever the relevant OS command is. In your opinion, is this a reasonable way to accomplish this task? Would there perhaps be a better way than creating a system call? Not presently. I've long thought we should have a qmf shutdown command on the broker but never actually did anything about it. Mucking about with extra processes is painful for such a basic task. There is a denial of service security concern, my feeling is that adding a shutdown permission to the ACL rules would cover that. Note you can already delete anything from provisioning via QMF (until prevented by authentication and ACLs). This in my eyes is bigger security concern than possibility to shut down a broker. As deleting a durable queue, one loses all (durable) messages in. Also, what is the difference between connecting to AMQP listening port, providing proper (stolen?) credentials and shutting the broker via QMF, and doing the same via ssh killall qpidd? (until you disable ssh from outside network where AMQP is allowed) So I would be in favour of adding the shutdown QMF command and relevant ACL action. Anyone think this is a bad idea, or have a better idea? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: REQEST FEEDBACK Re: How to test the performance quid c++ broker
- Original Message - I think that RelWithDebInfo is more generally useful - it gives you nearly all the optimisation you want and debugging symbols for when you screw up! Works for me. Did some quick benchmarks and the perf differences between Release and RelWithDebInfo are not big. One test was 2% off on throughput, 10% off on latency, but other tests didn't show any significant difference. I'll change it on Tues if nobody else objects. I would vote for whatever of these options to be the default, but with mentioning what the default is and describe its options. I.e. update qpid/cpp/INSTALL, part Note that there are 4 different predefined cmake build types (maybe worth mentioning it at the beginning of Paragraph 2) - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Why performance of sending durable messages to qpid queue is really bad?
Durable messages will be everytime (much) slower to process than transient, as disk I/O operations are much slower than keeping the message just in memory. I would rather attempt to improve general I/O performance of the filesystem/disk you use for journals. I.e. older versions of ext4 filesystems provided poor performance for direct AIO unaligned operations the store module performs. (newer versions should be fine, afaik). Apart from that, on qpidd level, you can try tuning one broker option: --wcache-page-size N (32) Size of the pages in the write page cache in KiB. Allowable values - powers of 2: 1, 2, 4, ... , 128. Lower values decrease latency at the expense of throughput. Kind regards, Pavel - Original Message - From: smartdog jwjjj1...@gmail.com To: users@qpid.apache.org Sent: Tuesday, June 17, 2014 11:35:22 PM Subject: Re: Why performance of sending durable messages to qpid queue is really bad? It is the qpid c++ broker. The BDB store is used. Is there a way to improve the performance while preserving the persistence? qpidd.conf data-dir=/var/spool/qpid mgmt-enable=yes load-module=/usr/local/phonefactor/bin/legacystore.so load-module=/usr/local/phonefactor/bin/qpid/store/store.so From logs: [Store] info Default files per journal: 8 [Store] info Default journal file size: 24 (wpgs) [Store] info Default write cache page size: 32 (KiB) [Store] info Default number of write cache pages: 32 [Store] info TPL files per journal: 8 [Store] info TPL journal file size: 24 (wpgs) [Store] info TPL write cache page size: 4 (KiB) [Store] info TPL number of write cache pages: 64 -- View this message in context: http://qpid.2158936.n2.nabble.com/Why-performance-of-sending-durable-messages-to-qpid-queue-is-really-bad-tp7609368p7609374.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
[C++ broker] Worth removing FTD related stats?
Hi all, getting some queue statistics, I realized there are legacy stats like: byteFtdDepth byteFtdDequeues byteFtdEnqueues msgFtdDepth msgFtdDequeues msgFtdEnqueues that refer to already removed flow-to-disk queue limit policy. As far as I checked, the code changing the stats is a dead code. Is it worth removing the stats counters from management schema (together with the dead code), or does it make sense to keep it there due to some backward compatibility reasons? Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: [HA] debug Backup Rejected connection - problem with setting up qpid HA
Hi Radek, did you promote either of brokers to be a primary? See qpid-ha promote command. Kind regards, Pavel - Original Message - From: Radek Smigielski radek.smigiel...@ymail.com To: users@qpid.apache.org Sent: Friday, May 23, 2014 6:14:45 PM Subject: [HA] debug Backup Rejected connection - problem with setting up qpid HA Hi, I'm tryin to make pair of qpids (qpid0.26) working as a HA pair. Here is my very simplistic config, identical for both nodes: worker-threads=10 mgmt-qmf2=yes mgmt-qmf1=yes enable-timestamp=yes log-enable=info+ log-time=yes log-category=yes ha-queue-replication=yes ha-replicate=all ha-cluster=yes ha-brokers-url=amqp:qpid1,qpid2 link-maintenance-interval=1 link-heartbeat-interval=3 I am using examples/messaging/hello_world to check if I can connect to any of the nodes and getting the same error on both nodes: [radek@qpid1 ~]$ /home/radek/src/qpid-cpp-0.26/build/examples/messaging/hello_world 2014-05-23 11:35:02 [Client] warning Connection [127.0.0.1:42581-localhost:5672] closed Failed to connect (reconnect disabled) This is what I see on the servers, the same when I am trying connect to any of nodes: [HA] debug Backup: Rejected connection qpid.127.0.0.1:5672-127.0.0.1:42556 And this is the full qpidd trace output 2014-05-23 11:28:05 [Network] info Set TCP_NODELAY on connection to 127.0.0.1:42556 2014-05-23 11:28:05 [System] debug RECV [qpid.127.0.0.1:5672-127.0.0.1:42556]: INIT(0-10) 2014-05-23 11:28:05 [Security] debug External ssf=0 and auth= 2014-05-23 11:28:05 [Security] debug min_ssf: 0, max_ssf: 256, external_ssf: 0 2014-05-23 11:28:05 [Security] info SASL: Mechanism list: ANONYMOUS 2014-05-23 11:28:05 [Broker] debug LinkRegistry::notifyConnection(); key=qpid.127.0.0.1:5672-127.0.0.1:42556 2014-05-23 11:28:05 [Model] trace Mgmt create connection. id:qpid.127.0.0.1:5672-127.0.0.1:42556 2014-05-23 11:28:05 [Management] debug Management object (V1) added: org.apache.qpid.broker:connection:qpid.127.0.0.1:5672-127.0.0.1:42556 2014-05-23 11:28:05 [Protocol] trace SENT [qpid.127.0.0.1:5672-127.0.0.1:42556]: INIT(0-10) 2014-05-23 11:28:05 [Protocol] trace SENT [qpid.127.0.0.1:5672-127.0.0.1:42556]: Frame[BEbe; channel=0; {ConnectionStartBody: server-properties={qpid.federation_tag:V2:36:str16(48096762-a992-4f4d-9836-b9895965c499)}; mechanisms=str16{V2:9:str16(ANONYMOUS)}; locales=str16{V2:5:str16(en_US)}; }] 2014-05-23 11:28:05 [Protocol] trace RECV [qpid.127.0.0.1:5672-127.0.0.1:42556]: Frame[BEbe; channel=0; {ConnectionStartOkBody: client-properties={qpid.client_pid:F4:int32(456),qpid.client_ppid:F4:int32(342),qpid.client_process:V2:11:str16(hello_world),qpid.session_flow:F4:int32(1)}; mechanism=ANONYMOUS; response=xx; locale=en_US; }] 2014-05-23 11:28:05 [Security] info SASL: Starting authentication with mechanism: ANONYMOUS 2014-05-23 11:28:05 [Management] debug SEND raiseEvent (v1) class=org.apache.qpid.broker.clientConnect 2014-05-23 11:28:05 [Management] debug SEND raiseEvent (v2) class=org.apache.qpid.broker.clientConnect 2014-05-23 11:28:05 [Model] debug Create connection. user:anonymous@QPID rhost:qpid.127.0.0.1:5672-127.0.0.1:42556 2014-05-23 11:28:05 [Security] info qpid.127.0.0.1:5672-127.0.0.1:42556 SASL: Authentication succeeded for: anonymous@QPID 2014-05-23 11:28:05 [Protocol] trace SENT [qpid.127.0.0.1:5672-127.0.0.1:42556]: Frame[BEbe; channel=0; {ConnectionTuneBody: channel-max=32767; max-frame-size=65535; heartbeat-min=0; heartbeat-max=120; }] 2014-05-23 11:28:05 [Protocol] trace RECV [qpid.127.0.0.1:5672-127.0.0.1:42556]: Frame[BEbe; channel=0; {ConnectionTuneOkBody: channel-max=32767; max-frame-size=65535; heartbeat=0; }] 2014-05-23 11:28:05 [Protocol] trace RECV [qpid.127.0.0.1:5672-127.0.0.1:42556]: Frame[BEbe; channel=0; {ConnectionOpenBody: virtual-host=; capabilities=void{}; insist=1; }] 2014-05-23 11:28:05 [HA] debug Backup: Rejected connection qpid.127.0.0.1:5672-127.0.0.1:42556 2014-05-23 11:28:05 [System] debug DISCONNECTED [qpid.127.0.0.1:5672-127.0.0.1:42556] 2014-05-23 11:28:05 [Model] trace Mgmt destroying connection. id:qpid.127.0.0.1:5672-127.0.0.1:42556 Statistics: {bytesFromClient:226, bytesToClient:49, closing:False, framesFromClient:3, framesToClient:2, msgsFromClient:0, msgsToClient:0} 2014-05-23 11:28:05 [Management] debug SEND raiseEvent (v1) class=org.apache.qpid.broker.clientDisconnect 2014-05-23 11:28:05 [Management] debug SEND raiseEvent (v2) class=org.apache.qpid.broker.clientDisconnect 2014-05-23 11:28:05 [Model] debug Delete connection. user:anonymous@QPID rhost:qpid.127.0.0.1:5672-127.0.0.1:42556 Thanks for advice, - Radoslaw Smigielski - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Closing connection with broker is not releasing Memory ?
Hi, do you refer to a memory issue in (C++) broker or in Java client? And what version of broker+client do you use? If you refer to Java client memory issue, did you check Garbage collector logs? Could you add JVM options: -Xloggc:/path/to/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCDetails and wait until Full Garbage Collector occurs (and check the GC log then)? Kind regards, Pavel - Original Message - From: RajendraNPTL prr...@gmail.com To: users@qpid.apache.org Sent: Monday, May 12, 2014 6:26:47 AM Subject: Closing connection with broker is not releasing Memory ? Hello, i am using Qpid c++ broker. While doing performance test with the Java client, i observed that starting a connection with broker taking nearly 50 to 60MB. It's not releasing the memory by closing the connection . It's releasing memory only after exiting the program. Anybody faced the same issue ? - Time and tide waits for NONE ..!! -- View this message in context: http://qpid.2158936.n2.nabble.com/Closing-connection-with-broker-is-not-releasing-Memory-tp7608007.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Qpid client crashes in heartbeat operation
That sounds like https://issues.apache.org/jira/browse/QPID-3553 but that is fixed in 0.13.. - Original Message - From: trivedi_ravi13 trivedi_rav...@yahoo.co.in To: users@qpid.apache.org Sent: Friday, May 2, 2014 11:42:51 PM Subject: Re: Qpid client crashes in heartbeat operation I am using Qpid 0.24 standalone broker. Applications use Qpid C++ client APIs ( which are deprecated ) directly. I have observed this issue in 0.12 version as well. Command to start broker /usr/sbin/qpidd -p 5672 -d --config /etc/qpidd.conf --max-connections 5 --data-dir /tmp/qpid_data_trusted --pid-dir /tmp/qpid_pid_trusted --acl-file /etc/qpid/qpidacltrusted Broker and applications are cross compiled and running on powerPC -- View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-client-crashes-probably-in-heartbeat-operation-tp7607861p7607882.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Purging a big queue backed by linear store postpones journal files to be returned to EFP
Hi all, we identified some sub-optimal behaviour in the way linear store returns files to Empty File Pool (EFP) in one use case. I would like to get some feedback from the community if it would be valuable to implement some better way of returning the files to EFP. Currently, the only way to return a journal file to EFP during broker runtime is a check after every 100th dequeue on that queue/journal: while the _oldest_ journal file has no valid enqueue record, it is returned to EFP and removed from the journal. This has one surprising consequence when purging a queue with many messages: - you enqueue e.g. 199 messages to journal files numbered from 1 to let say 100 - you dequeue all of them: - dequeueing a message means writing to the journal a dequeue record and discarding the enqueue record. This means, dequeueing 100 messages means creating let say 10 new journal files (numbered 101 to 110) filled by dequeue events only - every 100th dequeue event checks if the oldest journal file can be returned to EFP. Dequeueing millionth message sees 100th journal file still has 99 remaining enqueues, so it does not return it to EFP. While the next 9 or 10 journal files with just dequeue records are not checked at all - at the end, we have an empty queue backed up by 10 journal files. While one file would be sufficient. Note the above is _not_ a journal file leak, the store only postpones moving the files to EFP until next 100th dequeue is coming. So it only increases disk psace utilization, somehow. Also note the use case assumes the journal gets many dequeue events in a row (with no or very few enqueues in between), and no enqueue+dequeue activity follows for a longer time (as in the scenario above, next dequeue would move 10 journal files to EFP). In other cases, no (or at most one) journal file moving to EFP is postponed by some time. It would be possible to implement some time-based trigger that will - for example - checks all journal files in all journals if they can be returned to EFP. The question is how much valuable it would be (compared to adding some complexity to the code). My own attitude is disk space is cheap, don't implement it, but if somebody has some solid use case where such feature would be much appreciated, please respond. Kind regards, Pavel Moravec - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
session.sync() on AMQP 1.0 does not notify broker
Hello, I have spotted a performance-impacting lack in AMQP 1.0 implementation of session.sync() in C++ client. On AMQP 0-10, the method sends execution.sync frame to the broker, asking for prompt delivery status to be sent. But 1.0 client does not invoke any such command. For performance impact, see QPID-5701 I raised. My question is: what would be reasonable way of telling the broker please send me all dispositions now over AMQP 1.0? Any ideas? Checking AMQP 1.0 spec I dont see a direct way.. Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
[Dispatch router] addressing examples help
Hi all, I am trying to play with Dispatch Router but dont understand its addressing. In particular: 1) Having a router listening on port say 1, how to point qdstat to it? $ tools/qdstat -a -b amqp://0.0.0.0:1/ Failed: MessengerException - [-2]: unable to connect to amqp:/#: getaddrinfo(amqp, ): Name or service not known $ tools/qdstat -a -r amqp://0.0.0.0:1/ No response from router amqp://0.0.0.0:1/ after timeout $ tools/qdstat -a -r amqp://0.0.0.0:1/local/$management -t 1 No response from router amqp://0.0.0.0:1/local/ after timeout $ .. 2) Having a mobile listener on address my-address on router A inter-connected to router B, what tool and what address to use to send messages there? I tried qpid-send from qpid broker with addresses: amqp://0.0.0.0/my-address /my-address but apparently qpid-send is stuck to message.to/message.routingKey schema. [1] refers to send.py and recv.py but I cant locate them in either Dispatch or Qpid broker sources. Thanks in advance for hints. Kind regards, Pavel [1] http://qpid.apache.org/releases/qpid-dispatch-0.1/notes.html - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Message auditing in C++ broker
Hi all, I raised QPID-5619 to implement message auditing in C++ broker. As a new feature with more options of implementation and mainly configuration, I would like to discuss it first. Quoting the JIRA for discussion: There is a reasonable request to have message enqueue/dequeue audited. It can help troubleshooting or monitoring situations when the broker is suspected from discarding a message that was sent to it but never received by a consumer. Implementation options as to be discussed in qpid users mailing list: 1) Key feature: a) where to store/send the information about new message enqueue/dequeue? - store it in a configurable log-file (per queue? per broker?) - problem with format of binary data there - amend/modify replicated queues for this purpose - though replicated queues are designed not to replicate messages that are dequeued before they have been replicated - generate QMF event to be sent to qmf.default.topic/agent.ind.event.org_apache_qpid_broker.queue_name.audit.enqueue|dequeue topic - note this would have to change qpid.subject to route the message properly - have dedicated audit exchange for this purpose (of topic type, I expect) - again, qpid.subject would have to be changed b) how to trigger message auditing: - via QueueObserver (thanks Gordon for idea) 2) Options of auditing: a) audit enqueue+dequeue events only? or also acquire, release, reject? b) it should be possible to track only one fromenqueue | dequeue | acquire events, or all of them c) event/message should contain: - timestamp - original message (or just its header? or configurable?) - type of event (enqueue/dequeue/..) - identification of consumer acquiring/dequeueing the message (and also producer?) 3) Provisioning/managing auditing: a) have the possibility to turn on/off auditing per queue (via QMF) b) have some x-declare arguments for the auditing when declaring the queue? Thanks in advance for your feedback. Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Questions of qpid-route usage
Hi Kyrie, not sure if I understand your thoughts properly, but in normal A--B federation (without -s option), B broker behaves as a fake client and subscribes to A. When -s is used, A initiates the connection to B and requests it to connect to itself. The main reason for this push-like option is to bypass potential firewall blocking B-A connection establishment. About federation link reconnection (assuming A--B pull route where B connects to A): if A is down, B periodically retries to connect to A, usually according to TCP retry scheme with doubled delays between two subsequent retries. These are managed by operating system. Once OS gives up, I *suppose* qpid broker invokes another request to the OS (so another full TCP retry scheme is followed), and the broker does so forever. Hope that helps. Kind regards, Pavel Moravec - Original Message - From: Jie Gao (Kyrie, HPIT-DS-CDC) jie@hp.com To: users@qpid.apache.org Sent: Monday, February 24, 2014 9:16:10 AM Subject: Questions of qpid-route usage Hi all, I am really confused about this tool. Hope some kind soul can help to explain the mechanism to me. If we want to add a route between 2 brokers, we can use qpid-route route add ip:port ip:port exchange_type routingkey. Which means: A--B A is a fake client and B will be the server. B will store all the route information and also will be waiting for A's data. Inside qpid it uses TCP protocol, so I guess B will be the server in the lower level(am I right?). In this way, we can add a route without broker A. Which means, even if we didn't start A, the route would still be added successfully. The problem is: Can we always do this thing? Ummm...if we let A missing for a very long time (at this time B and the route will always exist), then we start A again. Does it still works? Also, if we let the route existed in B, will B being busy(waiting for A's data) all the time? Another question is when we use -s option in qpid-route. I saw the help document it said: Configure the route in the source broker (create a push route). So does it mean A will be the server and B will be the fake client here? -s [ --src-local ] Configure the route in the source broker (create a push route). Thanks, Kyrie - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
auto-delete in AMQP 1.0
Hi all, there is an ambiguity how to interpret auto-delete x-declare option for a 1.0 producer. Assuming the producer uses address string like: autodel_q;{create:sender, node: {x-declare: {auto-delete: True}}} There are two possibilities when the producer disconnects (while there has not been a consumer meantime): A) auto-delete in AMQP 1.0 terminology is understood as delete-if-unused / delete-on-no-links lifetime policy and the queue / (1.0 node) is deleted. - this is the current broker behaviour, mapping 0-10 auto-delete lifetime policy to 1.0 one B) auto-delete queue is understand in original AMQP 0-10 terminology as to be deleted after the last consumer unsubscribes. As no consumer has been subscribed so far, the queue should not be deleted now. - this would preserve 1.0 and 0-10 behaviour compatibility when using the address string Both options have their advantages and it is hard to say which one is least confusing. Do you have some preference how the broker should behave here? Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: how to bind exchange to exchange using C++ broker?
Hi, this is not possible. In any case, messages from producers get to an exchange. The exchange checks what bindings match to the message and then it enqueues the message to all queues of the bindings that matched. Every consumer gets a message from queues only. You might to set up alternate exchange, but that has a different purpose: if no binding matches the message, the message is re-routed to the alternate exchange. Hope that helps. Kind regards, Pavel - Original Message - From: kevency_poche kevencypo...@gmail.com To: users@qpid.apache.org Sent: Wednesday, February 12, 2014 9:34:28 AM Subject: how to bind exchange to exchange using C++ broker? Hello everyone.. Yesterday i learnt how to bind in between queue and exchange. Now i want to send messages from one exchange to another exchange in the same broker. When i try to bind two exchanges it is not allowing. Can any one help me out ? Thank you !! -- View this message in context: http://qpid.2158936.n2.nabble.com/how-to-bind-exchange-to-exchange-using-C-broker-tp7604322.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
AMQP 1.0 dissector to Wireshark now available
Hi all, I have finished implementing AMQP 1.0 dissector to Wireshark, to simplify our troubleshooting of issues with communication over the protocol. See [1] for the bug requesting inclusion in Wireshark. As I don't know when my work will appear in a Wireshark public release, feel free to build Wireshark locally (it's really simple) with the patch from the BZ applied. If you will have some issues with the dissector, let me know and I can write a patch for it. During implementing the dissector, I realized how 1.0 protocol version is much more coherent, well structured and with much more internal logic than 0-10 is. Great work done by the AMQP 1.0 committee! Kind regards, Pavel Moravec [1] https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9612 - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: AMQP 1.0 tcpdumps wanted
Hi Bruno, no repository except for my mailbox :) Cheers, Pavel - Original Message - From: Bruno Matos bruno.ma...@paradigmaxis.pt To: users@qpid.apache.org Sent: Monday, December 23, 2013 4:39:12 PM Subject: Re: AMQP 1.0 tcpdumps wanted Hi Pavel, Do you have a repository where I can put the files? Regards, Bruno Matos On 12/23/2013 02:42 PM, Pavel Moravec wrote: Hi all, I am implementing AMQP 1.0 dissector to Wireshark, as I feel it is needed for debugging qpid-clients communication. Before I commit the code to Wireshark upstream, I would like to test the implementation on as much various PDUs as possible. Therefore I am kindly asking you (as the community I am involved in) to send me tcpdumps with miscellaneous AMQP 1.0 traffic, e.g.: - non-anonymous SASL authentication - TLS - various application headers in various primitive types (e.g. timestamp, uuid, float,..) - transactions .. Surely, I will not share any information (like IP address or so) from the tcpdump, I just need them to check my almost-done dissector really works well for as variety PDUs as possible. As a result, you would get better Wireshark dissector of AMQP 1.0, that will help you to troubleshoot whatever qpid communication problem you have. Wishing you successful new year. Thanks in advance, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- Bruno Matos - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
AMQP 1.0 tcpdumps wanted
Hi all, I am implementing AMQP 1.0 dissector to Wireshark, as I feel it is needed for debugging qpid-clients communication. Before I commit the code to Wireshark upstream, I would like to test the implementation on as much various PDUs as possible. Therefore I am kindly asking you (as the community I am involved in) to send me tcpdumps with miscellaneous AMQP 1.0 traffic, e.g.: - non-anonymous SASL authentication - TLS - various application headers in various primitive types (e.g. timestamp, uuid, float,..) - transactions .. Surely, I will not share any information (like IP address or so) from the tcpdump, I just need them to check my almost-done dissector really works well for as variety PDUs as possible. As a result, you would get better Wireshark dissector of AMQP 1.0, that will help you to troubleshoot whatever qpid communication problem you have. Wishing you successful new year. Thanks in advance, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Deadlock when using amqp-1-0 client to connect to amqp-0-9-1 broker
Hi Brian, let raise it in upstream JIRA: https://issues.apache.org/jira/browse/QPID Kind regards, Pavel Moravec - Original Message - From: Brian acb.kyl...@hotmail.com To: users@qpid.apache.org Sent: Friday, December 6, 2013 3:40:24 AM Subject: Re: Deadlock when using amqp-1-0 client to connect to amqp-0-9-1 broker I wanted to report it as a bug for development team, but don't know how. Anyone can help? -- View this message in context: http://qpid.2158936.n2.nabble.com/Deadlock-when-using-amqp-1-0-client-to-connect-to-amqp-0-9-1-broker-tp7601480p7601532.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
How to debug AMQP 1.0?
Hi all, playing with AMQP 1.0 in C++ broker and client, I realized I can't debug the broker like I was used in 0.10 protocol version, as: 1) broker traces contain logs like: 2013-11-22 12:44:19 [Broker] trace qpid.127.0.0.1:5672-127.0.0.1:58187 process() 2013-11-22 12:44:19 [Network] trace qpid.127.0.0.1:5672-127.0.0.1:58187 canEncode(): 0 2013-11-22 12:44:19 [Broker] trace qpid.127.0.0.1:5672-127.0.0.1:58187 decode(30) 2013-11-22 12:44:19 [Network] debug qpid.127.0.0.1:5672-127.0.0.1:58187 decoded 30 bytes from 30 2013-11-22 12:44:19 [Broker] trace qpid.127.0.0.1:5672-127.0.0.1:58187 process() 2013-11-22 12:44:19 [Model] debug qpid.127.0.0.1:5672-127.0.0.1:58187 session begun 2013-11-22 12:44:19 [Model] trace Mgmt create session. id:0x7f5c480182a0 where I lack information provided for 0.10 connections like: 2013-11-22 12:25:51 [Protocol] trace RECV [qpid.127.0.0.1:5672-127.0.0.1:57185]: Frame[BEbe; channel=0; {ConnectionStartOkBody: client-properties={qpid.client_pid:F4:int32(3641),qpid.client_ppid:F4:int32(3436),qpid.client_process:V2:12:str16(qpid-receive),qpid.session_flow:F4:int32(1)}; mechanism=ANONYMOUS; response=xx; locale=en_US; }] or: 2013-11-22 12:25:51 [Protocol] trace SENT [qpid.127.0.0.1:5672-127.0.0.1:57185]: Frame[BEbe; channel=0; {ConnectionTuneBody: channel-max=32767; max-frame-size=65535; heartbeat-min=0; heartbeat-max=120; }] That makes qpidd traces for AMQP 1.0 quite useless, if I need to check protocol specific parameters. 2) there is apparently no AMQP 1.0 wireshark dissector, hence taking tcpdump and decoding it does not help. Is there a way to decode/debug AMQP 1.0 communication? As the current state prevents any in-depth investigation.. Thanks in advance for any feedback. Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: connection.close() in C++ client never returns after resource-limit-exceeded exception
Hi Jakub, I expect Gordon can comment this much better, but can't it be resolved by QPID-5288? Kind regards, Pavel - Original Message - From: Jakub Scholz ja...@scholz.cz To: users@qpid.apache.org Sent: Saturday, November 9, 2013 1:46:51 AM Subject: connection.close() in C++ client never returns after resource-limit-exceeded exception Hi, I have C++ client using the qpid.messaging API to send messages to a queue on the C++ broker. The code block where I'm sending the messages is surrounded with try-catch and when I catch an exception I simply try to close the connection and exit the program: try { ... sender.send(msg, false); ... } catch (...) { connection.close(); } When using AMQP 0.10 and sending a message into a queue which is already full, the connaction.close() call returns and I can exit the program. But with AMQP 1.0 when I receive the same exception, the connection.close() call seems to hang forever and never return. With other exceptions - for example with unauthorized access - the connection.close() returns fine. I believe the resource-limit-exceeded error does close the connection from the broker side. That might be the reason for the different behaviour. But I would still expect that the close() call should not hang. Is the error handling as I implemented it not correct - i.e. do I have to specifically catch the exceptions which close the connection? Or is this a bug in the AMQP 1.0 implementation? Thanks Regards Jakub - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Flow Control for JMS Durable Subscribers
What version of qpid Java client do you use? I tried 0.24 and it created the queue with proper options: $ ./run_example.sh org.apache.qpid.example.Drain -f -c 1 amq.fanout; {link:{durable:True, name:'my-subscription', x-declare:{auto-delete:True, arguments:{'qpid.auto_delete_timeout':10 $ qpid-stat -q Queues queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind = 204f1f75-93a4-4cdf-a78d-2bc0d0c62ff5:0.0 YY0 0 0 0 00 1 2 my-subscription YYY0 0 0 0 00 1 2 $ qpid-config queues my-subscription Queue Name Attributes my-subscription --durable --replicate=all auto-del excl --argument qpid.ha-uuid=True --argument qpid.auto_delete_timeout=10 --argument no-local=False $ Kind regards, Pavel - Original Message - From: Gordon Sim g...@redhat.com To: users@qpid.apache.org Sent: Monday, October 21, 2013 10:58:03 AM Subject: Re: Flow Control for JMS Durable Subscribers On 10/20/2013 10:19 PM, Andrew Grissom wrote: I took a look at the examples and that works just like I wanted, I'm not sure how my googling missed that page. I also saw that I can have the queue be auto-deleted after a certain time period, but it does not seem to work when I make the queues durable. For example: my-topic; {link:{durable:True, name:'my-subscription', x-declare:{auto-delete:True, arguments:{'qpid.auto_delete_timeout':30 In this case I see durable in qpid-config. If I remove durable:True, I see auto-delete. Is this the expected behavior and if it is, is there a way to have the broker automatically delete a durable subscription queue after a certain period of time? That is certainly not the behaviour I would expect. Using the same address with the c++ client for example, the queue created is both durable and auto-deleted (with delay as configured). Could you create a bug report for that: https://issues.apache.org/jira/browse/QPID - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Flow Control for JMS Durable Subscribers
Hi Andrew, I recommend reading https://cwiki.apache.org/confluence/display/qpid/Addressing+Examples where one example covers your requirement. When declaring the topic, use {link:{x-declare:{arguments:{'qpid.max_count':500,'qpid.policy_type':'ring' in its address string to create auxiliary queue of ring policy and of maximal message depth 500. Kind regards, Pavel - Original Message - From: Andrew Grissom andrew.grisso...@outlook.com To: users@qpid.apache.org Sent: Thursday, October 17, 2013 11:19:41 PM Subject: Flow Control for JMS Durable Subscribers I have a Java client using the JMS interfaces that connects to the qpid C++ broker to receive updates on a topic. I need to make this subscription durable, which is pretty easy using the straight JMS calls. However, the problem I have is how do I handle clients that do not cleanly remove their subscriptions or do not connect for an extended period of time? I can prevent this somewhat by using the TTL values but is there a way to switch the broker from rejecting messages when a queue is full to using the ring policy? Is there a better way to handle this situation? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
QMF method queueMoveMessages can loose one message
Hi all, I identified a use case when moving (via QMF) more messages than the destination queue can handle causes one message loss. See reproducer when attempting to move 300 messages to a queue with max count 100: $ qpid-receive -a ringQueue; {create:always, node:{ x-declare:{ arguments:{'qpid.max_count':100 $ qpid-send -a fromQueue; {create:always} -m 1000 $ qpid-tool Management Tool for QPID qpid: list broker Object Summary: ID Created Destroyed Index === 117 08:11:52 - amqp-broker qpid: call 117 queueMoveMessages fromQueue ringQueue 300 {} qpid: resource-limit-exceeded: Policy exceeded on ringQueue, policy: size: max=104857600, current=1000; count: max=100, current=100; type=reject (qpid/broker/QueuePolicy.cpp:92) (7) - {} qpid: quit Exiting... $ qpid-stat -q Queues queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind = 3d03d063-4f1e-4cc2-822f-00d95d7e42e9:0.0 YY0 0 0 0 00 1 2 fromQueue 899 1.00k 1018.99k 10.0k1.01k0 1 ringQueue 100 100 01.00k 1.00k 0 0 1 $ My question is: is it a bug or intended behaviour? On one side, the broker should not lose any message. On the other side, it is an operator mistake trying to route more messages than the destination queue can handle.. (and moreover, knowing the method responsible for moving the messages, it would not be trivial to fix this without performance impact, I guess). Any comments? Kind regards, Pavel Moravec - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Reappearing messages with qpidc and bdb persistency store
Hi Aleš, I haven't met this, but here are some hints to troubleshoot/investigate it: 1) Before each broker shutdown, run (and keep output of) commands: qpid-stat -q qpid-stat -u (to see subscriptions - you will see unacknowledged messages there) store_chk directory_with_journal_of_the_queue(to examine journal content) 2) Once the problem re-occurs, run the commands again and compare outputs. Hope that helps in you investigation. Kind regards, Pavel - Original Message - From: Aleš Trček ales.tr...@halcom.si To: users@qpid.apache.org Sent: Friday, August 30, 2013 12:41:02 AM Subject: Reappearing messages with qpidc and bdb persistency store Hi all, Every now and then, when I restart my qpidc broker (version 0.22) some messages reappear in the queues, although they were long since taken from the queues (and committed). Something like that (qpid-stat output): queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind == some_queue_or_another Y0 125 125 0 1.31k1.31k1 5 I then restart (qpidd --quit; qpidd --daemon --load-module ) the broker, and I get something like this: queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind == some_queue_or_another Y 125 125 01.31k 1.31k 01 5 Queues are created like this: qpid-config add queue some_queue_or_another --durable --argument=qpid.priorities=10 --file-count=16 --file-size=512 As said, sometimes the messages magically reappear, sometimes they don't. I didn't yet get the pattern when or why. I use transacted sessions, and I am sure, that all messages get committed OK. Did anybody else have some similar problem? Regards, Ales - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: C++ client question
Hi Petr, just a guess: aren't you destroying instance of your class meantime? I.e. having declared in a block and trying to use its Session outside the block or so? Can you access the instance of your class and its connection at the place when calling Send() method for the second time? Btw. you can even optimize our (currently not working) approach by sharing Sender object, instead of Session (until you need to send to various addresses). Kind regards, Pavel - Original Message - From: Petr Parýzek petrpary...@seznam.cz To: users@qpid.apache.org Sent: Thursday, June 27, 2013 3:37:28 PM Subject: C++ client question Hi, I was try to encapsulate qpid client into this simple class class MySender { private: Connection *pConn; public: MySender() {pConn = NULL;} ~MySender() { if (pConn != NULL) delete pConn;} void Send(const char *Addr, const char *Msg) {if (pConn == NULL) pConn = new Connection(URL); try {if (!pConn-isOpen()) pConn-open(); Session session = pConn-createSession(); qpid::messaging::Sender sender = session.createSender(Addr); sender.send(Message(Msg)); } catch(const std::exception error) {std::cerr error.what() std::endl; pConn-close(); } } }; In my program I have one global instance of this class and use its method Send(). It's simple and it works. However, it is not very effective. I would like to don't always create a new instance of Session and its Sender. But when I create a Session only once and save pointer to it in some member variable, it stop working. Second call of Send() method throws an exception. So I ask, how to keep one instance of a Session between calls? Thanks Petr - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Receiver fetch vs get
Hi Bruno, I think you hit https://issues.apache.org/jira/browse/QPID-4595 that is fixed in qpid 0.21. Kind regards, Pavel - Original Message - From: Bruno Matos bruno.ma...@paradigmaxis.pt To: users@qpid.apache.org Sent: Thursday, June 27, 2013 5:19:38 PM Subject: Receiver fetch vs get Hi, I'm using the C++ Qpid Broker v. 0.22 and client libraries 0.20. In the client I have a Receiver configured to listen events like bindings and unbindings with a defined capacity of 100. If I use the fetch method, after reading 100 messages or a little more, the following messages never reach to Receiver, growing the number of messages in the queue. If I use the get method everything work as expected. I was using the fetch method with Broker 0.10 and it never leaved messages in the queue. Any ideas why this is happening? Thank you, Regards. -- Bruno Matos - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: qpid-config issue
Hi Rajesh, that appears when python-qpid package is not installed (but I have no experience how to install it on Windows). As this package implements qpid.messaging library. Kind regards, Pavel - Original Message - From: Rajesh Khan rajeshkhan...@gmail.com To: users@qpid.apache.org Sent: Tuesday, June 11, 2013 9:19:33 PM Subject: qpid-config issue I am having the following issue with Qpid-config and I am unable to create queues. I am getting the following message. Any suggestions on how I could resolve this issue. C:\qpid-0.18\qpid-0.18\tools\src\pypython qpid-config Traceback (most recent call last): File qpid-config, line 31, in module from qpid.messaging import Connection ImportError: No module named qpid.messaging - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: SASL Error
Hi Jagan, see inline responses. Kind regards, Pavel - Original Message - From: Sree82 sys8...@hotmail.com To: users@qpid.apache.org Sent: Wednesday, May 29, 2013 2:04:11 PM Subject: Re: SASL Error Gordon, I m using qpid client and qpid messaging apis to open a connection. i)Is it sufficient to use auth mechanism as PLAIN (say) through the api or should the mechanism be present in the qpidd.conf file for sure ? Assuming you mean /etc/sasl2/qpidd.conf file (i.e. SASL config file, not /etc/qpidd.conf, the broker conf.file), then yes. ii)If more than one authentication mechanism is present in qpidd.conf file then should I mention the auth type through the api I am trying to connect ? It depends. Without specifying it, the client choses the most secure from the list. If that is suitable for your purposes, you dont need to specify the mechanism in API. But if e.g. the mech_list contains PLAIN and also DIGEST-MD5, and you would like to use the less secure PLAIN in some client, you have to specify it. What is the default authentication mechanism qpidd uses if no mechanism is present in the qpidd.conf file ? By default, ANONYMOUS, PLAIN, LOGIN, CRAM-MD5, DIGEST-MD5 and GSSAPI mechanisms (listed from least secure to most) are available and offered to a client. (technically, there is also EXTERNAL mechanism for SSL but I suppose that isnt relevant to you). Note that the list can be shorter if you dont have installed relevant cyrus-sasl libraries. Thanks Jagan. -- View this message in context: http://qpid.2158936.n2.nabble.com/SASL-Error-tp7593596p7593598.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Exception when calling close - Java client/C++ broker.
Hi Fraser, I got the error, it is coincidentally a bug I already filed: https://issues.apache.org/jira/browse/QPID-4524. (if one comments out in the reproducer the second producer.send(..) call, you get precisely what you reported) I already updated the JIRA per some finding (yours and mine). Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Tuesday, May 7, 2013 7:39:37 PM Subject: Re: Exception when calling close - Java client/C++ broker. Hi Pavel, Thanks for having a go reproducing this, I'm afraid that the detail is a bit second hand, but the issue was specifically with pretty big message size with a persistent queue using the c++ broker. In particular my colleague said that it only started becoming weird at a message size of 1MB (1024*1024) with (1024*1024 - 1) the issue didn't appear. Skimming through your code (and without a copy of my colleagues code handy) the only difference I can guess at is that he's likely to have done: byte[] data = new byte[1024*1024]; msg.writeBytes(data); to try this out - shouldn't make any odds though, you did try 1MB as msgSize?? I think he also explicitly closed the producer and session before the connection, again it shouldn't make any odds I think the main point is that it's in the middle of trying to persist a large durable message when close gets called. One other thing, in my post I said that He tried using sync_publish but reckons that didn't help well he got back to me today and said that this *does* actually work - he was setting it in the brokerList initially when he moved it to the main URL it worked, as did setting it as a JVM property. He's pretty adamant that he is seeing this behaviour and I've no reason to doubt him, but as it only happens when 1MB is reached and only for persistent messages I think its a race condition between the persistence and the close and as I say below it's weird that it's exactly 1024*1024 where it starts happening so I'm thinking there's a cache/buffer beyond which disk sync/flush is occurring that taking more time. I guess that if your machine is a different spec it might affect reproducibility??!!?? Thinking about it have you tried it *without* the logging between send() and close()?? It's just possible that the logging takes just enough time to make it work OK - my colleague tried a short sleep call and that fixed the problem too. I've also asked him to try down-casting Session to AMQSession and calling sync() on that to see if it makes any difference and also to try a transacted session (which should call sync() when commit() is called). Looking through the implementation methods though it looks like Session close() is calling an underlying sync method, but there's some guard code around it so it's possible that it's not actually being called in some circumstances. Not syncing seems the most likely 'cause especially given that sync_publish does appear to work and I'd bet his python example also calls sync() JMS obviously doesn't have an explicit sync() call hence the other thoughts above. Thanks again for having a go reproducing it. When I get a moment I'll have a go at reproducing it myself, though I'll need to build the persistence module 'cause I never use it myself. Frase On 07/05/13 11:58, Pavel Moravec wrote: Hi Fraser, I tried to reproduce it but havent succeeded. Having following code snippet: Properties properties = new Properties(); properties.setProperty(java.naming.factory.initial, org.apache.qpid.jndi.PropertiesFileInitialContextFactory); properties.setProperty(connectionfactory. + qpidConnectionfactory, amqp://guest:guest@clientid/test?brokerlist='tcp://localhost:5672?sasl_mechs='PLAIN''); Context ctx = new InitialContext(properties); ConnectionFactory conFac = (ConnectionFactory) ctx.lookup(qpidConnectionfactory); Connection connection = conFac.createConnection(guest, guest); Destination destination = new AMQAnyDestination(address); Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); BytesMessage msg = session.createBytesMessage(); for (int i=0; imsgSize; i++) msg.writeChar('A'); MessageProducer prod = session.createProducer(destination); log.info(Before send); prod.send(msg); log.info(After send); connection.close(); log.info(After close); and running it for various msgSize, I saw no exception being raised :-/ I guess there have to be something more (configuration-specific).. Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Friday, May 3, 2013 9:17:05 AM Subject: Exception when calling close
Re: Exception when calling close - Java client/C++ broker.
Hi Fraser, I tried to reproduce it but havent succeeded. Having following code snippet: Properties properties = new Properties(); properties.setProperty(java.naming.factory.initial, org.apache.qpid.jndi.PropertiesFileInitialContextFactory); properties.setProperty(connectionfactory. + qpidConnectionfactory, amqp://guest:guest@clientid/test?brokerlist='tcp://localhost:5672?sasl_mechs='PLAIN''); Context ctx = new InitialContext(properties); ConnectionFactory conFac = (ConnectionFactory) ctx.lookup(qpidConnectionfactory); Connection connection = conFac.createConnection(guest, guest); Destination destination = new AMQAnyDestination(address); Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); BytesMessage msg = session.createBytesMessage(); for (int i=0; imsgSize; i++) msg.writeChar('A'); MessageProducer prod = session.createProducer(destination); log.info(Before send); prod.send(msg); log.info(After send); connection.close(); log.info(After close); and running it for various msgSize, I saw no exception being raised :-/ I guess there have to be something more (configuration-specific).. Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Friday, May 3, 2013 9:17:05 AM Subject: Exception when calling close - Java client/C++ broker. Hi all, A colleague of mine mailed me the other day with some behaviour I hadn't come across before. in précis he was messing about with a Java client and C++ broker (Qpid 0.18 I believe but I doubt that's relevant here). The client was very simple: create Connection, create Session, create MessageProducer, create ByteMessage, send message, close Connection. What was interesting was when the message got larger. So he sent to a persistent queue and when the message hit 1MB (1024*1024) he got an Exception when closing. Interestingly when he tried with a message just one octet less (1024*1024 - 1) the Exception didn't occur. I can't recall the exact Exception he mentioned unfortunately, but it related to not closing cleanly. The message did actually end up on the queue, and if he loops with several messages all messages get sent and the issue is clearly caused at the end during the call to close. My response to him was that I suspected that it related to the default behaviour of Qpid, which is to use asynchronous delivery for efficiency, which kind of bends the JMS spec a little (IIRC send should block until a broker has taken responsibility for the message and close should block until the pending message has been delivered). The hitting 1MB was weird, but I reckoned that it related to caching behaviour, so below 1MB the broker has the message in cache/buffers and the flush/sync to actual disk doesn't immediately occur, so the acks go back to the client pretty much instantly thus close is happy, but when it hits 1MB a disk flush occurs which hits real disk and takes longer thus triggering the Exception due to closing while the transfer is in progress. I might be wrong, but it feels plausible and definitely felt like a race condition between persisting the message and closing the connection. What I'm not clear on is how to make such a scenario behave cleanly. I did suggest that if he was really sending one message and closing immediately then it might make sense to force synchronous behaviour. He tried using sync_publish but reckons that didn't help, which surprised me. Of course adding a delay between calling send and close makes it behave properly, but that feels like a cop-out solution for what is clearly a race condition. Can anyone suggest the correct solution for the case where you really want to send a (big durable) message and immediately close the connection? One other thing, my colleague mentioned that he'd tried the same scenario with a python client and he said that it didn't behave like this. thoughts? MTIA, Frase - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: JBoss AS 7 w/ qpid RA and RabbitMQ: NullPointerException when receiving a message
Hi James, this comes from UnprocessedMessage_0_8 where _exchange seems not to be initialized - no idea why. Anyway is there some reason to use AMQP 0.8 or BURL syntax (deprecated in favour of address strings)? It is probable the apparently bug is not present when using either address strings or higher AMQP protocol version (currently in qpid client, 0.10). Kind regards, Pavel Moravec - Original Message - From: James Gilbertson james.gilbert...@bitsinglass.com To: users@qpid.apache.org Sent: Friday, February 22, 2013 5:40:34 PM Subject: JBoss AS 7 w/ qpid RA and RabbitMQ: NullPointerException when receiving a message Hi, We're currently developing an app that needs to subscribe to a RabbitMQ queue, running on JBoss AS 7. I've tried using the quid RA, and have managed to get to the point where it's connecting to the queue. However, when a message is received, I receive the following exception: 09:18:44,446 ERROR [org.apache.qpid.client.BasicMessageConsumer] (Dispatcher-1-Conn-1) Caught exception (dump follows) - ignoring...: java.lang.NullPointerException at org.apache.qpid.client.message.AbstractAMQMessageDelegate.generateDestination(AbstractAMQMessageDelegate.java:117) at org.apache.qpid.client.message.AMQMessageDelegate_0_8.init(AMQMessageDelegate_0_8.java:127) at org.apache.qpid.client.message.AbstractJMSMessageFactory.create08MessageWithBody(AbstractJMSMessageFactory.java:103) at org.apache.qpid.client.message.AbstractJMSMessageFactory.createMessage(AbstractJMSMessageFactory.java:160) at org.apache.qpid.client.message.MessageFactoryRegistry.createMessage(MessageFactoryRegistry.java:127) at org.apache.qpid.client.BasicMessageConsumer_0_8.createJMSMessageFromUnprocessedMessage(BasicMessageConsumer_0_8.java:118) at org.apache.qpid.client.BasicMessageConsumer_0_8.createJMSMessageFromUnprocessedMessage(BasicMessageConsumer_0_8.java:44) at org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:712) at org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3362) at org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3301) at org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3088) at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3081) at org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54) at org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3224) at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_13] Here's the configuration I'm currently using: subsystem xmlns=urn:jboss:domain:resource-adapters:1.0 resource-adapters resource-adapter archive qpid-ra-0.20.rar /archive transaction-supportXATransaction/transaction-support config-property name=TransactionManagerLocatorClass org.apache.qpid.ra.tm.JBoss7TransactionManagerLocator /config-property config-property name=connectionURL amqp://guest:guest@/?brokerlist='tcp://localhost:5672' /config-property config-property name=TransactionManagerLocatorMethod getTm /config-property admin-objects admin-object class-name=org.apache.qpid.ra.admin.QpidQueueImpl jndi-name=java:/amqp/queue/ldav use-java-context=false pool-name=LdavAmqp config-property name=DestinationAddress BURL:direct://excAlarms//sps.queue?durable='true' /config-property /admin-object /admin-objects /resource-adapter /resource-adapters /subsystem Is there something wrong with the above configuration? Or is it a bug with looking up the exchange? Because when I patch generateDestination to use the default exchange name of when exchange is null, receiving messages work fine. -- James Gilbertson Software Architect/Developer Bits In Glass Inc. http://www.bitsinglass.com james.gilbert...@bitsinglass.com +1 780-436-4484 ext. 8002 - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: limit-policy question
Hi Connor, this is not possible either C++ broker version. Default limit policy reject is hardcoded. Note that since qpid 0.20 there are only two limit policies, reject and ring, until a replacement of flow-to-disk is implemented (https://issues.apache.org/jira/browse/QPID-4339). Kind regards, Pavel - Original Message - From: Connor Poske connor.po...@onrampwireless.com To: users@qpid.apache.org Sent: Tuesday, February 5, 2013 2:39:48 AM Subject: limit-policy question Hello, this is my first time mailing to this group. The qpid-tool --help page and other references mention that one of the options for setting queue limit-policy is: none (default) - Use broker's default policy. I have searched high and low throughout all the documentation and done much googling' but I cannot find any reference to setting a broker's default limit policy. The qpidd man page does not show this as an option. I've blindly tried --default-limit-policy and --default-policy-type only to find both of those are invalid options. Is it actually possible to set the default queue limit policy on the broker so that any queues created on this broker that don't specify limit-policy (or set it to none) receive this default? Thanks, Connor - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Could somebody *please* explain Java Connection URLs...:-)
Hi Fraser, sorry for later response, I was in a rush at work. Yes my patch talks about internal variables username and password. Your suggestion makes sense - could you please attach it as a comment to the JIRA? Thanks, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Friday, January 25, 2013 1:15:07 PM Subject: Re: Could somebody *please* explain Java Connection URLs...:-) Hi Pavel, Thanks so much for the reply! So I'm thinking that this has always been the case with Java (despite the Jira referring to 0.18)? I'm thinking this is why I always ended up using guest:guest@ URLs when I started out, but it was a long time ago :-) You mention in the Jira comments at one point about: New version of patch. It again sets username to anonymous and password to (only when the credentials are missing) As I mentioned in my ramble below I actually tried *explicitly* doing: amqp://anonymous:@clientid/test?brokerlist='tcp://localhost:5672' in one of my tests, but that resulted in: connectionfactory.ConnectionFactory=amqp://anonymous:@clientid/test?brokerlist='tcp://localhost:5672', destination.publishedAddress=amq.match} main 2013-01-25 11:48:53,809 WARN [apache.qpid.jndi.PropertiesFileInitialContextFactory] Unable to createFactories:Illegal character in authority between indicies 7 and 1 amqp://anonymous:@clientid/test?brokerlist='tcp://localhost:5672' ^ createJMSSession() connectionFactory lookup failed, retrying So I'm guessing that what you are talking about is under the hood and bypasses that particular test?? I personally think that this bug notice should be extended to cover implicit *and* explicit settings of anonymous - what d'you reckon? The reason that I'm particularly interested in this issue now is that Bruno Matos noticed an issue with my recent Qpid GUI (well really in the back-end REST Server). He was running with a broker set to the default --auth yes and got the exception: INFO org.apache.qpid.client.AMQConnection - Not a hard-error connection not closing: org.apache.qpid.AMQException: ch=1 id=7 ExecutionException(errorCode=UNAUTHORIZED_ACCESS, commandId=14, classCode=0, commandCode=0, fieldIndex=0, description=unauthorized-access: authorised user id : anonymous@QPID but user id in message declared as guest (qpid/broker/SemanticState.cpp:484), errorInfo={}) [error code 403: access refused] So my reckoning was that issue was down to my ConnectionHelper class constructing a ConnectionURL of the form amqp://guest:guest@clientid when no user/passwd is specified (which would be the case for a default URL created if no -a option had been specified). I was planning on looking into that (defaulting it to anonymous rather than guest), hence my investigations, but it's looking like the underlying issue is deeper than my code. I guess that even when your patch makes it into the code base I'm going to have to be careful 'cause any patch will only work for up-to-date versions on the client runtime. Do you have any thoughts for a way 'round this or is Bruno (and others) just going to have to add guest as well as anonymous (or use proper usernames/passwds) if they run with --auth yes Cheers (thanks for restoring my sanity :-)) Sorry Bruno - not sure there's much I can do to sort that particular issue that you'd seen unless Pavel has any good ideas here (though as I said previously you should be able to specify your own default connection using the -a option in QpidRestAPI so that the GUI will use that as the default - rather than having to add a new connection on the GUI) Frase On 25/01/13 10:41, Pavel Moravec wrote: Hi Fraser, there is a bug in Java client disallowing no credentials option, see https://issues.apache.org/jira/browse/QPID-3396. Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Friday, January 25, 2013 11:07:29 AM Subject: Could somebody *please* explain Java Connection URLs...:-) It's a little embarrassing 'cause I've kind of been using these for a couple of years, but as they are ever so slightly obtuse :- I've generally been lazy/pragmatic and did copy'n'paste of things that have worked. Because of the various URL differences (the ones used by the python tools, the qpid::messaging amqp URL and the Java ConnectionURL I also wrote a ConnectionHelper class to allow me to specify URLs in any format). This class basically creates a Java ConnectionURL with sensible values filled in when I specify one of the shorter (AKA more convenient!!) formats. I'm generally OK on the brokerList sub URLs and the options I put in there seem to be fine. However I've just been bitten. 'Cause I've been a bit lazy and mostly tested and tinkered
Re: Could somebody *please* explain Java Connection URLs...:-)
Hi Fraser, there is a bug in Java client disallowing no credentials option, see https://issues.apache.org/jira/browse/QPID-3396. Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Friday, January 25, 2013 11:07:29 AM Subject: Could somebody *please* explain Java Connection URLs...:-) It's a little embarrassing 'cause I've kind of been using these for a couple of years, but as they are ever so slightly obtuse :- I've generally been lazy/pragmatic and did copy'n'paste of things that have worked. Because of the various URL differences (the ones used by the python tools, the qpid::messaging amqp URL and the Java ConnectionURL I also wrote a ConnectionHelper class to allow me to specify URLs in any format). This class basically creates a Java ConnectionURL with sensible values filled in when I specify one of the shorter (AKA more convenient!!) formats. I'm generally OK on the brokerList sub URLs and the options I put in there seem to be fine. However I've just been bitten. 'Cause I've been a bit lazy and mostly tested and tinkered with qpidd --auth no or actually used a proper user/password I ended up having defaults of guest:guest in ConnectionHelper. This hasn't hitherto been an issue for me but standing up a vanilla qpidd with auth yes the unsecured authentication username is anonymous *not* guest. The documentation http://qpid.apache.org/books/0.20/Programming-In-Apache-Qpid/html/QpidJNDI.html#id2553965 says: amqp://[user:pass@][clientid]virtualhost[?option='value'[option='value']] But I'm not convinced that's accurate. I've been hacking around with a few ConnectionURLs in a JNDI file and # simple URL with user = guest #connectionfactory.ConnectionFactory = amqp://guest:guest@clientid/test?brokerlist='tcp://localhost:5672' # Omitting the clientid bit seems to work OK #connectionfactory.ConnectionFactory = amqp://guest:guest@/test?brokerlist='tcp://localhost:5672' # Omitting the virtualhost bit seems to work too, but the slash is important e.g. /test and / work test does not?? #connectionfactory.ConnectionFactory = amqp://guest:guest@/?brokerlist='tcp://localhost:5672' But I *cannot* get/figure out how to do anonymous from a Java Client to a C++ broker. According to the BNF form in the documentation connectionfactory.ConnectionFactory = amqp://clientid/test?brokerlist='tcp://localhost:5672' *should* work as [user:pass@] ought to be optional, but that just gives connectionFactory lookup failed, retrying with both --auth no and --auth yes Same for connectionfactory.ConnectionFactory = amqp://@clientid/test?brokerlist='tcp://localhost:5672' If I do connectionfactory.ConnectionFactory = amqp://:@clientid/test?brokerlist='tcp://localhost:5672' That actually connects to a broker with --auth no, but barfs with an error warning Failed to retrieve sasl username on a broker with --auth yes connectionfactory.ConnectionFactory = amqp://anonymous:@clientid/test?brokerlist='tcp://localhost:5672' Actually connects to a broker with --auth no, but barfs with no error on a broker with --auth yes and a connectionFactory lookup failed, retrying When I add info logging I get main 2013-01-25 09:51:26,539 INFO [apache.qpid.client.AMQConnection] Connection:amqp://anonymous:@clientid/test?brokerlist='tcp://localhost:5672' main 2013-01-25 09:51:26,973 INFO [qpid.client.protocol.AMQProtocolSession] Using ProtocolVersion for Session:0-10 main 2013-01-25 09:51:26,995 INFO [qpid.client.handler.ClientMethodDispatcherImpl] New Method Dispatcher:AMQProtocolSession[null] main 2013-01-25 09:51:27,016 INFO [apache.qpid.client.AMQConnection] Connecting with ProtocolHandler Version:0-10 main 2013-01-25 09:51:27,167 INFO [apache.qpid.client.AMQConnection] Unable to connect to broker at tcp://localhost:5672 org.apache.qpid.AMQException: *Cannot connect to broker: connection-forced: Authentication failed [error code 320: context in use*] at org.apache.qpid.client.AMQConnectionDelegate_0_10.makeBrokerConnection(AMQConnectionDelegate_0_10.java:203) at org.apache.qpid.client.AMQConnection.makeBrokerConnection(AMQConnection.java:609) at org.apache.qpid.client.A... Logging at DEBUG doesn't give any more useful info. I've tried connectionfactory.ConnectionFactory = amqp://anonymous:anonymous@clientid/test?brokerlist='tcp://localhost:5672' connectionfactory.ConnectionFactory = amqp://anonymous:@clientid/test?brokerlist='tcp://localhost:5672' connectionfactory.ConnectionFactory = amqp://anonymous:''@clientid/test?brokerlist='tcp://localhost:5672' connectionfactory.ConnectionFactory = amqp://anonymous: @clientid/test?brokerlist='tcp://localhost:5672' and they *all* barf out :-( So could someone please tell me what the magic incantation is on a Java ConnectionURL for
Re: Retrieving specific messages from Last Value Queue
Hi Rajesh, you are in fact asking for message selectors that work well in Java client but dont exist in C++ client or in broker (https://issues.apache.org/jira/browse/QPID-530). I *think* only Java / JMS client currently supports so. The only way to workaround is to have separate queues for each stock symbol (or at least separate queue for each symbol you wish to _select_) and subscribe consumers to all such queues (plus optionally consume from the designated queue for e.g. ORCL quote). Kind regards, Pavel - Original Message - From: Rajesh Khan rajeshkhan...@gmail.com To: users@qpid.apache.org Sent: Wednesday, January 9, 2013 10:53:30 PM Subject: Retrieving specific messages from Last Value Queue Hi I have a receiver in C# that is suppose to retrieve messages from a LV Queue. Currently I traverse through all the messages in that queue. AS follows while (receiver.Fetch(ref msg, timeout)) { * prop = msg.Properties;* } I wanted to know if there is a way to retrieve a message that has a specific value. For instance The sender sends the following value msga.SetProperty(stock-symbol, ORCL); sender.Send(msga); Is there any way that I could retrieve the ORCL message without iterating through other messages? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: How to run the hello world example? Or we may need a minimal and throughout Getting Started steps.
Hi Robin, what Cyrus SASL libraries do you have installed? I.e. what is output of rpm -qa | grep sasl command? Per the qpidd traces: 2012-12-28 09:24:03 [Protocol] trace SENT [127.0.0.1:5672-127.0.0.1:54004]: Frame[BEbe; channel=0; {ConnectionStartBody: server-properties={qpid.federation_tag:V2:36:str16(f4df1e6b-6d61-4522-94de-ade72447fa86)}; mechanisms=str16{V2:9:str16(ANONYMOUS), V2:5:str16(PLAIN)}; locales=str16{V2:5:str16(en_US)}; }] 2012-12-28 09:24:03 [Protocol] trace RECV [127.0.0.1:5672-127.0.0.1:54004]: Frame[BEbe; channel=0; {ConnectionCloseBody: reply-code=501; reply-text=internal-error: Sasl error: SASL(0): successful result: (qpid/SaslFactory.cpp:223); }] the broker properly sent to the client connection.start AMQP method requiring to authenticate using either ANONYMOUS or PLAIN mechanism (that is implied by auth=no), but the client closed the connection due to some internal error.. Also how did you compile your C++ example? Kind regards, Pavel - Original Message - From: Robin Lee robinlee.s...@gmail.com To: users@qpid.apache.org Sent: Friday, December 28, 2012 2:49:04 AM Subject: Re: How to run the hello world example? Or we may need a minimal and throughout Getting Started steps. I just ran the Python version of Hello-world[1], and it just worked with broker started as qpidd --auth=no. Any trick or bug in C++ client? [1] http://qpid.apache.org/books/0.18/Programming-In-Apache-Qpid/html/ch02s02.html robin On Fri, Dec 28, 2012 at 9:30 AM, Robin Lee robinlee.s...@gmail.com wrote: Still not work with --no-data-dir option. qpidd -t ouput: http://paste.stg.fedoraproject.org/2790 robin On Fri, Dec 28, 2012 at 1:24 AM, Chuck Rolke cro...@redhat.com wrote: Hello Robin Lee, You may have some persistent configuration working against you. Try the command qpidd --auth no --no-data-dir to disable persistent configuration. If that fails please post the results of broker trace by using switch '-t' on the command line. -Chuck - Original Message - From: Robin Lee robinlee.s...@gmail.com To: users@qpid.apache.org Sent: Tuesday, December 25, 2012 5:08:28 AM Subject: Fwd: How to run the hello world example? Or we may need a minimal and throughout Getting Started steps. Hi, all! I am getting started Qpid today and failed to run the hello world example. I install qpid-cpp-server on Feodra 18 through Yum. Current version is qpid-cpp-server-0.18-6.fc18.x86_64. Then I run server with the command qpidd. Then I compiled and ran the hello world example[1]. But it showed: 2012-12-25 17:18:55 [Client] warning Connection [127.0.0.1:40303-localhost:5672] closed Failed to connect (reconnect disabled) Then if I ran the server with qpidd --auth=no and ran the example again. The client showed: 2012-12-25 17:20:27 [Client] warning Closing connection due to internal-error: Sasl error: SASL(0): successful result: (qpid/SaslFactory.cpp:223) internal-error: Sasl error: SASL(0): successful result: (qpid/SaslFactory.cpp:223) So I think the Getting Started page is not so comprehensive. We, or I, may need a Getting Started instruction with minimal steps that make things just work, but meanwhile cover the setup of both server and client. After all, I am too new for Qpid. Sorry for my brute request. Cheers, robin [1] http://qpid.apache.org/books/0.18/Programming-In-Apache-Qpid/html/ch02.html#id2531116 - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Last Value Queue - Information Required
Hi Khan, no, one queue is enough. See my favourite example below: # create a LVQ named MyLVQ with values stored in header field stock-symbol qpid-config add queue MyLVQ --argument qpid.last_value_queue_key=stock-symbol # generate some messages with stock prices $ qpid-send -a MyLVQ --property 'stock-symbol'='RHT' --property 'stock-value'='50' -m 1 $ qpid-send -a MyLVQ --property 'stock-symbol'='ORCL' --property 'stock-value'='50' -m 1 $ qpid-send -a MyLVQ --property 'stock-symbol'='RHT' --property 'stock-value'='53' -m 1 $ qpid-send -a MyLVQ --property 'stock-symbol'='MSFT' --property 'stock-value'='50' -m 1 $ qpid-send -a MyLVQ --property 'stock-symbol'='RHT' --property 'stock-value'='52' -m 1 # read the queue content - only the latest stock-symbol value will be printed $ qpid-receive -a MyLVQ --print-headers yes --print-content no -m 10 Properties: {sn:1, stock-symbol:ORCL, stock-value:50, ts:1356948468924902877, x-amqp-0-10.routing-key:MyLVQ} Properties: {sn:1, stock-symbol:MSFT, stock-value:50, ts:1356948468994528314, x-amqp-0-10.routing-key:MyLVQ} Properties: {sn:1, stock-symbol:RHT, stock-value:52, ts:1356948469024261662, x-amqp-0-10.routing-key:MyLVQ} $ Kind regards, Pavel - Original Message - From: Rajesh Khan rajeshkhan...@gmail.com To: users@qpid.apache.org Sent: Monday, December 31, 2012 11:43:55 AM Subject: Last Value Queue - Information Required After reading about the LV queue. I wanted to implement the following scenario. I have 3 symbols A,B and C and I want the queue to always hold the latest prices of these symbols. My question is will I have to create a separate queue for each symbol ? Any suggestions would be appreciated. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: limit policy on bound temporary queues
Hi Lance, let use in address string of the consumers just the exchange name, i.e.: amq.fanout qpid client will detect from an answer of qpid broker that amq.fanout is an existing exchange and it will create an auxiliary, auto-delete exclusive queue named like amq.fanout_18efcccb-3abb-4db4-8ff0-7bb6a8fe0044 (exchange name, '_' and random UUID) and will bind it to the exchange. You can also specify routing key (that has no sense for fanout exchange, though) to be used in the binding, when using address string like amq.direct/routingKey. Kind regards, Pavel Moravec - Original Message - From: Lance D. lan...@gmail.com To: users@qpid.apache.org Sent: Tuesday, December 18, 2012 4:09:00 AM Subject: limit policy on bound temporary queues Hello, I'm trying to figure out a problem and I'm hoping to find the right way to make this happen. I've got a data producer that publishes to an fanout exchange. My system requires that under no circumstances can the producer get throttled because even if 1 user can't keep up, the other consumers need the data as soon as possible. My thought is that if I just force all of the consumers to connect with a 'ring' limit policy, then I should be okay because slow consumers will just drop data on the floor. So I'd like to be able to make that the default policy. That said, I'm fairly certain that isn't possible, so I'd at least like to setup my clients with the correct settings. I've read through the documentation, but I can't seem to figure out how to register to receive from an exchange, using a temporary (implicit) queue. Thanks in advance for the help. -Lance - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Problem using a configuration file
Hi Adam, try to put to the file just lines in parameter=value syntax, i.e.: auth=no Syntax qpidd --auth=no is used for command line parameters (like you used for --config option). Kind regards, Pavel - Original Message - From: Adam Zedan zedan...@gmail.com To: users@qpid.apache.org Sent: Tuesday, December 4, 2012 3:30:15 AM Subject: Problem using a configuration file I want to start using a configuration file with my broker. Initially I just have one parameter in the file . This is how it looks Filename : C:\qpidd.conf It only has one line in it which is qpidd --auth=no Now I am running the broker as such c:\...\debug qpidd --config c:\\qpidd.conf However I dont think the config is being read how can I resolve this issue ? Any help would definitely be appreciated. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: All cluster members die simultaneously with store state dirty
Hi, qpidd brokers stopped as they run on un-quorable cluster: Nov 23 23:14:09 server_name corosync[27510]: [CMAN ] quorum lost, blocking activity Nov 23 23:14:09 server_name corosync[27510]: [QUORUM] This node is within the non-primary component and will NOT provide any services. Nov 23 23:14:09 server_name corosync[27510]: [QUORUM] Members[1]: 1 Nov 23 23:14:09 server_name qpidd[29703]: 2012-11-23 23:14:09 critical Lost contact with cluster quorum. That is proper reaction to network split-brain (i.e. something that qpid can't affect). When starting a clustered broker with dirty store as the first broker within the cluster, the broker: - stops itself with the Cannot recover, no clean store. error, expecting some other peer will start up with clear (i.e. up-to-date) store - backs-up store Then second step is redundant and might cause the possible store content removal. That is unwanted behavior but not planned to be fixed, as current cluster solution is being replaced by new one in upstream (https://issues.apache.org/jira/browse/QPID-3603). Kind regards, Pavel Moravec - Original Message - From: 3.lis...@adminlinux.com.br lis...@adminlinux.com.br To: users@qpid.apache.org Sent: Wednesday, November 28, 2012 6:06:57 PM Subject: All cluster members die simultaneously with store state dirty Hi, I have a Qpid cluster (3 members) with this configuration: Ubuntu12.04 qpidd-0.14-2 qpidd-msgstore-0.14-1 cman-3.1.7 corosync-1.4.2-2 libtotem-pg4-1.4.2-2 My cluster works perfectly for months. But sometimes all members die simultaneously. Cluster store state gets dirty in all members and the daemon runs again only after this procedure: cd data-dir mv rhm rhm.bak cp -a _cluster.bak./rhm . qpid-cluster-store -c data-dir Therefore, messages are lost in my queues. I added ulimit-c unlimited in /etc/init.d/qpid but no core has been generated. Here are the logs from the time of the crash: http://www.adminlinux.com.br/logs Can anyone help me? Thanks. -- Thiago Henrique adminlinux.com.br - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: How would you receive this C++ List in C#
Hi Rajesh, I have almost no experience with C# bindings but I suppose you must use method void TypeTranslator::NativeToManaged(::qpid::types::Variant::Map qpidMap, QpidMap ^ dict) { // For each object in the message map, // create a .NET object and add it to the dictionary. (and TypeTranslator::ManagedToNative for reverse conversion) Kind regards, Pavel Moravec - Original Message - From: Rajesh Khan rajeshkhan...@gmail.com To: users@qpid.apache.org Sent: Tuesday, October 23, 2012 12:40:50 AM Subject: How would you receive this C++ List in C# Hi , I have something like this in my C++ program qpid::types::Variant::Map content; content[keyA] = ValA; content[KeyB] = ValB qpid::types::Variant::List object; object.push_back(content); encode(object,message); sender_.send(message); Now in my C# I have something like this if (amqp/list == message_.ContentType) { Collectionobject content = new Collectionobject(); message_.GetContent(content); //How could I get a Dictinary from this ? } - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Sending objects to receivers
+1, I have seen this working already in a qpid project. - Original Message - From: Andy Goldstein andy.goldst...@redhat.com To: users@qpid.apache.org Sent: Tuesday, October 16, 2012 6:48:15 PM Subject: Re: Sending objects to receivers You could use something like Google Protocol Buffers to encode your object as a binary string to send it over the wire. Andy On Oct 16, 2012, at 12:46 PM, Rajesh Khan wrote: Thats an idea. However I was curious is there a better way to do this ? What if my object has lots of properties On Tue, Oct 16, 2012 at 10:44 AM, Andy Goldstein andy.goldst...@redhat.comwrote: Could you just take your class's properties and place them in a map? Andy On Oct 16, 2012, at 12:29 PM, Rajesh Khan wrote: While studying QPIDD I realized that I could send maps and lists to receivers like this qpid::types::Variant::Map content; content[id] = 987654321; content[name] = Widget; content[percent] = 0.99; encode(content, message); sender.send(message); I wanted to know How could I send a customized object I created from a class. ? Any suggestions ? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: active-passive cluster
Hi Terence, that is far away of my knowledge. I *think* CMAN nodes have to be in the same LAN, definitely latency between a pair of nodes has to be relatively low (something like round-trip time below 2ms). I recommend asking on cman mailing list for this. Kind regards, Pavel - Original Message - From: Terance Dias tera...@socialtwist.com To: users@qpid.apache.org Sent: Tuesday, October 16, 2012 3:03:42 PM Subject: Re: active-passive cluster I tried giving the hostname -- IP address translation to /etc/hosts but it did not work either. Then I tried giving the internal IP address of the node where I was trying to run the cman service. The cman service did run but it was not able to resolve the other node (which lies in another network) with its static/external IP. Does cman only works with internal IPs? Are all the nodes required to be in the same network for the cluster to work? Thanks, Terance. On Mon, Oct 15, 2012 at 9:14 PM, Pavel Moravec pmora...@redhat.com wrote: That seems to be ok.. Source code of cman seems to be not straightforward at that stage, maybe the cman preconfig does not fit with DNS.. Let try to write the hostname - IP address translation to /etc/hosts or ask in cman mailing list: http://sources.redhat.com/cluster/cman/ Kind regards, Pavel - Original Message - From: Terance Dias tera...@socialtwist.com To: users@qpid.apache.org Sent: Monday, October 15, 2012 8:31:53 AM Subject: Re: active-passive cluster Thanks a lot for your reply Pavel. Below is my cluster.conf file. --- ?xml version=1.0? !-- This is an example of a cluster.conf file to run qpidd HA under rgmanager. NOTE: fencing is not shown, you must configure fencing appropriately for your cluster. -- cluster name=test-qpid-cluster config_version=18 !-- The cluster has 2 nodes. Each has a unique nodid and one vote for quorum. -- clusternodes clusternode name=ec2-node-1.amazonaws.com nodeid=1/ clusternode name=ec2-node-2.amazonaws.com nodeid=2/ /clusternodes cman two_node=1 expected_votes=1 /cman !-- Resouce Manager configuration. -- rm !-- There is a failoverdomain for each node containing just that node. This lets us stipulate that the qpidd service should always run on each node. -- failoverdomains failoverdomain name=east-domain restricted=1 failoverdomainnode name=ec2-node-1.amazonaws.com/ /failoverdomain failoverdomain name=west-domain restricted=1 failoverdomainnode name=ec2-node-2.amazonaws.com/ /failoverdomain /failoverdomains resources !-- This script starts a qpidd broker acting as a backup. -- script file=/usr/local/etc/init.d/qpidd name=qpidd/ !-- This script promotes the qpidd broker on this node to primary. -- script file=/usr/local/etc/init.d/qpidd-primary name=qpidd-primary/ /resources !-- There is a qpidd service on each node, it should be restarted if it fails. -- service name=east-qpidd-service domain=east-domain recovery=restart script ref=qpidd/ /service service name=west-qpidd-service domain=west-domain recovery=restart script ref=qpidd/ /service !-- There should always be a single qpidd-primary service, it can run on any node. -- service name=qpidd-primary-service autostart=1 exclusive=0 recovery=relocate script ref=qpidd-primary/ /service /rm /cluster -- The exact message I get when I start cman service is : - Starting cluster: Checking Network Manager... [ OK ] Global setup... [ OK ] Loading kernel modules... [ OK ] Mounting configfs... [ OK ] Starting cman... Cannot find node name in cluster.conf Unable to get the configuration Cannot find node name in cluster.conf cman_tool: corosync daemon didn't start - Thanks, Terance. On Fri, Oct 12, 2012 at 7:38 PM, Pavel Moravec pmora...@redhat.com wrote: Hi Terance, could you paste here your cluster.conf file? (I know I had the same issue some time ago, since I specify the hostname in /etc/hosts I dont have problems further - but that may be irrelevant). - Original Message - From: Terance Dias tera...@socialtwist.com To: users@qpid.apache.org Sent: Friday, October 12, 2012 3:49:47 PM Subject: active-passive cluster Hi, I'm using qpid trunk
Re: active-passive cluster
That seems to be ok.. Source code of cman seems to be not straightforward at that stage, maybe the cman preconfig does not fit with DNS.. Let try to write the hostname - IP address translation to /etc/hosts or ask in cman mailing list: http://sources.redhat.com/cluster/cman/ Kind regards, Pavel - Original Message - From: Terance Dias tera...@socialtwist.com To: users@qpid.apache.org Sent: Monday, October 15, 2012 8:31:53 AM Subject: Re: active-passive cluster Thanks a lot for your reply Pavel. Below is my cluster.conf file. --- ?xml version=1.0? !-- This is an example of a cluster.conf file to run qpidd HA under rgmanager. NOTE: fencing is not shown, you must configure fencing appropriately for your cluster. -- cluster name=test-qpid-cluster config_version=18 !-- The cluster has 2 nodes. Each has a unique nodid and one vote for quorum. -- clusternodes clusternode name=ec2-node-1.amazonaws.com nodeid=1/ clusternode name=ec2-node-2.amazonaws.com nodeid=2/ /clusternodes cman two_node=1 expected_votes=1 /cman !-- Resouce Manager configuration. -- rm !-- There is a failoverdomain for each node containing just that node. This lets us stipulate that the qpidd service should always run on each node. -- failoverdomains failoverdomain name=east-domain restricted=1 failoverdomainnode name=ec2-node-1.amazonaws.com/ /failoverdomain failoverdomain name=west-domain restricted=1 failoverdomainnode name=ec2-node-2.amazonaws.com/ /failoverdomain /failoverdomains resources !-- This script starts a qpidd broker acting as a backup. -- script file=/usr/local/etc/init.d/qpidd name=qpidd/ !-- This script promotes the qpidd broker on this node to primary. -- script file=/usr/local/etc/init.d/qpidd-primary name=qpidd-primary/ /resources !-- There is a qpidd service on each node, it should be restarted if it fails. -- service name=east-qpidd-service domain=east-domain recovery=restart script ref=qpidd/ /service service name=west-qpidd-service domain=west-domain recovery=restart script ref=qpidd/ /service !-- There should always be a single qpidd-primary service, it can run on any node. -- service name=qpidd-primary-service autostart=1 exclusive=0 recovery=relocate script ref=qpidd-primary/ /service /rm /cluster -- The exact message I get when I start cman service is : - Starting cluster: Checking Network Manager... [ OK ] Global setup... [ OK ] Loading kernel modules... [ OK ] Mounting configfs... [ OK ] Starting cman... Cannot find node name in cluster.conf Unable to get the configuration Cannot find node name in cluster.conf cman_tool: corosync daemon didn't start - Thanks, Terance. On Fri, Oct 12, 2012 at 7:38 PM, Pavel Moravec pmora...@redhat.com wrote: Hi Terance, could you paste here your cluster.conf file? (I know I had the same issue some time ago, since I specify the hostname in /etc/hosts I dont have problems further - but that may be irrelevant). - Original Message - From: Terance Dias tera...@socialtwist.com To: users@qpid.apache.org Sent: Friday, October 12, 2012 3:49:47 PM Subject: active-passive cluster Hi, I'm using qpid trunk code and trying to set up the active-passive cluster mentioned at http://qpid.apache.org/books/trunk/AMQP-Messaging-Broker-CPP-Book/html/chap-Messaging_User_Guide-Active_Passive_Cluster.html . I'm having some trouble setting up cman and rpmanager 2 node cluster on our Ubuntu-11.4 64 bit servers. Has anybody successfully set up this cluster on Ubuntu? It would be great if you could share some kind of resource or document that explains how this can be done. Right now if I start cman I get an error saying Starting cman... Cannot find node name in cluster.conf . The node names mentioned in my cluster.conf file is accessible through DNS lookup and I've opened the following ports between my servers. TCP - 41966,41967,41968,41969,50006,50008,50009,21064 UPD - 50007,5404,5405 I'm not sure what I'm doing wrong. Thanks, Terance. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Cannot start qpidd
Hi, you reached journal limit (reaching approx. 80% of the journal capacity), I assume due to number of journal files less than 8 - see http://qpid.2158936.n2.nabble.com/Message-store-library-from-qpidcomponents-org-generates-JERR-JCNTL-RECOVERJFULL-error-during-recovery-td7345136.html for detailed description. I recommend resizing the journal before starting the qpid broker, see e.g. https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_MRG/2/html/Messaging_Installation_and_Configuration_Guide/Resize_the_Journal.html for reference. To prevent repeating the same problem in future, I recommend either increase journal files to at least 8 if relevant (or wait for the fix to be included either in 0.18 or 0.20, not sure now), or if you really reached the journal capacity by many enqueues, read https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_MRG/2/html/Messaging_Installation_and_Configuration_Guide/sect-Determining_Journal_Size.html to properly determine your journal size. Kind regards, Pavel Moravec - Original Message - From: 3.lis...@adminlinux.com.br lis...@adminlinux.com.br To: users@qpid.apache.org Sent: Monday, October 8, 2012 8:22:16 PM Subject: Cannot start qpidd Hi, I have a ubuntu12.04 + qpidd-0.14-2 + qpidd-msgstore-0.14-1 cluster. My cluster was working perfectly. I turned it off without problems. But now the command service qpidd start does not turn on the daemon. The only problem seems to be in a queue: Oct 8 17:32:14 net17 qpidd[12316]: 2012-10-08 17:32:14 critical Unexpected error: Queue thiago.test: recoverQueues() failed: jexception 0x0205 jcntl::recover() threw JERR_JCNTL_RECOVERJFULL: Journal data files full, cannot write. (MessageStoreImpl.cpp:826) I need to start the cluster and I can't lose the messages of the other queues. Is there a way to make qpidd daemon to ignore this problem with the queue thiago.test? Is there a way to make qpidd to ignore the thiago.test queue and start the cluster? Thanks. -- Thiago Henrique adminlinux.com.br - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Log rolling in Qpid C++ broker?
Hi Parkirat, C++ broker does not offer this feature. There is a workaround in logging to syslog that redirects the logs to a separate file, and use logrotate for the file. Kind regards, Pavel Moravec - Original Message - From: ParkiratBagga parkiratba...@gmail.com To: users@qpid.apache.org Sent: Saturday, September 22, 2012 9:25:46 AM Subject: Log rolling in Qpid C++ broker? Hi, How to do log rolling in C++ broker for Qpid? I don't see any qpid parameters to do log rolling. Regards, Parkirat Singh Bagga -- View this message in context: http://qpid.2158936.n2.nabble.com/Log-rolling-in-Qpid-C-broker-tp7582502.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Dynamic topic/queue creation
Hi Andrew, qpid uses address string syntax for that, see Programming in Apache QPID (http://qpid.apache.org/books/0.16/Programming-In-Apache-Qpid/pdf/Programming-In-Apache-Qpid.pdf), section 2.4. For example, to create a queue and automatically delete it when the session is closed, use address string: queueName; { create:always, delete:always, node:{type:queue }} See above document and also https://cwiki.apache.org/confluence/display/qpid/Qpid+extensions+to+AMQP for a complete list of parameters one can declare. Kind regards, Pavel - Original Message - From: Andrew Erickson andrew.erick...@sungard.com To: users@qpid.apache.org Sent: Friday, August 31, 2012 2:58:48 PM Subject: Dynamic topic/queue creation Hi, We have a JMS based application using ActiveMQ which we are migrating to QPid. I have successfully tested the application with QPid except for one issue. On startup I receive this warning - 14:16:02,505 WARN Setup of JMS message listener invoker failed for destination 'ServiceCallQueue' - trying to recover. Cause: Error registering consumer: org.apache.qpid.AMQException: The name 'ServiceCallQueue' supplied in the address doesn't resolve to an exchange or a queue If I manually add the queue to the config.xml then the warning is gone and everything works fine. Questions: 1. Does QPid support dynamic queue/topic creation at runtime? 2. How do I configure QPid for dynamic queue/topic creation? We are using QPid 0.16. Thanks in advance, Andrew. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Setting up Kerberos security
Hi Davide, add to /etc/sasl2/qpidd.conf: mech_list: GSSAPI to stick qpid broker in enforcing just that type of SASL mechanism. If you require to set it in the client, then it depends on what client/tool do you use. I.e. qpid-config --sasl-mechanism=GSSAPI .. C++ spout/drain: ./drain --connection-options '{sasl_mechanisms:GSSAPI, username:guest, password:guest }' .. Java client: ./run_example.sh org.apache.qpid.example.Drain --broker-option=sasl_mechs=GSSAPI Kind regards, Pavel - Original Message - From: Davide Anastasia davide.anasta...@qualitycapital.com To: users@qpid.apache.org Sent: Thursday, August 9, 2012 3:52:49 PM Subject: RE: Setting up Kerberos security Hi Gordon, I'm using the C++ broker, v. 0.14, the one shipped with RHEL6. I have used the guide in your page to set up the Kerberos server, without success. How can I explicitly require GSSAPI as the mechanism? How can I change it in the /etc/sasl2/qpidd.conf file? I'm just checking whether the cyrus-sasl is available. Best, Davide -Original Message- From: Gordon Sim [mailto:g...@redhat.com] Sent: 09 August 2012 10:05 To: users@qpid.apache.org Subject: Re: Setting up Kerberos security On 08/08/2012 05:00 PM, Davide Anastasia wrote: Hi All, Can anybody point me to a guide that could help me to setup Kerberos to work with Qpid? Which broker are you using? qpidd (i.e. c++) or qpid-server (Java). If the former, is http://qpid.apache.org/books/0.16/AMQP-Messaging-Broker-CPP-Book/html/ch ap-Messaging_User_Guide-Security.html the text you have already seen? I've been trying to follow the steps described in the User Manual, but Qpid still uses its SASL database for authentication. Which client are you using, and are you explicitly requesting GSSAPI as the mechanism? If using qpidd, does your /etc/sasl2/qpidd.conf file have the mech_list set and does the list include GSSAPI? Do you have the cyrus-sasl gssapi module installed? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Writing wrong filter in a QMF method causes the filter is ignored
Hi, I realized one particular (for me) unexpected behavior when using filters in QMF methods (in C++ broker). Invoking e.g. method for moving messages from a queue to another with an invalid filter like: (in qpid-tool) call 120 queueMoveMessages SrcQueue DstQueue 0 {'qpid.subject':'key2'} will produce an error in qpidd log: 2012-07-31 11:50:14 error Ignoring unrecognized message filter: '{qpid.subject:key2}' but the messages are moved from SrcQueue to DstQueue. So having an invalid filter in a QMF method raises an error log but otherwise the filter is ignored and method applied. And the method returns zero as usual. Do you treat that as correct behavior? I would expect rather not executing such a method and some error response code to be returned. (Philosophically: rather than perform a method with wrong arguments, it is safer to ignore its execution) Kind regards, Pavel Moravec - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Writing wrong filter in a QMF method causes the filter is ignored
Hi Ken, see https://issues.apache.org/jira/browse/QPID-4177, by default assigned to Ted Ross (havent found a way how to change it). Cheers, Pavel - Original Message - From: Ken Giusti kgiu...@redhat.com To: users@qpid.apache.org Sent: Tuesday, July 31, 2012 3:44:43 PM Subject: Re: Writing wrong filter in a QMF method causes the filter is ignored Pavel - I agree that behavior is unexpected - the command should fail without modifying the queue. Can you raise a JIRA and assign it to me? thanks -K - Original Message - Hi, I realized one particular (for me) unexpected behavior when using filters in QMF methods (in C++ broker). Invoking e.g. method for moving messages from a queue to another with an invalid filter like: (in qpid-tool) call 120 queueMoveMessages SrcQueue DstQueue 0 {'qpid.subject':'key2'} will produce an error in qpidd log: 2012-07-31 11:50:14 error Ignoring unrecognized message filter: '{qpid.subject:key2}' but the messages are moved from SrcQueue to DstQueue. So having an invalid filter in a QMF method raises an error log but otherwise the filter is ignored and method applied. And the method returns zero as usual. Do you treat that as correct behavior? I would expect rather not executing such a method and some error response code to be returned. (Philosophically: rather than perform a method with wrong arguments, it is safer to ignore its execution) Kind regards, Pavel Moravec - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Chain of dynamic routes does not work?
Hello qpid users, I am trying to send a message over multiple dynamic routes but I fail so. Trivial setup: * 3 nodes train1, train2 and train3 * create dynamic routes train1 - train2 - train3: qpid-route dynamic add train1 train2 amq.direct qpid-route dynamic add train2 train3 amq.direct * On each broker, create a queue and bind it to the amq.direct exchange: (node1): qpid-config add queue train1 qpid-config bind amq.direct train1 train1 (do the above analogously on train2 and train3) * Try to send a message to amq.direct/train3 address to train1 broker. amq.direct exchange on train1 drops it. Why the exchange dropped the message? Or is dynamic route restricted in the forwarding of bindings from destination broker to source broker to one hop only? Thanks in advance for your feedback. Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Chain of dynamic routes does not work?
Oops, a typo in the commands: qpid-route dynamic add train3 train2 amq.direct qpid-route dynamic add train2 train1 amq.direct (I.e. creating routes train1 - train2 - train3 and sending a message to train1). - Original Message - From: Pavel Moravec pmora...@redhat.com To: users@qpid.apache.org Sent: Monday, May 14, 2012 3:31:36 PM Subject: Chain of dynamic routes does not work? Hello qpid users, I am trying to send a message over multiple dynamic routes but I fail so. Trivial setup: * 3 nodes train1, train2 and train3 * create dynamic routes train1 - train2 - train3: qpid-route dynamic add train1 train2 amq.direct qpid-route dynamic add train2 train3 amq.direct * On each broker, create a queue and bind it to the amq.direct exchange: (node1): qpid-config add queue train1 qpid-config bind amq.direct train1 train1 (do the above analogously on train2 and train3) * Try to send a message to amq.direct/train3 address to train1 broker. amq.direct exchange on train1 drops it. Why the exchange dropped the message? Or is dynamic route restricted in the forwarding of bindings from destination broker to source broker to one hop only? Thanks in advance for your feedback. Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Creation of Request and Response quese
Hi Sumi, it is enough to set reply-to in the message request and have a subscriber of that response queue. See attached C++ example program (its purpose is to send a QMF query to get some queue details and requesting to get the response to the given queue). So the key commands are: //create a receiver of the response queue; I recommend having there {create:always, delete:always} part to automatically create it now and delete it once the receiver is closed Receiver r = session.createReceiver(response-queue); //in the request message m, set reply-to m.setReplyTo(Address(r.getName())); (sending the request message) //fetch / receive the response, wait for it at most 3 seconds m = r.fetch(3 * Duration::SECOND); To ensure unique and exclusive access to the response queue, it makes sense to have the queue named with some uuid (i.e. use uuid_generate method in C++) and to use exclusive queue (to grant the only access to the queue only for this subscriber). Kind regards, Pavel - Original Message - From: Sumi subha.manjun...@newedge.com To: users@qpid.apache.org Sent: Thursday, April 19, 2012 7:32:14 AM Subject: Creation of Request and Response quese Hi, I am very new to Qpid , How do i create Request and response queues For ex:request queue , durable , direct exchange response queue : is an temporary queue and what should be my reply to property and how to bind these two queues so that when my producer send message to request queue , the consumer should be able to recieve from the response queue ? Please help this might be basic question but i am very new to qpid and have been asked to create req , response queues? -- View this message in context: http://qpid.2158936.n2.nabble.com/Creation-of-Request-and-Response-quese-tp7479418p7479418.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org #include qpid/messaging/Connection.h #include qpid/messaging/Session.h #include qpid/messaging/Sender.h #include qpid/messaging/Receiver.h #include qpid/messaging/Message.h #include qpid/messaging/Address.h #include iostream using namespace std; using namespace qpid::messaging; using namespace qpid::types; int main(int argc, char** argv) { if (argc 2) { cerr Missing queue name to print. endl; return 1; } string queue_name = argv[1]; Connection c(argc2?argv[2]:localhost:5672); c.open(); Session session = c.createSession(); Receiver r = session.createReceiver(#qlister; {create:always, delete:always}); Sender s = session.createSender(qmf.default.direct/broker); Message m; m.setReplyTo(Address(r.getName())); m.setProperty(x-amqp-0-10.app-id, qmf2); m.setProperty(qmf.opcode, _query_request); Variant::Map request; request[_what] = OBJECT; Variant::Map schemaId; schemaId[_class_name] = queue; request[_schema_id] = schemaId; encode(request, m); s.send(m); m = r.fetch(3 * Duration::SECOND); session.acknowledge(m); Variant::List response; decode(m, response); for(Variant::List::iterator iter = response.begin(); iter != response.end(); iter++) { Variant::Map map = iter-asMap(); Variant::Map values = map[_values].asMap(); //cout values[name] : map endl endl; // this will print out the queue name and its complete map of all attributes and values if (values[name] == queue_name) { cout queue values[name] : qpid.max_count= values[arguments].asMap()[qpid.max_count] , qpid.max_size= values[arguments].asMap()[qpid.max_size] , msgDepth= values[msgDepth] , byteDepth= values[byteDepth] endl; } } r.close(); s.close(); session.close(); c.close(); return 0; } - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Reduce Sender traffic
Hi Bruno, what broker (C++ / Java) and client (C++ / Java / Python / ..) are you using? Generally, I would recommend creating more AMQP sessions where messages to potentially blocked queues are sent on one session, time to time synchronize those sessions and use producer flow control that would block sender from sending messages to a queue occupied by 80% (default value) or more. Kind regards, Pavel - Original Message - From: Bruno Matos bruno.ma...@paradigmaxis.pt To: users@qpid.apache.org Sent: Wednesday, April 18, 2012 12:19:52 PM Subject: Reduce Sender traffic Hi, I have a sender that publishes about 10MB of messages per second for different topics. Only about 1% of that traffic is delivered because only about 1% of the topics are bound at a time. Is it possible, or would it be desirable, that the sender only effectively send the messages for witch there is a bound queue? Tia. -- Bruno Matos - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: 0.14 cluster never survives more than an hour or so.
Hi Paul, both errors occur under very similar circumstances. I recommend enabling debug logs of cluster component by adding: log-enable=debug+:cluster log-enable=notice+ to qpidd.conf and post the logs to a new JIRA. (you can try enabling trace logs that might provide more verbose output but running traces for 1/2 hour would require some nontrivial disk space) To alleviate consequences, I think disabling management shall help (but some other problems can arise later on somewhere else, as this just prevents the consequence and not the root cause bug). And some QMF based services (like qpid-tool) won't work with management disabled. To disable management stuff, add to qpidd.conf: mgmt-enable=no Alternatively, one can setup frequency of management updates (that are processed by the periodicProcessing task), see mgmt-pub-interval option (set by default to 10 seconds). Setting it to e.g. 2 hours, your qpid cluster will run for at least 2 hours without the error. But again, some QMF based services rely on the updates. Kind regards, Pavel Moravec - Original Message - From: Paul Colby p...@colby.id.au To: users@qpid.apache.org Sent: Friday, April 13, 2012 11:02:14 AM Subject: Re: 0.14 cluster never survives more than an hour or so. Alas the patch at https://issues.apache.org/jira/browse/QPID-3369 has not fixed the issue. Interestingly though, it did move the error to a different line, but with a very similar message. eg Apr 13 17:04:17 gateway02 qpidd[32258]: 2012-04-13 17:04:17 critical Error delivering frames: Cluster timer wakeup non-existent task ManagementAgent::periodicProcessing (qpid/cluster/ClusterTimer.cpp:112) So it's moved from ClusterTimer::deliverDrop to ClusterTimer::deliverWakeup instead... but with the same effectual result. pc http://colby.id.au On Fri, Apr 13, 2012 at 9:30 AM, Paul Colby p...@colby.id.au wrote: Thanks Pavel and Gordon, I really appreciate you guys getting back to me so quickly :) I'm not currently using cman. I hadn't been using it on 0.12 either. I suspect that split-brain is not the case, since the test cluster in question on on virtual machines all within a single host, with *very* reliable virtual networking between them. After reading your response, I did have a quick look at setting up cman to verify either way, but that's not proving to be quick and easy, so I'll come back to it shortly. The https://issues.apache.org/jira/browse/QPID-3369 issue does look interesting. I'll apply the patch suggested there and see what difference it makes. Thanks again. I'll let you know how it goes :) pc http://colby.id.au On Thu, Apr 12, 2012 at 9:39 PM, Pavel Moravec pmora...@redhat.comwrote: Hi Paul, this usually happens as a consequence of cluster split-brain. Are you using CMAN (Cluster Manager)? (Technically, when split brain occurs, two (or more) qpid brokers think they are the elder nodes (elder node = the managing node, usually the node that is oldest in the cluster). But there can be just one elder node in a cluster, as the elder node periodically invokes periodicProcessing task cluster-wide that can run just one at a time. When more elder nodes are present, all invokes the task on every cluster member, causing more tasks to be executed - that is prevented by broker shutdown.) Kind regards, Pavel Moravec - Original Message - From: Paul Colby p...@colby.id.au To: users@qpid.apache.org Sent: Thursday, April 12, 2012 5:08:01 AM Subject: 0.14 cluster never survives more than an hour or so. Hi guys, I'm having an issue with my new 0.14 cluster, where the same configuration was fine with 0.12. The cluster starts up, and all brokers are happy. Then, with no client activity at all, after some seemingly random amount time (usually around 30 minutes to an hour) all brokers in the cluster (three, in this case) report the following error: critical Error delivering frames: Cluster timer drop non-existent task ManagementAgent::periodicProcessing (qpid/cluster/ClusterTimer.cpp:128) Then they all shutdown, leaving their respective stores dirty :( Any ideas what might be going wrong here? Thanks, pc http://colby.id.au - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Does the qpid-cluster command support SSL?
Hi Paul, I think the answer is no, as all qpid-* tools are Python-based and SSL support in Python client is not resolved fully - see https://issues.apache.org/jira/browse/QPID-3175 for details. Kind regards, Pavel Moravec - Original Message - From: Paul Colby p...@colby.id.au To: users@qpid.apache.org Sent: Thursday, April 12, 2012 4:20:55 AM Subject: Does the qpid-cluster command support SSL? Hi, Just wondering, is the qpid-cluster command line utility meant to support SSL somehow? ie If my brokers are configured to require encryption, can I still use the qpid-cluster command? Interested in the answer for both 0.14 (I'm using now), and 0.16 (would love to use soon :) It looks to me like the answer is no for both 0.14 and 0.16... but please tell me I'm wrong ;) Thanks! :) pc http://colby.id.au - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: 0.14 cluster never survives more than an hour or so.
Hi Paul, this usually happens as a consequence of cluster split-brain. Are you using CMAN (Cluster Manager)? (Technically, when split brain occurs, two (or more) qpid brokers think they are the elder nodes (elder node = the managing node, usually the node that is oldest in the cluster). But there can be just one elder node in a cluster, as the elder node periodically invokes periodicProcessing task cluster-wide that can run just one at a time. When more elder nodes are present, all invokes the task on every cluster member, causing more tasks to be executed - that is prevented by broker shutdown.) Kind regards, Pavel Moravec - Original Message - From: Paul Colby p...@colby.id.au To: users@qpid.apache.org Sent: Thursday, April 12, 2012 5:08:01 AM Subject: 0.14 cluster never survives more than an hour or so. Hi guys, I'm having an issue with my new 0.14 cluster, where the same configuration was fine with 0.12. The cluster starts up, and all brokers are happy. Then, with no client activity at all, after some seemingly random amount time (usually around 30 minutes to an hour) all brokers in the cluster (three, in this case) report the following error: critical Error delivering frames: Cluster timer drop non-existent task ManagementAgent::periodicProcessing (qpid/cluster/ClusterTimer.cpp:128) Then they all shutdown, leaving their respective stores dirty :( Any ideas what might be going wrong here? Thanks, pc http://colby.id.au - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Selective message acknowledgment in Java client (and C++ broker)?
Hello, having Java client and C++ broker, I want to implement selective consumption of messages from a queue (something like JMS message selectors in Java broker). I.e.: 1) Having CLIENT_ACKNOWLEDGE mode, 2) In a loop through all messages of the queue: a) receive a message b) decide if to consume or not c) if so, acknowledge it, if not so, ignore it 3) Close the session to retrieve the unacknowledged messages back to the queue. However, I see no way of doing so. Method Message.acknowledge(); is not suitable as it acknowledges also all previous messages, following JMS specification. Using method AbstractJMSMessage.acknowledgeThis(), I got the same results. Though the method name would suggest, it shall acknowledge just this message, no else. But checking source code: public void acknowledgeThis() throws JMSException { // the JMS 1.1 spec says in section 3.6 that calls to acknowledge are ignored when client acknowledge // is not specified. In our case, we only set the session field where client acknowledge mode is specified. if (_session != null _session.getAcknowledgeMode() == Session.CLIENT_ACKNOWLEDGE) { if (_session.getAMQConnection().isClosed()) { throw new javax.jms.IllegalStateException(Connection is already closed); } // we set multiple to true here since acknowledgment implies acknowledge of all previous messages // received on the session _session.acknowledgeMessage(_deliveryTag, true); } } See the latest comment - it acknowledges all previous messages on purpose. Having the last command: _session.acknowledgeMessage(_deliveryTag, false); only this message will be acknowledged, no else. Is there some reason for this behavior? If so, how can I achieve a real single message acknowledgment? Thanks in advance for your thoughs. Kind regards, Pavel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Default queues on qpid c++ broker
Hi, what broker do you use? Java or C++? And what version of qpid-stat are you using? As in the most often case (C++ broker, qpid-stat from qpid 0.12 or older - haven't checked newer versions), qpid-stat (as well as any other qpid-* tool) creates few queues to receive data from the broker. In particular: qmfc-v2-hostname.PID.1 qmfc-v2-hb-hostname.PID.1 qmfc-v2-ui-hostname.PID.1 reply-hostname.PID.1 topic-hostname.PID.1 Can you see also the other queues (qmfc-v2-*) ? These are used solely for communication between qpid-stat and the broker and they are automatically deleted when qpid-stat finishes. Let re-run the command and you will see a new bunch of queues (with different PID reffering to PID of the process running qpid-stat). Kind regards, Pavel Moravec - Original Message - From: sandeep sndee...@gmail.com To: users@qpid.apache.org Sent: Tuesday, April 3, 2012 8:47:13 AM Subject: Default queues on qpid c++ broker Does qpid create default queues. I see two queues created when i start the qpid and check the queues using qpid-stat command. These queues are topic-LSS01.6410.1 and reply-LSS01.6410.1. I see some messages also posted on these queues. Can any one please provide some information on these queues. -- View this message in context: http://qpid.2158936.n2.nabble.com/Default-queues-on-qpid-c-broker-tp7431958p7431958.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: python QMF synchronization
Hi Martin, you can set up ReplyTo address where qpid shall send its response. I.e. something like C++ code below (that invokes queue deletion and fetches response in 30seconds limit): Connection connection(url); try { connection.open(); Session session = connection.createSession(); Sender sender = session.createSender(qmf.default.direct/broker); Address responseQueue(#reply-queue; {create:always, node:{x-declare:{auto-delete:true}}}); Receiver receiver = session.createReceiver(responseQueue); Message message; Variant::Map content; Variant::Map OID; Variant::Map arguments; OID[_object_name] = org.apache.qpid.broker:broker:amqp-broker; arguments[type] = queue; arguments[name] = queue_name; content[_object_id] = OID; content[_method_name] = delete; content[_arguments] = arguments; encode(content, message); message.setReplyTo(responseQueue); message.setProperty(x-amqp-0-10.app-id, qmf2); message.setProperty(qmf.opcode, _method_request); sender.send(message, true); Message response; if (receiver.fetch(response,qpid::messaging::Duration(3)) == true) { qpid::types::Variant::Map recv_props = response.getProperties(); if (recv_props[x-amqp-0-10.app-id] == qmf2) if (recv_props[qmf.opcode] == _method_response) std::cout Response: OK std::endl; else if (recv_props[qmf.opcode] == _exception) std::cerr Error: response.getContent() std::endl; else std::cerr Invalid response received! std::endl; else std::cerr Invalid response not of qmf2 type received! std::endl; } else std::cout Timeout: No response received within 30 seconds! std::endl; connection.close(); return 0; } catch(const std::exception error) { std::cout error.what() std::endl; connection.close(); } Kind regards, Pavel - Original Message - From: MartiN Beneš martinben...@gmail.com To: users users@qpid.apache.org Sent: Wednesday, March 21, 2012 12:26:59 PM Subject: python QMF synchronization Hi, I cannot find information about qmf synchronization. Or is every call synchronous? For instance if I purge a queue: queue.purge(0) how can i tell it was done? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: python QMF synchronization
.. and now also for python client (that I overlooked in email subject): conn = Connection(broker) try: conn.open() ssn = conn.session() snd = ssn.sender(qmf.default.direct/broker) reply_to = reply-queue; {create:always, node:{x-declare:{auto-delete:true}}} rcv = ssn.receiver(reply_to) content = { _object_id: {_object_name: org.apache.qpid.broker:broker:amqp-broker}, _method_name: delete, _arguments: {type:queue, name:queue_name} } request = Message(reply_to=reply_to, content=content) request.properties[x-amqp-0-10.app-id] = qmf2 request.properties[qmf.opcode] = _method_request snd.send(request) try: response = rcv.fetch(timeout=30) if response.properties['x-amqp-0-10.app-id'] == 'qmf2': if response.properties['qmf.opcode'] == '_method_response': print Response: print response.content['_arguments'] elif response.properties['qmf.opcode'] == '_exception': raise Exception(Error: %s % response.content['_values']) else: raise Exception(Invalid response received, unexpected opcode: %s % m) else: raise Exception(Invalid response received, not a qmfv2 method: %s % m) except Empty: print No response received! except Exception, e: print e except ReceiverError, e: print e except KeyboardInterrupt: pass conn.close() - Original Message - From: Pavel Moravec pmora...@redhat.com To: users@qpid.apache.org Sent: Wednesday, March 21, 2012 12:33:28 PM Subject: Re: python QMF synchronization Hi Martin, you can set up ReplyTo address where qpid shall send its response. I.e. something like C++ code below (that invokes queue deletion and fetches response in 30seconds limit): Connection connection(url); try { connection.open(); Session session = connection.createSession(); Sender sender = session.createSender(qmf.default.direct/broker); Address responseQueue(#reply-queue; {create:always, node:{x-declare:{auto-delete:true}}}); Receiver receiver = session.createReceiver(responseQueue); Message message; Variant::Map content; Variant::Map OID; Variant::Map arguments; OID[_object_name] = org.apache.qpid.broker:broker:amqp-broker; arguments[type] = queue; arguments[name] = queue_name; content[_object_id] = OID; content[_method_name] = delete; content[_arguments] = arguments; encode(content, message); message.setReplyTo(responseQueue); message.setProperty(x-amqp-0-10.app-id, qmf2); message.setProperty(qmf.opcode, _method_request); sender.send(message, true); Message response; if (receiver.fetch(response,qpid::messaging::Duration(3)) == true) { qpid::types::Variant::Map recv_props = response.getProperties(); if (recv_props[x-amqp-0-10.app-id] == qmf2) if (recv_props[qmf.opcode] == _method_response) std::cout Response: OK std::endl; else if (recv_props[qmf.opcode] == _exception) std::cerr Error: response.getContent() std::endl; else std::cerr Invalid response received! std::endl; else std::cerr Invalid response not of qmf2 type received! std::endl; } else std::cout Timeout: No response received within 30 seconds! std::endl; connection.close(); return 0; } catch(const std::exception error) { std::cout error.what() std::endl; connection.close(); } Kind regards, Pavel - Original Message - From: MartiN Beneš martinben...@gmail.com To: users users@qpid.apache.org Sent: Wednesday, March 21, 2012 12:26:59 PM Subject: python QMF synchronization Hi, I cannot find information about qmf synchronization. Or is every call synchronous? For instance if I purge a queue: queue.purge(0) how can i tell it was done? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: How to receive specific response
Hi Todd, if you wish to non-destructively read messages in a queue, you can use browse mode, i.e. address string would be something like: queue-name; {mode:browse} If your question is extended by .. and how to consume the message I am interested in (and no other), then I don't know if it is possible with C++ broker (Java broker has message selectors that I am not familiar with). Kind regards, Pavel - Original Message - From: Todd Herman t...@apx-labs.com To: users@qpid.apache.org Sent: Monday, March 12, 2012 3:20:34 PM Subject: FW: How to receive specific response I send this question out earlier but didn't get any response so I figured I would send it again. The main thing I am trying to see is how to read a message from a queue without it being automatically deleted. I have no control over how the queue was created. I need to be able to check all the messages on the queue because only certain ones will be for me. I can't have the messages not for me being deleted because I read them. Any thoughts? From: Todd Herman [mailto:t...@apx-labs.com] Sent: Thursday, March 08, 2012 11:48 AM To: users@qpid.apache.org Subject: How to receive specific response I am using the Qpid C++ client. Well, technically I am writing this in C# so I am using the .net bindings for the c++ client. I need to retrieve a specific message from a queue. Basically, multiple clients will watch a certain queue for responses but they each only want there (based on the correlation id) message. It looks like you can do this with the Java client because CreateConsumer allows you to provide a message selector, which I assume allows you to filter the messages to just the one you want. So, how do I do the equivalent when using the c++ (and .net binding) client? I could grab each message and check the correlation id but won't doing that remove it from the queue? I need to make sure I don't remove the wrong one from the queue. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: How to know all the available topics
Hi Davide, does the command: qpid-config exchanges amq.topic -b fulfill your needs? To have it implemented in a client, it is possible to invoke a QMF command to list all bindings. Something like the C++ program below (not sure how to write the query to filter bindings from just some exchange). Kind regards, Pavel // begin of list_binginds.cpp #include qpid/messaging/Connection.h #include qpid/messaging/Session.h #include qpid/messaging/Sender.h #include qpid/messaging/Receiver.h #include qpid/messaging/Message.h #include qpid/messaging/Address.h #include iostream using namespace std; using namespace qpid::messaging; using namespace qpid::types; int main(int argc, char** argv) { Connection c(argc1?argv[1]:localhost:5672); c.open(); Session session = c.createSession(); Receiver r = session.createReceiver(#qlister; {create:always, delete:always}); Sender s = session.createSender(qmf.default.direct/broker); Message m; m.setReplyTo(Address(r.getName())); m.setProperty(x-amqp-0-10.app-id, qmf2); m.setProperty(qmf.opcode, _query_request); Variant::Map request; request[_what] = OBJECT; Variant::Map schemaId; schemaId[_class_name] = binding; request[_schema_id] = schemaId; encode(request, m); s.send(m); m = r.fetch(3 * Duration::SECOND); session.acknowledge(m); Variant::List response; decode(m, response); for(Variant::List::iterator iter = response.begin(); iter != response.end(); iter++) { Variant::Map map = iter-asMap(); cout map endl; Variant::Map values = map[_values].asMap(); cout values[name] endl endl endl; } r.close(); s.close(); session.close(); c.close(); return 0; } // end of list_binginds.cpp - Original Message - From: Davide Anastasia davide.anasta...@qualitycapital.com To: users@qpid.apache.org Sent: Wednesday, March 7, 2012 12:11:50 PM Subject: How to know all the available topics Is there a way to discover all the available topics to a certain exchange (let's say, amq.topic). Thanks, Davide Anastasia Analyst, Research Development Quality Capital Management Ltd. QCM House * Horizon Business Village No. 1 Brooklands Road Weybridge * Surrey KT13 0TJ United Kingdom Tel: +44 (0) 1932 334 400 Fax: +44 (0) 1932 334 415 Email: davide.anasta...@qualitycapital.com mailto:davide.anasta...@qualitycapital.com www.qualitycapital.com http://www.qualitycapital.com/ This email and any attachments are confidential and intended solely for the use of the individual(s) to whom it is addressed. Any views or opinions presented are solely those of the author and do not necessarily represent those of Quality Capital Management Ltd. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, printing, forwarding or copying of this email is strictly prohibited. Please contact the sender if you have received this email in error. You should also be aware that emails are susceptible to interference and you should not assume that the contents of this email originated from the sender above or that they have been accurately reproduced in their original form. Quality Capital Management Ltd is authorised and regulated by the Financial Services Authority in the UK and is a member of the National Futures Association in the US. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Creating Comon single Queue
Hi Sinduja, does replication event queue fulfills your requirements? See https://cwiki.apache.org/qpid/queue-state-replication.html for details. Kind regards, Pavel - Original Message - From: sinduja ramaraj sinduja.rama...@gmail.com To: users@qpid.apache.org Sent: Thursday, March 1, 2012 11:39:05 AM Subject: Fwd: Creating Comon single Queue ie., I have a broker1 listening at lets say 5672 and another broker2 listening at 5673. I've client of broker1 listening at 5672 who is creating message queue lets say abc_queue. I've another client application of broker2 listening at 5673 who will use the same message queue of broker1 created by client1. I need to push data from client2 into this message queue abc_queue created by client1. ir., the message queue abc_queue should be a common single queue for all the brokers.And all the client applications will push data into this message queue. How to achieve this scenario? Someone help. -- Forwarded message -- From: sinduja.rama...@gmail.com Date: Thu, Mar 1, 2012 at 3:18 PM Subject: Creating Comon single Queue To: users@qpid.apache.org Hi, I have a scenario where I will be creating n no. of brokers listening at different ports. How to create a common single queue for all the brokers? Regards, Sinduja.R - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: maxprefecth / capacity concerns
Hi Gordon, the default prefetch is 1 is my conclusion from Section 2.6: By default, a receiver requests the next message from the server in response to each fetch call, resulting in messages being sent to the receiver one at a time. Anyway, that is rather an explanation of prefetch, whereas default values are later on provided (500 for Java client). I agree that prefetch and capacity is something else (though having something common in consumer side). What I am missing in Java client (and what in fact triggered writing of my original email) is ability to automatically apply/follow producer flow control in Java client. Using the plain programming skeleton from Spout.java (or spout.cpp), we are just sending messages with no session synchronization (like session.sync() ). In C++, when I setup capacity to e.g. 50, then: - assume the broker applies producer flow control by reaching a queue its 80% capacity - so it stops ACKing messages from the producer but further messages are still put to the queue - so then C++ send method is blocked by C++ client library after sending 50 further messages While in Java client without ability to setup capacity, Java client can send to the queue any further number of messages, simply ignoring it hasn't received ACKs. Only when the client would try to close the session or connection, it is blocked (in AUTO ACK mode) until ACKs are received. Applying prefetch has no impact. So the only way (I found so far) is adding session.sync() after sending a bunch of messages. What is not ideal not only from performance point of view. I know there are more important issues with Java client, but is the request of adding capacity to Java client reasonable? Kind regards, Pavel - Original Message - From: Gordon Sim g...@redhat.com To: users@qpid.apache.org Sent: Friday, February 10, 2012 8:47:33 PM Subject: Re: maxprefecth / capacity concerns On 02/03/2012 11:51 AM, Pavel Moravec wrote: Hello qpid users, I am somehow confused about difference between capacity and maxprefetch. Both define maximum number of messages that can be sent without receiving a response to the first msg, but maxprefetch is applicable to consumer part of the communication only. My questions: 1) maxprefetch is applicable rather to Java client only while capacity rather to C++, python and .NET clients. Correct. 2) How to setup capacity in Java client (to limit producers to qpid)? I tried using address string small-queue;{link:{capacity:'5'}} but without an effect. I spotted non-JMS method setProducerCapacity from org.apache.qpid.client.messaging.address package (within class Link) but I am unable to use it via Connection, Session or MessageProducer objects. I don't believe you can at present. The closest it comes is that you can specify a limit to the command 'buffer' which is similiar but not quite the same thing. 3) Why Programming in Apache Qpid talks about prefetch having impact to performance, while I can't find a method to setting it in C++ client? The performance tips state: 'Consider prefetching messages for receivers (see Section 2.6, “Receiver Capacity (Prefetch)”)' where section 2.6 is http://qpid.apache.org/books/trunk/Programming-In-Apache-Qpid/html/prefetch.html which - in my view - gives a reasonable description of how prefetch relates to receiver capacity. I'd be interested to hear if that is or is not clear. Receiver capacity and prefetch are essentially the same thing. The receivers capacity determines how many messages it is prepared to buffer pending fetch() calls. 4) The same doc says prefetch is by default 1, Where is that? The default receiver capacity for python , c++ and .NET is actually 0 (i.e. no prefetch is enabled). while https://cwiki.apache.org/qpid/use-priority-queues.html states it has default value 5000 - what is correct then? 5) Are there some reasons to have those two features separated? And with different default values (capacity has 50)? I don't think there are two features here, merely a difference in how the JMS client works and how the messaging API implementations work. Clearly we want to move towards more commonality but JMS is still quite a different beast. Hope this helps... - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: maxprefecth / capacity concerns
know how you get on Frase. On 03/02/12 11:51, Pavel Moravec wrote: Hello qpid users, I am somehow confused about difference between capacity and maxprefetch. Both define maximum number of messages that can be sent without receiving a response to the first msg, but maxprefetch is applicable to consumer part of the communication only. My questions: 1) maxprefetch is applicable rather to Java client only while capacity rather to C++, python and .NET clients. 2) How to setup capacity in Java client (to limit producers to qpid)? I tried using address string small-queue;{link:{capacity:'5'}} but without an effect. I spotted non-JMS method setProducerCapacity from org.apache.qpid.client.messaging.address package (within class Link) but I am unable to use it via Connection, Session or MessageProducer objects. 3) Why Programming in Apache Qpid talks about prefetch having impact to performance, while I can't find a method to setting it in C++ client? 4) The same doc says prefetch is by default 1, while https://cwiki.apache.org/qpid/use-priority-queues.html states it has default value 5000 - what is correct then? 5) Are there some reasons to have those two features separated? And with different default values (capacity has 50)? Thanks in advance for your feedback. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
maxprefecth / capacity concerns
Hello qpid users, I am somehow confused about difference between capacity and maxprefetch. Both define maximum number of messages that can be sent without receiving a response to the first msg, but maxprefetch is applicable to consumer part of the communication only. My questions: 1) maxprefetch is applicable rather to Java client only while capacity rather to C++, python and .NET clients. 2) How to setup capacity in Java client (to limit producers to qpid)? I tried using address string small-queue;{link:{capacity:'5'}} but without an effect. I spotted non-JMS method setProducerCapacity from org.apache.qpid.client.messaging.address package (within class Link) but I am unable to use it via Connection, Session or MessageProducer objects. 3) Why Programming in Apache Qpid talks about prefetch having impact to performance, while I can't find a method to setting it in C++ client? 4) The same doc says prefetch is by default 1, while https://cwiki.apache.org/qpid/use-priority-queues.html states it has default value 5000 - what is correct then? 5) Are there some reasons to have those two features separated? And with different default values (capacity has 50)? Thanks in advance for your feedback. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: QUERY ON LOG FILES
Hi Sinduja, you mean something like: qpid-config queues qpid-stat -q ? Kind regards, Pavel - Original Message - From: Senthil Kumar, Sinduja (GE Healthcare) sinduja.senthilku...@ge.com To: users@qpid.apache.org Cc: pmora...@redhat.com Sent: Friday, February 3, 2012 6:35:50 AM Subject: RE: QUERY ON LOG FILES Hi, Can anyone tell me how to get the information about the list of queues available with a particular broker? Regards, Sinduja.R -Original Message- From: Pavel Moravec [mailto:pmora...@redhat.com] Sent: Thursday, February 02, 2012 6:09 PM To: users@qpid.apache.org Subject: Re: QUERY ON LOG FILES Hi Sinduja, see my response inline. Kind regards, Pavel - Original Message - From: Senthil Kumar, Sinduja (GE Healthcare) sinduja.senthilku...@ge.com To: users@qpid.apache.org Sent: Thursday, February 2, 2012 10:56:53 AM Subject: QUERY ON LOG FILES Hi, Can someone explain about the contents of the following three files created by the QPID broker. 1. .mbrokerdata The file relevant to data directory contains three strings: uuid bootSequence nextRemoteBank where uuid is unique ID (as returned by uuid_generate) for the broker storage module that is using this data directory, bootSequence is number of (re)starts of the broker and nextRemoteBank seems to be 10 everytime (don't know its purpose, somehow relevant with storage module) 2. Lock en empty file locked by the currently running qpid broker to prevent another qpid in using this data-dir in parallel 3. systemId. UUID (as returned by uuid_generate) of the broker. Regards, Sinduja.R - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: QUERY ON LOG FILES
Hi, I recommend reading http://qpid.apache.org/books/0.10/Programming-In-Apache-Qpid/html-single/Programming-In-Apache-Qpid.html#section-addresses. Basically, a consumer subscribes itself to a queue so you should use queue name in the address string (optionally followed by semicolon and queue attributes like queue_name; { create:always }). While a producer produces messages (with some routing key) via an exchange. So for producers you should specify their address in form exchange_name/routing_key, again optionally extended by semicolon and node properties. To make the basic flow complete, don't forget to create appropriate bindings to let exchanges route messages into relevant queues. Kind regards, Pavel Moravec - Original Message - From: Senthil Kumar, Sinduja (GE Healthcare) sinduja.senthilku...@ge.com To: users@qpid.apache.org Cc: pmora...@redhat.com Sent: Friday, February 3, 2012 1:01:26 PM Subject: RE: QUERY ON LOG FILES Hi, Thanks, I found out that. But actually Im to trying to implement the following scenario :Receiving Messages from Multiple Sources Server.cpp My address1 : Receiver receiver1 = session.createReceiver(service_queue/usa.news;{create:always}); My address2: Receiver receiver2 = session.createReceiver(service_queue/europe.news;{create:always}); Client1.cpp Sender sender = session.createSender(service_queue/europe.*); Client2.cpp Sender sender = session.createSender(service_queue/usa.*); Is this the way of setting two different addresses? Please help. Kindly tell me how to set two different addresses in this particular scenario. Regards, Sinduja.R -Original Message- From: Pavel Moravec [mailto:pmora...@redhat.com] Sent: Friday, February 03, 2012 5:27 PM To: users@qpid.apache.org Subject: Re: QUERY ON LOG FILES Hi Sinduja, you mean something like: qpid-config queues qpid-stat -q ? Kind regards, Pavel - Original Message - From: Senthil Kumar, Sinduja (GE Healthcare) sinduja.senthilku...@ge.com To: users@qpid.apache.org Cc: pmora...@redhat.com Sent: Friday, February 3, 2012 6:35:50 AM Subject: RE: QUERY ON LOG FILES Hi, Can anyone tell me how to get the information about the list of queues available with a particular broker? Regards, Sinduja.R -Original Message- From: Pavel Moravec [mailto:pmora...@redhat.com] Sent: Thursday, February 02, 2012 6:09 PM To: users@qpid.apache.org Subject: Re: QUERY ON LOG FILES Hi Sinduja, see my response inline. Kind regards, Pavel - Original Message - From: Senthil Kumar, Sinduja (GE Healthcare) sinduja.senthilku...@ge.com To: users@qpid.apache.org Sent: Thursday, February 2, 2012 10:56:53 AM Subject: QUERY ON LOG FILES Hi, Can someone explain about the contents of the following three files created by the QPID broker. 1. .mbrokerdata The file relevant to data directory contains three strings: uuid bootSequence nextRemoteBank where uuid is unique ID (as returned by uuid_generate) for the broker storage module that is using this data directory, bootSequence is number of (re)starts of the broker and nextRemoteBank seems to be 10 everytime (don't know its purpose, somehow relevant with storage module) 2. Lock en empty file locked by the currently running qpid broker to prevent another qpid in using this data-dir in parallel 3. systemId. UUID (as returned by uuid_generate) of the broker. Regards, Sinduja.R - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: QUERY ON LOG FILES
Hi Sinduja, see my response inline. Kind regards, Pavel - Original Message - From: Senthil Kumar, Sinduja (GE Healthcare) sinduja.senthilku...@ge.com To: users@qpid.apache.org Sent: Thursday, February 2, 2012 10:56:53 AM Subject: QUERY ON LOG FILES Hi, Can someone explain about the contents of the following three files created by the QPID broker. 1. .mbrokerdata The file relevant to data directory contains three strings: uuid bootSequence nextRemoteBank where uuid is unique ID (as returned by uuid_generate) for the broker storage module that is using this data directory, bootSequence is number of (re)starts of the broker and nextRemoteBank seems to be 10 everytime (don't know its purpose, somehow relevant with storage module) 2. Lock en empty file locked by the currently running qpid broker to prevent another qpid in using this data-dir in parallel 3. systemId. UUID (as returned by uuid_generate) of the broker. Regards, Sinduja.R - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
QMF creating huge durable queue causes cluster desync
Hi all, it seems to me that sending QMF methods to create a queue over-passes cluster error checking mechanism (if a sole node detects an error, it preventive leaves the cluster). Having a simple C++ client to create a huge durable queue: Connection connection(amqp:tcp:127.0.0.1:5672); connection.open(); Session session = connection.createSession(); Sender sender = session.createSender(qmf.default.direct/broker); Address responseQueue(#reply-queue; {create:always, node:{x-declare:{auto-delete:true}}}); Receiver receiver = session.createReceiver(responseQueue); Message message; Variant::Map content; Variant::Map OID; Variant::Map arguments; Variant::Map properties; properties[durable] = true; properties[qpid.file_count] = 3000; arguments[type] = queue; arguments[name] = HugeDurableQueue; arguments[properties] = properties; OID[_object_name] = org.apache.qpid.broker:broker:amqp-broker; content[_object_id] = OID; content[_method_name] = create; content[_arguments] = arguments; encode(content, message); message.setReplyTo(responseQueue); message.setProperty(x-amqp-0-10.app-id, qmf2); message.setProperty(qmf.opcode, _method_request); message.setContentType(amqp/map); sender.send(message, true); and running the client to a clustered pair of brokers creates a durable queue with 3000 files in its journal. However, if one of the brokers (let say node1) has insufficient free disk space, qpid on node1 shall shut down due to the cluster error checking mechanism. At least, that I see when I create the queue via qpid-config or using proper address string and spout/drain. But running the above QMF method leaves the cluster in state where node1 has not have the queue created while node2 has it created - a substantial cluster de-sync. Did I do something wrong or is it a bug? Thanks in advance for your answers. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
QMF raising event when defaut binding is created?
Hi all, I realized that currently QMF does not raise an event when a default binding is created (by default binding I mean a binding of type {exchange:'', queue:queuename, key:queuename} that is created automatically when a queue is created). To check that behavior, let subscribe to qmf.default.topic/agent.ind.event.org_apache_qpid_broker.bind.# and create a queue - no message will be send to the subscription. Any other binding crated is notified by sending a message to the subscription. My question is: is that a inconsistency that shall be fixed or a desired behavior? The answer might not be so plain as the '' exchange and the default bindings are rather hacks to allow sending messages to queues directly. So while technically the defualt bindings are a real bindings, logically they are not. Does somebody (plan to) benefit either by raising QMF event when a default binding is created or not raising so? Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Weird behavior of flow_to_disk policy
Hi Gordon, - Original Message - From: Gordon Sim g...@redhat.com To: users@qpid.apache.org Sent: Thursday, December 15, 2011 3:33:22 PM Subject: Re: Weird behavior of flow_to_disk policy On 12/13/2011 03:33 PM, Pavel Moravec wrote: Hi all, I noticed a weird behavior of flow-to-disk policy. When the relevant queue is utilized over its max-queue-count and when I am sending and receiving messages there, my sender gets Enqueue capacity threshold. One particular scenario: qpid-config add queue ToFillQueue --durable --max-queue-count=1000 --limit-policy flow-to-disk --file-count 2 --file-size 2 ./spout -c 1000 ToFillQueue ./spout -c 1000 ToFillQueue (due to flow control, we have to break the program) ./drain -c 1000 ToFillQueue At this point the journal still has 1000 messages in it. The queue also has 1000 messages, so any further messages will be written to journal as part of the flow-to-disk behaviour... I was under the impression that flow_to_disk is applied when max-queue-count messages are in memory, not in the whole queue.. Then the behavior makes sense. So, this means that once FTD is applied, messages (msg bodies) will be stored to disk and not to memory until the queue depth drops below max-queue-count. And there will be messages on disk while memory not fully utilized(*) until there will be no message in the journal. (*)memory fully utilized = memory keeps max-queue-count messages. That is unexpected but in my opinion acceptable behavior - the second option in having max-queue-count in memory every time would gain some performance but more complex to implement. ./spout -c 1000 ToFillQueue 2011-12-13 16:15:50 warning Broker closed connection: 501, Enqueue capacity threshold exceeded on queue ToFillQueue. (JournalImpl.cpp:587) 2011-12-13 16:15:50 error session-busy: Session detached by peer session-busy: Session detached by peer I think this is simply a journal capacity problem. It cannot accommodate 2000 messages. Journal size is 2*64k*2=256k, which assuming the smallest enqueue record of 128 bytes would allow for only 2000 in theory and in practice you get less (I believe due to other control records plus file boundaries etc). Yes, I chosen the journal size just for this example. The journal itself stores 1536 of the smallest messages. qpid-tool shows: arguments {u'qpid.file_size': 2, u'qpid.file_count': 2, u'qpid.max_count': 1000, u'qpid.policy_type': u'flow_to_disk'} altExchangeabsent msgTotalEnqueues 2536 msgTotalDequeues 1000 msgTxnEnqueues 0 msgTxnDequeues 0 msgPersistEnqueues 0 msgPersistDequeues 0 msgDepth 1536 Interestingly, when I pump messages to the queue (without any draining) to fill the journals, the exception is raised when exactly the same number of msgTotalEnqueues appears. Playing with more scenarios, sometimes the msgTotalEnqueues is higher, sometimes even lower than the total capacity of the queue(+its journals). Almost the same result I achieve when sending durable messages (with the main difference that all messages go to journal direclt, not storing any in memory). Anyway, it seems for me that consuming a message from flow-to-disk queue (when the policy is already applied) is wrongly implemented. I noticed this on qpid 0.10 and 0.12 while 0.8 seems ok. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Trying to get persistence plugin for qpid 0.10 on REHL
Hi Daryoush, RPM for persistence is named qpid-cpp-server-store, it should be of the same version as qpid-cpp-server package is. How have you installed qpid? As in both usual ways (compiling from source and installing via yum), store RPM shall be already present. Kind regards, Pavel - Original Message - From: Daryoush Mehrtash dmehrt...@gmail.com To: users@qpid.apache.org Sent: Tuesday, December 13, 2011 7:24:22 PM Subject: Trying to get persistence plugin for qpid 0.10 on REHL I am running qpidd (qpidc) version 0.10 on Red Hat Enterprise Linux Server release 6.1 (Santiago) I need to add persistence plugin to get durability.Is there an RPM that I can install?If not, is there instruction page on what (is there a specific version of the store that you would need for 0.10?) to build from source? Daryoush - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Weird behavior of flow_to_disk policy
Hi all, I noticed a weird behavior of flow-to-disk policy. When the relevant queue is utilized over its max-queue-count and when I am sending and receiving messages there, my sender gets Enqueue capacity threshold. One particular scenario: qpid-config add queue ToFillQueue --durable --max-queue-count=1000 --limit-policy flow-to-disk --file-count 2 --file-size 2 ./spout -c 1000 ToFillQueue ./spout -c 1000 ToFillQueue (due to flow control, we have to break the program) ./drain -c 1000 ToFillQueue ./spout -c 1000 ToFillQueue 2011-12-13 16:15:50 warning Broker closed connection: 501, Enqueue capacity threshold exceeded on queue ToFillQueue. (JournalImpl.cpp:587) 2011-12-13 16:15:50 error session-busy: Session detached by peer session-busy: Session detached by peer qpid-tool shows: arguments {u'qpid.file_size': 2, u'qpid.file_count': 2, u'qpid.max_count': 1000, u'qpid.policy_type': u'flow_to_disk'} altExchangeabsent msgTotalEnqueues 2536 msgTotalDequeues 1000 msgTxnEnqueues 0 msgTxnDequeues 0 msgPersistEnqueues 0 msgPersistDequeues 0 msgDepth 1536 Interestingly, when I pump messages to the queue (without any draining) to fill the journals, the exception is raised when exactly the same number of msgTotalEnqueues appears. Playing with more scenarios, sometimes the msgTotalEnqueues is higher, sometimes even lower than the total capacity of the queue(+its journals). Almost the same result I achieve when sending durable messages (with the main difference that all messages go to journal direclt, not storing any in memory). Anyway, it seems for me that consuming a message from flow-to-disk queue (when the policy is already applied) is wrongly implemented. I noticed this on qpid 0.10 and 0.12 while 0.8 seems ok. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: A JMS Connection failing with asynchronous exception doesn't seem to close cleanly.
Hi Frase, the bug is already tracked under https://issues.apache.org/jira/browse/QPID-3575. In fact, there are two somehow joined issues: 1) the session and connection leak problem itself, where a session becomes into detached state but cant be used further any way. 2) any exception raised on a session leads to the session and connection closure, though many exceptions do not need to end up in connection closure. That is raised as https://issues.apache.org/jira/browse/QPID-3289 / https://issues.apache.org/jira/browse/QPID-3234. Currently I am not aware of a workaround. Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Sunday, December 11, 2011 1:05:01 PM Subject: A JMS Connection failing with asynchronous exception doesn't seem to close cleanly. I've just noticed some odd behaviour when a connection got closed on exception with JMS (Qpid 0.12 using C++ broker). In my test I've got an ExceptionListener registered and I deliberately send a big payload in order to get resource-limit-exceeded. The client log says: IoReceiver - localhost/127.0.0.1:5672 2011-12-11 11:39:29,557 INFO [apache.qpid.client.AMQConnection] Closing AMQConnection due to :org.apache.qpid.AMQException: ch=1 id=4 ExecutionException(errorCode=RESOURCE_LIMIT_EXCEEDED, commandId=11, classCode=0, commandCode=0, fieldIndex=0, description=resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on TempQueuee8e2ed69-63ce-41bd-9459-28cb17d1d629, policy: size: max=104857600, current=0; count: unlimited; type=reject (qpid/broker/QueuePolicy.cpp:87), errorInfo={}) [error code 506: resource error] JMSException 506 caught: BigPayloadAgentTest failed IoReceiver - localhost/127.0.0.1:5672 2011-12-11 11:39:29,557 INFO [apache.qpid.client.BasicMessageConsumer] Closing consumer:1[1625215216] IoReceiver - localhost/127.0.0.1:5672 2011-12-11 11:39:29,557 INFO [apache.qpid.client.BasicMessageConsumer] Closing consumer:2[1147629119] Dispatcher-Channel-0 2011-12-11 11:39:29,557 INFO [qpid.client.AMQSession.Dispatcher] Dispatcher-Channel-0 thread terminating for channel 0:org.apache.qpid.client.AMQSession_0_10@a383118 So *in theory* the connection should be closed, but in practice it doesn't seem to be. Checking with qpid-printevents shows that there is no clientDisconnect Event and checking with qpid-config shows that exclusive queues bound to the connection remain. There seems to be no clean way to properly close a connection in this state (I've tried explicitly closing it and even closing the underlying Session and MessageProducers/MessageConsumers using the connection). The clientDisconnect Event only occurs when I actually terminate the client. This behaviour seems wrong to me and results in a potential resource leak, unless I'm missing something it looks like something isn't getting correctly tidied up when this exception occurs which results in the underlying socket connection remaining. Does anyone know of a workaround? I actually wanted to handle this in my ExceptionListener and re-establish connections. Cheers, Frase - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Qpid clustering with cman
Hi, this is a known issue, see https://issues.apache.org/jira/browse/QPID-3483. Some background: corosync has not authenticated qpidd process to access its CPG methods. Without CMAN in use, the access is allowed by /etc/corosync/uidgid.d/qpidd file. However, CMAN configuration (cluster.conf) simply ignores the file at all, not providing an option to set it up within the conf file. Workaround: run qpidd under root (as root user has access to corosync CPG). Kind regards, Pavel Moravec - Original Message - From: David Zhou dayda...@gmail.com To: users@qpid.apache.org Sent: Thursday, December 8, 2011 8:33:44 AM Subject: Qpid clustering with cman (nabble link: http://apache-qpid-users.2158936.n2.nabble.com/Qpid-clustering-with-cman-td7073352.html ) Hi all, I set up a 2-nodes Qpid cluster (0.12) which is synchronized by corosync (1.4.2). The cluster works fine except running with cman to avoid split-brain. It's said that settings of corosync.conf would be replaced by cluster.conf, but from the following trial and error it seems that corosync doesn't fetch settings from cluster.conf. I wonder there are problems with my configurations. Could you please help me out? Thank you!! First here apply the configuration sample from Qpid wikihttps://cwiki.apache.org/qpid/configuring-qpidd-with-cluster-manager.html : --cluster.conf-- ?xml version=1.0 ? cluster name=David config_version=1 cman two_node=1 expected_votes=1 /cman clusternodes clusternode name=c1q1 nodeid=1 /clusternode clusternode name=c1q2 nodeid=2 /clusternode /clusternodes rm failoverdomains failoverdomain name=only_broker1 nofailback=0 ordered=0 restricted=1 failoverdomainnode name=c1q1 priority=1/ /failoverdomain failoverdomain name=only_broker2 nofailback=0 ordered=0 restricted=1 failoverdomainnode name=c1q2 priority=1/ /failoverdomain /failoverdomains resources script name=qpidd file=/etc/init.d/qpidd / /resources service name=qpidd_broker1 domain=only_broker1 script ref=qpidd / /service service name=qpidd_broker2 domain=only_broker2 script ref=qpidd / /service /rm /cluster --- = Starting cman OK, but starting qpidd get the following error: --log- Dec 7 19:02:45 c1q2 qpidd[6893]: notice Journal TplStore: Created Dec 7 19:02:45 c1q2 qpidd[6893]: notice Store module initialized; store-dir=/var/spool/qpid Dec 7 19:02:45 c1q2 qpidd[6893]: notice Initializing CPG Dec 7 19:02:45 c1q2 corosync[6572]: [IPC ] Invalid IPC credentials. Dec 7 19:02:45 c1q2 qpidd[6893]: critical Unexpected error: Failed to initialize CPG.: security (100) --- I think it's because the bindnetaddr is not set in cluster.conf (Without cman, corosync + qpidd works fine), so I add the syntax found at herehttps://bugzilla.redhat.com/show_bug.cgi?id=613880 --cluster.conf-- ?xml version=1.0 ? cluster name=David config_version=1 cman two_node=1 expected_votes=1 /cman totem interface ringnumber=0 bindnetaddr=192.168.0.0 mcastaddr=226.94.1.1 mcastport=5405 / /totem clusternodes clusternode name=c1q1 nodeid=1 /clusternode clusternode name=c1q2 nodeid=2 /clusternode /clusternodes rm failoverdomains failoverdomain name=only_broker1 nofailback=0 ordered=0 restricted=1 failoverdomainnode name=c1q1 priority=1/ /failoverdomain failoverdomain name=only_broker2 nofailback=0 ordered=0 restricted=1 failoverdomainnode name=c1q2 priority=1/ /failoverdomain /failoverdomains resources script name=qpidd file=/etc/init.d/qpidd / /resources service name=qpidd_broker1 domain=only_broker1 script ref=qpidd / /service service name=qpidd_broker2 domain=only_broker2 script ref=qpidd / /service /rm /cluster --- The result is not OK. corosync finds no multicast address specified. --log-- Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Corosync Cluster Engine ('1.4.2'): started and ready to provide service. Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Corosync built-in features: nss Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Successfully read config from /etc/cluster/cluster.conf Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Successfully parsed cman config Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Successfully configured openais services to load Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] parse error in config
Re: how to delete queue from jms client
Hi Gastón, you can use QMF method delete with parameters type (queue in our case) and name (name of the queue). Here is the code snippet from C++ program I use: Connection connection(url/*, connectionOptions*/); try { connection.open(); Session session = connection.createSession(); Sender sender = session.createSender(qmf.default.direct/broker); Address responseQueue(#reply-queue; {create:always, node:{x-declare:{auto-delete:true}}}); Receiver receiver = session.createReceiver(responseQueue); Message message; Variant::Map content; Variant::Map OID; Variant::Map arguments; OID[_object_name] = org.apache.qpid.broker:broker:amqp-broker; arguments[type] = queue; arguments[name] = queue_name; // a string of the queue name content[_object_id] = OID; content[_method_name] = delete; content[_arguments] = arguments; encode(content, message); message.setReplyTo(responseQueue); message.setProperty(x-amqp-0-10.app-id, qmf2); message.setProperty(qmf.opcode, _method_request); sender.send(message, true); /* check response if the queue was properly deleted */ Message response; if (receiver.fetch(response,qpid::messaging::Duration(3)) == true) { qpid::types::Variant::Map recv_props = response.getProperties(); if (recv_props[x-amqp-0-10.app-id] == qmf2) if (recv_props[qmf.opcode] == _method_response) std::cout Response: OK std::endl; else if (recv_props[qmf.opcode] == _exception) std::cerr Error: response.getContent() std::endl; else std::cerr Invalid response received! std::endl; else std::cerr Invalid response not of qmf2 type received! std::endl; } else std::cout Timeout: No response received within 30 seconds! std::endl; connection.close(); return 0; } catch(const std::exception error) { std::cout error.what() std::endl; connection.close(); } Kind regards, Pavel - Original Message - From: Gaston Quezada quezada.gas...@gmail.com To: users@qpid.apache.org Sent: Monday, December 5, 2011 9:14:10 PM Subject: how to delete queue from jms client Hi, Im using qpid-client 0.12, and need delete queue from JMS client. the queue is not auto delete regards -- Gastón Quezada - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: How to clear the Journal so I can restart C++ Broker?
Hi, There is a tool for (even offline) journal resizing, see e.g. http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/2/html/Messaging_User_Guide/sect-Messaging_User_Guide-Determining_Journal_Size-Resizing_the_Journal.html (I hope also some qpid documentation covers that): /usr/libexec/qpid/resize --help The DIR argument shall be directory store-directory/rhn/jnrl/queue_name, check settings of --data-dir and/or --store-dir options (or, when running as a service, check directory /var/lib/qpidd). The simplest way (e.g. if you don't know parameters of the current journal) is to increase the number of journal files (-N parameter of resize command). Hope that helps. Kind regards, Pavel - Original Message - From: dmounessa dmoune...@yahoo.com To: users@qpid.apache.org Sent: Thursday, December 1, 2011 10:29:03 PM Subject: How to clear the Journal so I can restart C++ Broker? I get the following error when starting the qpidd Daemon startup failed: Queue N200: recoverQueues() failed: jexception 0x0205 jcntl::recover() threw JERR_JCNTL_RECOVERJFULL: Journal data files full, cannot write. (MessageStoreImpl.cpp:820) How do I clear the journal so I can start it up? How do I delete the queues with /usr/bin/qpid-config -- since it can not connect to it? is there a way to expand the journal file? -- View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/How-to-clear-the-Journal-so-I-can-restart-C-Broker-tp7052224p7052224.html Sent from the Apache Qpid users mailing list archive at Nabble.com. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: How do I get Qpid's log messages?
Hi Daryoush, you need to use option log-to-file: log-to-file=/tmp/qpidd.log See qpidd --help | grep log or man qpidd for more logging options. Esp. for log-enable that can define (different) verbosity of logs for different areas. Please note that by default logs are sent to syslog, so they are written to /var/log/messages by default. Kind regards, Pavel - Original Message - From: Daryoush Mehrtash dmehrt...@gmail.com To: users@qpid.apache.org Sent: Friday, October 7, 2011 2:14:34 AM Subject: How do I get Qpid's log messages? I have installed qpid package in REHL using yum: sudo yum install qpid-cpp-server.x86_64 qmf.x86_64 python-qpid.noarch qpid-qmf.x86_64 qpid-cpp-server-ssl.x86_64 qpid-cpp-client-ssl.x86_64 python-qpid-qmf.x86_64 qpid-tools.noarch I then add a line to /etc/qpidd.conf to get log out put ... # (Note: no spaces on either side of '='). Using default settings: # qpidd --help or man qpidd for more details. cluster-mechanism=ANONYMOUS auth=yes log-output=/tmp/qpidd.log When I restart qpidd using the service command (sudo service qpidd restart) I don't get any log messages. What do I need to do to see qpid's debug messages? Daryoush - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Is it possible to set authentication to only authenticate consumers?
Hi Frase, qpidd chooses from the available mechanisms the most secure (from available) every time, when possible. I think SASL library itself does not have a priority list itself, the decision is made by qpidd only. Kind regards, Pavel - Original Message - From: Fraser Adams fraser.ad...@blueyonder.co.uk To: users@qpid.apache.org Sent: Friday, October 7, 2011 1:32:02 PM Subject: Re: Is it possible to set authentication to only authenticate consumers? I think I'm sorted now. I added: mech_list: anonymous plain to my qpidd.conf and that seems to work. out of curiosity does sasl choose the mechanisms in order here? Without specifying mech_list the broker trace indicated that it was supporting a wide range of mechanisms including anonymous yet it chose MD5-DIGEST (I think) when it was initially failing with my c++ client Frase Fraser Adams wrote: Gordon Sim wrote: On 10/03/2011 06:42 PM, Fraser Adams wrote: Is it possible to set authentication to only authenticate consumers so producers can connect in without needing authentication? You can allow both anonymous- and known- users to connect, and then use ACLs to only allow the known users to consume while allowing everyone (including anonymous users) to publish. Hi Gordon, How would I go about enabling anonymous authentication? I've successfully authenticated my basic Java client using the guest/guest username/password - I'm guessing that's not anonymous though as it clearly has a name :-). I've just run up a basic C++ client and that asks for a password. It appears to be sending the account name as the username (in other words in my case it's saying Authentication failed for fadams@QPID:SASL(-13): authentication failure: client response doesn't match what we generated). My client is pretty basic and has string broker = localhost:5672; string connectionOptions = {reconnect: true}; Now I think that I can add username/password to the connection options and I noticed a |sasl_mechanisms |connection option so I may be able to explicitly set that to anonymous But both of these would require code changes. That's fine in my case here where I can change the code, but in a real world scenario I've got a lot of producers (and I'm not convinced that the developers have necessarily made the connection options configurable) currently connecting to a broker with authentication disabled. I'd like to be able to authenticate without them having to change and to add ACL rules to only allow them to produce. I'd have thought that anonymous would have been something that I could enable on the broker config. Have I missed something? MTIA Frase - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
ring queue needs max-queue-size specified?
Hello, I noticed that declaring ring queue with just max-queue-count parameter prevents me sending messages with nonempty content: # qpid-config add queue Test2 --max-queue-count=100 --limit-policy=ring # ./spout -c 100 --content a Test2 2011-10-05 10:53:18 warning Exception received from broker: resource-limit-exceeded: resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on Test2, policy: size: unlimited; count: max=100, current=0; type=ring (qpid/broker/QueuePolicy.cpp:87) [caused by 3 \x00:\x00] 2011-10-05 10:53:18 warning Exception received from broker: resource-limit-exceeded: resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on Test2, policy: size: unlimited; count: max=100, current=0; type=ring (qpid/broker/QueuePolicy.cpp:87) [caused by 1 \x00:\x00] Failed to connect (reconnect disabled) # ./spout -c 100 Test2 # While when I define also max-queue-size, nonempty messages are accepted: # qpid-config del queue Test2 --force # qpid-config add queue Test2 --max-queue-count=100 --max-queue-size=10 --limit-policy=ring # ./spout -c 100 --content a Test2 # Debugging the failing case, qpidd really rejects the messages due to size: 2011-10-05 10:53:18 debug Message too large for ring queue Test2 [size: unlimited; count: max=100, current=0; type=ring] : message size = 1 bytes: max queue size = 0 bytes Is that intentional or not? As I would assume not specifying max-queue-* parameter means I don't want to restrict the queue in that manner.. I will create JIRA and (try to) propose a patch for it if you see it as a bug as well. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: ring queue needs max-queue-size specified?
Hi Gordon, thanks for pointing me out to the JIRA case - I should search there first and then ask.. I am using old good 0.10 so without the fix yet. Kind regards, Pavel - Original Message - From: Gordon Sim g...@redhat.com To: users@qpid.apache.org Sent: Wednesday, October 5, 2011 12:33:01 PM Subject: Re: ring queue needs max-queue-size specified? On 10/05/2011 10:08 AM, Pavel Moravec wrote: Hello, I noticed that declaring ring queue with just max-queue-count parameter prevents me sending messages with nonempty content: # qpid-config add queue Test2 --max-queue-count=100 --limit-policy=ring # ./spout -c 100 --content a Test2 2011-10-05 10:53:18 warning Exception received from broker: resource-limit-exceeded: resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on Test2, policy: size: unlimited; count: max=100, current=0; type=ring (qpid/broker/QueuePolicy.cpp:87) [caused by 3 \x00:\x00] 2011-10-05 10:53:18 warning Exception received from broker: resource-limit-exceeded: resource-limit-exceeded: resource-limit-exceeded: Policy exceeded on Test2, policy: size: unlimited; count: max=100, current=0; type=ring (qpid/broker/QueuePolicy.cpp:87) [caused by 1 \x00:\x00] Failed to connect (reconnect disabled) # ./spout -c 100 Test2 # While when I define also max-queue-size, nonempty messages are accepted: # qpid-config del queue Test2 --force # qpid-config add queue Test2 --max-queue-count=100 --max-queue-size=10 --limit-policy=ring # ./spout -c 100 --content a Test2 # Debugging the failing case, qpidd really rejects the messages due to size: 2011-10-05 10:53:18 debug Message too large for ring queue Test2 [size: unlimited; count: max=100, current=0; type=ring] : message size = 1 bytes: max queue size = 0 bytes Is that intentional or not? As I would assume not specifying max-queue-* parameter means I don't want to restrict the queue in that manner.. I will create JIRA and (try to) propose a patch for it if you see it as a bug as well. It is a bug, but I believe it should already be fixed in 0.12: https://issues.apache.org/jira/browse/QPID-3180. What version were you using? - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Priority queues with JMS Client/ Java Broker
From: Alan Conway acon...@redhat.com To: users@qpid.apache.org Cc: Pavel Moravec pmora...@redhat.com Sent: Wednesday, October 5, 2011 3:35:14 PM Subject: Re: Priority queues with JMS Client/ Java Broker On 09/27/2011 03:37 AM, Pavel Moravec wrote: Hi Praveen and all, just an additional info to prevent some confusion if you wish to use browse mode: priorities work when consuming the messages, while when browsing the queue, messages are ordered based on their enqueueing time. I.e. when sending messages: - first with priority 2 - then with priority 1 - latest with priority 3 then browsing the queue shows messages in ordering priority 2, priority 1, priority 3 message. While consuming messages consumes them in ordering priority 3, priority 2, priority 1 message. That sounds like a bug to me. Is it intentional? I think so - I discussed it with somebody (Gordon?). At present it is intentional as the change would not be trivial afaik. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Logging
Hello, I can confirm the below. It occurs when qpidd runs as a daemon: qpidd --log-to-syslog no --log-to-file /tmp/qpidd.txt -d causes logs go to syslog, while: qpidd --log-to-syslog no --log-to-file /tmp/qpidd.txt doesn't log to syslog. I think the issue is caused by -d flag overriding log-to-syslog option: Daemon options: -d [ --daemon ] Run as a daemon. Logs to syslog by default in this mode. I have already created https://issues.apache.org/jira/browse/QPID-3482 for it, I suppose potential fix can be quite easy. Kind regards, Pavel Moravec GSS SEG Red Hat - Original Message - From: Bradley Scalio brad.sca...@noaa.gov To: users@qpid.apache.org Sent: Tuesday, September 13, 2011 2:28:14 PM Subject: Logging Greetings, Trying to turn off logging to /var/log/messages and send qpidd logging to /var/log/qpidd.log I tried the following in our qpidd.conf: sed -i '$a log-to-file=/var/log/qpidd.log' qpidd.conf sed -i '$a log-to-syslog=no' qpidd.conf I get logging to /var/log/qpidd.log but I still am receiving duplicates to /var/log/messages. I know there are ample log sink options and variations, and have tested a few to no avail, including syslog-name=qpidd.log Is there a basic setup to not log to syslog? thanks -- Bradley M Scalio, RHCE RHCSS Principal Engineer w/Honors AWIPS Mission Assurance Mission Operations Services Raytheon Company (o) +301-495-2277 (c) +301-830-0207 (f) +301-713-1950 - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
List of areas to log/debug/trace
Hello, is there a list of areas we can modify logging level in cpp broker? I mean, when using --log-enable debug:framing option, list of verbosity levels is printed in qpidd help. But I can't find anywhere the list of areas to debug. (I suppose the list is list of directories in qpid source, i.e.: acl agent amqp_0_10 broker client cluster console framing log management messaging replication store sys types xml but that is only my guess based on my experience so far..) Thanks in advance for information. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Unification of qpid tools authentication options
Hello all, it has been noticed that qpid tools like qpid-config, qpid-cluster etc. use different options for setting authentication stuff (username, password, sasl mechanism). Moreover some tools don't have some options available. Plus few tools behave wrongly in some particular cases. I would like to unify that, so I created https://issues.apache.org/jira/browse/QPID-3437. I attached there a table with options each tool uses for setting up credentials. What remains to decide (and why I write here) is what options to se for setting up username / password / SASL mechanism. Basically, these three types are present at some tools and one of them should be used: qpid-perftest --username guest --password passwd -b broker --mechanism MECH qpid-config -a guest/passwd@broker --sasl-mechanism=MECH qpid-stat guest/passwd@broker -sasl-mechanism=MECH Are there some preferences among qpid users? Or factical reasons what is better / worse ? Thanks in advance for your opinion. Kind regards, Pavel - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: How to set subject in Java spout?
Hi Rajith and Alan, thanks for responses. I was aware of the address syntax but forgot to mention it in my email. My question was rather to -P parameter usage and its syntax in general, that Rajith answered. Kind regards, Pavel - Original Message - From: Rajith Attapattu rajit...@gmail.com To: users@qpid.apache.org Sent: Wednesday, August 3, 2011 5:44:19 PM Subject: Re: How to set subject in Java spout? In addition to what Alan suggested (which is the preferred way) you can do the following as well. java -cp $QP org.apache.qpid.example.Spout -P=qpid.subject=hello demoQueue If you type -h or --help, it will tell you how to specify these properties. regards, Rajith On Wed, Aug 3, 2011 at 9:34 AM, Alan Conway acon...@redhat.com wrote: On 08/03/2011 06:47 AM, Pavel Moravec wrote: Hi all, comparing C++ and Java spout programs, I realized I am unable to set subject of a message using Java Spout program. C++ has syntax: ./spout -b localhost:5672 -c 1 -P '{subject:Some_subject}' testQueue but trying the same in Java client does not work: # cd /usr/share/doc/qpid-java-0.10/examples # ./run_example.sh org.apache.qpid.example.Spout -c 1 -P '{subject:Some_subject}' testQueue {P={subject:Some_subject}, c=1} Exception in thread main java.lang.StringIndexOutOfBoundsException: String index out of range: -1 at java.lang.String.substring(String.java:1937) at org.apache.qpid.example.Spout.setProperties(Spout.java:135) at org.apache.qpid.example.Spout.createMessage(Spout.java:124) at org.apache.qpid.example.Spout.init(Spout.java:97) at org.apache.qpid.example.Spout.main(Spout.java:146) # No variation to the -P parameter worked for me. Any idea how to set subject there? Kind regards, Pavel address_string ::= address [ / subject ] [ ; options ] From http://qpid.apache.org/books/trunk/Programming-In-Apache-Qpid/html/ch02s04.html#id2661206 - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Configuration of CRAM-MD5 SASL method?
Hi Gordon, thanks a lot. Checking source code was the next step in my investigation here ;-) Kind regards, Pavel - Original Message - From: Gordon Sim g...@redhat.com To: users@qpid.apache.org Sent: Thursday, August 4, 2011 2:53:39 PM Subject: Re: Configuration of CRAM-MD5 SASL method? On 08/04/2011 09:47 AM, Pavel Moravec wrote: Hi all, does somebody know how to configure CRAM-MD5 SASL authentication method? I tried the following: # cat /etc/sasl2/qpidd.conf pwcheck_method: auxprop auxprop_plugin: sasldb sasldb_path: /var/lib/qpidd/qpidd.sasldb #following line stops spurious 'sql_select option missing' errors when #cyrus-sql-sasl plugin is installed sql_select: dummy select mech_list: cram-md5 # qpid-perftest --count 100 --username guest --password guest --mechanism CRAM-MD5 2011-08-04 10:34:49 warning Broker closed connection: 320, connection-forced: Authentication failed connection-forced: Authentication failed # qpid debug has: 2011-08-04 10:33:05 info SASL: Mechanism list: CRAM-MD5 2011-08-04 10:33:05 debug Management object (V1) added: org.apache.qpid.broker:connection:127.0.0.1:5672-127.0.0.1:54123 2011-08-04 10:33:05 debug SASL: Starting authentication with mechanism: CRAM-MD5 2011-08-04 10:33:05 warning Failed to retrieve sasl username 2011-08-04 10:33:05 info SASL: Authentication failed (no username available):SASL(-6): can't request info until later in exchange: Information that was requested is not yet available. 2011-08-04 10:33:05 debug Exception constructed: Authentication failed 2011-08-04 10:33:05 warning Failed to retrieve sasl username The same (error 320 and SASL(-6)) I received when using Java HelloWorld program specifying sasl_mechs='CRAM-MD5' . Any suggestions what do I wrong? As when I replace CRAM-MD5 by DIGEST-MD5 in sasl config file and perftest command line, the authentication passes.. (well, it does not in Java HelloWorld program, but that is another story). You aren't doing anything wrong, this appears to be a bug in the broker. I have raised a JIRA (QPID-3393) and have a fix that I'll commit shortly. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
How to set subject in Java spout?
Hi all, comparing C++ and Java spout programs, I realized I am unable to set subject of a message using Java Spout program. C++ has syntax: ./spout -b localhost:5672 -c 1 -P '{subject:Some_subject}' testQueue but trying the same in Java client does not work: # cd /usr/share/doc/qpid-java-0.10/examples # ./run_example.sh org.apache.qpid.example.Spout -c 1 -P '{subject:Some_subject}' testQueue {P={subject:Some_subject}, c=1} Exception in thread main java.lang.StringIndexOutOfBoundsException: String index out of range: -1 at java.lang.String.substring(String.java:1937) at org.apache.qpid.example.Spout.setProperties(Spout.java:135) at org.apache.qpid.example.Spout.createMessage(Spout.java:124) at org.apache.qpid.example.Spout.init(Spout.java:97) at org.apache.qpid.example.Spout.main(Spout.java:146) # No variation to the -P parameter worked for me. Any idea how to set subject there? Kind regards, Pavel
Re: Too many open files
Hi Sergey, no, this error log comes from OS, not from qpidd process. The system does not allow to open a new file descriptor as the qpidd process has opened maximum allowed FDs already. What is your ulimit settings (ulimit -a for qpidd user)? What returns the following lsof command? lsof -p $(pgrep qpidd) | nl Kind regards, Pavel - Original Message - From: Zhemzhitsky Sergey sergey_zhemzhit...@troika.ru To: users@qpid.apache.org Sent: Tuesday, August 2, 2011 9:39:12 AM Subject: RE: Too many open files Hi Pavel, I have the following errors in the log file error Could not accept socket: Too many open files (qpid/sys/posix/Socket.cpp:220) error Could not accept socket: Too many open files (qpid/sys/posix/Socket.cpp:220) error Could not accept socket: Too many open files (qpid/sys/posix/Socket.cpp:220) Could it be connected with max connection count? Best Regards, Sergey Zhemzhitsky Information Technology Division Troika Dialog, 4, Romanov lane, Moscow 125009, Russia Phone. +7 495 2580500 ext. 1246 -Original Message- From: Pavel Moravec [mailto:pmora...@redhat.com] Sent: Monday, August 01, 2011 3:42 PM To: users@qpid.apache.org Subject: Re: Too many open files Hi Sergey, what precise log did appear? Something like: Client max connection count limit exceeded: 500 connection refused ? If so, let see option --max-connections N (500) Sets the maximum allowed connections Kind regards, Pavel - Original Message - From: Zhemzhitsky Sergey sergey_zhemzhit...@troika.ru To: users@qpid.apache.org Sent: Monday, August 1, 2011 12:53:09 PM Subject: Too many open files Hi there, We have already set ulimit to be able to open 16384 files, but the error still happens from time to time. Could you help us to identify the possible sources of this issue. Best Regards, Sergey Zhemzhitsky Information Technology Division Troika Dialog, 4, Romanov lane, Moscow 125009, Russia Phone. +7 495 2580500 ext. 1246 ___ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org ___ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
qpid-config slow
Hi all, I realised on every qpid version system I used a poor performance of qpid-config tool. It usually takes 2 to 3 seconds to execute a command, e.g.: # time qpid-config add queue test real 0m2.243s user 0m0.161s sys 0m0.027s # time qpid-config del queue test real 0m2.303s user 0m0.208s sys 0m0.037s # I wrote a simple C++ program to emulate some parts of qpid-config tool by connecting to the broker and sending proper QMF message(s). The program executes equivalent stuff almost instantly. Why is qpid-config so slow? Is there some timeout waiting for (potentially negative) response? Does it make sense / would not it be usefull to speed it up? Kind regards, Pavel