Re: Best practice for sending UTF-16 wstring and Binary blobs via messaging API for C++ & Python ?
On 06/09/2010 07:01 PM, Kerry Bonin wrote: Is the recommended practice for sending a UTF-16 string to convert to UTF-8 and send as std::string? Ideally, is there an example available to send& receive a wstring via the messaging API? I understand the amqp 0-10 codec supports this type, but its not clear how to send something not directly supported by the Variant class. On a related note, what is the recommended practice for sending a binary blob using the messaging API? It wasn't clear from QPID-2452 if it is safe to stuff a binary array into a string type. Is there any example available to send& receive a binary blob via the messaging API? The string Variant type can hold anything a std::string can, so binary is fine. You can signal any encoding via Variant::setEncoding() (if no encoding is specified the string will be transferred as an AMQP vbin16). There is no special wide string support as yet. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Best practice for sending UTF-16 wstring and Binary blobs via messaging API for C++ & Python ?
Is the recommended practice for sending a UTF-16 string to convert to UTF-8 and send as std::string? Ideally, is there an example available to send & receive a wstring via the messaging API? I understand the amqp 0-10 codec supports this type, but its not clear how to send something not directly supported by the Variant class. On a related note, what is the recommended practice for sending a binary blob using the messaging API? It wasn't clear from QPID-2452 if it is safe to stuff a binary array into a string type. Is there any example available to send & receive a binary blob via the messaging API? And my apologies if this has appeared in recent example code, I'm using a slightly dated trunk... Thanks! - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Prototype web site - now with most of the content
I would also remove the links to the old .NET clients from the download page. These clients are buggy and unsupported. It's best we not provide anything at all, rather then providing something that is known to be buggy and unsupported. This is not to disrespect the individuals who have put a lot of effort into these clients, but rather being pragmatic about what we are able to do as a community. With the WCF client and the new .NET binding (over c++) it is unlikely that the old clients will receive any TLC. So lets get rid of them from the download pages and the code tree (You could always get it from the svn history if needed). Rajith On Tue, Jun 8, 2010 at 10:41 AM, Jonathan Robie wrote: > On Tue, 2010-06-08 at 14:27 +0100, Robbie Gemmell wrote: >> On the same page, the link to the wiki should point to >> http://cwiki.apache.org/confluence/display/qpid >> >> Overall looks good, I echo Gordon's thoughts, lets get this live ASAP >> and improve as we go rather than spending too much time polishing it >> beforehand: anythings better than what we have now and everything is >> still reachable as long as we link to the wiki. >> >> One minor nit, the bullet pointed feature box things on the front page >> dont display in the right place in IE6 and IE7 (or 8 in compatibility >> mode), the entire containing div drops down the page to where the menu >> stops. I imagine its just a margin/padding+float CSS bug, ill try to >> take a look tonight if I get a chance, they are still around 1/3rd of >> all used browsers afterall..now, back to Firefox ;) > > Nice catch - and there's a CSS validation error for feature_box in > the .css stylesheet. I fixed that, I'm asking #asfinfra to refresh (it's > not yet set up for me to do that automagically). > > We'll see if that fixes the problem ... > > Jonathan > > > - > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:dev-subscr...@qpid.apache.org > > -- Regards, Rajith Attapattu Red Hat http://rajith.2rlabs.com/ - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2653) Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
[
https://issues.apache.org/jira/browse/QPID-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877119#action_12877119
]
Sorin Suciu commented on QPID-2653:
---
Agreed Robbie, here comes a better patch I hope.
> Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
> ---
>
> Key: QPID-2653
> URL: https://issues.apache.org/jira/browse/QPID-2653
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
>Affects Versions: 0.7
>Reporter: Sorin Suciu
>Priority: Minor
> Fix For: 0.7
>
> Attachments:
> 0001-Double-checked-locking-in-org.apache.qpid.server.tra.patch,
> qpid-2653.patch
>
>
> org.apache.qpid.server.transport.ThreadPoolFilter has a double check locking
> in this method:
> private SessionBuffer getSessionBuffer(IoSession session)
> {
> final Map buffers = this.buffers;
> SessionBuffer buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> synchronized (buffers)
> {
> buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> buf = new SessionBuffer(session);
> buffers.put(session, buf);
> }
> }
> }
> return buf;
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-2653) Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
[
https://issues.apache.org/jira/browse/QPID-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sorin Suciu updated QPID-2653:
--
Attachment: qpid-2653.patch
> Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
> ---
>
> Key: QPID-2653
> URL: https://issues.apache.org/jira/browse/QPID-2653
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
>Affects Versions: 0.7
>Reporter: Sorin Suciu
>Priority: Minor
> Fix For: 0.7
>
> Attachments:
> 0001-Double-checked-locking-in-org.apache.qpid.server.tra.patch,
> qpid-2653.patch
>
>
> org.apache.qpid.server.transport.ThreadPoolFilter has a double check locking
> in this method:
> private SessionBuffer getSessionBuffer(IoSession session)
> {
> final Map buffers = this.buffers;
> SessionBuffer buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> synchronized (buffers)
> {
> buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> buf = new SessionBuffer(session);
> buffers.put(session, buf);
> }
> }
> }
> return buf;
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2653) Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
[
https://issues.apache.org/jira/browse/QPID-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877095#action_12877095
]
Robbie Gemmell commented on QPID-2653:
--
The original code, whilst a bit odd looking, does ensure that only 1 new buffer
is added to the map. The patch does not, as it synchronizes after checking
whether tehre was anything in the map and so multiple threads waiting on the
lock could then add to the map. I would look at synchronizing/locking around
the get and put.
> Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
> ---
>
> Key: QPID-2653
> URL: https://issues.apache.org/jira/browse/QPID-2653
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
>Affects Versions: 0.7
>Reporter: Sorin Suciu
>Priority: Minor
> Fix For: 0.7
>
> Attachments:
> 0001-Double-checked-locking-in-org.apache.qpid.server.tra.patch
>
>
> org.apache.qpid.server.transport.ThreadPoolFilter has a double check locking
> in this method:
> private SessionBuffer getSessionBuffer(IoSession session)
> {
> final Map buffers = this.buffers;
> SessionBuffer buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> synchronized (buffers)
> {
> buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> buf = new SessionBuffer(session);
> buffers.put(session, buf);
> }
> }
> }
> return buf;
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-2653) Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
[
https://issues.apache.org/jira/browse/QPID-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sorin Suciu updated QPID-2653:
--
Attachment: 0001-Double-checked-locking-in-org.apache.qpid.server.tra.patch
> Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
> ---
>
> Key: QPID-2653
> URL: https://issues.apache.org/jira/browse/QPID-2653
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
>Affects Versions: 0.7
>Reporter: Sorin Suciu
>Priority: Minor
> Fix For: 0.7
>
> Attachments:
> 0001-Double-checked-locking-in-org.apache.qpid.server.tra.patch
>
>
> org.apache.qpid.server.transport.ThreadPoolFilter has a double check locking
> in this method:
> private SessionBuffer getSessionBuffer(IoSession session)
> {
> final Map buffers = this.buffers;
> SessionBuffer buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> synchronized (buffers)
> {
> buf = (SessionBuffer) buffers.get(session);
> if (buf == null)
> {
> buf = new SessionBuffer(session);
> buffers.put(session, buf);
> }
> }
> }
> return buf;
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Created: (QPID-2653) Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
Double checked locking in org.apache.qpid.server.transport.ThreadPoolFilter
---
Key: QPID-2653
URL: https://issues.apache.org/jira/browse/QPID-2653
Project: Qpid
Issue Type: Bug
Components: Java Broker
Affects Versions: 0.7
Reporter: Sorin Suciu
Priority: Minor
Fix For: 0.7
org.apache.qpid.server.transport.ThreadPoolFilter has a double check locking
in this method:
private SessionBuffer getSessionBuffer(IoSession session)
{
final Map buffers = this.buffers;
SessionBuffer buf = (SessionBuffer) buffers.get(session);
if (buf == null)
{
synchronized (buffers)
{
buf = (SessionBuffer) buffers.get(session);
if (buf == null)
{
buf = new SessionBuffer(session);
buffers.put(session, buf);
}
}
}
return buf;
}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-2650) DerbyStore only ever uses auto-commit connections
[ https://issues.apache.org/jira/browse/QPID-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated QPID-2650: - Status: Ready To Review (was: In Progress) > DerbyStore only ever uses auto-commit connections > - > > Key: QPID-2650 > URL: https://issues.apache.org/jira/browse/QPID-2650 > Project: Qpid > Issue Type: Bug > Components: Java Broker MessageStore - DerbyStore >Affects Versions: M3, M4, 0.5, 0.6 >Reporter: Robbie Gemmell >Assignee: Robbie Gemmell >Priority: Critical > Fix For: 0.7 > > > The DerbyStore currently only ever uses auto-commit connections. The store > does explicitly commit and rollback connections in the appropriate locations, > however the default auto-commit behaviour of the created connections is never > disabled, meaning that upon broker startup the VirtualHost recovery process > can recover messages published in a transactional session which was never > committed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-2651) replace the deprecated Derby 10.3.2.1 (data corruption bug) with a newer version
[ https://issues.apache.org/jira/browse/QPID-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated QPID-2651: - Status: Ready To Review (was: In Progress) > replace the deprecated Derby 10.3.2.1 (data corruption bug) with a newer > version > > > Key: QPID-2651 > URL: https://issues.apache.org/jira/browse/QPID-2651 > Project: Qpid > Issue Type: Improvement > Components: Java Broker MessageStore - DerbyStore >Affects Versions: M3, M4, 0.5, 0.6 >Reporter: Robbie Gemmell >Assignee: Robbie Gemmell >Priority: Critical > Fix For: 0.7 > > > The specific Derby 10.3.2.1 release we currently ship was pulled and > deprecated due to discovery of an unrecoverable data corruption bug and > should be replaced. It is also more than a few years old and should be > upgraded to a newer version anyway. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Created: (QPID-2652) Performance problems
Performance problems
Key: QPID-2652
URL: https://issues.apache.org/jira/browse/QPID-2652
Project: Qpid
Issue Type: Bug
Components: Python Client
Environment: Debian GNU Linux squeeze, QPID 0.6/trunk
Reporter: Cajus Pollmeier
When using the python client, I noticed that there's a significant performance
difference when using python and c++ services. Using the client below runs ~10s
with the c++ request/reponse and ~40s with the python one. A factor 4 slowdown
doesn't seem to recommend the python libs.
C++ request/response:
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
using namespace qpid::messaging;
using std::stringstream;
using std::string;
Connection connection;
void * run(void *conn_ptr) {
Session session = connection.newSession();
Receiver receiver = session.createReceiver("requests; {create: always}");
receiver.setCapacity(2);
while (true) {
Message request = receiver.fetch();
try {
const Address& address = request.getReplyTo();
if (address) {
std::string s = request.getContent();
std::cout << "Processing request: "
<< request.getContent()
<< " -> "
<< request.getReplyTo().getName() << std::endl;
Sender sender =
session.createSender(request.getReplyTo().getName() );
std::transform(s.begin(), s.end(), s.begin(), toupper);
Message response(s);
sender.send(response);
std::cout << "Processed request: "
<< request.getContent()
<< " -> "
<< response.getContent() << std::endl;
session.acknowledge();
} else {
std::cerr << "Error: no reply address specified for request: "
<< request.getContent() << std::endl;
session.reject(request);
}
} catch(const std::exception& error) {
std::cout << "Something went wrong: " << error.what() << std::endl;
session.reject(request);
}
}
}
int main(int argc, char** argv) {
const char* url = argc>1 ? argv[1] : "amqp:tcp:127.0.0.1:5672";
Variant::Map amqp_options;
amqp_options["username"] = "admin";
amqp_options["password"] = "secret";
connection = Connection::open(url, amqp_options);
int i = 0;
pthread_t threads[10];
// Create the threads
for(i = 0; i<10; i++) {
pthread_create(&threads[i], NULL, run, (void*) NULL);
}
// Get the threads back
for(i = 0; i<10; i++) {
pthread_join(threads[i], NULL);
}
connection.close();
return 0;
}
Python request/response:
import traceback
from qpid.messaging import *
from threading import Thread
class Processor(Thread):
def __init__(self, ssn):
Thread.__init__(self)
self.setDaemon(True)
self.ssn = ssn
def run(self):
while True:
msg = self.ssn.next_receiver().fetch()
try:
to, reply = self.process(msg)
snd = ssn.sender(to)
snd.send(reply)
snd.close()
self.ssn.acknowledge(msg)
except:
traceback.print_exc()
print "rejecting: %s" % msg
self.ssn.acknowledge(msg, Disposition(REJECTED))
def process(self, msg):
print "processing %s" % msg
return msg.reply_to, Message("echo %s" % msg.content)
conn = Connection.establish("localhost", username='admin', password='secret')
processors = []
for i in range(10):
ssn = conn.session()
rcv = ssn.receiver("requests", capacity=10)
proc = Processor(ssn)
proc.start()
processors.append(proc)
# this may look a bit weird, but the timed join keeps python from
# swallowing interrupts
while True:
for p in processors:
p.join(3)
---
Test-Client:
import traceback
from qpid.messaging import *
from threading import Thread
conn = Connection.establish("localhost")
ssn = conn.session(str(uuid4()))
snd = ssn.sender("requests")
rcv = ssn.receiver('reply-%s; {create:always, delete:always}' % ssn.name)
for i in range(1000):
msg = Message("The quick brown fox tests the performance")
msg.reply_to = 'reply-%s' % ssn.name
snd.send(msg)
res = rcv.fetch()
print res.content
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
