Re: jprofiler hotspot results for java engine clients
Thanks Rafi. I do indeed see a difference in how long it takes to send the messages when I use all the credit at once. However, I was seeing an overall slowdown in how long it takes to settle all the messages. The rest of this message is just a commentary and theory on the slowdown. I'm sending 100,000 messages. The receiver is calling .drain(10) Scenario 1 - I send a single message in onFlow: - The total time it takes before all messages are SENT is approx 6 seconds - The total time it takes before all messages are SETTLED is approx 6 seconds - The settlements come in as the messages are sent Scenario 2 - I send as many messages as I have credit for in onFlow: - The total time it takes before all messages are SENT is approx 1.5 seconds - The total time it takes before all messages are SETTLED is approx 22 seconds - The settlements start after all messages are sent One clue as to the reason for the dramatic increase in total settlement time is that the rate of settlement increases as the number of deliveries remaining to be settled decreases. In my receiver, I'm printing a '*' for every 1000 settlements. At first the *s come about .5 seconds apart, but they increase in speed in what appears to be a linear fashion. From this I'm guessing there is a list traversal of all unsettled deliveries happening in the engine. And indeed, if I decrease the receiver's credit window to .drain(1000), the settlements come in as expected. Scenario 3 - I send as many messages as I have credit for in onFlow but the credit window is 1000 - The total time it takes before all messages are SENT is approx 4 seconds - The total time it takes before all messages are SETTLED is approx 4 seconds - The settlements come in as each batch of messages are sent I'll do a jprofiler run on the receiver for scenario 2 and see what I find. - Original Message - From: Rafael Schloming r...@alum.mit.edu To: proton@qpid.apache.org Sent: Friday, October 10, 2014 12:55:45 PM Subject: Re: jprofiler hotspot results for java engine clients I think if you change the innermost if statement in your onFlow handler to a while you should see significantly improved performance. Flow events are generated when credit levels change. This happens at two different times, 1) when the receiver grants more credit to the sender, and 2) when the sender actually uses some of that credit by writing a message to the wire. The way you have coded your event handler, even if the receiver sends you 10,000 credits, you are only going to send a single message until you get another flow event, and this won't happen until you write the first message out onto the wire. This is going to introduce some added processing and latency and therefore be significantly less efficient than if you use up your credit in larger chunks than one message at a time. --Rafael On Thu, Oct 9, 2014 at 2:48 PM, Ernest Allen eal...@redhat.com wrote: - Original Message - From: Rafael Schloming r...@alum.mit.edu To: proton@qpid.apache.org Sent: Thursday, October 9, 2014 10:43:09 AM Subject: Re: jprofiler hotspot results for java engine clients Can you include a bit more context? The 6-7 seconds for 100,000 messages seems a bit slow compared to what I've been able to measure. Was it over loopback or a real network? Was the profiler running when those measurements were made? How big are the messages, etc? It was just on my laptop using 0.0.0.0. I'll try on a more powerful lab machine. The profiler was not running. I ran both from command line. The messages are small: Hello World. I'm starting the clock after all connections are made, just before I send the 1st message. I stop the clock when the last message is remotelySettled in the onDelivery event handler. I'm just sending a single message every time the onFlow handler is called. I could put a loop in onFlow and send all the messages at once. I was able to reduce the time to a little over 5 seconds by removing all string manipulations out of the onFlow event handler. ---Rafael On Thu, Oct 9, 2014 at 10:10 AM, Ernest Allen eal...@redhat.com wrote: There are some java apps based on the 0.8 engine api available at https://github.com/ErnieAllen/jprofile-clients. They are based on the clients at https://github.com/rhs/qpid-proton-demo They will send/receive 100,000 messages and measure the elapsed time. They are averaging between 6 and 7 seconds for the 100,000 messages. I used these clients to run jprofiler. Reports showing which java methods are hotspots are at http://misc-eapages.rhcloud.com/jprofiler-results/send/Hot_Spots.html http://misc-eapages.rhcloud.com/jprofiler-results/recv/Hot_Spots.html If anyone has suggestions as to how to improve the clients' throughput or would like to see a different jprofiler graph, please let me know. -Ernie
jprofiler hotspot results for java engine clients
There are some java apps based on the 0.8 engine api available at https://github.com/ErnieAllen/jprofile-clients. They are based on the clients at https://github.com/rhs/qpid-proton-demo They will send/receive 100,000 messages and measure the elapsed time. They are averaging between 6 and 7 seconds for the 100,000 messages. I used these clients to run jprofiler. Reports showing which java methods are hotspots are at http://misc-eapages.rhcloud.com/jprofiler-results/send/Hot_Spots.html http://misc-eapages.rhcloud.com/jprofiler-results/recv/Hot_Spots.html If anyone has suggestions as to how to improve the clients' throughput or would like to see a different jprofiler graph, please let me know. -Ernie
Re: jprofiler hotspot results for java engine clients
- Original Message - From: Rafael Schloming r...@alum.mit.edu To: proton@qpid.apache.org Sent: Thursday, October 9, 2014 10:43:09 AM Subject: Re: jprofiler hotspot results for java engine clients Can you include a bit more context? The 6-7 seconds for 100,000 messages seems a bit slow compared to what I've been able to measure. Was it over loopback or a real network? Was the profiler running when those measurements were made? How big are the messages, etc? It was just on my laptop using 0.0.0.0. I'll try on a more powerful lab machine. The profiler was not running. I ran both from command line. The messages are small: Hello World. I'm starting the clock after all connections are made, just before I send the 1st message. I stop the clock when the last message is remotelySettled in the onDelivery event handler. I'm just sending a single message every time the onFlow handler is called. I could put a loop in onFlow and send all the messages at once. I was able to reduce the time to a little over 5 seconds by removing all string manipulations out of the onFlow event handler. ---Rafael On Thu, Oct 9, 2014 at 10:10 AM, Ernest Allen eal...@redhat.com wrote: There are some java apps based on the 0.8 engine api available at https://github.com/ErnieAllen/jprofile-clients. They are based on the clients at https://github.com/rhs/qpid-proton-demo They will send/receive 100,000 messages and measure the elapsed time. They are averaging between 6 and 7 seconds for the 100,000 messages. I used these clients to run jprofiler. Reports showing which java methods are hotspots are at http://misc-eapages.rhcloud.com/jprofiler-results/send/Hot_Spots.html http://misc-eapages.rhcloud.com/jprofiler-results/recv/Hot_Spots.html If anyone has suggestions as to how to improve the clients' throughput or would like to see a different jprofiler graph, please let me know. -Ernie
Re: how can I get proton-api version 1.0-SNAPSHOT for api-reconciliation?
Thanks Robbie. In order to get the proton-jni jar I naively checked out and built the branch named jni-binding and copied the contents of the build/proton-c/bindings/java dir. Since the api-reconciliation tool is no longer viable, are there any example java apps that use the engine api? -Ernie - Original Message - From: Robbie Gemmell robbie.gemm...@gmail.com To: proton@qpid.apache.org Sent: Thursday, October 2, 2014 11:14:29 AM Subject: Re: how can I get proton-api version 1.0-SNAPSHOT for api-reconciliation? On 2 October 2014 16:13, Robbie Gemmell robbie.gemm...@gmail.com wrote: Hi Ernie, The proton-api module no longer exists, it was merged with proton-j-impl to form the current proton-j module, so there are snapshots * no snapshots (which are confusingly named 1.0-SNAPSHOT all the time currently) being made for it now. The JNI bits were also removed around the same time. I'm afraid I never ran the tool that you are trying to use, but I would assume that it no longer works given the above. What version are you using if you managed to have a jni jar? Robbie On 1 October 2014 19:16, Ernest Allen eal...@redhat.com wrote: There is probably a simple solution to this, but I'm trying to run the api-reconciliation tool and I'm getting errors. Here is what I've done: - built proton - did a source config.sh from the build dir - switched to the design/api-reconciliation dir - ran ./generate-c-functions.sh - verified that target/cfunctions.txt exists - ran mvn clean install -U - ran mvn compile - ran mvn exec:java Here is the output from the exec: [INFO] Scanning for projects... [INFO] [INFO] [INFO] Building proton-api-reconciliation 1.0-SNAPSHOT [INFO] [INFO] [INFO] exec-maven-plugin:1.2.1:java (default-cli) @ proton-api-reconciliation [INFO] [INFO] exec-maven-plugin:1.2.1:java (default-cli) @ proton-api-reconciliation [INFO] [INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ proton-api-reconciliation --- [WARNING] The POM for org.apache.qpid:proton-api:jar:1.0-SNAPSHOT is missing, no dependency information available [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 1.563s [INFO] Finished at: Wed Oct 01 14:06:17 EDT 2014 [INFO] Final Memory: 8M/103M [INFO] [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project proton-api-reconciliation: Execution default-cli of goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java failed: Plugin org.codehaus.mojo:exec-maven-plugin:1.2.1 or one of its dependencies could not be resolved: Failure to find org.apache.qpid:proton-api:jar:1.0-SNAPSHOT in http://snapshots.repository.codehaus.org was cached in the local repository, resolution will not be reattempted until the update interval of codehaus.org has elapsed or updates are forced - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException From this I'm assuming that I'm missing the correct proton-api.jar, but I'm unclear on how to build/install it. I found a pom.xml for proton-api version 0.6, but api-reconciliation is looking for version 1.0-SNAPSHOT. When I modify the api-reconciliation pom to use the 0.6 proton.api.jar, it crashes. Any suggestions? P.S. I should mention that I built the proton-jni jar file separately and copied to my build/proton-c/bindings/java directory Thanks -Ernie
Re: how can I get proton-api version 1.0-SNAPSHOT for api-reconciliation?
Thanks all. It's good to have options. - Original Message - From: Clebert Suconic csuco...@redhat.com To: proton@qpid.apache.org Sent: Thursday, October 2, 2014 1:38:43 PM Subject: Re: how can I get proton-api version 1.0-SNAPSHOT for api-reconciliation? On Oct 2, 2014, at 1:01 PM, Bozo Dragojevic bo...@digiverse.si wrote: On 2. 10. 14 17:31, Ernest Allen wrote: Since the api-reconciliation tool is no longer viable, are there any example java apps that use the engine api? contrib/proton-hawtdispatch is also using the engine. True, but let me point that HornetQ's ProtonPlug doesn't have any specific HornetQ usage. It has some Netty buffer dependencies though as it's using a buffer pool in some places. Bozzo
how can I get proton-api version 1.0-SNAPSHOT for api-reconciliation?
There is probably a simple solution to this, but I'm trying to run the api-reconciliation tool and I'm getting errors. Here is what I've done: - built proton - did a source config.sh from the build dir - switched to the design/api-reconciliation dir - ran ./generate-c-functions.sh - verified that target/cfunctions.txt exists - ran mvn clean install -U - ran mvn compile - ran mvn exec:java Here is the output from the exec: [INFO] Scanning for projects... [INFO] [INFO] [INFO] Building proton-api-reconciliation 1.0-SNAPSHOT [INFO] [INFO] [INFO] exec-maven-plugin:1.2.1:java (default-cli) @ proton-api-reconciliation [INFO] [INFO] exec-maven-plugin:1.2.1:java (default-cli) @ proton-api-reconciliation [INFO] [INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ proton-api-reconciliation --- [WARNING] The POM for org.apache.qpid:proton-api:jar:1.0-SNAPSHOT is missing, no dependency information available [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 1.563s [INFO] Finished at: Wed Oct 01 14:06:17 EDT 2014 [INFO] Final Memory: 8M/103M [INFO] [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project proton-api-reconciliation: Execution default-cli of goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java failed: Plugin org.codehaus.mojo:exec-maven-plugin:1.2.1 or one of its dependencies could not be resolved: Failure to find org.apache.qpid:proton-api:jar:1.0-SNAPSHOT in http://snapshots.repository.codehaus.org was cached in the local repository, resolution will not be reattempted until the update interval of codehaus.org has elapsed or updates are forced - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException From this I'm assuming that I'm missing the correct proton-api.jar, but I'm unclear on how to build/install it. I found a pom.xml for proton-api version 0.6, but api-reconciliation is looking for version 1.0-SNAPSHOT. When I modify the api-reconciliation pom to use the 0.6 proton.api.jar, it crashes. Any suggestions? P.S. I should mention that I built the proton-jni jar file separately and copied to my build/proton-c/bindings/java directory Thanks -Ernie
Javascript bindings: calling subscribe(address) from within a browser
I'm getting an error when trying to receive messages from within a browser. The
same code runs fine when run from the command line using node.
The error message is:
Module.MessengerError {name: MessengerError, message: listen: Not
supported, constructor: function, toString: function}
The error happens when calling messenger.subscribe(address);
Is it the case that subscribing is actually not supported from within a
browser, or am I possibly doing something wrong?
I basically just took the code from recv.js and put it in a HTML page.
Here is the code:
!DOCTYPE html !-- HTML5 doctype --
html
head
titleSimple Proton Messenger Send Example/title
meta http-equiv=content-type content=text/html;charset=utf-8 /
script type=text/javascript
src=../../../node_modules/qpid-proton/lib/proton.js/script
script type=text/javascript
var address = amqp://~0.0.0.0;
var message = new proton.Message();
var messenger = new proton.Messenger();
var pumpData = function() {
while (messenger.incoming()) {
var t = messenger.get(message);
console.log(Address: + message.getAddress());
console.log(Subject: + message.getSubject());
// body is the body as a native JavaScript Object, useful for most real
cases.
//console.log(Content: + message.body);
// data is the body as a proton.Data Object, used in this case because
// format() returns exactly the same representation as recv.c
console.log(Content: + message.data.format());
messenger.accept(t);
}
};
console.log(address = ' + address + ');
messenger.setIncomingWindow(1024);
messenger.on('error', function(error) {console.log(error);});
messenger.on('work', pumpData);
messenger.start();
messenger.subscribe(address);
messenger.recv(); // Receive as many messages as messenger can buffer.
/script
/head
body
/body
/html
Regards,
-Ernie
Re: javascript binding in a browser
Hi Fraser,
Lots of great info. Thanks for being so thorough in your explanations, it
really helps. After adding in the missing tracker=, everything works as
expected.
What I meant by browserify the proton.js file was the browserify utility
(http://browserify.org/). I incorrectly thought my problem was that the
proton.js file needed to be modified to run in a browser environment. That
utility will bundle all required modules into a single file along with the
calling javascript. As it turns out, your module is already stand-alone and
works in either environment.
I'll grab your latest and use that in my experiments. Thanks again for the help
and the excellent work. It is fun the rediscover javascript again.
-Ernie
- Original Message -
From: Fraser Adams fraser.ad...@blueyonder.co.uk
To: proton@qpid.apache.org
Sent: Monday, September 1, 2014 2:45:46 PM
Subject: Re: javascript binding in a browser
Hi again Ernie,
I've just committed a few improvements.
In terms of debugging if you specify
cmake .. -DCMAKE_BUILD_TYPE=Debug
or
cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo
it will do a build with no JavaScript optimisations and no closure
compiler so any JavaScript errors will be a bit more human readable. The
default build does full Clang optimisations and JavaScript/asm.js
optimisations and minifies/optimises with the closure compiler.
I added code to the status and isBuffered methods that checks for a null
tracker being passed, and if so they now retrieve the outgoing tracker
and use that.
I've also added a soak test and a little send.html
I need to take a look at error handling - if I try to send when there's
no receiver it gets in a state where it won't subsequently send messages
when the receiver is restarted. I might just need to stop and restart
Messenger - TBH I can't claim to really understand the Messenger API.
Hope that you are having fun with this stuff.
Frase
On 30/08/14 16:25, Ernest Allen wrote:
Hi Fraser,
I'm trying to run the proton javascript bindings from within a browser. I'm
getting various errors depending on the way I'm doing it and I was hoping you
could shed some light on the correct approach.
I have node recv.js running in a separate window.
The first way I tried was just including proton.js directly. When I load the
page the console shows the following for the HTML listed below:
pre-main prep time: 6 ms
proton is [object Object]
before put
Uncaught TypeError: Cannot read property 'fa' of null
Uncaught TypeError: Cannot read property 'fa' of null
Here is the HTML:
!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Strict//EN
http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd;
html xmlns=http://www.w3.org/1999/xhtml;
head
titleAn XHTML 1.0 Strict standard template/title
meta http-equiv=content-type
content=text/html;charset=utf-8 /
script type=text/javascript
src=../../../node_modules/qpid-proton/lib/proton.js/script
script type=text/javascript
//![CDATA[
console.log(proton is + window.proton);
var address = amqp://0.0.0.0;
var subject = BrowserMessage;
var msgtext = Hello From Browser!;
var tracker = null;
var running = true;
var message = new proton.Message();
var messenger = new proton.Messenger();
var pumpData = function() {
var status = messenger.status(tracker);
if (status != proton.Status.PENDING) {
if (running) {
messenger.stop();
running = false;
}
}
if (messenger.isStopped()) {
message.free();
messenger.free();
}
};
messenger.on('error', function(error) {console.log(this is the error +
error);});
messenger.on('work', pumpData);
messenger.setOutgoingWindow(1024);
messenger.start();
message.setAddress(address);
message.setSubject(subject);
message.body = msgtext;
console.log(before put);
messenger.put(message);
//]]
/script
/head
body
/body
/html
The output in the window running node recv.js is:
[0x535398]:ERROR[-2] AMQP header mismatch: '' (connection aborted)
CONNECTION ERROR connection aborted (remote)
When I browserify the proton.js into qpid-proton-browser.js and load that
file instead, I get the following when the page loads:
Uncaught TypeError: Cannot read property 'replace' of undefined
proton is undefined
Uncaught ReferenceError: proton is not defined
The Uncaught TypeError is due to a referece to process.argv[1]. If I
manually edit the qpid-proton-browser.js file to define all the process
properties that it wants, I still get the proton is undefined error.
Do you know what I'm doing wrong?
Thanks,
-Ernie
javascript binding in a browser
Hi Fraser,
I'm trying to run the proton javascript bindings from within a browser. I'm
getting various errors depending on the way I'm doing it and I was hoping you
could shed some light on the correct approach.
I have node recv.js running in a separate window.
The first way I tried was just including proton.js directly. When I load the
page the console shows the following for the HTML listed below:
pre-main prep time: 6 ms
proton is [object Object]
before put
Uncaught TypeError: Cannot read property 'fa' of null
Uncaught TypeError: Cannot read property 'fa' of null
Here is the HTML:
!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Strict//EN
http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd;
html xmlns=http://www.w3.org/1999/xhtml;
head
titleAn XHTML 1.0 Strict standard template/title
meta http-equiv=content-type
content=text/html;charset=utf-8 /
script type=text/javascript
src=../../../node_modules/qpid-proton/lib/proton.js/script
script type=text/javascript
//![CDATA[
console.log(proton is + window.proton);
var address = amqp://0.0.0.0;
var subject = BrowserMessage;
var msgtext = Hello From Browser!;
var tracker = null;
var running = true;
var message = new proton.Message();
var messenger = new proton.Messenger();
var pumpData = function() {
var status = messenger.status(tracker);
if (status != proton.Status.PENDING) {
if (running) {
messenger.stop();
running = false;
}
}
if (messenger.isStopped()) {
message.free();
messenger.free();
}
};
messenger.on('error', function(error) {console.log(this is the error +
error);});
messenger.on('work', pumpData);
messenger.setOutgoingWindow(1024);
messenger.start();
message.setAddress(address);
message.setSubject(subject);
message.body = msgtext;
console.log(before put);
messenger.put(message);
//]]
/script
/head
body
/body
/html
The output in the window running node recv.js is:
[0x535398]:ERROR[-2] AMQP header mismatch: '' (connection aborted)
CONNECTION ERROR connection aborted (remote)
When I browserify the proton.js into qpid-proton-browser.js and load that file
instead, I get the following when the page loads:
Uncaught TypeError: Cannot read property 'replace' of undefined
proton is undefined
Uncaught ReferenceError: proton is not defined
The Uncaught TypeError is due to a referece to process.argv[1]. If I manually
edit the qpid-proton-browser.js file to define all the process properties that
it wants, I still get the proton is undefined error.
Do you know what I'm doing wrong?
Thanks,
-Ernie
javascript bindings connecting to a qpid c++ broker
I'm getting a connection error when using the javascript bindings example
recv.js to receive a message through a qpid c++ broker.
I have a qpid c++ broker running on a remote host with a queue named eaq. I'm
attempting to start up the javascript example recv.js using the remote
address/queue, but I'm getting a connection error:
[eallen@redhat javascript]$ node recv.js amqp://my.remote.host.com/eaq
that._checkErrors = true
error = 3,111,ECONNREFUSED: Connection refused
{ [MessengerError: ECONNREFUSED: Connection refused]
name: 'MessengerError',
message: 'ECONNREFUSED: Connection refused' }
I've tried variations on the address string (without the amqp://, with the ip
address, with a ~) but to no avail.
Notes:
- The proton c examples recv and send work just fine with my broker running on
the remote host and queue. I also tried the python and ruby bindings and they
worked.
- Also, the javascript bindings programs recv.js and send.js work fine together
when going through amqp://0.0.0.0
- When I start recv.js (using the default amqp://0.0.0.0), and send a message
using either the c send example or one of the other language bindings send
example programs, I get an error:
[eallen@redhat py]$ ./send.py Hello
[0xb1d650]:ERROR[-2] SASL header mismatch: ''
CONNECTION ERROR connection aborted (remote)
- I'm also getting the same connection refused error when running the
qpid-config.js example against a broker.
Could this be a problem with my build or environment? Or is this a bug?
Thanks,
-Ernie
Re: javascript bindings connecting to a qpid c++ broker
Ah, well that would explain it. Thanks!
- Original Message -
From: Fraser Adams fraser.ad...@blueyonder.co.uk
To: proton@qpid.apache.org
Sent: Wednesday, August 27, 2014 5:53:55 PM
Subject: Re: javascript bindings connecting to a qpid c++ broker
Hey Ernie,
You did read what I put in my previous response about the JavaScript
bindings using a *WebSocket* transport didn't you?
send.js and recv.js won't talk *directly* to the C++ broker, however
I've included a simple WebSocket-TCP Socket proxy.
If you stand up proxy.js that should proxy WS port 5673 to TCP port 5672
(you can change the defaults using command line parameters do node
proxy.js -h to see)
Something like
./proxy.js
Proxying ws - tcp
Forwarding port 5673 to 0.0.0.0:5672
./recv.js amqp://localhost:5673/eaq
./send.js -a amqp://localhost:5673/eaq
JavaScript is a bit funny like that :-) I mentioned on my TODO list that
I want to look at native TCP sockets for node, but the main reason for
doing JavaScript is really to get AMQP working in a browser and for that
WebSockets are pretty much the only reasonable transport - though Chrome
packaged apps allow TCP connections.
The Java Broker has a WebSocket transport so I can talk directly to
that, though that has its own quirks at the moment.
Almost certainly the fact that it's a WebSocket transport is the cause
of what you are seeing.
Frase
On 27/08/14 20:34, Ernest Allen wrote:
I'm getting a connection error when using the javascript bindings example
recv.js to receive a message through a qpid c++ broker.
I have a qpid c++ broker running on a remote host with a queue named eaq.
I'm attempting to start up the javascript example recv.js using the remote
address/queue, but I'm getting a connection error:
[eallen@redhat javascript]$ node recv.js amqp://my.remote.host.com/eaq
that._checkErrors = true
error = 3,111,ECONNREFUSED: Connection refused
{ [MessengerError: ECONNREFUSED: Connection refused]
name: 'MessengerError',
message: 'ECONNREFUSED: Connection refused' }
I've tried variations on the address string (without the amqp://, with the ip
address, with a ~) but to no avail.
Notes:
- The proton c examples recv and send work just fine with my broker running
on the remote host and queue. I also tried the python and ruby bindings and
they worked.
- Also, the javascript bindings programs recv.js and send.js work fine
together when going through amqp://0.0.0.0
- When I start recv.js (using the default amqp://0.0.0.0), and send a message
using either the c send example or one of the other language bindings
send example programs, I get an error:
[eallen@redhat py]$ ./send.py Hello
[0xb1d650]:ERROR[-2] SASL header mismatch: ''
CONNECTION ERROR connection aborted (remote)
- I'm also getting the same connection refused error when running the
qpid-config.js example against a broker.
Could this be a problem with my build or environment? Or is this a bug?
Thanks,
-Ernie
Re: proton javascript binding problem/question
Unfortunately, just changing the CMakeFiles.txt to
${PROJECT_SOURCE_DIR}/node_modules/qpid-proton/lib/
doesn't work. I get an error since the directory doesn't exist.
Shame it doesn't work, though I guess not surprising. I'm torn between
having the trailing slash or not, though you say that with it you get an
error. Perhaps getting an error is preferable? In your case you had a
silent issue originally, would an error have helped you track things
down more quickly?
It probably would have. I would have seen that the build didn't complete
instead of looking into possible environment problems with NODE_PATH. However,
once I found the file package.json and saw that it referenced
./lib/proton.js, the work-around was clear. However, this isn't really the
class of problem that you'd normally guard against.
When I create the lib dir before the build, everything works as expected.
The real solution is to create the empty lib dir in the Red Hat
repository. I'll look into doing that.
When I pull the source from the public svn link, everything builds just
fine.
I'm glad that the public svn link works, yay!!
Thanks for looking into this.
-Ernie Allen
No probs, it's been my personal little baby for a while now, so it's a
good thing that others are starting to take a peek. I think that quality
wise it's pretty decent and pretty well commented and I think that the
functional behaviour is largely all in place, but I haven't really
looked at optimisation/profiling yet.
What do you think of the approach that I've taken? My rationale for
compiling proton-c to JavaScript and using a thin (ish) binding layer
rather than doing a ground-up native JavaScript rewrite was primarily
about support. I figured that there was a lot of effort being put into
maintaining proton-c and that was what might be considered the
canonical or reference implementation, so tracking improvements would
end up being a pain with a separate code-base, whereas the binding to a
compiled library just needs to cover the public API and ultimately I
should pick up improvements to the core code transparently. If you
look at what I've done you'll hopefully see a startling similarity with
the SWIG bindings particularly the Python one. I spent some effort
contributing to emscripten too, putting things in place so that proton-c
compiles cleanly (if you look in
fadams-javascript-binding/bld/proton-c/bindings/javascript/examples you
will see recv-async.js and send-async.js, which are actually the results
of directly compiling recv-async.c and send-async.c. Those were the
first things I got working before I started on the binding stuff - I
figured that JavaScript programmers probably wouldn't enjoy using a C
API ;-)
IANAOPS (I am not an official Proton spokesperson) but I personally agree with
your approach. I doubt there is much of a downside to using a binding as
opposed to a native JavaScript implementation. And the upside is huge,
especially for bug fixes and enhancements.
I did a bit of work on the qpid-config port yesterday and it now does
everything the python one does (well except the xml binding support at
the moment as that loads a file). I'm currently trying to get it to work
with the Java Broker (see my Java Broker AMQP 1.0 support - is it by
default? post to the users list).
You've done some very impressive work here. I'm looking forward to trying it
out.
-Ernie
Re: proton javascript binding problem/question
-proton/
I'm seeing a directory called lib too, if you are seeing a file there
then I wonder if it's something weird with the git-svn bridge, but
I'm clutching at straws.
Hmmm re-reading your mail you say When I renamed the lib file to
proton.js, created a lib directory and moved the file under it, all
was well. that's suggesting that proton.js got copied to a file
called lib rather than into the directory called lib, so that second
CMake COMMAND:
COMMAND ${CMAKE_COMMAND}
-E copy
${CMAKE_CURRENT_BINARY_DIR}/proton.js
${PROJECT_SOURCE_DIR}/node_modules/qpid-proton/lib
certainly copies proton.js into node_modules/qpid-proton/lib for me on
my Linux box, but that is the most likely place for things behaving
differently between us...
BTW:
1. You shouldn't need to do a make install, just doing make should
copy this stuff I wanted to make sure a top level node_modules was in
place so that the tests and demos work without too much fuss (clearly
I was delusional :-D)
2. The spout.js and drain.js are currently *very* hacky (they just
print spout not implemented yet and exit ;-) the send.js and
recv.js and client.js and server.js should work pretty much as per the
normal proton examples of the same name and qpid-config.js is pretty
much complete (just the *generic* list mechanism to list AMQP 1.0 QMF
objects still to do).
3. One thing to bear in mind is that this stuff uses a *websocket*
transport, hopefully that's fairly obvious as it's JavaScript, but
perhaps not. I've tried it against the Java Broker's websocket
transport (though not for a couple of months and I've not tried
qpid-config.js with the QMF plugin yet either). If you want to talk to
the C++ broker I've included a simple node.js based WebSocket-TCP
proxy (proxy.js). On my TODO list is to put some more work into the
emscripten network library so that it's possible to configurably
choose websockets or TCP sockets (using the standard node net library)
Hopefully this gives you a few pointers, do let me know how you get
on, though I'm not getting a lot of computer time at the moment as I'm
digging loads of holes for fence posts :-(
Frase
On 22/08/14 19:34, Ernest Allen wrote:
I may have discovered a problem with the javascript bindings for
proton that I got from:
git+ssh://git.app.eng.bos.redhat.com/srv/git/rh-qpid-proton.git
branch origin/fadams-javascript-binding
After the build, I got an error when running the examples.
Specifically, this is what I got:
node spout.js
module.js:340
throw err;
^
Error: Cannot find module 'qpid-proton'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object.anonymous
(/home/ernie/jsbindings/rh-qpid-proton/examples/messenger/javascript/spout.js:24:14)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
After verifying my environment, I finally tracked it down to the
node_modules/qpid-proton directory. The build creates a file named
lib in that directory. It should create a directory named lib that
contains a file named proton.js in it.
When I renamed the lib file to proton.js, created a lib directory and
moved the file under it, all was well.
I did not do a make install, so I was just running the examples from
my git checkout directory.
I'm not familiar enough with the cmake files to track this down any
further, so I'm hoping someone here can provide the fix.
-Ernie Allen
proton javascript binding problem/question
I may have discovered a problem with the javascript bindings for proton that I got from: git+ssh://git.app.eng.bos.redhat.com/srv/git/rh-qpid-proton.git branch origin/fadams-javascript-binding After the build, I got an error when running the examples. Specifically, this is what I got: node spout.js module.js:340 throw err; ^ Error: Cannot find module 'qpid-proton' at Function.Module._resolveFilename (module.js:338:15) at Function.Module._load (module.js:280:25) at Module.require (module.js:364:17) at require (module.js:380:17) at Object.anonymous (/home/ernie/jsbindings/rh-qpid-proton/examples/messenger/javascript/spout.js:24:14) at Module._compile (module.js:456:26) at Object.Module._extensions..js (module.js:474:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Function.Module.runMain (module.js:497:10) After verifying my environment, I finally tracked it down to the node_modules/qpid-proton directory. The build creates a file named lib in that directory. It should create a directory named lib that contains a file named proton.js in it. When I renamed the lib file to proton.js, created a lib directory and moved the file under it, all was well. I did not do a make install, so I was just running the examples from my git checkout directory. I'm not familiar enough with the cmake files to track this down any further, so I'm hoping someone here can provide the fix. -Ernie Allen
map values in message body
Does proton messenger support sending complex maps as the message body? That
is, maps whose values are other maps or lists and not just strings?
When I send a message and the body is encoded as qpid::proton::MAP, the map
values appear to have been converted to strings.
For example (in perl)
In the sender
$msg-set_body({foo = 2, bar = [3]}, qpid::proton::MAP);
In the receiver when I dump the body, I'm getting
$VAR1 = \{
'bar' = 'ARRAY(0xa2aad0)',
'foo' = '2'
};
It appears that the array has been converted to the string 'ARRAY(0x...)'. And
the 2 is now a string '2'.
I don't think this is just the way the receiver is dumping the message body as
I'm not able to convert the results to the expected type or variable.
When I send a perl HASH instead of an array as one of the values, it is coming
through as the string'HASH(0x)'.
-Ernie
[jira] [Created] (PROTON-325) Python lists are truncated when None is encountered
Ernest Allen created PROTON-325: --- Summary: Python lists are truncated when None is encountered Key: PROTON-325 URL: https://issues.apache.org/jira/browse/PROTON-325 Project: Qpid Proton Issue Type: Bug Components: proton-c Reporter: Ernest Allen When receiving a message that has a list with a None, all subsequent values in the list are lost. To reproduce: - modify program send.py by adding msg.properties = [one, two, None, three, four] before the mng.put(msg) line - launch example program recv.py - run modified send.py and notice the list truncation -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
