Re: Is Proton a lightweight alternative to Qpid?
A couple of things... I believe cmake places an 'rpath' on the libraries when they are built. This is removed when you run 'make install' and the libraries are copied to their installed location. The .so and .so.1 files are soft links to the .so.1.0.0 file. Depending on how you copy them, they may act in unexpected ways. You may need to set the environment variable LD_LIBRARY_PATH to your local directory so the loader can find the library file if it's not in a standard install location. I hope this helps, -Ted On 01/22/2013 06:02 PM, Eagy, Taylor wrote: Ted, This is a follow up of our earlier discussion. After building Proton using cmake as described, I copied and placed those four files into another directory: ~/myproject/proton/proton.py ~/myproject/proton/cproton.py ~/myproject/proton/_cproton.so ~/myproject/proton/libqpid-proton.so.1.0.0 ~/myproject/proton/libqpid-proton.so.1 I found that I also needed a link libqpid-proton.so.1 to point to libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build directory, when I try to import proton.py, it spits out: import proton Traceback (most recent call last): File stdin, line 1, in module File proton.py, line 33, in module from cproton import * File cproton.py, line 26, in module _cproton = swig_import_helper() File cproton.py, line 22, in swig_import_helper _mod = imp.load_module('_cproton', fp, pathname, description) ImportError: libqpid-proton.so.1: cannot open shared object file: No such file or directory Is there something in cmake that is adding absolute paths to the library files? When I built again, importing proton worked fine. Is there a cmake variable that I need to turn off to remove any references to the build directory? The reason I need to do this is because when I copy it over to a clean install of the same Linux version, it spits out that import error. Thanks, Taylor From: Eagy, Taylor [te...@blackbirdtech.com] Sent: Wednesday, January 16, 2013 12:13 PM To: proton@qpid.apache.org Subject: RE: Is Proton a lightweight alternative to Qpid? Rafael and Ted, Thanks for your help on this. I'm excited to see that proton is getting a Windows port since I wasn't able to build it in VS2012 successfully. Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:13 PM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? If you run cmake this way you can build the minimal code needed for just the proton library and its python bindings: cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF -DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree A quick test on my system shows that a make install based on the above build works out to about 1.4MB. Stripping out header files and some package config stuff would get you down to about 1.2MB if you want to go super barebones. --Rafael On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote: Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
RE: Is Proton a lightweight alternative to Qpid?
Thanks a lot Ted. It's working now. I appreciate all your help. Thanks, Taylor From: Ted Ross [tr...@redhat.com] Sent: Wednesday, January 23, 2013 8:26 AM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? A couple of things... I believe cmake places an 'rpath' on the libraries when they are built. This is removed when you run 'make install' and the libraries are copied to their installed location. The .so and .so.1 files are soft links to the .so.1.0.0 file. Depending on how you copy them, they may act in unexpected ways. You may need to set the environment variable LD_LIBRARY_PATH to your local directory so the loader can find the library file if it's not in a standard install location. I hope this helps, -Ted On 01/22/2013 06:02 PM, Eagy, Taylor wrote: Ted, This is a follow up of our earlier discussion. After building Proton using cmake as described, I copied and placed those four files into another directory: ~/myproject/proton/proton.py ~/myproject/proton/cproton.py ~/myproject/proton/_cproton.so ~/myproject/proton/libqpid-proton.so.1.0.0 ~/myproject/proton/libqpid-proton.so.1 I found that I also needed a link libqpid-proton.so.1 to point to libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build directory, when I try to import proton.py, it spits out: import proton Traceback (most recent call last): File stdin, line 1, in module File proton.py, line 33, in module from cproton import * File cproton.py, line 26, in module _cproton = swig_import_helper() File cproton.py, line 22, in swig_import_helper _mod = imp.load_module('_cproton', fp, pathname, description) ImportError: libqpid-proton.so.1: cannot open shared object file: No such file or directory Is there something in cmake that is adding absolute paths to the library files? When I built again, importing proton worked fine. Is there a cmake variable that I need to turn off to remove any references to the build directory? The reason I need to do this is because when I copy it over to a clean install of the same Linux version, it spits out that import error. Thanks, Taylor From: Eagy, Taylor [te...@blackbirdtech.com] Sent: Wednesday, January 16, 2013 12:13 PM To: proton@qpid.apache.org Subject: RE: Is Proton a lightweight alternative to Qpid? Rafael and Ted, Thanks for your help on this. I'm excited to see that proton is getting a Windows port since I wasn't able to build it in VS2012 successfully. Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:13 PM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? If you run cmake this way you can build the minimal code needed for just the proton library and its python bindings: cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF -DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree A quick test on my system shows that a make install based on the above build works out to about 1.4MB. Stripping out header files and some package config stuff would get you down to about 1.2MB if you want to go super barebones. --Rafael On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote: Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
RE: Is Proton a lightweight alternative to Qpid?
Ted, This is a follow up of our earlier discussion. After building Proton using cmake as described, I copied and placed those four files into another directory: ~/myproject/proton/proton.py ~/myproject/proton/cproton.py ~/myproject/proton/_cproton.so ~/myproject/proton/libqpid-proton.so.1.0.0 ~/myproject/proton/libqpid-proton.so.1 I found that I also needed a link libqpid-proton.so.1 to point to libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build directory, when I try to import proton.py, it spits out: import proton Traceback (most recent call last): File stdin, line 1, in module File proton.py, line 33, in module from cproton import * File cproton.py, line 26, in module _cproton = swig_import_helper() File cproton.py, line 22, in swig_import_helper _mod = imp.load_module('_cproton', fp, pathname, description) ImportError: libqpid-proton.so.1: cannot open shared object file: No such file or directory Is there something in cmake that is adding absolute paths to the library files? When I built again, importing proton worked fine. Is there a cmake variable that I need to turn off to remove any references to the build directory? The reason I need to do this is because when I copy it over to a clean install of the same Linux version, it spits out that import error. Thanks, Taylor From: Eagy, Taylor [te...@blackbirdtech.com] Sent: Wednesday, January 16, 2013 12:13 PM To: proton@qpid.apache.org Subject: RE: Is Proton a lightweight alternative to Qpid? Rafael and Ted, Thanks for your help on this. I'm excited to see that proton is getting a Windows port since I wasn't able to build it in VS2012 successfully. Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:13 PM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? If you run cmake this way you can build the minimal code needed for just the proton library and its python bindings: cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF -DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree A quick test on my system shows that a make install based on the above build works out to about 1.4MB. Stripping out header files and some package config stuff would get you down to about 1.2MB if you want to go super barebones. --Rafael On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote: Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
RE: Is Proton a lightweight alternative to Qpid?
Rafael and Ted, Thanks for your help on this. I'm excited to see that proton is getting a Windows port since I wasn't able to build it in VS2012 successfully. Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:13 PM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? If you run cmake this way you can build the minimal code needed for just the proton library and its python bindings: cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF -DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree A quick test on my system shows that a make install based on the above build works out to about 1.4MB. Stripping out header files and some package config stuff would get you down to about 1.2MB if you want to go super barebones. --Rafael On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote: Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
RE: Is Proton a lightweight alternative to Qpid?
For anyone interested in a Windows port of the qpid amqp 1.0, I have Visual Studio 10 snapshots available on my github webpage. (Visual Studio port of proton and a qpid port loading proton) I placed my Visual Studio port of the qpid project on my github website. https://github.com/MaryDHinton/qpid/tree/qpidWinSnap You can download a zip file by selecting the ZIP button on the Webpage. To compile and run this code you will also need to download, compile, and run the Snapshot code of the proton Windows port. It is located at: https://github.com/MaryDHinton/qpid-proton/tree/protonWinSnap Both projects are set up for Visual Studio 10 and debug mode. I don't know any reason why they wouldn't work for Visual Studio 12. == Proton === To compile and run the proton port code, all you should need to do is: Select the zip button on the page above and extract the code. Select the proton.sln file in the proton-c folder and open it in Visual Studio 2010. Compile the qpid-proton and proton projects. Go to the Debug folder and run proton (the debug version is the only mode set up, the release will have to be set up). I set up an environment variable for the proton after building the proton dll, then added this environment variable to the qpid project in the C/C++ properties page (Additional Include Directories). Set the environment variable PROTON_INCLUDE --- to your environment. Mine is set to: C:\proton\protonWinSnap\qpid-proton\proton-c\include;C:\proton\protonWinSnap \qpid-proton\proton-c I also have the path to proton executable set in my environment path c:\qpid\qpid\proton\proton-c\Debug After the proton dll and proton executable are compiled, you can run proton in server and client mode. Server mode: Open up a dialog box and change to the proton path and run proton Client mode: Open up a second dialog box and change to the proton path and run proton -c 127.0.0.1 You should be able to see messages passed between a proton client and server. QPID AMQP 1.0 === For QPID, I used Boost 1.51.0. I have a solution file (qpid-cpp-Snap.sln) for the qpid port that includes only the projects to compile, run, and test the qpid amqp 1.0 for Visual Studio. projects included: amqp amqpc hello_world qpid-client-test qpidbroker qpidclient qpidcommon qpidd qpidmessaqing qpidtypes You may need to check the Visual Studio Properties for the 10 projects in this solution and make sure the paths are correct on your pc. I used the path C:\qpidWin to put the extracted files from the zip. After building both the proton Windows port and the QPID port for the server and client, the amqp 1.0 can be loaded in the server with the following command from the path where your dlls and executables are located. The path setup in this solution is: c:\qpidWin\Debug server c:\qpidWin\qpidWinSnap\qpid\qpid\cpp\output\Debugqpidd --load-module amqpd --auth no clients set the environment variable QPID_LOAD_MODULE to amqpcd hello_world client hello_world qpid-client-test client qpid-client-test --verbose response: Opened connection. Opened session. Declared exchange. Declared queue. Bound queue to exchange. Published message: abcdefghijklmnop... Received the exepected message. Closed session. Closed connection. If you are using Norton's antivirus, you may have to disable checking for the Suspicious.Cloud.7.F signature. Original email on these snapshots was sent as: Windows Snapshot of QPID that loads the AMQP 1.0 module (proton windows port) on 12/18/2012 to d...@qpid.apache.org and Proton Port for Windows on 11/14/2012 proton@qpid.apache.org So both ports need to be rebased. I'm planning to do this soon, since a new release is out for proton Thanks, Mary Hinton -Original Message- From: Eagy, Taylor [mailto:te...@blackbirdtech.com] Sent: Wednesday, January 16, 2013 12:13 PM To:
Re: Is Proton a lightweight alternative to Qpid?
On Mon, Jan 14, 2013 at 11:14 AM, Eagy, Taylor te...@blackbirdtech.comwrote: Hi guys, I've been using Qpid for the past several months and I really like it. However, I've mainly just been using it to pass messages between several Python processes running on the same machine, so using Qpid is probably overkill. Then I noticed Proton and got excited. Ideally I'm looking for a fast, lightweight, and portable queueing library preferrably in Python. Are there any roadmap plans to create a Proton Python broker/engine? I looked at RabbitMQ, but read the performance wasn't as good as Qpid. I was looking into the examples and noticed you could implement your own queueing server in Python, but it's not exactly portable since I'd have to build it for Linux and Windows. Hi, To answer the question in your subject first, Proton isn't an alternative to Qpid per/se, rather it's a component of Qpid. The latest release of the cpp broker uses proton to provide AMQP 1.0 support, and we plan to use it in future releases of the Java broker also. That said, Proton is definitely intended to be used independently of either the cpp or Java broker, and can be used without either (i.e. peer to peer), so in that sense the answer to your question is yes. There is definitely interest in building a lightweight queuing component that works well with Proton and can be flexibly deployed in a variety of topologies, and even dynamically/transparently redeployed at runtime. A python prototype has been discussed as a starting point for some of this work, and there has been other work ongoing both in terms of re-factoring the cpp broker and in terms of prototyping new servers that may ultimately contribute. Why do you think your own queuing server built in python wouldn't be portable? Can you describe a little bit more about your scenario, e.g. do you need persistence, transactions, etc ...? Do you have any particular performance requirements? Can you describe your messaging topology at all? --Rafael
Re: Is Proton a lightweight alternative to Qpid?
Taylor, Another possibility to look at is using ActiveMQ as your broker. They've added an AMQP transport (based on Proton-J) to their project recently. -Ted On 01/15/2013 12:37 PM, Eagy, Taylor wrote: Rafael, Thanks for responding. The only reason why I said it wouldn't be portable is because when I saw the CMake files I thought I'd have to build it to bind C functions from the engine to Python. However, if I can perform the p2p messaging just using the proton.py then that would work. My scenario is the following: I have three Python modules running. One is a web service that takes incoming requests and places the request data as a message on an incoming queue, then another Python service listens on the incoming queue and processes the data in the messages. After it finishes processing the data it passes the results as another message onto an outgoing queue which is then grabbed by the last Python service and sent back. Previously I was using the Java Qpid broker because I need persistence enabled so that if the Qpid broker or one of these Python services were to fail during a queue transaction, the services, when restarted, would be able to pickup the durable message and continue the data flow. While it works great with the Qpid broker, the main issue is that the Qpid broker is just too heavy. I need a fast lightweight version that still offers the basic persistence (I was using Derby store) and ideally written in Python since all of my code is in Python. I don't have any specific performance requirements other than saying the faster the better. Right now all of the queueing is done locally between these services, but I would like to have the ability to extend it easily to work over networked machines which I know Proton can do. Do you think it's overkill to use something like Qpid or Proton in my scenario? Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:44 AM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? On Mon, Jan 14, 2013 at 11:14 AM, Eagy, Taylor te...@blackbirdtech.comwrote: Hi guys, I've been using Qpid for the past several months and I really like it. However, I've mainly just been using it to pass messages between several Python processes running on the same machine, so using Qpid is probably overkill. Then I noticed Proton and got excited. Ideally I'm looking for a fast, lightweight, and portable queueing library preferrably in Python. Are there any roadmap plans to create a Proton Python broker/engine? I looked at RabbitMQ, but read the performance wasn't as good as Qpid. I was looking into the examples and noticed you could implement your own queueing server in Python, but it's not exactly portable since I'd have to build it for Linux and Windows. Hi, To answer the question in your subject first, Proton isn't an alternative to Qpid per/se, rather it's a component of Qpid. The latest release of the cpp broker uses proton to provide AMQP 1.0 support, and we plan to use it in future releases of the Java broker also. That said, Proton is definitely intended to be used independently of either the cpp or Java broker, and can be used without either (i.e. peer to peer), so in that sense the answer to your question is yes. There is definitely interest in building a lightweight queuing component that works well with Proton and can be flexibly deployed in a variety of topologies, and even dynamically/transparently redeployed at runtime. A python prototype has been discussed as a starting point for some of this work, and there has been other work ongoing both in terms of re-factoring the cpp broker and in terms of prototyping new servers that may ultimately contribute. Why do you think your own queuing server built in python wouldn't be portable? Can you describe a little bit more about your scenario, e.g. do you need persistence, transactions, etc ...? Do you have any particular performance requirements? Can you describe your messaging topology at all? --Rafael
Re: Is Proton a lightweight alternative to Qpid?
Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor