Re: Messaging protocol with Stratos python cartridge agent

2014-09-04 Thread Imesh Gunaratne
Hi, Thank you very much Paul! I completely agree with you and +1 for the refactoring of the topic hierarchy. Regarding the reliable delivery in MQTT, I just checked v3.1 specification [1] and it has three levels of QoS: *Three qualities of service for message delivery:* *- "At most once", where

Re: Messaging protocol with Stratos python cartridge agent

2014-09-04 Thread Paul Fremantle
Folks I am not suggesting we move to MQTT necessarily. However, if we want a simple protocol with easy native python support, its not bad. STOMP is also good. It is pretty limiting that MQTT doesn't support headers. My only comment was that we are not actually using headers properly here. The ref

Re: Messaging protocol with Stratos python cartridge agent

2014-09-04 Thread Imesh Gunaratne
> > Based on the above discussion, I guess we need to change the the existing > Stratos messaging model to support the Pub/Sub model and MQTT protocol. > I think this statement is not correct, currently we use Pub/Sub model in Stratos messaging system. AFAIU we are looking at two concerns in this

Re: Messaging protocol with Stratos python cartridge agent

2014-09-03 Thread Lakmal Warusawithana
IMHO, we should do this, I know it is costly but need to correct without going further. I can help on finalized the event schema structure. On Thu, Sep 4, 2014 at 10:18 AM, Gayan Gunarathne wrote: > > Thanks Paul. > > Based on the above discussion, I guess we need to change the the existing > S

Re: Messaging protocol with Stratos python cartridge agent

2014-09-03 Thread Gayan Gunarathne
Thanks Paul. Based on the above discussion, I guess we need to change the the existing Stratos messaging model to support the Pub/Sub model and MQTT protocol.AFAIU following areas we need to modify in the Stratos messaging model. Step 1 : We need to remove the header based message distinguished.

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Paul Fremantle
Gayan Normally you would structure your topic hierarchy to be capture the structure of events and the schema e.g. topic /metrics/datacentre1/system1/cpu message { json format for cpu data } Paul On 2 September 2014 11:40, Gayan Gunarathne wrote: > Hi Paul, > > AFAIU with the existing message

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Gayan Gunarathne
Hi Paul, AFAIU with the existing message design and with your points with the Pub/Sub ,+1 for improve our Startos messaging model as far as we can as it will improve the performance as well as standards.As far as I understood the current messaging model , we can thinking about following two sugges

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Paul Fremantle
Gayan I don't understand this. I think this is a mistake on how to use Pub/Sub. The headers timestamp and expiry should probably be in the message (not sure) The destination is part of the pub/sub network The event-class should be implicitly based on the topic. In other words in a pub/sub network

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Dinesh Bandara
We can use custom JSON serializer, deserializer from Stratos end to avoid the message header. On Tue, Sep 2, 2014 at 2:14 PM, Gayan Gunarathne wrote: > Hi Paul, > > Please find the example of existing message header and body.AFAIK > "event-class-name" is the header value used for distinguish th

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Gayan Gunarathne
Hi Paul, Please find the example of existing message header and body.AFAIK "event-class-name" is the header value used for distinguish the messages. header: key expires , value 0 header: key timestamp , value 1409646637558 header: key destination , value /topic/topology header: key persistent , v

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Paul Fremantle
Can you post an example of the headers and body? Paul On 2 September 2014 09:17, Gayan Gunarathne wrote: > Hi Paul, > > Currently we have existing java based message publisher with the headers. > Here we try to use those published messages with the python client. If we > use JSON structured me

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Gayan Gunarathne
Hi Paul, Currently we have existing java based message publisher with the headers. Here we try to use those published messages with the python client. If we use JSON structured message with headers enclosed we have to change all other current implementations as well. AFAIK We will distinguished t

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Paul Fremantle
What headers do you need to pass? The way of doing headers with MQTT is to use a structured message (e.g. JSON) and put some headers into the JSON. Paul On 2 September 2014 07:58, Gayan Gunarathne wrote: > > We did some analysis regarding the MQTT protocol as well.Following are our > findings

Re: Messaging protocol with Stratos python cartridge agent

2014-09-02 Thread Gayan Gunarathne
We did some analysis regarding the MQTT protocol as well.Following are our findings regarding the MQTT. MQTT is low bandwidth consumption messaging protocol which supports pub/sub protocol with the following limitations. - It has fixed header (2 bytes) . Specifications mentioning that it can

Re: Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Gayan Gunarathne
Thanks Paul. We will check on this paho-mgtt python library as well. Thanks, Gayan On Mon, Sep 1, 2014 at 4:17 PM, Paul Fremantle wrote: > Python has excellent pure python MQTT support. > > > pip install paho-mqtt > > https://pypi.python.org/pypi/paho-mqtt > > Paul > > > On 1 September 2014

Re: Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Dinesh Bandara
+1 for STOMP. As per the [2] it can be used to communicate with RabbitMQ as well. AFAIK even with the current cartridge agent implementation (java) we do not use AMQP as the messaging protocol. And the same time we can do some research on MQTT as well [1] [1] http://mqtt.org/wiki/doku.php/librari

Re: Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Paul Fremantle
Python has excellent pure python MQTT support. > pip install paho-mqtt https://pypi.python.org/pypi/paho-mqtt Paul On 1 September 2014 11:40, Gayan Gunarathne wrote: > Hi Paul, > > Here we try to connect the our existing Apache ActiveMQ with the pure > python.(We try to replace our java cart

Re: Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Gayan Gunarathne
Hi Paul, Here we try to connect the our existing Apache ActiveMQ with the pure python.(We try to replace our java cartridge agent with the python as after that we can remove the java dependency from the cartridge itself). In the initial research we didn't notice such a pure python libraries with t

Re: Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Jayanath Liyanage
thanks On Mon, Sep 1, 2014 at 3:40 PM, Gayan Gunarathne wrote: > > Hi All, > > As per our initial research on the python based messaging implementation > of python cartridge agent and discussion[1] we can summarize our findings > as following. > > *1.AMQP* > > Apache ActiveMQ used the AMQP 1.0

Re: Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Paul Fremantle
Have you looked at mqtt? Paul On 1 Sep 2014 12:10, "Gayan Gunarathne" wrote: > > Hi All, > > As per our initial research on the python based messaging implementation > of python cartridge agent and discussion[1] we can summarize our findings > as following. > > *1.AMQP* > > Apache ActiveMQ used

Messaging protocol with Stratos python cartridge agent

2014-09-01 Thread Gayan Gunarathne
Hi All, As per our initial research on the python based messaging implementation of python cartridge agent and discussion[1] we can summarize our findings as following. *1.AMQP* Apache ActiveMQ used the AMQP 1.0 protocol. We found C based client library (proton) [2] which can be used with python