We can use custom JSON serializer, deserializer from Stratos end to avoid
the message header.
On Tue, Sep 2, 2014 at 2:14 PM, Gayan Gunarathne <gay...@wso2.com> wrote:
> Hi Paul,
>
> Please find the example of existing message header and body.AFAIK
> "event-class-name" is the header value used for distinguish the messages.
>
> header: key expires , value 0
> header: key timestamp , value 1409646637558
> header: key destination , value /topic/topology
> header: key persistent , value true
> header: key priority , value 4
> header: key event-class-name , value
> org.apache.stratos.messaging.event.topology.CompleteTopologyEvent
>
> Message body:
>
>
> {"topology":{"serviceMap":{"gateway":{"serviceName":"gateway","serviceType":"SingleTenant","clusterIdClusterMap":{"gatewaygateway.am.wso2.com.dom":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","autoscalePolicyName":"auto-scl-pol1","deploymentPolicyName":"
> dep-pol1","hostNames":["foo.org"],"tenantRange":"1-*","isLbCluster":false,"memberMap":{"gatewaygateway.am.wso2.com.dom-member-2":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"gatewaygateway.am.wso2.com.dom-member-2","portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}},"memberPublicIp":"127.0.0.1
> public","status":"Activated","memberIp":"127.0.0.1
> private","properties":{"mykey1xxxxxxxxx":"myvalweccccccccccccccc"}},"gatewaygateway.am.wso2.com.dom-member-1":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"gatewaygateway.am.wso2.com.dom-member-1","portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}},"memberPublicIp":"127.0.0.1
> public","status":"Activated","memberIp":"127.0.0.1
> private","properties":{"mykey1xxxxxxxxx":"myvalweccccccccccccccc"}},"gatewaygateway.am.wso2.com.dom5863cf8d-2746-4be1-91e9-da98c2bcb5a9":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"gatewaygateway.am.wso2.com.dom5863cf8d-2746-4be1-91e9-da98c2bcb5a9","portMap":{},"memberPublicIp":"public
> IP","memberIp":"private
> IP"},"lb-member-id-1":{"serviceName":"lb","clusterId":"lbisuruh.lk.domain","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"lb-member-id-1","portMap":{},"memberPublicIp":"public
> IP of apistore","memberIp":"private IP of
> apistore"}},"properties":{"mykey1weeee":"myvalweeeeeeeee1","mykey1tttttttt":"myvalweetttttttttttt1"}}},"portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}},"properties":{"mykey1weeee":"myvalweeeeeeeee1","mykey1tttttttt":"myvalweetttttttttttt1"}},"apistore":{"serviceName":"apistore","serviceType":"SingleTenant","clusterIdClusterMap":{"apistorestore.am.wso2.com.doma":{"serviceName":"apistore","clusterId":"apistorestore.am.wso2.com.doma","autoscalePolicyName":"auto-scl-pol1","deploymentPolicyName":"
> dep-pol1","hostNames":["foo.org"],"tenantRange":"1-*","isLbCluster":false,"memberMap":{"apistorestore.am.wso2.com.doma298e2959-a3b4-45a0-a11f-7671606158ed":{"serviceName":"apistore","clusterId":"apistorestore.am.wso2.com.doma","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"apistorestore.am.wso2.com.doma298e2959-a3b4-45a0-a11f-7671606158ed","portMap":{},"memberPublicIp":"pub
> ip","status":"Activated","memberIp":"ip","lbClusterId":"lbmongolb594224131.mongolb.dom"}}}},"portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}}},"lb":{"serviceName":"lb","serviceType":"SingleTenant","clusterIdClusterMap":{"lbisuruh.lk.domain":{"serviceName":"lb","clusterId":"lbisuruh.lk.domain","autoscalePolicyName":"auto-scl-pol1","deploymentPolicyName":"
> dep-pol1","hostNames":["foo.org
> "],"tenantRange":"1-*","isLbCluster":false,"memberMap":{}}},"portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}}}},"initialized":false}}
>
>
> Thanks,
> Gayan
>
>
>
> On Tue, Sep 2, 2014 at 1:49 PM, Paul Fremantle <p...@wso2.com> wrote:
>
>> Can you post an example of the headers and body?
>>
>> Paul
>>
>>
>> On 2 September 2014 09:17, Gayan Gunarathne <gay...@wso2.com> wrote:
>>
>>> Hi Paul,
>>>
>>> Currently we have existing java based message publisher with the
>>> headers. Here we try to use those published messages with the python
>>> client. If we use JSON structured message with headers enclosed we have to
>>> change all other current implementations as well. AFAIK We will
>>> distinguished the messages with their header value.So my point is if we
>>> used this sort of JSON structure for messages, we need to change all the
>>> current messaging model.
>>>
>>> OTOH with the Stomp protocol with python client we were able to get
>>> those header values and used those messages as it is.
>>>
>>> Thanks,
>>> Gayan
>>>
>>>
>>> On Tue, Sep 2, 2014 at 1:04 PM, Paul Fremantle <p...@wso2.com> wrote:
>>>
>>>> What headers do you need to pass?
>>>>
>>>> The way of doing headers with MQTT is to use a structured message (e.g.
>>>> JSON) and put some headers into the JSON.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 2 September 2014 07:58, Gayan Gunarathne <gay...@wso2.com> wrote:
>>>>
>>>>>
>>>>> We did some analysis regarding the MQTT protocol as well.Following are
>>>>> our findings regarding the MQTT.
>>>>>
>>>>> MQTT is low bandwidth consumption messaging protocol which
>>>>> supports pub/sub protocol with the following limitations.
>>>>>
>>>>> - It has fixed header (2 bytes) . Specifications mentioning that
>>>>> it can be extended.But AFAIU clients[1] didn't support for this header
>>>>> extension. We have checked the MQTT with the messages published with
>>>>> headers and we didn't receive the header on those.Only receive the
>>>>> message
>>>>> body.
>>>>> - The payload size for a MQTT message is 256MB.I guess design
>>>>> point for MQTT usage is targeted at more constrained devices.[3]. MQTT
>>>>> is
>>>>> better at a high volume of low size messages.
>>>>> - It is supports only topic.This will not be a issue currently in
>>>>> our scenario as we used only topic currently AFAIK.[4]
>>>>>
>>>>>
>>>>> I guess it build with above limitations and it is very good choice for
>>>>> the mobile and small application as it is very lightweight and low
>>>>> power.[5] WDYT?
>>>>>
>>>>> Please share your thoughts on this.
>>>>>
>>>>> [1] https://pypi.python.org/pypi/paho-mqtt
>>>>> [2[ https://github.com/iwanbk/nyamuk
>>>>> [3] http://stephendnicholas.com/archives/1217
>>>>> [4] http://www.mqtechconference.com/sessions_v2013/MQTC_v2013_MQTT.pdf
>>>>> [5]
>>>>> http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Gayan
>>>>>
>>>>>
>>>>> On Mon, Sep 1, 2014 at 4:32 PM, Gayan Gunarathne <gay...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Paul.
>>>>>>
>>>>>> We will check on this paho-mgtt python library as well.
>>>>>>
>>>>>> Thanks,
>>>>>> Gayan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 1, 2014 at 4:17 PM, Paul Fremantle <p...@wso2.com> wrote:
>>>>>>
>>>>>>> Python has excellent pure python MQTT support.
>>>>>>>
>>>>>>> > pip install paho-mqtt
>>>>>>>
>>>>>>> https://pypi.python.org/pypi/paho-mqtt
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 1 September 2014 11:40, Gayan Gunarathne <gay...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Paul,
>>>>>>>>
>>>>>>>> Here we try to connect the our existing Apache ActiveMQ with the
>>>>>>>> pure python.(We try to replace our java cartridge agent with the
>>>>>>>> python as after that we can remove the java dependency from the
>>>>>>>> cartridge
>>>>>>>> itself).
>>>>>>>> In the initial research we didn't notice such a pure python
>>>>>>>> libraries with the MQTT which can connect with the ActiveMQ. Also
>>>>>>>> with the
>>>>>>>> initial reading I guess they are recommend it for the mobile
>>>>>>>> applications
>>>>>>>> as its low bandwidth consumption.Also I am noticed lot of web socket
>>>>>>>> based
>>>>>>>> applications are developed through MQTT.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Gayan
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Sep 1, 2014 at 3:42 PM, Paul Fremantle <p...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Have you looked at mqtt?
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>> On 1 Sep 2014 12:10, "Gayan Gunarathne" <gay...@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> As per our initial research on the python based messaging
>>>>>>>>>> implementation of python cartridge agent and discussion[1] we can
>>>>>>>>>> summarize
>>>>>>>>>> our findings as following.
>>>>>>>>>>
>>>>>>>>>> *1.AMQP*
>>>>>>>>>>
>>>>>>>>>> Apache ActiveMQ used the AMQP 1.0 protocol. We found C based
>>>>>>>>>> client library (proton) [2] which can be used with python.
>>>>>>>>>> But it is a platform dependent library and every time we need to
>>>>>>>>>> install the libraries based on the architecture and OS version.IMO
>>>>>>>>>> this
>>>>>>>>>> will be difficult us to manage as we need to install the libraries
>>>>>>>>>> based on
>>>>>>>>>> the platform.
>>>>>>>>>>
>>>>>>>>>> Other python libraries(txAMQP ,pika etc) supports AMQP 0.9 which
>>>>>>>>>> fails to connect with the ActiveMQ.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *2.STOMP*
>>>>>>>>>>
>>>>>>>>>> We can connect to the ActiveMQ with the STOMP protocol by using
>>>>>>>>>> the STOMP python client library[3].IMO we can used STOMP with
>>>>>>>>>> ActiveMQ
>>>>>>>>>> without worry about the platform based library installation.By
>>>>>>>>>> considering
>>>>>>>>>> our scope of the cartridge agent I think it is better to have a
>>>>>>>>>> platform
>>>>>>>>>> independence cartridge as far as we can. Then, IMO STOMP based
>>>>>>>>>> messaging
>>>>>>>>>> queue implementation will be more manageable when we increase our
>>>>>>>>>> number
>>>>>>>>>> of platform supported.
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> When comparing the performance of theses two protocols it looks
>>>>>>>>>> like there are no such huge difference[4].I guess the performance
>>>>>>>>>> also
>>>>>>>>>> tightly coupled with the message broker you are using as well.OTOH
>>>>>>>>>> Apache
>>>>>>>>>> ActiveMQ recommended way also to used the STOMP with the python[5].
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please share your thoughts on this.
>>>>>>>>>>
>>>>>>>>>> [1] Stratos Python Cartridge Agent
>>>>>>>>>> [2] https://github.com/apache/qpid-proton
>>>>>>>>>> [3] https://github.com/jasonrbriggs/stomp.py
>>>>>>>>>> [4]
>>>>>>>>>> http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html
>>>>>>>>>> [5] http://activemq.apache.org/python.html
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Gayan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Gayan Gunarathne
>>>>>>>>>> Technical Lead
>>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>>> email : gay...@wso2.com | mobile : +94 766819985
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards,
>>>>>>>>
>>>>>>>> Gayan Gunarathne
>>>>>>>> Technical Lead
>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>> email : gay...@wso2.com | mobile : +94 766819985
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Paul Fremantle
>>>>>>> CTO and Co-Founder, WSO2
>>>>>>> OASIS WS-RX TC Co-chair, Apache Member
>>>>>>>
>>>>>>> UK: +44 207 096 0336
>>>>>>>
>>>>>>> blog: http://pzf.fremantle.org
>>>>>>> twitter.com/pzfreo
>>>>>>> p...@wso2.com
>>>>>>>
>>>>>>> wso2.com Lean Enterprise Middleware
>>>>>>>
>>>>>>> Disclaimer: This communication may contain privileged or other
>>>>>>> confidential information and is intended exclusively for the
>>>>>>> addressee/s.
>>>>>>> If you are not the intended recipient/s, or believe that you may have
>>>>>>> received this communication in error, please reply to the sender
>>>>>>> indicating
>>>>>>> that fact and delete the copy you received and in addition, you should
>>>>>>> not
>>>>>>> print, copy, retransmit, disseminate, or otherwise use the information
>>>>>>> contained in this communication. Internet communications cannot be
>>>>>>> guaranteed to be timely, secure, error or virus-free. The sender does
>>>>>>> not
>>>>>>> accept liability for any errors or omissions.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>>
>>>>>> Gayan Gunarathne
>>>>>> Technical Lead
>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>> email : gay...@wso2.com | mobile : +94 766819985
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>>
>>>>> Gayan Gunarathne
>>>>> Technical Lead
>>>>> WSO2 Inc. (http://wso2.com)
>>>>> email : gay...@wso2.com | mobile : +94 766819985
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Paul Fremantle
>>>> CTO and Co-Founder, WSO2
>>>> OASIS WS-RX TC Co-chair, Apache Member
>>>>
>>>> UK: +44 207 096 0336
>>>>
>>>> blog: http://pzf.fremantle.org
>>>> twitter.com/pzfreo
>>>> p...@wso2.com
>>>>
>>>> wso2.com Lean Enterprise Middleware
>>>>
>>>> Disclaimer: This communication may contain privileged or other
>>>> confidential information and is intended exclusively for the addressee/s.
>>>> If you are not the intended recipient/s, or believe that you may have
>>>> received this communication in error, please reply to the sender indicating
>>>> that fact and delete the copy you received and in addition, you should not
>>>> print, copy, retransmit, disseminate, or otherwise use the information
>>>> contained in this communication. Internet communications cannot be
>>>> guaranteed to be timely, secure, error or virus-free. The sender does not
>>>> accept liability for any errors or omissions.
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>>
>>> Gayan Gunarathne
>>> Technical Lead
>>> WSO2 Inc. (http://wso2.com)
>>> email : gay...@wso2.com | mobile : +94 766819985
>>>
>>>
>>
>>
>>
>> --
>> Paul Fremantle
>> CTO and Co-Founder, WSO2
>> OASIS WS-RX TC Co-chair, Apache Member
>>
>> UK: +44 207 096 0336
>>
>> blog: http://pzf.fremantle.org
>> twitter.com/pzfreo
>> p...@wso2.com
>>
>> wso2.com Lean Enterprise Middleware
>>
>> Disclaimer: This communication may contain privileged or other
>> confidential information and is intended exclusively for the addressee/s.
>> If you are not the intended recipient/s, or believe that you may have
>> received this communication in error, please reply to the sender indicating
>> that fact and delete the copy you received and in addition, you should not
>> print, copy, retransmit, disseminate, or otherwise use the information
>> contained in this communication. Internet communications cannot be
>> guaranteed to be timely, secure, error or virus-free. The sender does not
>> accept liability for any errors or omissions.
>>
>
>
>
> --
> Best Regards,
>
> Gayan Gunarathne
> Technical Lead
> WSO2 Inc. (http://wso2.com)
> email : gay...@wso2.com | mobile : +94 766819985
>
>
--
*Dinesh Bandara*
Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware
