Re: C++ Messaging API questions...
On Tue, Mar 16, 2010 at 1:29 PM, Gordon Sim wrote: > On 03/16/2010 06:05 PM, Kerry Bonin wrote: > >> r/e headers - I do see the same thing you listed (when I use quotation >> marks, I'm on Win32). Your comment implies the basis for my question - >> digging into the source I now see properties are internally split into >> MessageProperties and DeliveryProperties, and I was expecting to see those >> fields when I called getHeaders(). >> > > The messaging API is intended to abstract out protocol specifics. So there > are explicit setters/getters for attributes of a message (e.g. userid or > ttl) and these will be mapped on to the underlying protocol used (in the > case of AMQP 0-10, onto fields in Message- or Delivery- Properties). Then > there are generic properties available to applications for their own > purposes. These are name value pairs. In AMQP 0-10 they map onto > MessageProperites::applicationHeaders. > > > r/e timestamps >> - One common use case we have is to have clients synchronize distributed >> data sets, which we do by providing a snapshot via request - response as >> well as an update broadcast via publish - subscribe. We use timestamps to >> determine when an update event was sent relative to the snapshot. We were >> including our own timestamp field in the message body for this, and >> thought >> we could use message timestamps. >> - Another common use case is to broadcast events. We care when these were >> sent, and historically have included a UTC event time in the message that >> is >> used (in part) to identify the event. Again, we could include our own >> timestamp in the message body for this, but we thought we could use the >> message timestamp for this - so long as it is the time of creation or >> first >> transmission. We haven't seen documentation on the expected behavior of >> message timestamps across federated and/or clustered systems, or through >> durable queues. >> > > So the timestamp is set by the sender and refers to the creation of the > message or to the time of the event the message represents? (i.e. the > detailed semantics are application specific?) Yes, that's how we've used message timestamps historically - our send API would set the timestamp to the current time if it had not been previously set during message construction, and this value would be propagated to all recipients. If this concept differs from the meaning of timestamp in AMQP, we can certainly return to an application field for timestamps. In either case, it would be useful to clarify formally (somewhere) what timestamps are used for in AMQP, and if they have any value to an application use it would seem reasonable to expose them. > > r/e the asynchronous APIs, this is more of a nice to have, but we have C++ >> and Python developers who liked not having to create service threads to >> read >> asynchronously for the previous API. Event loops are common, but not >> always >> used... >> > > I agree its nice to have something there out of the box. However even in > the old 0-10 specific API the only really usable solution was > SubscriptionManager::run(). I think you could replace the functionality of > that method with: > > typedef std::map Callbacks; > Callbacks callbacks; > //populate per subscription listeners > > Listener* default = //default callback > > while (true) { >Receiver r = session.nextReceiver(); >Callbacks::const_iterator i = callbacks.find(r.getName()); >if (i == callbacks.end()) { > default->received(r.fetch()); >} else { > i->second->received(r.fetch()); >} > } > > You can also customise that loop more, e.g. by timing out if nothing is > received after a certain time, or having different strategies for > dispatching the messages to listeners/callbacks. > > The above of course still requires a thread per session, but that is no > worse than the old API. The plan is to add the ability to tie in waiting on > one or more connection to the same pattern. > >
Re: C++ Messaging API questions...
On 03/16/2010 06:05 PM, Kerry Bonin wrote: r/e headers - I do see the same thing you listed (when I use quotation marks, I'm on Win32). Your comment implies the basis for my question - digging into the source I now see properties are internally split into MessageProperties and DeliveryProperties, and I was expecting to see those fields when I called getHeaders(). The messaging API is intended to abstract out protocol specifics. So there are explicit setters/getters for attributes of a message (e.g. userid or ttl) and these will be mapped on to the underlying protocol used (in the case of AMQP 0-10, onto fields in Message- or Delivery- Properties). Then there are generic properties available to applications for their own purposes. These are name value pairs. In AMQP 0-10 they map onto MessageProperites::applicationHeaders. r/e timestamps - One common use case we have is to have clients synchronize distributed data sets, which we do by providing a snapshot via request - response as well as an update broadcast via publish - subscribe. We use timestamps to determine when an update event was sent relative to the snapshot. We were including our own timestamp field in the message body for this, and thought we could use message timestamps. - Another common use case is to broadcast events. We care when these were sent, and historically have included a UTC event time in the message that is used (in part) to identify the event. Again, we could include our own timestamp in the message body for this, but we thought we could use the message timestamp for this - so long as it is the time of creation or first transmission. We haven't seen documentation on the expected behavior of message timestamps across federated and/or clustered systems, or through durable queues. So the timestamp is set by the sender and refers to the creation of the message or to the time of the event the message represents? (i.e. the detailed semantics are application specific?) r/e the asynchronous APIs, this is more of a nice to have, but we have C++ and Python developers who liked not having to create service threads to read asynchronously for the previous API. Event loops are common, but not always used... I agree its nice to have something there out of the box. However even in the old 0-10 specific API the only really usable solution was SubscriptionManager::run(). I think you could replace the functionality of that method with: typedef std::map Callbacks; Callbacks callbacks; //populate per subscription listeners Listener* default = //default callback while (true) { Receiver r = session.nextReceiver(); Callbacks::const_iterator i = callbacks.find(r.getName()); if (i == callbacks.end()) { default->received(r.fetch()); } else { i->second->received(r.fetch()); } } You can also customise that loop more, e.g. by timing out if nothing is received after a certain time, or having different strategies for dispatching the messages to listeners/callbacks. The above of course still requires a thread per session, but that is no worse than the old API. The plan is to add the ability to tie in waiting on one or more connection to the same pattern. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: C++ Messaging API questions...
r/e headers - I do see the same thing you listed (when I use quotation marks, I'm on Win32). Your comment implies the basis for my question - digging into the source I now see properties are internally split into MessageProperties and DeliveryProperties, and I was expecting to see those fields when I called getHeaders(). r/e timestamps - One common use case we have is to have clients synchronize distributed data sets, which we do by providing a snapshot via request - response as well as an update broadcast via publish - subscribe. We use timestamps to determine when an update event was sent relative to the snapshot. We were including our own timestamp field in the message body for this, and thought we could use message timestamps. - Another common use case is to broadcast events. We care when these were sent, and historically have included a UTC event time in the message that is used (in part) to identify the event. Again, we could include our own timestamp in the message body for this, but we thought we could use the message timestamp for this - so long as it is the time of creation or first transmission. We haven't seen documentation on the expected behavior of message timestamps across federated and/or clustered systems, or through durable queues. r/e the asynchronous APIs, this is more of a nice to have, but we have C++ and Python developers who liked not having to create service threads to read asynchronously for the previous API. Event loops are common, but not always used... On Tue, Mar 16, 2010 at 12:20 PM, Gordon Sim wrote: > On 03/16/2010 03:58 PM, Kerry Bonin wrote: > >> I'm looking at migrating to messaging, and have a few questions. >> >> r/e Message : >> - If headers has an accessor, why doesn't it work? It only returns an >> empty >> map. >> > > It works for me (are you using trunk?): > > $ ./examples/messaging/spout --address 'my-queue; {create: always}' > --property my-property=my-value > $ ./examples/messaging/drain --address 'my-queue; {create: always}' > Message(properties={my-property:my-value, > spout-id:eab53080-8f16-48db-b4cf-5cf9bb200ec0:0}, content='') > > The above uses Message::getHeaders(), what do you see when you run those > examples? > > (Note I'd actually like to change that to getProperties() for consistency > with the python client). > > > - Why isn't timestamp exposed? (Trust& TZ issues aside, it is common to >> want this data, why require recreation?) >> > > How would you expect the timestamp to be used? > > > r/e Receiver : >> - Why did we lose the callback pattern? I now have to (at the least) spin >> a >> thread across all my receivers to collect asynchronously. >> > > The idea is that some sort of generic dispatcher would be layered on top of > the current api. The plan is to extend or generalise the > Session::nextReceiver() pattern so that Sessions and indeed Connections > could be integrated into a common event loop. > > > FWIW, I'd be very happy to help fix these, as I need them myself... >> > > Excellent, and thanks for the feedback, the API could do with some more > eyes on it. > > > - > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:dev-subscr...@qpid.apache.org > >
Re: C++ Messaging API questions...
On 03/16/2010 01:20 PM, Gordon Sim wrote: On 03/16/2010 03:58 PM, Kerry Bonin wrote: I'm looking at migrating to messaging, and have a few questions. r/e Message : - If headers has an accessor, why doesn't it work? It only returns an empty map. It works for me (are you using trunk?): $ ./examples/messaging/spout --address 'my-queue; {create: always}' --property my-property=my-value $ ./examples/messaging/drain --address 'my-queue; {create: always}' Message(properties={my-property:my-value, spout-id:eab53080-8f16-48db-b4cf-5cf9bb200ec0:0}, content='') The above uses Message::getHeaders(), what do you see when you run those examples? (Note I'd actually like to change that to getProperties() for consistency with the python client). - Why isn't timestamp exposed? (Trust& TZ issues aside, it is common to want this data, why require recreation?) How would you expect the timestamp to be used? r/e Receiver : - Why did we lose the callback pattern? I now have to (at the least) spin a thread across all my receivers to collect asynchronously. The idea is that some sort of generic dispatcher would be layered on top of the current api. The plan is to extend or generalise the Session::nextReceiver() pattern so that Sessions and indeed Connections could be integrated into a common event loop. IMO we need the API for that sooner rather than later, as it will be a blocker to porting existing test programs. I will try to take a shot at it this week. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: C++ Messaging API questions...
On 03/16/2010 03:58 PM, Kerry Bonin wrote: I'm looking at migrating to messaging, and have a few questions. r/e Message : - If headers has an accessor, why doesn't it work? It only returns an empty map. It works for me (are you using trunk?): $ ./examples/messaging/spout --address 'my-queue; {create: always}' --property my-property=my-value $ ./examples/messaging/drain --address 'my-queue; {create: always}' Message(properties={my-property:my-value, spout-id:eab53080-8f16-48db-b4cf-5cf9bb200ec0:0}, content='') The above uses Message::getHeaders(), what do you see when you run those examples? (Note I'd actually like to change that to getProperties() for consistency with the python client). - Why isn't timestamp exposed? (Trust& TZ issues aside, it is common to want this data, why require recreation?) How would you expect the timestamp to be used? r/e Receiver : - Why did we lose the callback pattern? I now have to (at the least) spin a thread across all my receivers to collect asynchronously. The idea is that some sort of generic dispatcher would be layered on top of the current api. The plan is to extend or generalise the Session::nextReceiver() pattern so that Sessions and indeed Connections could be integrated into a common event loop. FWIW, I'd be very happy to help fix these, as I need them myself... Excellent, and thanks for the feedback, the API could do with some more eyes on it. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
C++ Messaging API questions...
I'm looking at migrating to messaging, and have a few questions. r/e Message : - If headers has an accessor, why doesn't it work? It only returns an empty map. - Why isn't timestamp exposed? (Trust & TZ issues aside, it is common to want this data, why require recreation?) r/e Receiver : - Why did we lose the callback pattern? I now have to (at the least) spin a thread across all my receivers to collect asynchronously. FWIW, I'd be very happy to help fix these, as I need them myself...