Page edited by Ted Ross

Added up-to-date information about routing keys

Note
This page is being updated with protocol changes introduced in M4 (and are unchanged in release 0.5)

Protocol Header

QMF messages are composed of sequences of binary-encoded data fields, in a manner consistent with the 0-10 version of the AMQP specification.

All QMF messages begin with a message header:

          octet 0      1         2         3         4         5         6         7
        +---------+---------+---------+---------+---------+---------+---------+---------+
        |   'A'   |   'M'   |   '2'   | op-code |                sequence               |
        +---------+---------+---------+---------+---------+---------+---------+---------+

The first three octets contain the protocol magic number "AM2" which is used to identify the type and version of the message.

The opcode field identifies the operation represented by the message

Mapping QMF Messages to AMQP Messages

QMF messages are carried in the body segments of AMQP messages. An AMQP message body may contain 1 or more QMF messages. QMF messages do not span AMQP messages, each QMF messages must be entirely contained within a single AMQP message body.

Protocol Exchange Patterns

The following patterns are followed in the design of the protocol:

Request-Response
Query-Indication
Unsolicited Indication

The Request-Response Pattern

In the request-response pattern, a requestor sends a request message to one of its peers. The peer then does one of two things: If the request can be successfully processed, a single response message is sent back to the requestor. This response contains the requested results and serves as the positive acknowledgement that the request was successfully completed.

If the request cannot be successfully completed, the peer sends a command complete message back to the requestor with an error code and error text describing what went wrong.

The sequence number in the response or command complete message is the same as the sequence number in the request.

    Requestor                                                          Peer
        |                                                               |
        | --- Request (seq) ------------------------------------------> |
        |                                                               |
        | <----------------------------------------- Response (seq) --- |
        |                                                               |

    Requestor                                                          Peer
        |                                                               |
        | --- Request (seq) ------------------------------------------> |
        |                                                               |
        | <-------------------------- Command Complete (seq, error) --- |
        |                                                               |

The Query-Indication Pattern

The query-indication pattern is used when there may be zero or more answers to a question. In this case, the requestor sends a query message to its peer. The peer processes the query, sending as many indication messages as needed back to the requestor (zero or more). Once the last indication has been sent, the peer then sends a command complete message with a success code indicating that the query is complete.

If there is an error in the query, the peer may reply with a command complete message containg an error code. In this case, no indication messages may be sent.

All indication and command complete messages shall have the same sequence number that appeared in the query message.

    Requestor                                                          Peer
        |                                                               |
        | --- Query (seq) --------------------------------------------> |
        |                                                               |
        | <--------------------------------------- Indication (seq) --- |
        | <--------------------------------------- Indication (seq) --- |
        | <--------------------------------------- Indication (seq) --- |
        | <--------------------------------------- Indication (seq) --- |
        | <--------------------------------------- Indication (seq) --- |
        |                                                               |
        | <------------------------ Command Complete (seq, success) --- |
        |                                                               |

    Requestor                                                          Peer
        |                                                               |
        | --- Query (seq) --------------------------------------------> |
        |                                                               |
        | <-------------------------- Command Complete (seq, error) --- |
        |                                                               |

The Unsolicited-Indication Pattern

The unsolicited-indication pattern is used when one peer needs to send unsolicited information to another peer, or to broadcast information to multiple peers via a topic exchange. In this case, indication messages are sent with the sequence number field set to zero.

      Peer                                                             Peer
        |                                                               |
        | <----------------------------------- Indication (seq = 0) --- |
        | <----------------------------------- Indication (seq = 0) --- |
        | <----------------------------------- Indication (seq = 0) --- |
        | <----------------------------------- Indication (seq = 0) --- |
        |                                                               |

Object Identifiers

Manageable objects are tagged with a unique 128-bit object identifier. The object identifier space is owned and managed by the management broker. Objects managed by a single management broker shall have unique object identifiers. Objects managed by separate management brokers may have the same object identifier.

If a management console is designed to manage multiple management brokers, it must use the broker identifier as well as the object identifier to ensure global uniqueness.

first uint64:
         63   60 59              48 47                   28 27                           0
        +-------+------------------+-----------------------+------------------------------+
        | flags |     sequence     |      broker bank      |          agent bank          |
        +-------+------------------+-----------------------+------------------------------+

second uint64:
         63                                                                              0
        +---------------------------------------------------------------------------------+
        |                                     object                                      |
        +---------------------------------------------------------------------------------+

Field	Size (bits)	Description
flags	4	Reserved, must be zero
sequence	12	Boot sequence of the agent, or zero for persistent IDs
broker bank	20	Bank number unique to the broker
agent bank	28	Bank number unique to the agent
object	64	Identifier assigned by the agent

For persistent IDs, sequence is zero
For non-persistent IDs, sequence is a number which increments each time the management broker is restarted.

Establishing Communication Between Client and Agent

Communication is established between the management client and management agent using normal AMQP procedures. The client creates a connection to the broker and then establishes a session with its corresponding channel.

A private (exclusive/auto-delete) queue is then declared and bound to the qpid.management exchange. A binding with key "schema.#" will subscribe to all schema-related information and a second binding with key "console.#" will subscribe to all management data.

A binding must also be established to the "amq.direct" exchange using the queue's name as the binding key. This will be used as a reply-to address for requests sent to the broker and to agents.

When a client successfully binds to the qpid.management exchange, the management agent schedules a schema broadcast to be sent to the exchange. The agent will publish, via the exchange, a description of the schema for all manageable objects in its control.

      Client                                                             Broker
        |                                                                  |
        | --- AMQP Connection and Session Setup -------------------------> |
        |                                                                  |
        | --- Queue.declare (private data queue) ------------------------> |
        | --- Bind queue to exchange 'qpid.management' key 'schema.#' ---> |
        | --- Bind queue to exchange 'qpid.management' key 'console.#' --> |
        | --- Bind queue to exchange 'amq.direct' -----------------------> |
        |                                                                  |
        | --- Broker Request --------------------------------------------> |
        | <------------------------------------------- Broker Response --- |
        |                                                                  |
        |                                                                  |
        |                                                                  |
        | <---------- Management schema via exchange 'qpid.management' --- |
        |                                                                  |

Broadcast of Configuration and Instrumentation Updates

The management agent will periodically publish updates to the configuration and instrumentation of management objects under its control. Under normal circumstances, these updates are published only if they have changed since the last time they were published. Configuration updates are only published if configuration has changed and instrumentation updates are only published if instrumentation has changed. The exception to this rule is that after a management client binds to the qpid.management exchange, all configuration and instrumentation records are published as though they had changed whether or not they actually did.

  Client                                                                          Broker
    |                                                                               |
    | <--- Object properties via 'console.obj.1.<agent-bank>.<package>.<class>' --- | |
    | <--- Object statistics via 'console.obj.1.<agent-bank>.<package>.<class>' --- | |
    |                                                                               | |
    |                                                                               | | Publish
    |                                                                               | | Interval
    |                                                                               | |
    |                                                                               | V
    | <--- Object properties via 'console.obj.1.<agent-bank>.<package>.<class>' --- |
    | <--- Object statistics via 'console.obj.1.<agent-bank>.<package>.<class>' --- |
    |                                                                               |

Invoking a Method on a Managed Object

When the management client wishes to invoke a method on a managed object, it sends a method request message to the qpid.management exchange. The routing key contains the object class and method name (refer to Routing Key Structure below). The method request must have a header entry (reply-to) that contains the name of the method-reply queue so that the method response can be properly routed back to the requestor.

The method request contains a sequence number that is copied to the method reply. This number is opaque to the management agent and may be used by the management client to correlate the reply to the request. The asynchronous nature of requests and replies allows any number of methods to be in-flight at a time. Note that there is no guarantee that methods will be replied to in the order in which they were requested.

      Client                                                          Broker
        |                                                               |
        | --- Method Request (to exchange 'qpid.management') ---------> |
        |                                                               |
        |                                                               |
        | <--------------- Method Reply (via exchange 'amq.direct') --- |
        |                                                               |

Details of QMF Message Types

opcode	message	handled by	description
'B'	Broker Request	broker	This message contains a broker request, sent from the management console to the broker to initiate a management session.
'b'	Broker Response	console	This message contains a broker response, sent from the broker in response to a broker request message.
'z'	Command Completion	all	This message is sent to indicate the completion of a request.
'Q'	Class Query	broker, agent	Class query messages are used by a management console to request a list of schema classes that are known by the management broker.
'q'	Class Indication	console, broker	Sent by the management broker, a class indication notifies the peer of the existence of a schema class.
'S'	Schema Request	broker, agent	Schema request messages are used to request the full schema details for a class.
's'	Schema Response	console, broker	Schema response message contain a full description of the schema for a class.
'h'	Heartbeat Indication	console	This message is published once per publish-interval. It can be used by a client to positively determine which objects did not change during the interval (since updates are not published for objects with no changes).
'c', 'i', 'g'	Content Indication	console	This message contains a content record. Content records contain the values of all properties or statistics in an object. Such records are broadcast on a periodic interval if 1) a change has been made in the value of one of the elements, or 2) if a new management client has bound a queue to the management exchange.
'G'	Get Query	agent	Sent by a management console, a get query requests that the management broker provide content indications for all objects that match the query criteria.
'M'	Method Request	agent	This message contains a method request.
'm'	Method Response	console	This message contains a method result.
'P'	Package Query	broker, agent	This message contains a schema package query request, requesting that the broker dump the list of known packages
'p'	Package Indication	console, broker	This message contains a schema package indication, identifying a package known by the broker
'A'	Agent Attach Request	broker	This message is sent by a remote agent when it wishes to attach to a management broker
'a'	Agent Attach Response	agent	The management broker sends this response if an attaching remote agent is permitted to join
'x'	Console Added Indication	agent	This message is sent to all remote agents by the management broker when a new console binds to the management exchange

Broker Request Message

When a management console first establishes contact with the broker, it sends a Broker Request message to initiate the exchange.

        routing_key: broker
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'B' |           0           |
        +-----+-----+-----+-----+-----------------------+

The Broker Request message has no payload.

Broker Response Message

When the broker receives a Broker Request message, it responds with a Broker Response message. This message contains an identifier unique to the broker.

        routing_key: <reply_to from request>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'b' |           0           |
        +-----+-----+-----+-----+-----------------------+----------------------------+
        | brokerId (uuid)                                                            |
        +----------------------------------------------------------------------------+

Command Completion Message

        routing_key: <reply_to from request>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'z' |          seq          |
        +-----+-----+-----+-----+----+------------------+
        |  Completion Code (uint32)  |
        +----------------------------+------------------------------------+
        |  Completion Text (str8)                                         |
        +-----------------------------------------------------------------+

Class Query

        routing_key: broker
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'Q' |          seq          |
        +-----+-----+-----+-----+-----------------------+----------+
        |  package name (str8)                                     |
        +----------------------------------------------------------+

Class Indication

        routing_key: <reply_to from request> (if in reply to a request)
                     schema.package          (if unsolicited)
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'q' |          seq          |
        +-----+-----+-----+-----+---------------+-------+
        | class kind (uint8) 1=Object, 2=Event  |
        +---------------------------------------+------------------+
        | package name (str8)                                      |
        +----------------------------------------------------------+
        | class name (str8)                                        |
        +----------------------------------------------------------+
        | schema hash (bin128)                                     |
        +----------------------------------------------------------+

Schema Request

        routing_key: broker
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'S' |          seq          |
        +-----+-----+-----+-----+-----------------------+----------+
        | packageName (str8)                                       |
        +----------------------------------------------------------+
        | className (str8)                                         |
        +----------------------------------------------------------+
        | schema-hash (bin128)                                     |
        +----------------------------------------------------------+

Schema Response

        routing_key: <reply_to from request> (if in reply to a request)
                     schema.package          (if unsolicited)
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 's' |          seq          |
        +-----+-----+-----+-----+------------+----------+
        | kind (uint8) 1=Object, 2=Event     |
        +------------------------------------+---------------------+
        | packageName (str8)                                       |
        +----------------------------------------------------------+
        | className (str8)                                         |
        +----------------------------------------------------------+
        | schema-hash (bin128)                                     |
        +-----------------------------------------------+----------+
        | propCount (uint16)                            |
        +-----------------------------------------------+
        | statCount (uint16)                            |
        +-----------------------------------------------+
        | methodCount (uint16)                          |
        +-----------------------------------------------+----------------------------+
        | propCount property records                                                 |
        +----------------------------------------------------------------------------+
        | statCount statistic records                                                |
        +----------------------------------------------------------------------------+
        | methodCount method records                                                 |
        +----------------------------------------------------------------------------+

Each property record is an AMQP map with the following fields. Optional fields may optionally be omitted from the map.

field name	optional	description
name	no	Name of the property
type	no	Type code for the property
refPackage	yes	Name of referenced package (for objectReference and object types)
refClass	yes	Name of referenced class (for objectReference and object types)
access	no	Access code for the property
index	no	1 = index element, 0 = not an index element
optional	no	1 = optional element (may be not present), 0 = mandatory (always present)
unit	yes	Units for numeric values (i.e. seconds, bytes, etc.)
min	yes	Minimum value for numerics
max	yes	Maximum value for numerics
maxlen	yes	Maximum length for strings
desc	yes	Description of the property

Each statistic record is an AMQP map with the following fields:

field name	optional	description
name	no	Name of the statistic
type	no	Type code for the statistic
unit	yes	Units for numeric values (i.e. seconds, bytes, etc.)
desc	yes	Description of the statistic

method records contain a main map that describes the method or header followed by zero or more maps describing arguments. The main map contains the following fields:

field name	optional	description
name	no	Name of the method or event
argCount	no	Number of argument records to follow
desc	yes	Description of the method or event

Argument maps contain the following fields:

field name	optional	description
name	no	Argument name
type	no	Type code for the argument
refPackage	yes	Name of referenced package (for objectReference and object types)
refClass	yes	Name of referenced class (for objectReference and object types)
dir	yes	Direction code for method arguments
unit	yes	Units for numeric values (i.e. seconds, bytes, etc.)
min	yes	Minimum value for numerics
max	yes	Maximum value for numerics
maxlen	yes	Maximum length for strings
desc	yes	Description of the argument
default	yes	Default value for the argument

type codes are numerics with the following values:

value	type	Encoding
1	uint8	uint8
2	uint16	uint16
3	uint32	uint32
4	uint64	uint64
6	str8	str8
7	str16	str16
8	absTime	uint64
9	deltaTime	uint64
10	objectReference	bin128
11	boolean	boolean
12	float	float
13	double	double
14	uuid	uuid
15	map	map
16	int8	int8
17	int16	int16
18	int32	int32
19	int64	int64
20	object	package-name(str8) + class-name(str8) + hash(bin128) + object-indication-encoding
21	list	list
22	array	array

access codes are numerics with the following values:

value	access
1	Read-Create access
2	Read-Write access
3	Read-Only access

direction codes are strings with the following values:

value	direction
"I"	Input (from client to broker)
"O"	Output (from broker to client)
"IO"	IO (bidirectional)

Heartbeat Indication

        routing_key: console.heartbeat.1.<agent_bank>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'h' |           0           |
        +-----+-----+-----+-----+-----------------------+
        | timestamp of current interval (datetime)      |
        +-----------------------------------------------+

Configuration and Instrumentation Content Messages

Content messages are published when changes are made to the values of properties or statistics or when new management clients bind a queue to the management exchange.

        for 'g':     routing_key: <reply_to from request>
        for 'c','i': routing_key: console.obj.1.<agent_bank>.<package_name>.<class_name>
        +-----+-----+-----+-------+-----------------------+
        | 'A' | 'M' | '2' |'g/c/i'|          seq          |
        +-----+-----+-----+-------+-----------------------+--------+
        |                packageName (str8)                        |
        +----------------------------------------------------------+
        |                className (str8)                          |
        +----------------------------------------------------------+
        |                class hash (bin128)                       |
        +-----+-----+-----+-----+-----+-----+-----+-----+----------+
        | timestamp of current sample (datetime)        |
        +-----+-----+-----+-----+-----+-----+-----+-----+
        | time object was created (datetime)            |
        +-----+-----+-----+-----+-----+-----+-----+-----+
        | time object was deleted (datetime)            |
        +-----+-----+-----+-----+-----+-----+-----+-----+
        | objectId (bin128)                             |
        +-----+-----+-----+-----+-----+-----+-----+-----+
        | presence bitmasks (0 or more uint8 fields)    |
        +-----+-----+-----+-----+-----+-----+-----+-----+------------------------+
        | config/inst values (in schema order)                                   |
        +------------------------------------------------------------------------+

All timestamps are uint64 values representing nanoseconds since the epoch (January 1, 1970). The objectId is a bin128 value that uniquely identifies this object instance.

If any of the properties in the object are defined as optional, there will be 1 or more "presence bitmask" octets. There are as many octets as are needed to provide one bit per optional property. The bits are assigned to the optional properties in schema order (first octet first, lowest order bit first).

For example: If there are two optional properties in the schema called "option1" and "option2" (defined in that order), there will be one presence bitmask octet and the bits will be assigned as bit 0 controls option1 and bit 1 controls option2.

If the bit for a particular optional property is set (1), the property will be encoded normally in the "values" portion of the message. If the bit is clear (0), the property will be omitted from the list of encoded values and will be considered "NULL" or "not present".

The element values are encoded by their type into the message in the order in which they appeared in the schema message.

Get Query Message

A Get Request may be sent by the management console to cause a management agent to immediately send content information for objects of a class.

        routing_key: agent.1.<agent_bank>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'G' |          seq          |
        +-----+-----+-----+-----+-----------------------+----------+
        | Get request field table (map)                            |
        +----------------------------------------------------------+

The content of a get request is a field table that specifies what objects are being requested. Most of the fields are optional and are available for use in more extensive deployments.

Field Key	Type	Description
"_class"	str8	The name of the class of objects being requested.
"_package"	str8	The name of the extension package the class belongs to. If omitted, the package defaults to "qpid" for access to objects in the connected broker.
_objectid	bin128	The object ID of the object being requested

When the management agent receives a get request, it sends content messages describing the requested objects. Once the last content message is sent, it then sends a Command Completion message with the same sequence number supplied in the request to indicate to the requestor that there are no more messages coming.

Method Request

Method request messages have the following structure. The sequence number is opaque to the management agent. It is returned unchanged in the method reply so the calling client can correctly associate the reply to the request. The objectId is the unique ID of the object on which the method is to be executed.

        routing_key: agent.1.<agent_bank>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'M' |          seq          |
        +-----+-----+-----+-----+-----------------------+
        |  objectId (bin128)                            |
        +-----------------------------------------------+
        |  package name (str8)                          |
        +-----------------------------------------------+
        |  class name (str8)                            |
        +-----------------------------------------------+
        |  class hash (bin128)                          |
        +-----------------------------------------------+
        |  methodName (str8)                            |
        +-----------------------------------------------+------------------------+
        |  input and bidirectional argument values (in schema order)             |
        +------------------------------------------------------------------------+

Method Response

Method reply messages have the following structure. The sequence number is identical to that supplied in the method request. The status code (and text) indicate whether or not the method was successful and if not, what the error was. Output and bidirectional arguments are only included if the status code was 0 (STATUS_OK).

        routing_key: <reply_to from request>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'm' |          seq          |
        +-----+-----+-----+-----+-----------------------+
        |  status code (uint32) |
        +-----------------------+----------------------------------+
        |  status text (str16)                                     |
        +-----------------------+----------------------------------+-------------+
        |  output and bidirectional argument values (in schema order)            |
        +------------------------------------------------------------------------+

status code values are:

value	description
0	STATUS_OK - successful completion
1	STATUS_UNKNOWN_OBJECT - objectId not found in the agent
2	STATUS_UNKNOWN_METHOD - method is not known by the object type
3	STATUS_NOT_IMPLEMENTED - method is not currently implemented
4	STATUS_INVALID_PARAMETER - input argument is invalid
5	STATUS_FEATURE_NOT_IMPLEMENTED
6	STATUS_FORBIDDEN - operation is forbidden by Access Control List
7	STATUS_EXCEPTION - exception caught during method execution
8	STATUS_UNKNOWN_PACKAGE - package name not found
9	STATUS_UNKNOWN_CLASS - class name not found in package

Messages for Extended Scenario

Extended Management Protocol

Qpid supports management extensions that allow the management broker to be a central point for the management of multiple external entities with their own management schemas.

      Broker                                                       Remote Agent
        |                                                               |
        | <----------------------------------------- Attach Request --- |
        | --- Attach Response ----------------------------------------> |
        |                                                               |
        | <------------------------------------- Package Indication --- |
        | <------------------------------------- Package Indication --- |
        |                                                               |
        | <--------------------------------------- Class Indication --- |
        | <--------------------------------------- Class Indication --- |
        | <--------------------------------------- Class Indication --- |
        | <--------------------------------------- Class Indication --- |
        | <--------------------------------------- Class Indication --- |
        |                                                               |
        | --- Schema Request (class key) -----------------------------> |
        | <---------------------------------------- Schema Response --- |
        |                                                               |
        | --- Schema Request (class key) -----------------------------> |
        | <---------------------------------------- Schema Response --- |
        |                                                               |
        |                                                               |

Package Query

        routing_key: agent.1.<agent_bank>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'P' |          seq          |
        +-----+-----+-----+-----+-----------------------+

Package Indication

        routing_key: <reply_to from request> (if in reply to a request)
                     schema.package          (if unsolicited)
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'p' |          seq          |
        +-----+-----+-----+-----+-----------------------+----------+
        |  package name (str8)                                     |
        +----------------------------------------------------------+

Agent Attach Request

        routing_key: broker
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'A' |          seq          |
        +-----+-----+-----+-----+-----------------------+----------+
        |  label (str8)                                            |
        +----------------------------------------------------------+
        |  system-id (uuid)                                        |
        +---------------------------------+------------------------+
        |  requested broker bank (uint32) |
        +---------------------------------+
        |  requested agent bank (uint32)  |
        +---------------------------------+

Agent Attach Response (success)

        routing_key: <reply_to from request>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'a' |          seq          |
        +-----+-----+-----+-----+--------+--------------+
        |  assigned broker bank (uint32) |
        +--------------------------------+
        |  assigned agent bank (uint32)  |
        +--------------------------------+

Console Added Indication

        routing_key: agent.1.<agent_bank>
        +-----+-----+-----+-----+-----------------------+
        | 'A' | 'M' | '2' | 'x' |          seq          |
        +-----+-----+-----+-----+-----------------------+

Change Notification Preferences

View Online | View Change | Add Comment

--------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:commits-subscr...@qpid.apache.org

[CONF] Apache Qpid > QMF Protocol

QMF Protocol