[CONF] Apache Qpid: QMF Python Console Tutorial (page edited)

[confluence]

Page Edited : qpid : QMF Python Console Tutorial QMF Python Console Tutorial has been edited by Ted Ross (Jan 30, 2009). (View changes) Content: Prerequisite - Install Qpid Messaging Synchronous Console Operations Creating a QMF Console Session and Attaching to a Broker Accessing Managed Objects Viewing Properties and Statistics of an Object Invoking Methods on an Object Asynchronous Console Operations Creating a Console Class to Receive Asynchronous Data Receiving Events Receiving Objects Discovering what Kinds of Objects are Available Prerequisite - Install Qpid Messaging QMF uses Qpid Messaging as its means of communication. To use QMF, Qpid messaging must be installed somewhere in the network. Qpid can be downloaded as source from Apache, is packaged with a number of Linux distributions, and can be purchased from commercial vendors that use Qpid. Please see Download for information as to where to get Qpid Messaging. Qpid Messaging includes a message broker (qpidd) which typically runs as a daemon on a system. It also includes client bindings in various programming languages. The Python-language client library includes the QMF console libraries needed for this tutorial. Please note that Qpid Messaging has two broker implementations. One is implemented in C++ and the other in Java. At press time, QMF is supported only by the C++ broker. If the goal is to get the tutorial examples up and running as quickly as possible, all of the Qpid components can be installed on a single system (even a laptop). For more realistic deployments, the broker can be deployed on a server and the client/QMF libraries installed on other systems. Synchronous Console Operations The Python console API for QMF can be used in a synchronous style, an asynchronous style, or a combination of both. Synchronous operations are conceptually simple and are well suited for user-interactive tasks. All operations are performed in the context of a Python function call. If communication over the message bus is required to complete an operation, the function call blocks and waits for the expected result (or timeout failure) before returning control to the caller. Creating a QMF Console Session and Attaching to a Broker For the purposes of this tutorial, code examples will be shown as they are entered in an interactive python session. $ python Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38) [GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> We will begin by importing the required libraries. If the Python client is properly installed, these libraries will be found normally by the Python interpreter. >>> from qmf.console import Session We must now create a Session object to manage this QMF console session. >>> sess = Session() If no arguments are supplied to the creation of Session, it defaults to synchronous-only operation. It also defaults to user-management of connections. More on this in a moment. We will now establish a connection to the messaging broker. If the broker daemon is running on the local host, simply use the following: >>> broker = sess.addBroker() If the messaging broker is on a remote host, supply the URL to the broker in the addBroker function call. Here's how to connect to a local broker using the URL. >>> broker = sess.addBroker("amqp://localhost") The call to addBroker is synchronous and will return only after the connection has been successfully established or has failed. If a failure occurs, addBroker will raise an exception that can be handled by the console script. >>> try: ... broker = sess.addBroker("amqp://localhost:1000") ... except: ... print "Connection Failed" ... Connection Failed >>> This operation fails because there is no Qpid Messaging broker listening on port 1000 (the default port for qpidd is 5672). If preferred, the QMF session can manage the connection for you. In this case, addBroker returns immediately and the session attempts to establish the connection in the background. This will be covered in detail in the section on asynchronous operations. Accessing Managed Objects The Python console API provides access to remotely managed objects via a proxy model. The API gives the client an object that serves as a proxy representing the "real" object being managed on the agent application. Operations performed on the proxy result in the same operations on the real object. The following examples assume prior knowledge of the kinds of objects that are actually available to be managed. There is a section later in this tutorial that describes how to discover what is manageable on the QMF bus. Proxy objects are obtained by calling the Session.getObjects function. To illustrate, we'll get a list of objects representing queues in the message broker itself. >>> queues = sess.getObjects(_class="queue", _package="org.apache.qpid.broker") queues is an array of proxy objects representing real queues on the message broker. A proxy object can be printed to display a description of the object. >>> for q in queues: ... print q ... org.apache.qpid.broker:queue[0-1537-1-0-58] 0-0-1-0-1152921504606846979:reply-localhost.localdomain.32004 org.apache.qpid.broker:queue[0-1537-1-0-61] 0-0-1-0-1152921504606846979:topic-localhost.localdomain.32004 >>> Viewing Properties and Statistics of an Object Let us now focus our attention on one of the queue objects. >>> queue = queues[0] The attributes of an object are partitioned into properties and statistics. Though the distinction is somewhat arbitrary, properties tend to be fairly static and may also be large and statistics tend to change rapidly and are relatively small (counters, etc.). There are two ways to view the properties of an object. An array of properties can be obtained using the getProperties function: >>> props = queue.getProperties() >>> for prop in props: ... print prop ... (vhostRef, 0-0-1-0-1152921504606846979) (name, u'reply-localhost.localdomain.32004') (durable, False) (autoDelete, True) (exclusive, True) (arguments, {}) >>> The getProperties function returns an array of tuples. Each tuple consists of the property descriptor and the property value. A more convenient way to access properties is by using the attribute of the proxy object directly: >>> queue.autoDelete True >>> queue.name u'reply-localhost.localdomain.32004' >>> Statistics are accessed in the same way: >>> stats = queue.getStatistics() >>> for stat in stats: ... print stat ... (msgTotalEnqueues, 53) (msgTotalDequeues, 53) (msgTxnEnqueues, 0) (msgTxnDequeues, 0) (msgPersistEnqueues, 0) (msgPersistDequeues, 0) (msgDepth, 0) (byteDepth, 0) (byteTotalEnqueues, 19116) (byteTotalDequeues, 19116) (byteTxnEnqueues, 0) (byteTxnDequeues, 0) (bytePersistEnqueues, 0) (bytePersistDequeues, 0) (consumerCount, 1) (consumerCountHigh, 1) (consumerCountLow, 1) (bindingCount, 2) (bindingCountHigh, 2) (bindingCountLow, 2) (unackedMessages, 0) (unackedMessagesHigh, 0) (unackedMessagesLow, 0) (messageLatencySamples, 0) (messageLatencyMin, 0) (messageLatencyMax, 0) (messageLatencyAverage, 0) >>> or alternatively: >>> queue.byteTotalEnqueues 19116 >>> The proxy objects do not automatically track changes that occur on the real objects. For example, if the real queue enqueues more bytes, viewing the byteTotalEnqueues statistic will show the same number as it did the first time. To get updated data on a proxy object, use the update function call: >>> queue.update() >>> queue.byteTotalEnqueues 19783 >>> Be Advised The update method was added after the M4 release of Qpid/Qmf. It may not be available in your distribution. Invoking Methods on an Object Up to this point, we have used the QMF Console API to find managed objects and view their attributes, a read-only activity. The next topic to illustrate is how to invoke a method on a managed object. Methods allow consoles to control the managed agents by either triggering a one-time action or by changing the values of attributes in an object. First, we'll cover some background information about methods. A QMF object class (of which a QMF object is an instance), may have zero or more methods. To obtain a list of methods available for an object, use the getMethods function. >>> methodList = queue.getMethods() getMethods returns an array of method descriptors (of type qmf.console.SchemaMethod). To get a summary of a method, you can simply print it. The _repr_ function returns a string that looks like a function prototype. >>> print methodList [purge(request)] >>> For the purposes of illustration, we'll use a more interesting method available on the broker object which represents the connected Qpid message broker. >>> br = sess.getObjects(_class="broker", _package="org.apache.qpid.broker")[0] >>> mlist = br.getMethods() >>> for m in mlist: ... print m ... echo(sequence, body) connect(host, port, durable, authMechanism, username, password, transport) queueMoveMessages(srcQueue, destQueue, qty) >>> We have just learned that the broker object has three methods: echo, connect, and queueMoveMessages. We'll use the echo method to "ping" the broker. >>> result = br.echo(1, "Message Body") >>> print result OK (0) - {'body': u'Message Body', 'sequence': 1} >>> print result.status 0 >>> print result.text OK >>> print result.outArgs {'body': u'Message Body', 'sequence': 1} >>> In the above example, we have invoked the echo method on the instance of the broker designated by the proxy "br" with a sequence argument of 1 and a body argument of "Message Body". The result indicates success and contains the output arguments (in this case copies of the input arguments). To be more precise... Calling echo on the proxy causes the input arguments to be marshalled and sent to the remote agent where the method is executed. Once the method execution completes, the output arguments are marshalled and sent back to the console to be stored in the method result. You are probably wondering how you are supposed to know what types the arguments are and which arguments are input, which are output, or which are both. This will be addressed later in the "Discovering what Kinds of Objects are Available" section. Asynchronous Console Operations QMF is built on top of a middleware messaging layer (Qpid Messaging). Because of this, QMF can use some communication patterns that are difficult to implement using network transports like UDP, TCP, or SSL. One of these patterns is called the Publication and Subscription pattern (pub-sub for short). In the pub-sub pattern, data sources publish information without a particular destination in mind. Data sinks (destinations) subscribe using a set of criteria that describes what kind of data they are interested in receiving. Data published by a source may be received by zero, one, or many subscribers. QMF uses the pub-sub pattern to distribute events, object creation and deletion, and changes to properties and statistics. A console application using the QMF Console API can receive these asynchronous and unsolicited events and updates. This is useful for applications that store and analyze events and/or statistics. It is also useful for applications that react to certain events or conditions. Note that console applications may always use the synchronous mechanisms. Creating a Console Class to Receive Asynchronous Data Asynchronous API operation occurs when the console application supplies a Console object to the session manager. The Console object (which overrides the qmf.console.Console class) handles all asynchronously arriving data. The Console class has the following methods. Any number of these methods may be overridden by the console application. Any method that is not overridden defaults to a null handler which takes no action when invoked. Method Arguments Invoked when... brokerConnected broker a connection to a broker is established brokerDisconnected broker a connection to a broker is lost newPackage name a new package is seen on the QMF bus newClass kind, classKey a new class (event or object) is seen on the QMF bus newAgent agent a new agent appears on the QMF bus delAgent agent an agent disconnects from the QMF bus objectProps broker, object the properties of an object are published objectStats broker, object the statistics of an object are published event broker, event an event is published heartbeat agent, timestamp a heartbeat is published by an agent brokerInfo broker information about a connected broker is available to be queried Supplied with the API is a class called DebugConsole. This is a test Console instance that overrides all of the methods such that arriving asynchronous data is printed to the screen. This can be used to see all of the arriving asynchronous data. Receiving Events We'll start the example from the beginning to illustrate the reception and handling of events. In this example, we will create a Console class that handles broker-connect, broker-disconnect, and event messages. We will also allow the session manager to manage the broker connection for us. Begin by importing the necessary classes: >>> from qmf.console import Session, Console Now, create a subclass of Console that handles the three message types: >>> class EventConsole(Console): ... def brokerConnected(self, broker): ... print "brokerConnected:", broker ... def brokerDisconnected(self, broker): ... print "brokerDisconnected:", broker ... def event(self, broker, event): ... print "event:", event ... >>> Make an instance of the new class: >>> myConsole = EventConsole() Create a Session class using the console instance. In addition, we shall request that the session manager do the connection management for us. Notice also that we are requesting that the session manager not receive objects or heartbeats. Since this example is concerned only with events, we can optimize the use of the messaging bus by telling the session manager not to subscribe for object updates or heartbeats. >>> sess = Session(myConsole, manageConnections=True, rcvObjects=False, rcvHeartbeats=False) >>> broker = sess.addBroker() >>> Once the broker is added, we will begin to receive asynchronous events (assuming there is a functioning broker available to connect to). brokerConnected: Broker connected at: localhost:5672 event: Thu Jan 29 19:53:19 2009 INFO org.apache.qpid.broker:bind broker=localhost:5672 ... Receiving Objects To illustrate asynchronous handling of objects, a small console program is supplied. The entire program is shown below for convenience. We will then go through it part-by-part to explain its design. # Import needed classes from qmf.console import Session, Console from time import sleep # Declare a dictionary to map object-ids to queue names queueMap = {} # Customize the Console class to receive object updates. class MyConsole(Console): # Handle property updates def objectProps(self, broker, record): # Verify that we have received a queue object. Exit otherwise. classKey = record.getClassKey() if classKey.getClassName() != "queue": return # If this object has not been seen before, create a new mapping from objectID to name oid = record.getObjectId() if oid not in queueMap: queueMap[oid] = record.name # Handle statistic updates def objectStats(self, broker, record): # Ignore updates for objects that are not in the map oid = record.getObjectId() if oid not in queueMap: return # Print the queue name and some statistics print "%s: enqueues=%d dequeues=%d" % (queueMap[oid], record.msgTotalEnqueues, record.msgTotalDequeues) # if the delete-time is non-zero, this object has been deleted. Remove it from the map. if record.getTimestamps()[2] > 0: queueMap.pop(oid) # Create an instance of the QMF session manager. Set userBindings to True to allow # this program to choose which objects classes it is interested in. sess = Session(MyConsole(), manageConnections=True, rcvEvents=False, userBindings=True) # Register to receive updates for broker:queue objects. sess.bindClass("org.apache.qpid.broker", "queue") broker = sess.addBroker() # Suspend processing while the asynchronous operations proceed. try: while True: sleep(1) except: pass # Disconnect the broker before exiting. sess.delBroker(broker) Discovering what Kinds of Objects are Available

Powered by Atlassian Confluence (Version: 2.2.9 Build:#527 Sep 07, 2006) - Bug/feature request

Unsubscribe or edit your notifications preferences

--------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:commits-subscr...@qpid.apache.org