Hello, I am new to Twisted. I am extending the twisted Echo server example by sending the data that the server receives from telnet into a child process that processes the data. The function outReceived in censoringProtocol is called when the child writes the result and the result is received. The process of passing data from parent to child and back to parent is done by having a while loop of stdin.readline() and stdout.write() in the child process. Specifically, I generate a deferred object each time I write to the child using the function sendData (which then calls writeToChild) and use a queue to manage the deferred objects. I add a callBack (printData) that will broadcast the result to all instances of Echo when outReceived is called.
I have two questions: 1/ Is this the right approach to communicate with the child process in order to achieve what I want? Are there any gotcha's? 2/ This server will be exposed to high loads. There will be many instances of Echo sending data to the child at a high rate. Are there any limitations that I have to watch for, such as the limit on child process's pipe sizes or anything else? My code is down below, as well as in this link: https://gist.github.com/2920032 Thank You for your time. Quan Nguyen from sys import executable from os import environ from twisted.internet import reactor, defer from twisted.internet.protocol import Protocol from twisted.internet.protocol import Factory from twisted.internet import protocol from twisted.protocols.basic import LineReceiver import sys import time import Queue import os implementation = """ import os import time import sys import string def censor(): BW = ["sfs", "fsdfwe", "tryt", "sfsvx"] while (True): a = sys.stdin.readline() if a: for bannedWord in BW: a = string.replace(a, bannedWord , '<banned>') sys.stdout.write(a) sys.stdout.flush() """ class Echo(LineReceiver): def lineReceived(self, data): d = self.factory.process.sendData(data+'\r\n') #reactor.callLater(3,d.addCallback, printData) d.addCallback(printData) prot.transport.write(data) def createProcess(): pp = censoringProtocol() Process = reactor.spawnProcess(pp, executable, [executable, "-c", implementation], env=environ, childFDs = {0:"w", 1:"r", 2:"r"}) pp.process = Process return pp class EchoFactory(Factory): protocol = Echo telnetConnectionList = [] process = None deferQ = None def __init__(self): EchoFactory.process = createProcess() EchoFactory.deferQ = Queue.Queue() def buildProtocol (self, addr): newProtocol = Factory.buildProtocol(self, addr) EchoFactory.telnetConnectionList.append(newProtocol) return newProtocol class censoringProtocol(protocol.ProcessProtocol): def __init__(self): self.process = None def connectionMade(self): print "connectionMade!" def outReceived(self, data): EchoFactory.deferQ.get().callback(data) def sendData(self,data): d = defer.Deferred() EchoFactory.deferQ.put(d) self.process.writeToChild(0, data) return d reactor.listenTCP(11111, EchoFactory()) print 'in parent', os.getpid() reactor.run() _______________________________________________ Twisted-Python mailing list [email protected] http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
