Yeah, actually I have started working on a new syslog consumer implementation. As part of this I am also planning to write brand new TCP and UDP transports(for high volume syslog message deployments). I will report any progress as I go forward.
Rajika On Fri, Aug 9, 2013 at 3:15 PM, Hiranya Jayathilaka <[email protected]>wrote: > > On Aug 9, 2013, at 4:57 AM, "Wright, Peter" <[email protected]> > wrote: > > > Hi Hiranja, > > > > Great, thanks a lot, now appears to be working correctly!! > > That's great. > > I think it would be good to have some syslog mediation support built into > Synapse. The correct solution is probably to implement a syslog message > builder and a formatter. There's a long standing jura issue regarding this > as well. We'll consider this feature for a future release. > > Thanks, > Hiranya > > > I changed my class mediator to generate the XML as you suggested, and > then had to > > include the namespace in the iterate definition and include it in the > xpath expression > > before it worked correctly. > > > > Best regards, > > Peter > > > > ================================= > > > > 1. XML message returned by my class mediator > > > > <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><text xmlns=" > http://ws.apache.org/commons/ns/payload";>This is msg number 1</text><text > xmlns="http://ws.apache.org/commons/ns/payload";>This is msg number > 2</text><text xmlns="http://ws.apache.org/commons/ns/payload";>This is msg > number 3</text></soapenv:Body></soapenv:Envelope> > > > > 2. synapse.xml fragment (iterator definition) > > > > <iterate id="syslogInterator" preservePayload="false" sequential="true" > xmlns:m0="http://ws.apache.org/commons/ns/payload"; expression="//m0:text"> > > <target> > > <sequence> > > <send> > > <endpoint> > > <address .../> > > </endpoint> > > </send> > > </sequence> > > </target> > > </iterate> > > > > -----Original Message----- > > From: Hiranya Jayathilaka [mailto:[email protected]] > > Sent: Freitag, 9. August 2013 01:54 > > To: [email protected] > > Subject: Re: Synapse: tcp-listener: handling of multiple syslog messages > in one TCP message? > > > > > > On Aug 7, 2013, at 4:53 AM, "Wright, Peter" <[email protected]> > wrote: > > > >> Hi Hiranja, > >> > >> I have now implemented a class mediator that parses the (multiple) > >> syslog plain text messages and makes an XML payload from them. > >> I've also got an iterate mediator that splits the messages and > >> sends them individually as JMS messages to the messaging provider. > >> So far so good... > >> > >> However, the individual JMS messages are now in XML form rather > >> than plain text (with all the XML headers that I don't want). > >> How can I get just the plain text payload to be sent? > >> > >> Thanks > >> Peter > >> > >> ===================== > >> > >> 1. Input: Dummy (syslog) message sent to TCP listener (individual > messages separated by NL) > >> 20 This is msg number 1 > >> 20 This is msg number 2 > >> 20 This is msg number 3 > >> > >> 2. SOAP message logged by synapse > >> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><text xmlns=" > http://ws.apache.org/commons/ns/payload";>20 This is msg number 1 > >> 20 This is msg number 2 > >> 20 This is msg number 3 > >> </text></soapenv:Body></soapenv:Envelope> > >> > >> 3. XML message returned by my class mediator > >> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><syslogMsgs><msg>This > is msg number 1</msg><msg>This is msg number 2</msg><msg>This is msg number > 3</msg></syslogMsgs></soapenv:Body></soapenv:Envelope> > > > > This should create a message with multiple <text> elements. See the > output from step 2 above. It has a single <text> element with all the > content. You should break it down to multiple <text> elements. The plain > text serializer of Axis2 looks for this specific <text> element when > sending plain text messages. > > > > Pay attention to the namespace (http://ws.apache.org/commons/ns/payload) > of the <text> element too. It's also required. > > > > Thanks, > > Hiranya > > > >> > >> 4. JMS messages sent to the messaging provider > >> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><msg>This is msg > number 1</msg></soapenv:Body></soapenv:Envelope> > >> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><msg>This is msg > number 2</msg></soapenv:Body></soapenv:Envelope> > >> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><msg>This is msg > number 3</msg></soapenv:Body></soapenv:Envelope> > >> > >> 5. JMS messages that I want to be sent > >> This is msg number 1 > >> This is msg number 2 > >> This is msg number 3 > >> > >> 6. synapse.xml > >> <definitions xmlns="http://ws.apache.org/ns/synapse";> > >> <proxy name="proxyTcp2Jms" transports="tcp"> > >> <target> > >> <inSequence> > >> <!-- Define TCP listener expects plain text (syslog) > messages --> > >> <property name="messageType" value="text/plain" > scope="axis2"/> > >> <property name="OUT_ONLY" value="true"/> > >> <property name="TRANSPORT_HEADERS" scope="axis2" > action="remove"/> > >> <log level="full"/> > >> > >> <!-- Split any multiple syslog messages for the iterator > --> > >> <!-- and return as XML with child messages > --> > >> <class > name="com.sixtelekurs.classMediators.syslogMsgBuilder"> > >> <log level="full"/> > >> </class> > >> > >> <!-- Iterate over any multiple messages --> > >> <iterate id="syslogInterator" preservePayload="false" > sequential="true" expression="//msg"> > >> <log level="full"/> > >> <target> > >> <sequence> > >> <send> > >> <endpoint> > >> <address > uri="jms:/cn=sed.finesb.syslog?java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory&java.naming.provider.url=MY_LDAP_URL&transport.jms.ConnectionFactoryJNDIName=cn=MY_TCP&transport.jms.DestinationType=topic&java.naming.security.principal=MY_DN&java.naming.security.credentials=MY_PW"/> > >> </endpoint> > >> </send> > >> </sequence> > >> </target> > >> </iterate> > >> </inSequence> > >> <outSequence/> > >> <faultSequence> > >> <log level="full" category="ERROR" separator=","/> > >> </faultSequence> > >> </target> > >> <parameter name="transport.tcp.port">6060</parameter> > >> <parameter name="transport.tcp.contentType">text/plain</parameter> > >> </proxy> > >> > >> </definitions> > >> > >> > >> > >> -----Original Message----- > >> From: Hiranya Jayathilaka [mailto:[email protected]] > >> Sent: Freitag, 26. Juli 2013 20:05 > >> To: [email protected] > >> Subject: Re: Synapse: tcp-listener: handling of multiple syslog > messages in one TCP message? > >> > >> You should probably apply some kind of a content transformation to > transform this payload into a XML payload with multiple "text" elements > (one per each syslog message). But I'm not sure whether traditional > techniques like XSLT and XQuery support that kind of transformations. After > all you're trying to transform based on XML text and not the XML nodes. > That's almost always very difficult. > >> > >> IMO you're better off writing a custom mediator to do this > transformation. Then you can use the iterate mediator to split the message > into its constituents and send to the JMS queue. An even more elegant > solution would be to write a custom message builder that knows how to parse > the incoming batch requests and construct a proper XML payload out of it. > >> > >> Thanks, > >> Hiranya > >> > >> On Jul 26, 2013, at 6:10 AM, "Wright, Peter" < > [email protected]> wrote: > >> > >>> Hi, > >>> > >>> I got synapse working with a TCP-Listener accepting plain text messages > >>> as input, and the messages being sent as JMS messages to IBM Websphere > MQ. > >>> > >>> Now we're in the test phase with "real" syslog (RFC-5424) data, and > I'm having a problem > >>> because the TCP listener is receiving multiple syslog messages in one > TCP message. > >>> The multiple messages are being sent on as one JMS message. The > individual > >>> syslog messages are separated by NL and have the message length at the > start. > >>> > >>> Here an example message from my log file with 8 syslog messages: > >>> > >>> 2013-07-25T16:12:20,INFO ,mdzhsfesb01,MSGMED,MediatorLog.java:99,To: , > WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: > urn:uuid:1BC21F33F4BAA449111374761227027, Direction: request, Envelope: > >>> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><text xmlns=" > http://ws.apache.org/commons/ns/payload";> > >>> 186 <174>1 2013-07-25T16:08:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="1"] [HEARTBEAT@7410 Type="Heartbeat" > timestamp="1374761220" telegrams="5522922" BKs="632" LineIDs="441"] > >>> 400 <170>1 2013-07-25T16:09:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="2"] [Ticket_605@7410 Type="Feed Outage" Status="New" > BC="462" BK="842" Occured="1374760680" Reported="1374761340" > Summary="Outage of BC 462 at 2013-07-25 13:58:00 UTC (DAWD on mdzhldos02)" > Details="Outage of BC 462; BK 842; No Bid/Asks for 11 minutes; Occured: > 2013-07-25 13:58:00 UTC; Reported by DAWD on mdzhldos02"] > >>> 186 <174>1 2013-07-25T16:09:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="3"] [HEARTBEAT@7410 Type="Heartbeat" > timestamp="1374761280" telegrams="6001019" BKs="665" LineIDs="441"] > >>> 186 <174>1 2013-07-25T16:10:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="4"] [HEARTBEAT@7410 Type="Heartbeat" > timestamp="1374761340" telegrams="4908793" BKs="612" LineIDs="440"] > >>> 119 <174>1 2013-07-25T16:10:40+02:00 mdzhldos02 DaMon.DAWD 24438 - > [meta sequenceId="5"] [HEARTBEAT@7410 type="Heartbeat"] > >>> 362 <170>1 2013-07-25T16:11:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="6"] [Ticket_605@7410 Status="Close" BC="462" > BK="(null)" Occured="1374761340" Reported="1374761460" Summary="Problem > with BC 462 disappeared at 2013-07-25 14:09:00 UTC (DAWD on mdzhldos02)" > Details="Problem with BC 462 disappeared at 2013-07-25 14:09:00 UTC (DAWD > on mdzhldos02)"] > >>> 186 <174>1 2013-07-25T16:11:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="7"] [HEARTBEAT@7410 Type="Heartbeat" > timestamp="1374761400" telegrams="6099578" BKs="657" LineIDs="440"] > >>> 186 <174>1 2013-07-25T16:12:00+02:00 mdzhldos02 DaMon.DAWD 12355 - > [meta sequenceId="8"] [HEARTBEAT@7410 Type="Heartbeat" > timestamp="1374761460" telegrams="5570590" BKs="613" LineIDs="446"] > >>> </text></soapenv:Body></soapenv:Envelope> > >>> > >>> What is the recommended way of handling this? Use the Iterate-Mediator > to > >>> split the messages and send them individually? Do I have to use Xpath > here, > >>> or is it possible to use e.g. javascript? Any examples would be > appreciated. > >>> > >>> Thanks for any help, > >>> Peter > >>> > >>> > >>> The content of this e-mail is intended only for the confidential use > of the person addressed. > >>> If you are not the intended recipient, please notify the sender and > delete this email immediately. > >>> Thank you. > >> > >> -- > >> Hiranya Jayathilaka > >> Mayhem Lab/RACE Lab; > >> Dept. of Computer Science, UCSB; http://cs.ucsb.edu > >> E-mail: [email protected]; Mobile: +1 (805) 895-7443 > >> Blog: http://techfeast-hiranya.blogspot.com > >> > >> The content of this e-mail is intended only for the confidential use of > the person addressed. > >> If you are not the intended recipient, please notify the sender and > delete this email immediately. > >> Thank you. > > > > -- > > Hiranya Jayathilaka > > Mayhem Lab/RACE Lab; > > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > > Blog: http://techfeast-hiranya.blogspot.com > > > > The content of this e-mail is intended only for the confidential use of > the person addressed. > > If you are not the intended recipient, please notify the sender and > delete this email immediately. > > Thank you. > > -- > Hiranya Jayathilaka > Mayhem Lab/RACE Lab; > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > Blog: http://techfeast-hiranya.blogspot.com > >
