On Mon, Aug 26, 2013 at 7:44 PM, Hiranya Jayathilaka
<hiranya...@gmail.com> wrote:
> Hi,
>
> On Aug 26, 2013, at 4:48 AM, "Wright, Peter" <peter.wri...@six-group.com>
> wrote:
>
>> Hi Hiranja,
>>
>> No problem. Thanks for the support.
>>
>> Peter
>>
>> ===============================================================
>>
>> /*
>> * Filename: syslogMsgBuilder.java
>> * Author: P.Wright
>> * Date: 06.08.2013
>> * Description: Java class for Synapse class mediator.
>> * Class is embedded in Synapse, and called when a SYSLOG
>> message is received.
>> *
>> * The SYSLOG message can contain 1-n syslog messages, each
>> * separated by a NL, and the length of each message defined
>> * at the start of it. This is called TCP-Framing
>> "octet-counted".
>> * See: http://www.rsyslog.com/doc/omfwd.html
>> *
>> * Example input syslog message containing 3 messages
>> * 34 <1> This is dummy message number 1
>> * 34 <2> This is dummy message number 2
>> * 34 <3> This is dummy message number 3
>> *
>> * Example Synapse-SOAP message with 3 syslog messages in the
>> XML payload:
>> * <?xml version='1.0' encoding='utf-8'?>
>> * <soapenv:Envelope
>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";>
>> * <soapenv:Body>
>> * <text xmlns="http://ws.apache.org/commons/ns/payload";>
>> * 34 <1> This is dummy message number 1
>> * 34 <2> This is dummy message number 2
>> * 34 <3> This is dummy message number 3
>> * </text>
>> * </soapenv:Body>
>> * </soapenv:Envelope>
>> *
>> * Actions as follows:
>> * - Get the payload
>> * - Split the payload on NL
>> * - Remove the msglen at the start of each message
>> * - Replace any "<" with "<"
>> * - Replace the payload with an XML content with 3 messages
>> (see below)
>> *
>> * Output message as follows:
>> * <?xml version='1.0' encoding='utf-8'?>
>> * <soapenv:Envelope
>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";>
>> * <soapenv:Body>
>> * <text
>> xmlns="http://ws.apache.org/commons/ns/payload";><1>This is dummy message
>> number 1</text>
>> * <text
>> xmlns="http://ws.apache.org/commons/ns/payload";><2>This is dummy message
>> number 2</text>
>> * <text
>> xmlns="http://ws.apache.org/commons/ns/payload";><3>This is dummy message
>> number 3</text>
>> * </soapenv:Body>
>> * </soapenv:Envelope>
>> *
>> * Method Overview:
>> * - mediate ("main" method)
>> */
>> package com.sixtelekurs.classMediators;
>>
>> import java.util.Iterator;
>> import org.apache.axiom.om.OMElement;
>> import org.apache.axiom.om.util.AXIOMUtil;
>> import org.apache.axiom.soap.SOAPBody;
>> import org.apache.axiom.soap.SOAPEnvelope;
>> import org.apache.commons.logging.Log;
>> import org.apache.commons.logging.LogFactory;
>> import org.apache.synapse.MessageContext;
>> import org.apache.synapse.Mediator;
>>
>>
>> public class syslogMsgBuilder implements Mediator
>> {
>> private static String className = "syslogMsgBuilder";
>> private static final Log log = LogFactory.getLog(syslogMsgBuilder.class);
>> private String _desc = null;
>> private String _uid = null;
>>
>> //
>> ------------------------------------------------------------------------
>> public boolean mediate(MessageContext mc)
>> {
>> String logMsg = "===" + className + ".mediate called=============";
>> log.debug(logMsg);
>>
>> // Get the message content and process it
>> String syslogStr =
>> mc.getEnvelope().getBody().getFirstElement().getText();
>
> I believe this line if the issue. The call to getText() on the
> OMSourcedElement probably causes it to read the entire input stream until
> end-of-stream is encountered. But end-of-stream does not occur until the
> sender closes its output stream at the remote end.
>
> I think a proper solution should get the InputStream from the
> OMSourcedElement and try to read from it until a known number of bytes have
> been read or a particular delimiter character is encountered. In a case like
> HTTP, the HTTP message usually have the content-length header or a special
> delimiter sequence (chunk delimiter) which allows the server to know how much
> data to read from the input stream without reading all the way to the
> end-of-stream. In case of TCP transport, there are no special headers to look
> at, so you must rely on the information available on the message payload it
> self.
>
> That's just my take on it. May be there's a better/easier way to solve this
> problem at Axiom level (perhaps a special method in the OMSourcedElement
> API?). May be Andreas can shed some light on the matter.
For a message built by PlainTextBuilder, you can get the (charset
decoded character) stream of the original message using the
getTextAsStream(false) method on the wrapper element (i.e. the
OMSourcedElement). It is not exactly the original stream because there
is a character stream -> StAX events -> character stream conversion
involved here, but the content is the same, and in practice it can be
expected to have the same blocking behavior (i.e. it will only block
if no more data is available from the original stream), although Axiom
doesn't make such a guarantee.
However, I would strongly suggest not to use this feature in the use
case discussed here, for reasons that have nothing to do with Axiom.
More on this later.
Andreas
>
> Thanks,
> Hiranya
>
>> logMsg = "syslogStr='" + syslogStr + "'";
>> log.debug(logMsg);
>>
>> // Delete the existing payload
>> logMsg = "Deleting existing payload";
>> log.debug(logMsg);
>> SOAPBody soapBody = mc.getEnvelope().getBody();
>> for (Iterator itr = soapBody.getChildElements(); itr.hasNext();)
>> {
>> itr.next();
>> itr.remove();
>> }
>>
>> // Split the strings into N syslog messages, convert each to
>> // an OMElement, and append to the XML-Body
>> int msgCnt = 0;
>> try
>> {
>> // Split the string on NL
>> String[] lines = syslogStr.split("\\n");
>> logMsg = "syslog.input.cnt(NL)=" + lines.length;
>> log.debug(logMsg);
>> for (int ii=0; ii < lines.length; ii++)
>> {
>> // Now split on space (once only) to ignore the msglen
>> String[] msg = lines[ii].split(" ", 2);
>> if (msg.length == 2)
>> {
>> // Replace all "<" with "<" for XML
>> String syslogMsg = msg[1].replaceAll("<", "<");
>> int jj = ii+1;
>> logMsg = "syslogMsg." + jj + "='" + syslogMsg + "'";
>> log.debug(logMsg);
>> String xmlStr = "<text
>> xmlns=\"http://ws.apache.org/commons/ns/payload\";>";
>> xmlStr += syslogMsg;
>> xmlStr += "</text>";
>> msgCnt++;
>>
>> // Now add elem
>> OMElement xmlElem = AXIOMUtil.stringToOM(xmlStr);
>> logMsg = "xmlStr." + jj + "='" + xmlStr + "'";
>> log.debug(logMsg);
>> soapBody.addChild(xmlElem);
>> }
>> else
>> {
>> logMsg = "msg could not be split correctly [" + msg + "]";
>> log.error(logMsg);
>> }
>> }
>> }
>> catch (Exception e)
>> {
>> logMsg = "ERROR: " + e;
>> log.error(logMsg);
>> return(false);
>> }
>>
>> if (msgCnt == 0)
>> {
>> logMsg = "ERROR: no messages processed";
>> log.error(logMsg);
>> return(false);
>> }
>>
>> logMsg = "Made " + msgCnt + " messages";
>> log.info(logMsg);
>>
>> SOAPEnvelope envelope = mc.getEnvelope();
>> logMsg = "SOAP.Envelope: " + envelope;
>> log.debug(logMsg);
>>
>> return(true);
>> }
>>
>> // SET methods --------------------------------
>> public String getDescription()
>> {
>> String msg = className + ".desc";
>> return(msg);
>> }
>> public String getType()
>> {
>> String msg = className + ".type";
>> return(msg);
>> }
>> public int getTraceState()
>> {
>> return(0);
>> }
>>
>> // SET methods --------------------------------
>> public void setTraceState(int traceState)
>> {
>> traceState = 0;
>> }
>> public void setDescription(String desc)
>> {
>> _desc = desc;
>> }
>> public void setUid(String uid)
>> {
>> _uid = uid;
>> }
>> }
>>
>> -----Original Message-----
>> From: Hiranya Jayathilaka [mailto:hiranya...@gmail.com]
>> Sent: Freitag, 23. August 2013 08:30
>> To: user@synapse.apache.org
>> Subject: Re: Synapse: TCP syslog to JMS proxy: messages only processed in
>> synapse after client TCP connection is closed
>>
>> It sounds like something is trying to read from the input stream until the
>> end of stream is encountered. This may be the plain text builder or your
>> custom mediator. Can you share your custom mediator source code so we can
>> take a look?
>>
>> Thanks,
>> Hiranya
>>
>> On Aug 22, 2013, at 8:27 AM, "Wright, Peter" <peter.wri...@six-group.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have my TCP Syslog --> JMS proxy up and running (see previous emails),
>>> but now have encountered another problem.
>>> Up till now I have simply tested by using netstat to send 1-n messages
>>> stored in a file to the TCP server.
>>> In this case, the messages are being received and processed correctly by
>>> synapse (sent as JMS
>>> messages to the JMS server). It seems that this is working correctly,
>>> because the connection to the
>>> synapse TCP server is being closed by netstat after it sends the messages.
>>>
>>> In my real world case however, we have 1-n clients writing syslog messages
>>> to a
>>> local syslog server. This syslog server has been configured to send the
>>> messages on
>>> to the (remote) synapse TCP server. In this case the messages are NOT being
>>> received
>>> and processed by synapse. Only after the syslog server closes it connection
>>> to the synapse
>>> TCP server, are the messages being received and processed by synapse.
>>>
>>> Any ideas what the problem could be?
>>> Attached again my synapse.xml config file.
>>>
>>> Thanks,
>>> Peter
>>>
>>> ===============================================
>>>
>>> <definitions xmlns="http://ws.apache.org/ns/synapse";>
>>> <sequence name="fault">
>>> <makefault>
>>> <code xmlns:tns="http://www.w3.org/2003/05/soap-envelope";
>>> value="tns:Receiver"/>
>>> <reason value="Mediation failed."/>
>>> </makefault>
>>> <send/>
>>> </sequence>
>>> <sequence xmlns="http://ws.apache.org/ns/synapse"; name="main"
>>> onError="fault">
>>> <in>
>>> <log level="full"/>
>>> <send/>
>>> </in>
>>> <out>
>>> <send/>
>>> </out>
>>> </sequence>
>>>
>>> <proxy name="proxyTcp2Jms" transports="tcp">
>>> <target>
>>> <inSequence>
>>> <!-- Define TCP listener expects plain text (syslog) messages
>>> -->
>>> <property name="messageType" value="text/plain"
>>> scope="axis2"/>
>>> <property name="OUT_ONLY" value="true"/>
>>> <property name="TRANSPORT_HEADERS" scope="axis2"
>>> action="remove"/>
>>> <log level="full"/>
>>>
>>> <!-- Split any multiple syslog messages for the iterator -->
>>> <!-- and return as XML with child messages -->
>>> <class name="com.sixtelekurs.classMediators.syslogMsgBuilder">
>>> <log level="full"/>
>>> </class>
>>>
>>> <log level="full"/>
>>>
>>> <!-- Iterate over any multiple messages -->
>>> <iterate id="syslogInterator" preservePayload="false"
>>> sequential="true" xmlns:m0="http://ws.apache.org/commons/ns/payload";
>>> expression="//m0:text">
>>> <target>
>>> <sequence>
>>> <send>
>>> <endpoint>
>>> <address
>>> uri="jms:/cn=sed.finesb.syslog?java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory&java.naming.provider.url=LDAP_URL&transport.jms.ConnectionFactoryJNDIName=MY_TCF&transport.jms.DestinationType=topic&java.naming.security.principal=MY_DN&java.naming.security.credentials=MY_PASSWD"/>
>>> </endpoint>
>>> </send>
>>> </sequence>
>>> </target>
>>> </iterate>
>>> </inSequence>
>>> <outSequence/>
>>> <faultSequence>
>>> <log level="full" category="ERROR" separator=","/>
>>> </faultSequence>
>>> </target>
>>> <parameter name="transport.tcp.port">6060</parameter>
>>> <parameter name="transport.tcp.contentType">text/plain</parameter>
>>> </proxy>
>>>
>>> </definitions>
>>>
>>> The content of this e-mail is intended only for the confidential use of the
>>> person addressed.
>>> If you are not the intended recipient, please notify the sender and delete
>>> this email immediately.
>>> Thank you.
>>
>> --
>> Hiranya Jayathilaka
>> Mayhem Lab/RACE Lab;
>> Dept. of Computer Science, UCSB; http://cs.ucsb.edu
>> E-mail: hira...@cs.ucsb.edu; Mobile: +1 (805) 895-7443
>> Blog: http://techfeast-hiranya.blogspot.com
>>
>> The content of this e-mail is intended only for the confidential use of the
>> person addressed.
>> If you are not the intended recipient, please notify the sender and delete
>> this email immediately.
>> Thank you.
>
> --
> Hiranya Jayathilaka
> Mayhem Lab/RACE Lab;
> Dept. of Computer Science, UCSB; http://cs.ucsb.edu
> E-mail: hira...@cs.ucsb.edu; Mobile: +1 (805) 895-7443
> Blog: http://techfeast-hiranya.blogspot.com
>
