Hi Mohit, 2014-10-27 19:30 GMT+01:00 Mohit Durgapal <[email protected]>:
> > I am using rsyslog to send messages to flume nodes via AWS ELB. On flume > nodes I am using the source type *syslogtcp * where the ELB forwards the > messages. Now I see the messages that are over 2k in size are being broken > into chunks of size 2k when I receive them in flume. As my messages are in > JSON this breaks the structure and my application reading the logs from > hdfs see them as ill-formed JSON and skip those records. > You need to increase the message size both in rsyslog (as you did) *and* in your Flume agent. The Syslog source has an eventSize option that defaults to 2500 bytes. You can check the details in the docs: https://flume.apache.org/FlumeUserGuide.html#syslog-tcp-source Best, -- Santiago M. Mola <http://www.stratio.com/> Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón, Madrid Tel: +34 91 352 59 42 // *@stratiobd <https://twitter.com/StratioBD>*
