To add to what Hari says, you can try either creating your own avro client
using the Flume SDK (eg, you can grab the default RpcClient from the
factory class) or you could try using the avro-client (eg, bin/flume-ng
avro-client...). If you'd like to use the SDK, I suggest taking a look at:
1) The JavaDocs
2) The tests under flume-ng-sdk module (for executable documentation)
3) The source code under flume-ng-sdk module (to drill down deeper if
desired)
Cheers,
Will
On Wed, Jun 20, 2012 at 10:52 AM, Hari Shreedharan <
hshreedha...@cloudera.com> wrote:
> Hi Som,
>
> AvroSource is designed to received only Avro formatted events, so if you
> write arbitrary data to it, the AvroSource will behave in an undefined way.
> If you have a sink that writes to the AvroSource, it should be using the
> Flume SDK to send the event in the Flume Avro format. If you want to write
> arbitrary data, I suggest taking a look at the netcat source.
>
> Thanks
> Hari
>
> --
> Hari Shreedharan
>
> On Wednesday, June 20, 2012 at 10:48 AM, shekhar sharma wrote:
>
> Hello,
> I am encountering an exception when i am trying to transport the events to
> flume avro source.
> Basically i have created an agent consisting of avro source and my custom
> sink.
> Previously i was using exec source, but due to its limitations i am not
> using it anymore.
>
> Since Avro Source is listening on a port, so i have created a simple
> application, which opens a socket to that port and send the events through
> this socket.
> Socket connection is successful, but when i send the events i got the
> following error:
>
> 2012-06-20 10:34:15,767 INFO source.AvroSource: Avro source
> starting:AvroSource: { bindAddress:localhost port:41414 }
> 2012-06-20 10:35:52,588 INFO ipc.NettyServer: [id: 0x00f0b4a3, /
> 127.0.0.1:46240 => /127.0.0.1:41414] OPEN
> 2012-06-20 10:35:52,612 INFO ipc.NettyServer: [id: 0x00f0b4a3, /
> 127.0.0.1:46240 => /127.0.0.1:41414] BOUND: /127.0.0.1:41414
> 2012-06-20 10:35:52,612 INFO ipc.NettyServer: [id: 0x00f0b4a3, /
> 127.0.0.1:46240 => /127.0.0.1:41414] CONNECTED: /127.0.0.1:46240
> 2012-06-20 10:35:52,792 WARN ipc.NettyServer: Unexpected exception from
> downstream.
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.<init>(ArrayList.java:112)
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:154)
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:131)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
>
> Is there something else i need to perform?
>
> Regards,
> Som Shekhar Sharma
>
>
>
