RE: accumulo sink

[Martin, Ray]

Thank you.  You all have been so helpful.  I will get started on
creating the custom sink

 

From: shekhar sharma [mailto:shekhar2...@gmail.com] 
Sent: Thursday, June 21, 2012 2:34 AM
To: flume-user@incubator.apache.org
Subject: Re: accumulo sink

 

Well i would suggest instead of changing the flume base code, take core
jars from flume and build your own custom sink for flume.. and put the
jar inside the lib directory. When you create jar, make sure all the
dependencies are present in the jar.

A template for your any custom sink would look like as follows:

import org.apache.flume.Channel;
import org.apache.flume.ChannelException;

import org.apache.flume.Event;
import org.apache.flume.event.SimpleEvent;
import org.apache.flume.EventDeliveryException;
import org.apache.flume.Sink;
import org.apache.flume.Transaction;
import org.apache.flume.sink.AbstractSink;

/* your accumulo imports can go here*/

public class AccumuloSink extends AbstractSink /*This is required*/ {

public AccumuloSink()
{
//Your Accumulo initializers 
}
/* this method you need to override*/
@Override
  public Status process() throws EventDeliveryException {
    Status result = Status.READY;
Channel channel = getChannel();
    Transaction transaction = channel.getTransaction();
    Event event = null;

    try
        {
                 transaction.begin();
                 event = channel.take();
                 if(event != null)
                 {
                       //here you can send your events to accumulo sink
                 }
                transaction.commit();

      }
    catch (Exception ex)
     {
      transaction.rollback();
      throw new EventDeliveryException("Failed to log event: " + event,
ex);
     }
     finally
     {
      transaction.close();
    }
    return result;
  }
}

}

Hope this helps and expedite your process.

Regards,
Som Shekhar

On Thu, Jun 21, 2012 at 1:25 AM, Eric Sammer <esam...@cloudera.com>
wrote:

Ray:

 

Not off hand, no. That said, it shouldn't be terribly difficult to
build. I'm less familiar with the Accumulo client APIs, but it's so
close to HBase that I don't believe it would take you more than a day or
so with basic testing. Take a look at one of the sinks (one of the more
basic being the LoggerSink) to get a template to start from.

 

On Tue, Jun 19, 2012 at 1:14 PM, Martin, Ray <arma...@tycho.ncsc.mil>
wrote:

Is anyone aware of a Flume sink for Accumulo?

Thanx.





 

-- 
Eric Sammer
twitter: esammer
data: www.cloudera.com