RE: Using Hbase for Dedupping

2013-02-14 Thread Anoop Sam John
many times)? -Anoop- From: Rahul Ravindran [rahu...@yahoo.com] Sent: Friday, February 15, 2013 12:53 AM To: user@hbase.apache.org Subject: Using Hbase for Dedupping Hi, We have events which are delivered into our HDFS cluster which may be duplicated

Using Hbase for Dedupping

2013-02-14 Thread Rahul Ravindran
Hi,    We have events which are delivered into our HDFS cluster which may be duplicated. Each event has a UUID and we were hoping to leverage HBase to dedupe them. We run a MapReduce job which would perform a lookup for each UUID on HBase and then emit the event only if the UUID was absent and w