Re: how to resolve conflicts with jar dependencies

2013-03-13 Thread Jane Wayne
ue, Mar 12, 2013 at 6:49 AM, Jane Wayne wrote: > >> hi, >> >> i need to know how to resolve conflicts with jar dependencies. >> >> * first, my job requires Jackson JSON-processor v1.9.11. >> * second, the hadoop cluster has Jackson JSON-processor v1.5.2. the >&

how to resolve conflicts with jar dependencies

2013-03-12 Thread Jane Wayne
hi, i need to know how to resolve conflicts with jar dependencies. * first, my job requires Jackson JSON-processor v1.9.11. * second, the hadoop cluster has Jackson JSON-processor v1.5.2. the jars are installed in $HADOOP_HOME/lib. according to this link, http://blog.cloudera.com/blog/2011/01/ho

Re: Cumulative value using mapreduce

2012-10-05 Thread Jane Wayne
s you will have to come up with a composite key). when the data comes into the reducer, just keep a running count and emit each time. On Fri, Oct 5, 2012 at 11:21 AM, Jane Wayne wrote: > there's probably a million ways to do it, but it seems like it can be > done, per your question. off th

Re: Cumulative value using mapreduce

2012-10-05 Thread Jane Wayne
there's probably a million ways to do it, but it seems like it can be done, per your question. off the top of my head, you'd probably want to do the cumulative sum in the reducer. if you're savy, maybe even make the reducer reusable as a combiner (looks like this problem might have an associative a

Re: strategies to share information between mapreduce tasks

2012-09-26 Thread Jane Wayne
jay, thanks. i just needed a sanity check. i hope and expect that one day, hadoop will mature towards supporting a "shared-something" approach. the web service call is not a bad idea at all. that way, we can abstract what that ultimate data store really is. i'm just a little surprised that we are