Hey, I see that this is still in open and un-assigned - can you assign it to me so I can mark it as "patch available"? or do you want me just to mark it as "fixed"?
On 28 April 2015 at 08:37, Ido Hadanny <ido.hada...@gmail.com> wrote: > https://issues.apache.org/jira/browse/DATAFU-91 > > > On 27 April 2015 at 18:02, Matthew Hayes <matthew.terence.ha...@gmail.com> > wrote: > >> Great thanks :) Please file a JIRA and attach the patch there. >> >> -Matt >> >> On Apr 27, 2015, at 6:26 AM, Ido Hadanny <ido.hada...@gmail.com> wrote: >> >> Hey guys, >> patch is attached + tested on unit-tests + We're testing it on a >> 1000-nodes real hadoop cluster as we speak. >> Do you want us to create a jira issue for this, or is this good enough? >> Thanks, Ilia and Ido >> >> On 7 March 2015 at 23:09, Matthew Hayes <matthew.terence.ha...@gmail.com> >> wrote: >> >>> I don't remember if there was a particular reason I didn't implement >>> this as AlgebraicEvalFunc. It seems like it could be. I believe the Java >>> MapReduce version leverages the combiner. If you want to try making this >>> Algebraic we would be happy to accept a patch :) >>> >>> -Matt >>> >>> > On Mar 7, 2015, at 12:11 PM, Ido Hadanny <ido.hada...@gmail.com> >>> wrote: >>> > >>> > data.fu has a nice implementation of HyperLogLog for estimating >>> cardinality >>> > here >>> > < >>> https://github.com/apache/incubator-datafu/blob/master/datafu-pig/src/main/java/datafu/pig/stats/HyperLogLogPlusPlus.java >>> > >>> > >>> > However, it's implemented as Accumulator which means it will run only >>> at >>> > the reducer and not in the combiner (but it will never load the entire >>> set >>> > into memory as in normal EvalFunc). Why couldn't data.fu implement it >>> as >>> > Algebraic - and fill the registers at every combiner, then merge and >>> reduce >>> > the result? Am I missing something here? >>> > also available here: >>> > >>> http://stackoverflow.com/questions/28908217/why-is-data-fu-implementing-hyperloglog-as-an-accumulator-and-not-as-algebraic >>> > >>> > thanks! >>> > >>> > >>> > -- >>> > Sent from my androido >>> >> >> >> >> -- >> Sent from my androido >> >> <hyper-log-log-algebraic.diff> >> >> > > > -- > Sent from my androido > -- Sent from my androido
