plate,
> but I could be mistaken.
>
> Michael
>
> --
> *From:* leerho
> *Sent:* Tuesday, May 19, 2020 7:46 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic softwar
eerho
Sent: Tuesday, May 19, 2020 7:46 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academic software
Excellent work!
On Tue, May 19, 2020 at 4:04 PM Jon Malkin
mailto:jon.mal...@gmail.com>> wrote:
I also used k=160,
t has more to do with the compute overhead of generating the data since
>> Numpy tends to be faster when working in 1D vs multiple dimensions. But we
>> can see that it's possible to get runtimes much closer to C++ runtimes than
>> would be expected.
>>
>> Mic
> On Thu, May 14, 2020 at 6:54 PM Michael Himes
> wrote:
>
> Great, I'll be submitting the pull request shortly. The codebase I'm
> working with doesn't have any of the changes made in the past week or so,
> hopefully that isn't too much of a hassle to merge.
>
19, 2020 4:58 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academic software
Well, one thought was maybe we could always use the vectorized kll in python
and make it (relatively) easy to have it work with only 1 dimension. It lo
#x27;t have any of the changes made in the past week or so,
>>> hopefully that isn't too much of a hassle to merge.
>>>
>>> As an aside, my employer encourages us to contribute code to libraries
>>> like this, so I'm happy to work on additional features for t
t; As an aside, my employer encourages us to contribute code to libraries
>> like this, so I'm happy to work on additional features for the Python
>> interface as needed.
>>
>> Michael
>> ----------
>> *From:* Jon Malkin
>> *Sent:* T
ed.
>
> Michael
> --
> *From:* Jon Malkin
> *Sent:* Thursday, May 14, 2020 6:56 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> We've
his, so I'm happy to work on additional features for the Python interface as
needed.
Michael
From: Jon Malkin
Sent: Thursday, May 14, 2020 6:56 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
-
> *From:* Jon Malkin
> *Sent:* Monday, May 11, 2020 4:32 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> I didn't look in super close detail, but the code overall looks
It'd belong in the sketch description, as the type is defined when the
sketch is instantiated. Please create an issue if you find the
documentation lacking.
jon
On Mon, May 11, 2020 at 4:58 PM leerho wrote:
> Then we need clear documentation to explain that in the update method(s).
>
> On Mon
Then we need clear documentation to explain that in the update method(s).
On Mon, May 11, 2020 at 4:17 PM Jon Malkin wrote:
> C++ KLL is templatized so it can accept any user-defined type. 32-bit
> floats are only a requirement if data portability to Java is essential.
> There is no requirement
C++ KLL is templatized so it can accept any user-defined type. 32-bit
floats are only a requirement if data portability to Java is essential.
There is no requirement that every c++ kll_sketch created be portable. We
are unable to enforce that.
And I already create an issue for that NaN bug. Includ
------------------
>> *From:* Jon Malkin
>> *Sent:* Monday, May 11, 2020 4:32 PM
>> *To:* dev@datasketches.apache.org
>> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
>> open-source academic software
>>
>> I didn&#x
> *From:* Jon Malkin
> *Sent:* Monday, May 11, 2020 4:32 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> I didn't look in super close detail, but the code
__
From: Jon Malkin
Sent: Monday, May 11, 2020 4:32 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academic software
I didn't look in super close detail, but the code overall looks pretty good.
Comments are below
maybe this
> is a good update to the main branch.
>
> Michael
> --------------
> *From:* Jon Malkin
> *Sent:* Sunday, May 10, 2020 10:47 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-so
0:47 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academic software
My only comment without having looked at actual code is that the new class
would be more appropriate in the python wrapper. Maybe even drop it in as it's
own file, as
;> 4. That makes sense. Does using Numpy complicate that at all? My
>> thought is that since under the hood everything is using the existing
>> kll_sketch class, it would have full compatibility with the rest of the
>> library (once SerDe is added in).
>>
>> Michae
is added in).
>
> Michael
> --------------
> *From:* leerho
> *Sent:* Sunday, May 10, 2020 8:42 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> Thanks for the
it
would have full compatibility with the rest of the library (once SerDe is added
in).
Michael
From: leerho
Sent: Sunday, May 10, 2020 8:42 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academ
l data can also be used and one comparison test I would recommend is
>> comparing how long it takes to get approximate results using sketches
>> versus how long it would take to get exact results using brute force
>> methods. The bigger the data set is the better :)
>>
>&
compute resources permitting.
>
> Michael
> --
> *From:* leerho
> *Sent:* Sunday, May 10, 2020 5:32 PM
> *To:* dev@datasketches.apache.org
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
>
look at those tests you linked to and
see about running them, time and compute resources permitting.
Michael
From: leerho
Sent: Sunday, May 10, 2020 5:32 PM
To: dev@datasketches.apache.org
Subject: Re: Permission to use KLL streaming-quantiles code in free open-so
ry
> integrated into numpy/scipy would be awesome, I'm sure I could get some
> others in my field to begin using it.
>
> Michael
> --
> *From:* Lee Rhodes
> *Sent:* Saturday, May 9, 2020 5:06 PM
> *To:* Michael Himes ; dev@datasketches.apache.
aturday, May 9, 2020 5:06 PM
To: Michael Himes mailto:mhi...@knights.ucf.edu>>;
dev@datasketches.apache.org<mailto:dev@datasketches.apache.org>
mailto:dev@datasketches.apache.org>>
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academic software
This is just awes
rovements, I'm going to move some
> code around to follow the existing coding style, put it on Github, and
> submit a pull request.
>
> Michael
> --------------
> *From:* Lee Rhodes
> *Sent:* Saturday, May 9, 2020 4:22 PM
> *To:* Michael Himes
> *Subject
--
> *From:* Lee Rhodes
> *Sent:* Saturday, May 9, 2020 4:22 PM
> *To:* Michael Himes
> *Subject:* Fwd: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> Hi Michael,
> I don't think you saw this email as I doubt you are subscrib
thub, and submit a pull request.
Michael
From: Lee Rhodes mailto:lrho...@verizonmedia.com>>
Sent: Saturday, May 9, 2020 4:22 PM
To: Michael Himes mailto:mhi...@knights.ucf.edu>>
Subject: Fwd: Permission to use KLL streaming-quantiles code in free
ope
7, 2020 12:00 PM
> *To:* Michael Himes
> *Cc:* Edo Liberty ; dev@datasketches.apache.org <
> dev@datasketches.apache.org>; e...@edoliberty.com
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> If you do figure
th it.
Michael
From: Lee Rhodes
Sent: Thursday, May 7, 2020 12:00 PM
To: Michael Himes
Cc: Edo Liberty ; dev@datasketches.apache.org
; e...@edoliberty.com
Subject: Re: Permission to use KLL streaming-quantiles code in free open-source
academic software
If you do figure out how to do this, it
gt; *To:* Michael Himes
> *Cc:* dev@datasketches.apache.org ; Edo
> Liberty ; e...@edoliberty.com
>
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> Michael,
>
> One of my colleagues, Jon Malkin, pointed out that the vector-
ches.apache.org<mailto:dev@datasketches.apache.org>
mailto:dev@datasketches.apache.org>>
Cc: Edo Liberty mailto:edo.libe...@gmail.com>>;
e...@edoliberty.com<mailto:e...@edoliberty.com>
mailto:e...@edoliberty.com>>
Subject: Re: Permission to use KLL streaming-quantiles c
sary to loop over the streams since we can make use of Numpy's
>> broadcasting, which will handle the looping in its C++ code, as you
>> mentioned. I'll work on this and see if it really is as straight-forward
>> as it seems.
>>
>> If you have any advice on how
; the list-specific methods that are used, like .append(). Then, it isn't
>> necessary to loop over the streams since we can make use of Numpy's
>> broadcasting, which will handle the looping in its C++ code, as you
>> mentioned. I'll work on this and see if it
ode that would cause issues for my
> use case? Are the other methods offered in datasketches 'better' than the
> KLL implemented in streaming-quantiles? I'm quite out of my area of
> expertise, so I appreciate any advice you can offer, and I will of course
> acknowledge it in the publ
8:09 PM
> *To:* Lee Rhodes ; Michael Himes <
> mhi...@knights.ucf.edu>
> *Cc:* e...@edoliberty.com
> *Subject:* Re: Permission to use KLL streaming-quantiles code in free
> open-source academic software
>
> +Lee
>
> Hi Michael, Thanks for reaching out.
> While you
37 matches
Mail list logo
