On Tue, May 30, 2023 at 10:37 AM Kenneth Knowles wrote:
>
> On Sat, May 27, 2023 at 4:20 PM Stephan Hoyer via dev
> wrote:
>
>> On Fri, May 26, 2023 at 2:59 PM Robert Bradshaw
>> wrote:
>>
>>> Yes, with_hot_key_fanout only performs a single level of fanout. I don't
>>> think fanning out more th
On Sat, May 27, 2023 at 4:20 PM Stephan Hoyer via dev
wrote:
> On Fri, May 26, 2023 at 2:59 PM Robert Bradshaw
> wrote:
>
>> Yes, with_hot_key_fanout only performs a single level of fanout. I don't
>> think fanning out more than this has been explored, but I would imagine
>> that for most cases
On Fri, May 26, 2023 at 2:59 PM Robert Bradshaw wrote:
> Yes, with_hot_key_fanout only performs a single level of fanout. I don't
> think fanning out more than this has been explored, but I would imagine
> that for most cases the increased IO would negate most if not all of the
> benefits.
>
My
Yes, with_hot_key_fanout only performs a single level of fanout. I don't
think fanning out more than this has been explored, but I would imagine
that for most cases the increased IO would negate most if not all of the
benefits.
In particular, note that we already do "combiner lifting" to do as muc
We have some use-cases where we are combining over very large sets (e.g.,
computing the average of 1e5 to 1e6 elements, corresponding to hourly
weather observations over the past 50 years).
"with_hot_key_fanout" seems to be rather essential for performing these
calculations, but as far as I can te
