domain. One lesson we had was, it is hard to have a
>>>>>>> generic algorithm that worked for all cases.
>>>>>>>
>>>>>>> Regards
>>>>>>> kalyan.
>>>>>>>
>>>>>>>
>>>>&
gt;>>>>> 4. Stitch resource profile choices into query execution.
>>>>>>>> 5. Allow different DEA algo to be chosen for different queries
>>>>>>>> within the same spark application.
>>>>>>>> 6. Fall
gt;>>> I am aware of a few experiments carried out in this area by
>>>>>>> my friends in this domain. One lesson we had was, it is hard to have a
>>>>>>> generic algorithm that worked for all cases.
>>>>
>>>>>>> Thanks for pointing out this feature to me. I will have a look when
>>>>>>> I get there.
>>>>>>>
>>>>>>> Mich Talebzadeh,
>>>>>>> Solutions Architect/Engineering Lead
>>>>>
...@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks for pointing out this feature to me. I will have a look when
>>>>>>> I get there.
>>>>>>>
>>>>>>> Mich Talebz
gt;>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>>>>> for any loss, damage or destruction
t your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any m
>>>>
>>>> On Tue, 8 Aug 2023 at 11:44, roryqi(齐赫) wrote:
>>>>
>>>>> Spark 3.5 have added an method `supportsReliableStorage` in the `
>>>>> ShuffleDriverComponents` which indicate whether writing shuffle data
>>>>> to
t; https://github.com/apache/incubator-uniffle). It can enhance the
>>>> experience of Spark on K8S. After Spark 3.5 is released, Uniffle will
>>>> support the `ShuffleDriverComponents`. you can see [1].
>>>>
>>>> If you have interest about more d
m/apache/incubator-uniffle). It can enhance the
>>> experience of Spark on K8S. After Spark 3.5 is released, Uniffle will
>>> support the `ShuffleDriverComponents`. you can see [1].
>>>
>>> If you have interest about more details of Uniffle, you can see [2]
ffle%20-%20New%20chapter%20for%20the%20shuffle%20in%20the%20cloud%20native%20era
>>
>>
>>
>> *发件人**: *Mich Talebzadeh
>> *日期**: *2023年8月8日 星期二 06:53
>> *抄送**: *dev
>> *主题**: *[Internet]Re: Improving Dynamic Allocation Logic for Spark 4+
>>
>&
>
> *发件人**: *Mich Talebzadeh
> *日期**: *2023年8月8日 星期二 06:53
> *抄送**: *dev
> *主题**: *[Internet]Re: Improving Dynamic Allocation Logic for Spark 4+
>
>
>
> On the subject of dynamic allocation, is the following message a cause for
> concern when running Spark on k8s?
]
https://uniffle.apache.org/blog/2023/07/21/Uniffle%20-%20New%20chapter%20for%20the%20shuffle%20in%20the%20cloud%20native%20era
发件人: Mich Talebzadeh
日期: 2023年8月8日 星期二 06:53
抄送: dev
主题: [Internet]Re: Improving Dynamic Allocation Logic for Spark 4+
On the subject of dynamic allocation
13 matches
Mail list logo
