I am curious to know if this is resolved yet. I see that in the original
email you used "conf" : {"spark.dynamicAllocation.enabled":false
,"spark.shuffle.service.enabled":false}. But in the second email, you
used "conf"
: {"spark.dynamicAllocation.enabled":"false", "spark.
shuffle.service.enabled":"false"}. Was that the issue?
Thanks,
Meisam
On Fri, Nov 3, 2017 at 3:18 AM 王峰 <wangfengfight...@gmail.com> wrote:
> I have try to use livy 0.4 with POST requst
> POST :livyurl/sessions
> Content-Type: application/json; charset=utf-8
> {
> "kind" : "spark",
> "proxyUser" : "root",
> "executorMemory" : "4G",
> "executorCores": 4,
> "numExecutors" : 4,
> "conf" :
> {"spark.dynamicAllocation.enabled":"false","spark.shuffle.service.enabled":"false"}
>
> }
>
> But it doesn`t come into force...maybe I should try another way...
>
> 2017-11-03 17:54 GMT+08:00 Saisai Shao <sai.sai.s...@gmail.com>:
>
>> I think it should be worked, can you please test with 0.4 version of
>> Livy. Also "conf" should be a map of string key to string value.
>>
>> "conf" : {"spark.dynamicAllocation.enabled":"false","spark.shuffle.
>> service.enabled":"false"}
>>
>> Besides, please be aware in the current Livy we only tested on local and
>> yarn mode, we don't guarantee the correct behavior using Mesos cluster
>> manager.
>>
>> On Fri, Nov 3, 2017 at 5:36 PM, 王峰 <wangfengfight...@gmail.com> wrote:
>>
>>> Hello everyone , I have meet a problem about Livy-0.3 when I run `POST
>>> /sessions` to create a new interactive spark session
>>>
>>>
>>> here is the post body
>>> ```Content-Type: application/json; charset=utf-8
>>> {
>>> "kind" : "spark",
>>> "proxyUser" : "root",
>>> "executorMemory" : "4G",
>>> "executorCores": 4,
>>> "numExecutors" : 4,
>>> "conf" :
>>> {"spark.dynamicAllocation.enabled":false,"spark.shuffle.service.enabled":false}
>>> }
>>> ```
>>>
>>> However, I found that this Livy session allocated all of resources in
>>> Mesos UI as Pic shows
>>>
>>> [image: 内嵌图片 1]
>>>
>>> It seem like that `conf` did not worked but numExecutors , executorCores
>>> and executorMemory worked well..
>>>
>>> please help me thanks...
>>>
>>
>>
