Hi WeiWei, All,
I create a JIRA: YARN-8670. And add some new ideas.
If anyone who have doubt or advice, please feel free to comment.
Thanks
Best Regards
Sichen Zhao
________________________________________
From: Weiwei Yang <cheersy...@hotmail.com>
Sent: Thursday, August 16, 2018 1:18:15 PM
To: Sichen Zhao; Sunil G; Yufei Gu
Cc: Daniel Templeton; Hadoop Common; Wangda Tan
Subject: Re: YARN SLS improving idea
Hi Sichen
Sure, please go ahead creating JIRA to capture your ideas, we can move
discussion to smaller groups there.
At meanwhile, I am interested how you plan to support a large number of
“specific” resource requests, look forward to seeing more details.
Looks like 2.b depends on YARN-3409 but it is not yet merged, maybe that can be
another JIRA.
Thanks!
--
Weiwei
On 8/16/18, 12:06 PM, "Sichen Zhao" <zsc19940...@outlook.com> wrote:
Hi all,
Thanks for your advise.
1. I will create JIRA once the details are identified.
2. I read the patch YARN-8007. the scheduling request already done, but
there are some difference between the patch and my ideas:
a) the patch use the SYNTH as schedulingrequest input. But SYNTH can't
create a large number of specific resource requests. I wanna create a new
schedulingrequest input format for the more authentic input.
b) the patch doesn't add the attribute for the Node. I think this can
be done.
Best Regards
Sichen Zhao
________________________________________
From: Weiwei Yang <cheersy...@hotmail.com>
Sent: Tuesday, August 14, 2018 8:54:55 AM
To: Sunil G; Yufei Gu
Cc: Daniel Templeton; zsc19940...@outlook.com; Hadoop Common; Wangda Tan
Subject: Re: YARN SLS improving idea
Hi Sichen
Thanks for proposing this, this for sure a nice enhancement for SLS.
For SLS improvements, feel free to create JIRA tickets under
https://issues.apache.org/jira/browse/YARN-5065, that’s the umbrella we track
all related items.
For scheduling requests support, there is an existing JIRA and an existing
patch in https://issues.apache.org/jira/browse/YARN-8007, would you like to
take a look?
Thanks
--
Weiwei
From: Sunil G <sun...@apache.org>
Date: Tuesday, August 14, 2018 at 1:31 AM
To: Yufei Gu <flyrain...@gmail.com>
Cc: Daniel Templeton <dan...@cloudera.com>, YANG WEIWEI
<cheersy...@hotmail.com>, "zsc19940...@outlook.com" <zsc19940...@outlook.com>,
Hadoop Common <common-dev@hadoop.apache.org>, Wangda Tan <wheele...@gmail.com>
Subject: Re: YARN SLS improving idea
Hi Sichen
1. Add input support for the scheduling request format.
Yes. I suppose this change is in SLS end (client to generate requests)
2. Add support for scheduling request resource format in NMSim.
Makes sense.
3. Adding scheduling request support for the Capacity Scheduler(maybe it is
already done in current version).
This support is already there.
Adding to this, major challenge is to specify constrains per app level and
verifying results. I also suggest to cross check the scheduler invariants check
support added as per YARN-6547 and see how we can incorporate same to predict
o/p
- Sunil
On Mon, Aug 13, 2018 at 10:39 PM Yufei Gu
<flyrain...@gmail.com<mailto:flyrain...@gmail.com>> wrote:
+YANG WEIWEI<mailto:cheersy...@hotmail.com>
Make sense to me from SLS perspective, but I am not familiar with Placement
Constraints. Add WeiWei.
Best,
Yufei
`This is not a contribution`
On Sat, Aug 11, 2018 at 8:57 AM Daniel Templeton
<dan...@cloudera.com<mailto:dan...@cloudera.com>> wrote:
Yufei, Wangda, Sunil, any comments?
Daniel
On 8/11/18 8:48 AM, Sichen Zhao wrote:
> Hi,
> Is there anyone who can reply my ideas?
>
> Best Regards
> Sichen Zhao
>
> ________________________________________
> From: Sichen Zhao
<zsc19940...@outlook.com<mailto:zsc19940...@outlook.com>>
> Sent: Friday, August 10, 2018 11:10
> To: Hadoop Common
> Subject: YARN SLS improving idea
>
> Hi,
> I am a developer from AliBaBa China, i recently used SLS for scheduling
simulation, SLS currently supports multidimensional resource input(CPU, mem ,
other resources: disk). But SLS can't take scheduling request, which is
currently widely used in YARN, as input, so Placement Constraints and
attributes are not supported.
>
> So what i wanna improve the SLS: Add scheduling emulation for scheduling
request resource format.
>
> The specific work is as follows:
> 1. Add input support for the scheduling request format.
> 2. Add support for scheduling request resource format in NMSim.
> 3. Adding scheduling request support for the Capacity Scheduler(maybe it
is already done in current version).
>
> What do you think about my ideas?
>
>
> Best Regards
> Sichen Zhao
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
common-dev-unsubscr...@hadoop.apache.org<mailto:common-dev-unsubscr...@hadoop.apache.org>
> For additional commands, e-mail:
common-dev-h...@hadoop.apache.org<mailto:common-dev-h...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
common-dev-unsubscr...@hadoop.apache.org<mailto:common-dev-unsubscr...@hadoop.apache.org>
> For additional commands, e-mail:
common-dev-h...@hadoop.apache.org<mailto:common-dev-h...@hadoop.apache.org>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
