[jira] [Commented] (YARN-4905) Improve "yarn logs" command-line to optionally show log metadata also
[
https://issues.apache.org/jira/browse/YARN-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265637#comment-15265637
]
Hadoop QA commented on YARN-4905:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 37s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 47s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 7s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 5s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 46s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 5s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s {color} | {color:green} hadoop-yarn-project/hadoop-yarn: patch generated 0
new + 25 unchanged - 33 fixed = 25 total (was 58) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 58s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 2s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 13s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 15s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{colo
[jira] [Updated] (YARN-4905) Improve "yarn logs" command-line to optionally show log metadata also
[ https://issues.apache.org/jira/browse/YARN-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-4905: Attachment: YARN-4905.7.patch > Improve "yarn logs" command-line to optionally show log metadata also > - > > Key: YARN-4905 > URL: https://issues.apache.org/jira/browse/YARN-4905 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-4905.1.patch, YARN-4905.2.patch, YARN-4905.3.patch, > YARN-4905.4.patch, YARN-4905.5.patch, YARN-4905.6.1.patch, YARN-4905.7.patch, > YARN-4905.7.patch > > > Improve the Yarn log commandline to have "ls" command which can list > containers for which we have logs, list files within each container, along > with file size -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4905) Improve "yarn logs" command-line to optionally show log metadata also
[ https://issues.apache.org/jira/browse/YARN-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265598#comment-15265598 ] Xuan Gong commented on YARN-4905: - tick off the jenkins again > Improve "yarn logs" command-line to optionally show log metadata also > - > > Key: YARN-4905 > URL: https://issues.apache.org/jira/browse/YARN-4905 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-4905.1.patch, YARN-4905.2.patch, YARN-4905.3.patch, > YARN-4905.4.patch, YARN-4905.5.patch, YARN-4905.6.1.patch, YARN-4905.7.patch, > YARN-4905.7.patch > > > Improve the Yarn log commandline to have "ls" command which can list > containers for which we have logs, list files within each container, along > with file size -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5017) Relook at containerLaunchStartTime in the NM
[ https://issues.apache.org/jira/browse/YARN-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265587#comment-15265587 ] Sunil G commented on YARN-5017: --- Hi [~vvasudev], I could help to get this in, pls feel free to reassign if you already started. > Relook at containerLaunchStartTime in the NM > > > Key: YARN-5017 > URL: https://issues.apache.org/jira/browse/YARN-5017 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Vasudev > > With support for container re-launch, we need to re-look at > containerLaunchStartTime. We probably a need a new metric - one for the > overall the container run time and one for the current attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5017) Relook at containerLaunchStartTime in the NM
[ https://issues.apache.org/jira/browse/YARN-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G reassigned YARN-5017: - Assignee: Sunil G > Relook at containerLaunchStartTime in the NM > > > Key: YARN-5017 > URL: https://issues.apache.org/jira/browse/YARN-5017 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Vasudev >Assignee: Sunil G > > With support for container re-launch, we need to re-look at > containerLaunchStartTime. We probably a need a new metric - one for the > overall the container run time and one for the current attempt. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3401) [Security] users should not be able to create a generic TimelineEntity and associate arbitrary type
[ https://issues.apache.org/jira/browse/YARN-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265568#comment-15265568 ] Naganarasimha G R commented on YARN-3401: - We need to review back again the approaches taken in MAPREDUCE-6424 and MAPREDUCE-6688, Some approaches which i can think of are : # expose REST API's to push configs and metrics separately for the app # During *registration* of AM, AM can push some configurations to RM and RM can update ApplicationEntity with the configurations and during AM *unregister* with RM, AM can send the metrics at the job/app level. > [Security] users should not be able to create a generic TimelineEntity and > associate arbitrary type > --- > > Key: YARN-3401 > URL: https://issues.apache.org/jira/browse/YARN-3401 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Naganarasimha G R > > IIUC it is possible for users to create a generic TimelineEntity and set an > arbitrary entity type. For example, for a YARN app, the right entity API is > ApplicationEntity. However, today nothing stops users from instantiating a > base TimelineEntity class and set the application type on it. This presents a > problem in handling these YARN system entities in the storage layer for > example. > We need to ensure that the API allows only the right type of the class to be > created for a given entity type. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5016) Add support for a minimum retry interval
[ https://issues.apache.org/jira/browse/YARN-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Gong reassigned YARN-5016: -- Assignee: Jun Gong > Add support for a minimum retry interval > > > Key: YARN-5016 > URL: https://issues.apache.org/jira/browse/YARN-5016 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Vasudev >Assignee: Jun Gong > > The NM container re-launch feature should add support to specify a minimum > restart interval so that the minimum time between restarts can be set by > admins. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5015) Unify restart policies across AM and container restarts
[ https://issues.apache.org/jira/browse/YARN-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Gong reassigned YARN-5015: -- Assignee: Jun Gong > Unify restart policies across AM and container restarts > --- > > Key: YARN-5015 > URL: https://issues.apache.org/jira/browse/YARN-5015 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Vasudev >Assignee: Jun Gong > > We support AM restart and container restarts - however the two have slightly > different capabilities. We should unify them. There's no reason for them to > be different. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5015) Unify restart policies across AM and container restarts
[ https://issues.apache.org/jira/browse/YARN-5015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265566#comment-15265566 ] Jun Gong commented on YARN-5015: Thanks [~vvasudev] for creating the issue. I'd like to work on it if you are not working on it. Thanks! > Unify restart policies across AM and container restarts > --- > > Key: YARN-5015 > URL: https://issues.apache.org/jira/browse/YARN-5015 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Vasudev > > We support AM restart and container restarts - however the two have slightly > different capabilities. We should unify them. There's no reason for them to > be different. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3150) [Documentation] Documenting the timeline service v2
[ https://issues.apache.org/jira/browse/YARN-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265512#comment-15265512 ] Li Lu commented on YARN-3150: - No other concerns raised. Will commit shortly. > [Documentation] Documenting the timeline service v2 > --- > > Key: YARN-3150 > URL: https://issues.apache.org/jira/browse/YARN-3150 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Sangjin Lee > Labels: yarn-2928-1st-milestone > Attachments: TimelineServiceV2.pdf, YARN-3150-YARN-2928.01.patch, > YARN-3150-YARN-2928.02.patch, YARN-3150-YARN-2928.03.patch, > YARN-3150-YARN-2928.04.patch > > > Let's make sure we will have a document to describe what's new in TS v2, the > APIs, the client libs and so on. We should do better around documentation in > v2 than v1. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[ https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265434#comment-15265434 ] Varun Saxena commented on YARN-4447: bq. The only reason I suggested LinkedList is because we don't know the size of the stack/deque in advance, right? Fair point. Will use LinkedList. Thanks for the suggestion. > Provide a mechanism to represent complex filters and parse them at the REST > layer > -- > > Key: YARN-4447 > URL: https://issues.apache.org/jira/browse/YARN-4447 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265432#comment-15265432
]
Varun Saxena commented on YARN-4447:
bq. I understand the reader code now wants to use it, but it should be able to
use it from TimelineStorageUtils, right? I would be in favor of keeping it in
TimelineStorageUtils and have the reader code use it.
Ok...I am fine with that.
bq. That's a good reason. However, you didn't mean that we write the
expressions in the queries to the backend, right? We're talking about reading
the values from the storage as is and comparing them to the query expression as
is?
Yes, I meant that as from the writer side, configs,metrics,etc. are written
as-is, we cannot convert them to lowercase when they arrive in filters. The
responsibility of sending the config/metric in the correct case is
responsibility of client.
bq. Specifically for checking for OR and AND, I believe it should be completely
equivalent to expr.toLowerCase().indexOf("or ", offset) == offset
Sorry for the confusion.Yes, I agree with this suggestion of yours. Will change
that.
bq. If we were to avoid mutable filters, we would need to delay constructing
the current filter until the value is successfully parsed. It doesn't look very
obvious whether that is doable given the current shape of the parser code.
Will check on that. Current structure of code is such that they have to
mutable. If we were to make them immutable, as you said, I will have to keep
some extra variables and expose a single abstract method for setting
everything(key, compare op and value in one shot).
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265428#comment-15265428
]
Sangjin Lee commented on YARN-4447:
---
bq. IMO, we cannot lowercase the whole expression as
events/metrics/configs,etc.in filters have to be matched as-is. Thoughts ?
Per previous comment, I'm fine with that.
bq. Thanks for the pointer. Yes, synchronization is not required here, so
ArrayDeque can be used here.
The only reason I suggested {{LinkedList}} is because we don't know the size of
the stack/deque in advance, right? API-wise, both should be completely
identical (i.e. you should be able to use {{pop}} and {{push}} like a stack).
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[ https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265425#comment-15265425 ] Varun Saxena commented on YARN-4447: bq. Overall, catching Exception}}s is too broad Correct. Will tighten it. bq. l.34: angle brackets in javadoc should be escaped Ok. bq. l.69: I would recommend using the Deque interface (and LinkedList) over Stack, as Stack has unnecessary synchronization overhead Thanks for the pointer. Yes, synchronization is not required here, so ArrayDeque can be used here. bq. l.75: this is where you can lowercase the expression IMO, we cannot lowercase the whole expression as events/metrics/configs,etc.in filters have to be matched as-is. Thoughts ? bq. Does parseDataToRetrieve() merit its own TimelineParser() impl? It fits with the rest of the parser classes Hmm...We can have it inside a class as well. I am Ok with that. bq. How much negative testing (e.g. invalid expressions) is covered? Do we need to cover more of error cases? I have covered a few negative cases in TestTimelineReaderWebServicesUtils. More tests can be thought of and added. I have to add a few more test cases in the patch above. > Provide a mechanism to represent complex filters and parse them at the REST > layer > -- > > Key: YARN-4447 > URL: https://issues.apache.org/jira/browse/YARN-4447 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265424#comment-15265424
]
Sangjin Lee commented on YARN-4447:
---
{quote}
Spaces in the expression will have to be encoded while sending them in URL.
Colons were being used in previous expressions. If I am not wrong even in
ATSv1. It should be fine in query params I think. Colons are reserved
characters only in the scheme section of URI as per my understanding.
java.net.URI does not seem to have any objections with colons in query param
either.
{quote}
I'm fine as long as the expectation is set that clients/users escape spaces.
Ack on the colon.
{quote}
If you are talking about converting the whole expression to lower case, we
cannot convert the whole string to lowercase. Because
configs/metrics/events/info will have to be taken as-is. The reason being we do
not convert them to lower case when we write them to backend.
{quote}
That's a good reason. However, you didn't mean that we write the expressions in
the queries to the backend, right? We're talking about reading the values from
the storage as is and comparing them to the query expression as is?
{quote}
Sure, will add a note in javadoc. Had to make them mutable because I wanted to
set the variables as I was parsing the expression. Maybe you can have some
alternate suggestion when you go through the parsing code.
{quote}
If we were to avoid mutable filters, we would need to delay constructing the
current filter until the value is successfully parsed. It doesn't look very
obvious whether that is doable given the current shape of the parser code.
Making the filters mutable is a-ok, just be explicit about what it shouldn't do
(vis-a-vis HashMap/HashSet).
{quote}
Basically I just do not want to check for OR but also reject the expression if
ORR comes in place of OR. If code is confusing to read, I can consider changing
it.
{quote}
*Specifically* for checking for OR and AND, I believe it should be completely
equivalent to {{expr.toLowerCase().indexOf("or ", offset) == offset}}. I am
fine either way, though, as I don't think it is a serious issue.
{quote}
Wanted to check if metric values can have non numerical and non integral values
and if it has reject it while parsing itself. We can defer the checking to
storage layer as well though and in that case it can be moved back to
TimelineStorageUtils. Thoughts ?
{quote}
I understand the reader code now wants to use it, but it should be able to use
it from {{TimelineStorageUtils}}, right? I would be in favor of keeping it in
{{TimelineStorageUtils}} and have the reader code use it.
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[ https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265418#comment-15265418 ] Varun Saxena commented on YARN-4447: Sorry, substr is not required for AND/OR and your suggestion would work fine. Will do that. > Provide a mechanism to represent complex filters and parse them at the REST > layer > -- > > Key: YARN-4447 > URL: https://issues.apache.org/jira/browse/YARN-4447 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[ https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265415#comment-15265415 ] Varun Saxena commented on YARN-4447: Meant "Wanted to check if metric *filters* can have non numerical and non integral values" > Provide a mechanism to represent complex filters and parse them at the REST > layer > -- > > Key: YARN-4447 > URL: https://issues.apache.org/jira/browse/YARN-4447 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265414#comment-15265414
]
Varun Saxena commented on YARN-4447:
Thanks [~sjlee0] for the review.
bq. Do we have an escape/collision issue on the URL, or are they all properly
encoded/decoded? I'm concerned about colons and spaces specifically. Is it a
non-issue?
Spaces in the expression will have to be encoded while sending them in URL.
Colons were being used in previous expressions. If I am not wrong even in
ATSv1. It should be fine in query params I think. Colons are reserved
characters only in the scheme section of URI as per my understanding.
java.net.URI does not seem to have any objections with colons in query param
either.
This is what RFC 3986 ABNF states :
{panel}
URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
query = *( pchar / "/" / "?" )
{panel}
Here, query section does not have colon as reserved character.
However, I am not a 100% sure on this either.
bq. In the parser code, I see a lot of Character.toLowerCase() operations.
Since we need a fully lower-cased input string anyway, I would say simply
lower-case the input string and operate directly on that. That might make the
code less verbose with little performance impact.
If you are talking about converting the whole expression to lower case, we
cannot convert the whole string to lowercase. Because
configs/metrics/events/info will have to be taken as-is. The reason being we do
not convert them to lower case when we write them to backend.
bq. It looks like {{TimelineFilter}}s (and subclasses) are now mutable. Then we
need to be real careful as they should not be used as keys for {{HashMap}}s or
in {{HashSet}}s. This might be a little paranoid, but it would be great if you
could add a sentence in the javadoc that these are not immutable and cannot be
used in {{HashMap}}s or {{HashSet}}s.
Sure, will add a note in javadoc. Had to make them mutable because I wanted to
set the variables as I was parsing the expression. Maybe you can have some
alternate suggestion when you go through the parsing code.
bq. Why elaborate parsing based on chars? Can we not use
expr.toLowerCase().indexOf("or ") == offset? Was there a performance concern?
Or was it simply refactored out of the parser code that seems to be doing
similar things?
You can say that. I moved this code out of parsing code I had initially
written.
The main reason for parsing it character by character was to reject invalid
expressions whenever an unexpected character based on parsing state is
encountered. And yes, performance was a consideration as well.
However for OR/AND similar effect can be achieved with indexOf, substr and
trim, which although makes it slightly more expensive operation compared to
just checking character.
I do understand though, it will be a negligible penalty.
Basically I just do not want to check for OR but also reject the expression if
ORR comes in place of OR. If code is confusing to read, I can consider changing
it.
bq. Why did we move isIntegralValue() to TimelineReaderUtils?
Wanted to check if metric values can have non numerical and non integral values
and if it has reject it while parsing itself. We can defer the checking to
storage layer as well though and in that case it can be moved back to
TimelineStorageUtils. Thoughts ?
bq. This is an observation that the size of this test is growing tremendously
with this patch. Would it be an option to refactor filter-related tests into a
separate test class?
Yes, thats correct. We have a JIRA for refactoring test cases(with regards to
TestHBaseTimelineStorage). We can take up refactoring of this test class as
well there.
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265413#comment-15265413
]
Sangjin Lee commented on YARN-4447:
---
Some more feedback...
(TimelineParserForCompareExpr.java)
- Most of these comments apply to other parser classes...
- Overall, catching {{Exception}}s is too broad. This way, it can catch all
kinds of unexpected exceptions and handle them. Can we tighten exceptions that
can be thrown off of {{parse()}}? Perhaps a new exception like
{{ParseException}} (instead of {{IllegalArgumentException}}) would be clearer?
- l.34: angle brackets in javadoc should be escaped
- l.69: I would recommend using the {{Deque}} interface (and {{LinkedList}})
over {{Stack}}, as {{Stack}} has unnecessary synchronization overhead
- l.75: this is where you can lowercase the expression
(TimelineReaderWebServicesUtils.java)
- Does {{parseDataToRetrieve()}} merit its own {{TimelineParser()}} impl? It
fits with the rest of the parser classes
(test code)
- How much negative testing (e.g. invalid expressions) is covered? Do we need
to cover more of error cases?
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265389#comment-15265389
]
Sangjin Lee commented on YARN-4447:
---
bq. Can we not use expr.toLowerCase().indexOf("or ") == offset?
More precisely {{expr.toLowerCase().indexOf("or ", offset) == offset}}.
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265379#comment-15265379
]
Sangjin Lee commented on YARN-4447:
---
Thanks [~varun_saxena]! This is a very comprehensive (and important) piece of
work.
I'm trying to go over the patch, but I'm not quite sure if I can do a thorough
job in a day or two, given its size. Let me report some early feedback and
questions first. The main things I haven't fully gone through yet are the
parser classes.
Do we have an escape/collision issue on the URL, or are they all properly
encoded/decoded? I'm concerned about colons and spaces specifically. Is it a
non-issue?
In the parser code, I see a lot of {{Character.toLowerCase()}} operations.
Since we need a fully lower-cased input string anyway, I would say simply
lower-case the input string and operate directly on that. That might make the
code less verbose with little performance impact.
It looks like {{TimelineFilter}}s (and subclasses) are now mutable. Then we
need to be real careful as they should not be used as keys for {{HashMap}}s or
in {{HashSet}}s. This might be a little paranoid, but it would be great if you
could add a sentence in the javadoc that these are not immutable and cannot be
used in {{HashMap}}s or {{HashSet}}s.
(TimelineParseUtils.java)
- Why elaborate parsing based on chars? Can we not use
{{expr.toLowerCase().indexOf("or ") == offset}}? Was there a performance
concern? Or was it simply refactored out of the parser code that seems to be
doing similar things?
(TimelineStorageUtils.java)
- Why did we move {{isIntegratlValue()}} to {{TimelineReaderUtils}}? From the
name it sounds like {{TimelineStorageUtils}} should be a common utility, and if
both readers and writers have a need, it should stay in
{{TimelineStorageUtils}}?
(TimelineReaderWebServicesHBaseStorage.java)
- This is an observation that the size of this test is growing tremendously
with this patch. Would it be an option to refactor filter-related tests into a
separate test class? It doesn't have to be done as part of this JIRA, but it
would be great if we took time to do it later.
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> --
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4926) Change nodelabel rest API invalid reponse status to 400
[
https://issues.apache.org/jira/browse/YARN-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265293#comment-15265293
]
Hadoop QA commented on YARN-4926:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 37s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 16s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
patch generated 3 new + 38 unchanged - 0 fixed = 41 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 47m 34s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 48m 3s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 112m 47s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| JDK v1.8.0_91 Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes |
| JDK v1.7.0_95 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.
[jira] [Commented] (YARN-4925) ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression
[
https://issues.apache.org/jira/browse/YARN-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265278#comment-15265278
]
Hadoop QA commented on YARN-4925:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
39s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 1s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_92.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 15s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 145m 34s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_92 Failed junit tests | hadoop.yarn.client.TestGetGroups |
| | hadoop.yarn.client.api.impl.TestAMRMProxy |
| JDK v1.8.0_92 Timed out junit tests |
org.apache.hadoop.yarn.client.cli.TestYarnCLI |
| | org.apache.hadoop.yarn.client.api.impl.TestAMRMClient |
| | org.apache.hadoop.yarn.client.api.impl.TestYarnClient |
| | org.apache.hadoop.yarn.client.api.impl.TestNMClient |
| JDK v1.7.0_95 Failed junit tests | hadoop.yarn.client.TestGetGroups |
| | hadoop.yarn.client.api.impl.TestAMRMProxy |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.hadoop.yarn.client.cli.TestYarnCLI |
| | org.apache.hadoop.yarn.client.api.i
[jira] [Updated] (YARN-4926) Change nodelabel rest API invalid reponse status to 400
[ https://issues.apache.org/jira/browse/YARN-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated YARN-4926: --- Attachment: 0001-YARN-4926.patch Attaching patch after handling all the cases mentioned above. > Change nodelabel rest API invalid reponse status to 400 > --- > > Key: YARN-4926 > URL: https://issues.apache.org/jira/browse/YARN-4926 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4926.patch > > > For cases mentioned below need to change status from 404 to 400 > # Add level in using post request with invalid pattern > # Try modification of exclusivity for label using add request again > # Replace label of node with label that doesn’t exist in cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4925) ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression
[
https://issues.apache.org/jira/browse/YARN-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated YARN-4925:
---
Attachment: 0002-YARN-4925.patch
Attaching patch after adding testcase
> ContainerRequest in AMRMClient, application should be able to specify
> nodes/racks together with nodeLabelExpression
> ---
>
> Key: YARN-4925
> URL: https://issues.apache.org/jira/browse/YARN-4925
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4925.patch, 0002-YARN-4925.patch
>
>
> Currently with nodelabel AMRMClient will not be able to specify nodelabels
> with Node/Rack requests.For application like spark NODE_LOCAL requests cannot
> be asked with label expression.
> As per the check in {{AMRMClientImpl#checkNodeLabelExpression}}
> {noformat}
> // Don't allow specify node label against ANY request
> if ((containerRequest.getRacks() != null &&
> (!containerRequest.getRacks().isEmpty()))
> ||
> (containerRequest.getNodes() != null &&
> (!containerRequest.getNodes().isEmpty( {
> throw new InvalidContainerRequestException(
> "Cannot specify node label with rack and node");
> }
> {noformat}
> {{AppSchedulingInfo#updateResourceRequests}} we do reset of labels to that of
> OFF-SWITCH.
> The above check is not required for ContainerRequest ask /cc [~wangda] thank
> you for confirming
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
