[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16141034#comment-16141034
]
Wangda Tan commented on YARN-7072:
--
Thanks [~xgong],
Comments:
Overall suggestion:
- Please check appendable when LogAggregationIndexFileFormat is chosen.
- Suggest to put all TFile controller related implementation to
...filecontroller.tfile, and all Indexed controller impl to
...filecontroller.ifile (or some better name)
- IndexedFileAggregatedLogsBlock is a little bit lengthy, suggest to move it to
a separate file and break down render() method.
1) {{LogAggregationIndexedFileController#initializeWriter}}. IIUC, current
logic is:
a. Rolling aggregation append to remote file everytime.
b. There's a separate checksum file which records last succeeded write
location. It means last write succeeded if the checksum file doesn't exist.
c. When checksum file not existed, write last succeeded watermark. Otherwise
read meta information out.
Questions/comments:
- For c. I think we should read meta information from original file as well
when checksum file not existed (When we doing rolling aggregation and last
aggregation succeeded, since everytime we delete checksum file after
aggregation succeeded. see {{postWrite}}).
- {{Path remoteLogFile}}, should be final.
- IIUC, the dummyBytes is separator so we know what is the last succeeded
write. If so, probably a simple "\n" is not enough.
- Renames: fsDataOutputStream => checksumFileOutputStream, fsDataInputStream =>
checksumFileInputStream.
2) {{LogAggregationIndexedFileController#write}}
- It's better to make following warning log to be more specific:
{code}
if(fileLength < newLength) {
LOG.warn("Aggregated logs truncated by approximately "+
(newLength-fileLength) +" bytes.");
}
{code}
For example, we can report: "because log file is modified during
aggregation, so it might be trunked by X bytes."
- When IOException cached, it's better to log full stacktrace to log file
instead of only message.
{code}
outputStreamState.getOutputStream().write(
message.getBytes(Charset.forName("UTF-8")));
{code}
3) {{LogAggregationIndexedFileController#loadIndexedLogsMeta}}.
- It looks like loadIndexedLogsMeta did seek twice, is it possible to read last
x-MB (say, 64MB) data directly (which assumes in most cases total size of file
meta less than x-MB, so we don't have to do seek twice, seek operation could be
expensive. ByteArrayInputStream could be used to read from a cached memory.
4) {{LogAggregationIndexedFileController#readAggregatedLogs}}
- Why sort is needed? Could that possibly makes different sequence of file
content stored in log file (e.g. in serialized-file, we have container1_stdout,
container3_stderr, container2_stdout) which could lead to unnecessary seek
operation.
- Output format related logic should be common and shared by all controller
impl:
{code}
StringBuilder sb = new StringBuilder();
String endOfFile = "End of LogType:" + candidate.getFileName();
sb.append("\n" + endOfFile + "\n");
sb.append(StringUtils.repeat("*", endOfFile.length() + 50)
+ "\n\n");
{code}
Otherwise we may have different output format for different controller impl.
Others:
- It looks like {{getFilteredFiles}} could be getAllChecksumFiles, since suffix
never accept input other than CHECK_SUM_FILE_SUFFIX.
- Is it possible that there's more than two checksum files? Could we check it
inside {{getFilteredFiles}} and throw exception when we find such?
- {{LogAggregationFileController#createPrintStream}}, should use
{{LogCLIHelper#createPrintStream}} instead.
5) {{LogAggregationIndexedFileController#readAggregatedLogsMeta}}
Haven't reviewed details of the method yet, however I found it may have some
overlap of readAggregatedLogs, and some data structure looks very similar,
TODO: will review this part later.
> Add a new log aggregation file format controller
>
>
> Key: YARN-7072
> URL: https://issues.apache.org/jira/browse/YARN-7072
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144454#comment-16144454 ] Xuan Gong commented on YARN-7072: - Thanks for the review, [~leftnoteasy] bq. Please check appendable when LogAggregationIndexFileFormat is chosen. Right now, we do not have a fs api to check whether it supports the append or not. So, I am adding a temporary configuration for this. And will throw runtime exception if the user configure this configuration as false, and use LogAggregationIndexFileFormat. bq. Suggest to put all TFile controller related implementation to ...filecontroller.tfile, and all Indexed controller impl to ...filecontroller.ifile (or some better name) Done bq. IndexedFileAggregatedLogsBlock is a little bit lengthy, suggest to move it to a separate file and break down render() method. Done bq. It looks like getFilteredFiles could be getAllChecksumFiles, since suffix never accept input other than CHECK_SUM_FILE_SUFFIX. Make sense. Fixed it. bq. Is it possible that there's more than two checksum files? Could we check it inside getFilteredFiles and throw exception when we find such? I do not think so. Before we create checksum file, we need to make sure there is no checksum exists. If it does, we would read the existing checksum file instead of creating a new one. There should be only one checksum file for each NMs. bq. LogAggregationFileController#createPrintStream, should use LogCLIHelper#createPrintStream instead. Yes, move the createPrintStream to Utils class, and use it for both LogAggregationFileController and LogCLIHelper bq. Why sort is needed? Could that possibly makes different sequence of file content stored in log file (e.g. in serialized-file, we have container1_stdout, container3_stderr, container2_stdout) which could lead to unnecessary seek operation. Removed bq. Output format related logic should be common and shared by all controller impl: Make sense. Fixed bq. It looks like loadIndexedLogsMeta did seek twice, is it possible to read last x-MB (say, 64MB) data directly (which assumes in most cases total size of file meta less than x-MB, so we don't have to do seek twice, seek operation could be expensive. ByteArrayInputStream could be used to read from a cached memory. I think this should be fine. Seek operation is not that expensive. bq. When IOException cached, it's better to log full stacktrace to log file instead of only message. bq. For c. I think we should read meta information from original file as well when checksum file not existed (When we doing rolling aggregation and last aggregation succeeded, since everytime we delete checksum file after aggregation succeeded. see postWrite). Yes, in some cases, we do need to read the meta from the original file, but will not do it every time. bq. Path remoteLogFile, should be final. Done bq. IIUC, the dummyBytes is separator so we know what is the last succeeded write. If so, probably a simple "\n" is not enough. No, dummyBytes is not the separator. In current implementation, we do not have separator. For adding the dummyBytes, I just want to re-set the curator to the end of the file for appending the new logs. bq.Renames: fsDataOutputStream => checksumFileOutputStream, fsDataInputStream => checksumFileInputStream. Done > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144468#comment-16144468 ] Wangda Tan commented on YARN-7072: -- [~xgong], bq. Right now, we do not have a fs api to check whether it supports the append or not. So, I am adding a temporary configuration for this. And will throw runtime exception if the user configure this configuration as false, and use LogAggregationIndexFileFormat. I would prefer not to add a config. How about do a fs test when LogAggregationService start. Just create a file, append few bytes and fail fast when append fails. bq. I think this should be fine. Seek operation is not that expensive. Since this is just internal implementation, we can revisit this later. bq. No, dummyBytes is not the separator. In current implementation, we do not have separator. For adding the dummyBytes, I just want to re-set the curator to the end of the file for appending the new logs. I suggest to add a separator to this patch (a hard coded and randomly generated 128 bits string) if it is just a trivial change. This might be important for the future file format, update implementation is relatively simple but update format is painful. Will do another review later. > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, > YARN-7072-trunk.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145893#comment-16145893
]
Wangda Tan commented on YARN-7072:
--
Offline discussed with [~xgong].
bq. I suggest to add a separator to this patch (a hard coded and randomly
generated 128 bits string) if it is just a trivial change. This might be
important for the future file format, update implementation is relatively
simple but update format is painful.
We decided to remove uuid from file and append it after each partial
aggregation completed. It will be a const value.
Some rename suggestions:
1) PerContainerLogFileInfo => ContainerLogFileInfo
2) LogMeta => IndexedPerAggregationLogMeta
3) {{private Map> logMetas = new HashMap<>();}}, use
ContainerId as key.
4) FileLogMeta => IndexedFileLogMeta
Beyond this, I don't have more comments, I hope [~djp] could help to take a
look at the patch as well.
> Add a new log aggregation file format controller
>
>
> Key: YARN-7072
> URL: https://issues.apache.org/jira/browse/YARN-7072
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch,
> YARN-7072-trunk.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149618#comment-16149618 ] Xuan Gong commented on YARN-7072: - Uploaded a new patch for all the comments > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, > YARN-7072-trunk.003.patch, YARN-7072-trunk.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149972#comment-16149972
]
Hadoop QA commented on YARN-7072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
18s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
45s{color} | {color:red} hadoop-yarn-common in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
39s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 0s{color} | {color:orange} root: The patch generated 40 new + 61 unchanged
- 3 fixed = 101 total (was 64) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
32s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
generated 12 new + 0 unchanged - 0 fixed = 12 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
44s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-server-common in the patch failed. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
42s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
42s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
54s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
12s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
20s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 55s{color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| | Dead store to nodeFiles in
org.apache.hadoop.yarn.logaggregation
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151077#comment-16151077 ] Xuan Gong commented on YARN-7072: - Fix the findbugs and checkstyle issues, The testcase failures should be fixed in https://issues.apache.org/jira/browse/YARN-7148 > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, > YARN-7072-trunk.003.patch, YARN-7072-trunk.004.patch, > YARN-7072-trunk.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151252#comment-16151252
]
Hadoop QA commented on YARN-7072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
50s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
45s{color} | {color:red} hadoop-yarn-common in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
54s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 4s{color} | {color:orange} root: The patch generated 16 new + 60 unchanged
- 3 fixed = 76 total (was 63) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
30s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
49s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
30s{color} | {color:red} hadoop-yarn-server-common in the patch failed. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 17s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 37s{color}
| {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
50s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
39s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
11s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 23s{color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| | ja
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156169#comment-16156169 ] Xuan Gong commented on YARN-7072: - Rebase the patch and fix findbug issue > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, > YARN-7072-trunk.003.patch, YARN-7072-trunk.004.patch, > YARN-7072-trunk.005.patch, YARN-7072-trunk.006.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156296#comment-16156296
]
Hadoop QA commented on YARN-7072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
50s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
50s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 27s{color} | {color:orange} root: The patch generated 10 new + 60 unchanged
- 3 fixed = 70 total (was 63) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
48s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} hadoop-yarn-server-common in the patch failed. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
40s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
37s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
49s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
38s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
11s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
25s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885696/YARN-7072-trunk.006.patch
|
| Optional Tests | asflicense compile javac javado
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158305#comment-16158305
]
Junping Du commented on YARN-7072:
--
Patch LGTM in overall. The findbug warning is not related. However two javadoc
errors should get fixed:
{code}
...
[ERROR]
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/ContainerLogMeta.java:29:
error: reference not found
[ERROR] * A list of {@link PerContainerLogFileInfo}.
[ERROR] ^
...
[ERROR]
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/dao/ContainerLogsInfo.java:37:
error: reference not found
[ERROR] * A list of {@link PerContainerLogFileInfo}.
[ERROR] ^
{code}
Will +1 after javadoc warnings get fixed. Some NITs could get fixed later in
other following JIRAs.
> Add a new log aggregation file format controller
>
>
> Key: YARN-7072
> URL: https://issues.apache.org/jira/browse/YARN-7072
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch,
> YARN-7072-trunk.003.patch, YARN-7072-trunk.004.patch,
> YARN-7072-trunk.005.patch, YARN-7072-trunk.006.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159077#comment-16159077 ] Xuan Gong commented on YARN-7072: - Thanks for the review, [~djp] Attached a new patch for trunk. Will upload a fix for branch-2 shortly. > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, > YARN-7072-trunk.003.patch, YARN-7072-trunk.004.patch, > YARN-7072-trunk.005.patch, YARN-7072-trunk.006.patch, > YARN-7072-trunk.007.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159135#comment-16159135 ] Xuan Gong commented on YARN-7072: - upload a new patch for branch-2 as well > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159213#comment-16159213 ] Junping Du commented on YARN-7072: -- Thanks Xuan for updating the patch. Latest patch LGTM. +1 pending on Jenkins report. > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159390#comment-16159390
]
Hadoop QA commented on YARN-7072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
51s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
52s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 3s{color} | {color:orange} root: The patch generated 10 new + 60 unchanged
- 3 fixed = 70 total (was 63) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
2s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
59s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
16s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
21s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
19s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886141/YARN-7072-trunk.008.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e4a6171405ee 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159462#comment-16159462 ] Hudson commented on YARN-7072: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12828 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12828/]) YARN-7072. Add a new log aggregation file format controller. Contributed (junping_du: rev 3fddabc2fe4fbdb8ef3f9ce7558955c4f0794dcc) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/tfile/LogAggregationTFileController.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogToolUtils.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/BoundedRangeFileInputStream.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/LogAggregationIndexedFileController.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/TestLogAggregationIndexFileController.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogCLIHelpers.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/LogAggregationFileController.java * (delete) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/PerContainerLogFileInfo.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/dao/ContainerLogsInfo.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/ContainerLogFileInfo.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/IndexedFileAggregatedLogsBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/ContainerLogMeta.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/package-info.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/NMContainerLogsInfo.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/SimpleBufferedOutputStream.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebServices.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebServices.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[
https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159648#comment-16159648
]
Hadoop QA commented on YARN-7072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
14s{color} | {color:green} branch-2 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
33s{color} | {color:green} branch-2 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
20s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
22s{color} | {color:green} branch-2 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
46s{color} | {color:green} branch-2 passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
39s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
4s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 37s{color} | {color:orange} root: The patch generated 10 new + 62 unchanged
- 1 fixed = 72 total (was 63) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
32s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.8.0_144 with JDK
v1.8.0_144 generated 17 new + 4528 unchanged - 0 fixed = 4545 total (was 4528)
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
53s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 16s{color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_151. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 28s{color}
| {color:red} hadoop-yarn-common in the patch failed with JDK v1.7.0_151.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-server-common in the patch passed with
JDK v1.7.0_151. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 32m 31s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK
v1.7.0_151. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 8s{color}
| {color:red} hadoop-yarn-server-applicationhistoryservice in
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159687#comment-16159687 ] Junping Du commented on YARN-7072: -- Jenkins report indicates many UT get failed. [~xgong], can you quickly check if it is related to branch-2 patch here? > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159719#comment-16159719 ] Xuan Gong commented on YARN-7072: - [~djp] I do not think they are related. I can successfully run all the test cases locally. > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160695#comment-16160695 ] Junping Du commented on YARN-7072: -- I also verified locally. Most tests get passed locally as well, and the only failure is TestNodeStatusUpdater. It failed also without the patch, so I think it should be unrelated. We should file a separated JIRA to track the test failure on branch-2. +1. Committing branch-2 patch. > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160783#comment-16160783 ] Xuan Gong commented on YARN-7072: - Committed into branch-2. Thanks for the review, Junping > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.9.0, 3.1.0 > > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
[ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186709#comment-16186709 ] Wangda Tan commented on YARN-7072: -- Pushed to branch-3.0 as well. > Add a new log aggregation file format controller > > > Key: YARN-7072 > URL: https://issues.apache.org/jira/browse/YARN-7072 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.9.0, 3.1.0 > > Attachments: YARN-7072-branch-2.001.patch, YARN-7072-trunk.001.patch, > YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch, > YARN-7072-trunk.004.patch, YARN-7072-trunk.005.patch, > YARN-7072-trunk.006.patch, YARN-7072-trunk.007.patch, > YARN-7072-trunk.008.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
