[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[ https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455192#comment-16455192 ] Hudson commented on HDFS-12005: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14070 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14070/]) HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. (omalley: rev dcc21a4f939be587ea2642e44e51ccf53534f2e8) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/ksm-metrics.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/scm/scm-overview.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/OzoneHttpServer.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/ksm.js * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/scm/TestSCMMXBean.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/overview.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/rpc-metrics.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/index.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMMXBean.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/block/BlockManagerImpl.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/scm/scm.js * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/scm/index.html * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/ServiceRuntimeInfo.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/rpc-metrics.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/jvm.html * (delete) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/ServiceRuntimeInfoMBean.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/jvm.html * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/ServiceRuntimeInfoImpl.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/SCMMXBean.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/tools.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/block/BlockmanagerMXBean.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/ozone.js * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/StorageContainerManager.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/overview.html > Ozone: Web interface for SCM > > > Key: HDFS-12005 > URL: https://issues.apache.org/jira/browse/HDFS-12005 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-12005-HDFS-7240.001.patch, > HDFS-12005-HDFS-7240.002.patch > > > This is a propsal about how a web interface could be implemented for SCM (and > later for KSM) similar to the namenode ui. > 1. JS framework > There are three big option here. > A.) One is to use a full featured web framework with all the webpack/npm > minify/uglify magic. Build time the webpack/npm scripts should be run and the > result will be added to the jar file. > B.) It could be simplified if the generated minified/uglified js files are > added to the project on commit time. It requires an additional step for every > new patch (to generate the new minified javascripts) but doesn't require > additional JS build tools during the build. > C.) The third option is to make it as simple as possible similar to the > current namenode ui which uses javascript but every dependency is commited > (without JS minify/uglify and other preprocessing). > I prefer to the third one as: > * I have seen a lot of problems during frequent builds od older tez-ui > versions (bower version mismatch, npm version mismatch, npm transitive > dependency problems, proxy problem with older versions). All they could be > fixed but requires additional JS/NPM magic/knowledge. Without additional npm > build step the hdfs projects build could be kept more simple. > * The complexity of the planned SCM/KSM ui (hopefully it will remain simple) > doesn't require more sophisticated model. (Eg. we don't need JS require as we > need only a few controllers) > * HDFS developers mostly backend developers and not JS developers > 2. Frameworks > The big advantages of a more modern JS framework is the simplified > programming model (for example with two way databinding) I suggest to use a > more modern framework (not just j
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[ https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450943#comment-16450943 ] Hudson commented on HDFS-12005: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/]) HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. (aengineer: rev 57dfe49363949b08dd3b87064af76d02beb42aa5) * (delete) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/ServiceRuntimeInfoMBean.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/block/BlockManagerImpl.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/scm/index.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/overview.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/OzoneHttpServer.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/jvm.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMMXBean.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/overview.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/rpc-metrics.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/block/BlockmanagerMXBean.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/jvm.html * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/ServiceRuntimeInfoImpl.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/ksm.js * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/index.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/SCMMXBean.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/ozone.js * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/scm/TestSCMMXBean.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/scm/scm.js * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/StorageContainerManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/ksm-metrics.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/scm/scm-overview.html * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/ServiceRuntimeInfo.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/ksm/rpc-metrics.html * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/templates/tools.html > Ozone: Web interface for SCM > > > Key: HDFS-12005 > URL: https://issues.apache.org/jira/browse/HDFS-12005 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-12005-HDFS-7240.001.patch, > HDFS-12005-HDFS-7240.002.patch > > > This is a propsal about how a web interface could be implemented for SCM (and > later for KSM) similar to the namenode ui. > 1. JS framework > There are three big option here. > A.) One is to use a full featured web framework with all the webpack/npm > minify/uglify magic. Build time the webpack/npm scripts should be run and the > result will be added to the jar file. > B.) It could be simplified if the generated minified/uglified js files are > added to the project on commit time. It requires an additional step for every > new patch (to generate the new minified javascripts) but doesn't require > additional JS build tools during the build. > C.) The third option is to make it as simple as possible similar to the > current namenode ui which uses javascript but every dependency is commited > (without JS minify/uglify and other preprocessing). > I prefer to the third one as: > * I have seen a lot of problems during frequent builds od older tez-ui > versions (bower version mismatch, npm version mismatch, npm transitive > dependency problems, proxy problem with older versions). All they could be > fixed but requires additional JS/NPM magic/knowledge. Without additional npm > build step the hdfs projects build could be kept more simple. > * The complexity of the planned SCM/KSM ui (hopefully it will remain simple) > doesn't require more sophisticated model. (Eg. we don't need JS require as we > need only a few controllers) > * HDFS developers mostly backend developers and not JS developers > 2. Frameworks > The big advantages of a more modern JS framework is the simplified > programming model (for example with two way databinding) I suggest to use a > more modern framework (not just
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[ https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126558#comment-16126558 ] Anu Engineer commented on HDFS-12005: - +1, I will commit this shortly. > Ozone: Web interface for SCM > > > Key: HDFS-12005 > URL: https://issues.apache.org/jira/browse/HDFS-12005 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton > Attachments: HDFS-12005-HDFS-7240.001.patch, > HDFS-12005-HDFS-7240.002.patch > > > This is a propsal about how a web interface could be implemented for SCM (and > later for KSM) similar to the namenode ui. > 1. JS framework > There are three big option here. > A.) One is to use a full featured web framework with all the webpack/npm > minify/uglify magic. Build time the webpack/npm scripts should be run and the > result will be added to the jar file. > B.) It could be simplified if the generated minified/uglified js files are > added to the project on commit time. It requires an additional step for every > new patch (to generate the new minified javascripts) but doesn't require > additional JS build tools during the build. > C.) The third option is to make it as simple as possible similar to the > current namenode ui which uses javascript but every dependency is commited > (without JS minify/uglify and other preprocessing). > I prefer to the third one as: > * I have seen a lot of problems during frequent builds od older tez-ui > versions (bower version mismatch, npm version mismatch, npm transitive > dependency problems, proxy problem with older versions). All they could be > fixed but requires additional JS/NPM magic/knowledge. Without additional npm > build step the hdfs projects build could be kept more simple. > * The complexity of the planned SCM/KSM ui (hopefully it will remain simple) > doesn't require more sophisticated model. (Eg. we don't need JS require as we > need only a few controllers) > * HDFS developers mostly backend developers and not JS developers > 2. Frameworks > The big advantages of a more modern JS framework is the simplified > programming model (for example with two way databinding) I suggest to use a > more modern framework (not just jquery) which supports plain js (not just > ECMA2015/2016/typescript) and just include the required js files in the > projects (similar to the included bootstrap or as the existing namenode ui > works). > > * React could be a good candidate but it requires more library as it's just > a ui framework, even the REST calls need separated library. It could be used > with plain javascript instead of JSX and classes but not straightforward, and > it's more verbose. > > * Ember is used in yarnui2 but the main strength of the ember is the CLI > which couldn't be used for the simplified approach easily. I think ember is > best with the A.) option > * Angular 1 is a good candidate (but not so fancy). In case of angular 1 > the component based approach should be used (in that case later it could be > easier to migrate to angular 2 or react) > * The mainstream side of Angular 2 uses typescript, it could work with > plain JS but it could require additional knowledge, most of the tutorials and > documentation shows the typescript approach. > I suggest to use angular 1 or react. Maybe angular is easier to use as don't > need to emulate JSX with function calls, simple HTML templates could be used. > 3. Backend > I would prefer the approach of the existing namenode ui where the backend is > just the jmx endpoint. To keep it as simple as possible I suggest to try to > avoid dedicated REST backend if possible. Later we can use REST api of > SCM/KSM if they will be implemented. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[
https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124321#comment-16124321
]
Hadoop QA commented on HDFS-12005:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
17s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
19s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
14s{color} | {color:green} root: The patch generated 0 new + 4 unchanged - 1
fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
8s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 78m 10s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
34s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.cblock.TestCBlockReadWrite |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.cblock.TestBufferManager |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.ozone.web.client.TestKeys |
| Timed out junit tests | org.apache.hadoop.ozone.web.client.TestKeysRatis |
| | org.apache.hadoop.ozone.container.ozoneimpl.TestOzoneContainerRatis |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12005 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881544/HDFS-12005-HDFS-7240.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 55dfd25ed602 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / 531d9ce |
| Default Java | 1.8.0_144 |
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[ https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124079#comment-16124079 ] Elek, Marton commented on HDFS-12005: - TestSCMMXBean (and a few typo) is fixed. The other tests don't seem to be related... > Ozone: Web interface for SCM > > > Key: HDFS-12005 > URL: https://issues.apache.org/jira/browse/HDFS-12005 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton > Attachments: HDFS-12005-HDFS-7240.001.patch, > HDFS-12005-HDFS-7240.002.patch > > > This is a propsal about how a web interface could be implemented for SCM (and > later for KSM) similar to the namenode ui. > 1. JS framework > There are three big option here. > A.) One is to use a full featured web framework with all the webpack/npm > minify/uglify magic. Build time the webpack/npm scripts should be run and the > result will be added to the jar file. > B.) It could be simplified if the generated minified/uglified js files are > added to the project on commit time. It requires an additional step for every > new patch (to generate the new minified javascripts) but doesn't require > additional JS build tools during the build. > C.) The third option is to make it as simple as possible similar to the > current namenode ui which uses javascript but every dependency is commited > (without JS minify/uglify and other preprocessing). > I prefer to the third one as: > * I have seen a lot of problems during frequent builds od older tez-ui > versions (bower version mismatch, npm version mismatch, npm transitive > dependency problems, proxy problem with older versions). All they could be > fixed but requires additional JS/NPM magic/knowledge. Without additional npm > build step the hdfs projects build could be kept more simple. > * The complexity of the planned SCM/KSM ui (hopefully it will remain simple) > doesn't require more sophisticated model. (Eg. we don't need JS require as we > need only a few controllers) > * HDFS developers mostly backend developers and not JS developers > 2. Frameworks > The big advantages of a more modern JS framework is the simplified > programming model (for example with two way databinding) I suggest to use a > more modern framework (not just jquery) which supports plain js (not just > ECMA2015/2016/typescript) and just include the required js files in the > projects (similar to the included bootstrap or as the existing namenode ui > works). > > * React could be a good candidate but it requires more library as it's just > a ui framework, even the REST calls need separated library. It could be used > with plain javascript instead of JSX and classes but not straightforward, and > it's more verbose. > > * Ember is used in yarnui2 but the main strength of the ember is the CLI > which couldn't be used for the simplified approach easily. I think ember is > best with the A.) option > * Angular 1 is a good candidate (but not so fancy). In case of angular 1 > the component based approach should be used (in that case later it could be > easier to migrate to angular 2 or react) > * The mainstream side of Angular 2 uses typescript, it could work with > plain JS but it could require additional knowledge, most of the tutorials and > documentation shows the typescript approach. > I suggest to use angular 1 or react. Maybe angular is easier to use as don't > need to emulate JSX with function calls, simple HTML templates could be used. > 3. Backend > I would prefer the approach of the existing namenode ui where the backend is > just the jmx endpoint. To keep it as simple as possible I suggest to try to > avoid dedicated REST backend if possible. Later we can use REST api of > SCM/KSM if they will be implemented. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[
https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123858#comment-16123858
]
Hadoop QA commented on HDFS-12005:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
43s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
12s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
38s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
56s{color} | {color:green} root: The patch generated 0 new + 3 unchanged - 1
fixed = 3 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 5s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 34s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.shell.TestCopyFromLocal |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy |
| | hadoop.cblock.TestBufferManager |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.cblock.TestCBlockReadWrite |
| | hadoop.ozone.web.client.TestKeys |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.ozone.scm.TestSCMMXBean |
| Timed out junit tests | org.apache.hadoop.ozone.web.client.TestKeysRatis |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12005 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881457/HDFS-12005-HDFS-7240.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e4e2c61922e2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[
https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123353#comment-16123353
]
Elek, Marton commented on HDFS-12005:
-
First version is uploaded but as It depends on HDFS-12286, I wil not set the
"Patch available" status until the other on is not merged.
Summary of the changes:
1. The common (both scm/ksm) part (display rpc, display build info, uptime)
has been moved to a common js/html. (So the KSM part is slightly modified)
2. For SCM the defaults existing information is printed out: Chill mode
properties, available nodes per status...).
3. I added a new JMX interface for the BlockManager to print out the the number
of the openContainers.
4. The previously introduced ServiceRuntime interface has been renamed as it
was confusing for the JMX system when the parent interface does also follow the
*MXBean naming convention.
5. Metrics system is initialized for SCM.
6. Style/UX fixes (multiple rpc reports are separated with tabs, no external
page links for the tools as it's less confusing if it opens in the same window,
etc).
7. Additional small bugfixes what I noticed during the test. Some of them broke
the visibility of the KSM data. Should work after this patch.
Tested with corona and the {{hdfs oz}} command.
> Ozone: Web interface for SCM
>
>
> Key: HDFS-12005
> URL: https://issues.apache.org/jira/browse/HDFS-12005
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Elek, Marton
>Assignee: Elek, Marton
> Attachments: HDFS-12005-HDFS-7240.001.patch
>
>
> This is a propsal about how a web interface could be implemented for SCM (and
> later for KSM) similar to the namenode ui.
> 1. JS framework
> There are three big option here.
> A.) One is to use a full featured web framework with all the webpack/npm
> minify/uglify magic. Build time the webpack/npm scripts should be run and the
> result will be added to the jar file.
> B.) It could be simplified if the generated minified/uglified js files are
> added to the project on commit time. It requires an additional step for every
> new patch (to generate the new minified javascripts) but doesn't require
> additional JS build tools during the build.
> C.) The third option is to make it as simple as possible similar to the
> current namenode ui which uses javascript but every dependency is commited
> (without JS minify/uglify and other preprocessing).
> I prefer to the third one as:
> * I have seen a lot of problems during frequent builds od older tez-ui
> versions (bower version mismatch, npm version mismatch, npm transitive
> dependency problems, proxy problem with older versions). All they could be
> fixed but requires additional JS/NPM magic/knowledge. Without additional npm
> build step the hdfs projects build could be kept more simple.
> * The complexity of the planned SCM/KSM ui (hopefully it will remain simple)
> doesn't require more sophisticated model. (Eg. we don't need JS require as we
> need only a few controllers)
> * HDFS developers mostly backend developers and not JS developers
> 2. Frameworks
> The big advantages of a more modern JS framework is the simplified
> programming model (for example with two way databinding) I suggest to use a
> more modern framework (not just jquery) which supports plain js (not just
> ECMA2015/2016/typescript) and just include the required js files in the
> projects (similar to the included bootstrap or as the existing namenode ui
> works).
>
> * React could be a good candidate but it requires more library as it's just
> a ui framework, even the REST calls need separated library. It could be used
> with plain javascript instead of JSX and classes but not straightforward, and
> it's more verbose.
>
> * Ember is used in yarnui2 but the main strength of the ember is the CLI
> which couldn't be used for the simplified approach easily. I think ember is
> best with the A.) option
> * Angular 1 is a good candidate (but not so fancy). In case of angular 1
> the component based approach should be used (in that case later it could be
> easier to migrate to angular 2 or react)
> * The mainstream side of Angular 2 uses typescript, it could work with
> plain JS but it could require additional knowledge, most of the tutorials and
> documentation shows the typescript approach.
> I suggest to use angular 1 or react. Maybe angular is easier to use as don't
> need to emulate JSX with function calls, simple HTML templates could be used.
> 3. Backend
> I would prefer the approach of the existing namenode ui where the backend is
> just the jmx endpoint. To keep it as simple as possible I suggest to try to
> avoid dedicated REST backend if possible. Later we can u
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[ https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116767#comment-16116767 ] Elek, Marton commented on HDFS-12005: - The current plan is to share the common part of the KSM web interface with the SCM (jvm args, uptime, RPC latency). As a first step I would add the following SCM specific stats: * Size openContainers from the BlockManager * Table of the Nodes of the NodesManager * Aggregated NodeStats from NodeManager > Ozone: Web interface for SCM > > > Key: HDFS-12005 > URL: https://issues.apache.org/jira/browse/HDFS-12005 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton > > This is a propsal about how a web interface could be implemented for SCM (and > later for KSM) similar to the namenode ui. > 1. JS framework > There are three big option here. > A.) One is to use a full featured web framework with all the webpack/npm > minify/uglify magic. Build time the webpack/npm scripts should be run and the > result will be added to the jar file. > B.) It could be simplified if the generated minified/uglified js files are > added to the project on commit time. It requires an additional step for every > new patch (to generate the new minified javascripts) but doesn't require > additional JS build tools during the build. > C.) The third option is to make it as simple as possible similar to the > current namenode ui which uses javascript but every dependency is commited > (without JS minify/uglify and other preprocessing). > I prefer to the third one as: > * I have seen a lot of problems during frequent builds od older tez-ui > versions (bower version mismatch, npm version mismatch, npm transitive > dependency problems, proxy problem with older versions). All they could be > fixed but requires additional JS/NPM magic/knowledge. Without additional npm > build step the hdfs projects build could be kept more simple. > * The complexity of the planned SCM/KSM ui (hopefully it will remain simple) > doesn't require more sophisticated model. (Eg. we don't need JS require as we > need only a few controllers) > * HDFS developers mostly backend developers and not JS developers > 2. Frameworks > The big advantages of a more modern JS framework is the simplified > programming model (for example with two way databinding) I suggest to use a > more modern framework (not just jquery) which supports plain js (not just > ECMA2015/2016/typescript) and just include the required js files in the > projects (similar to the included bootstrap or as the existing namenode ui > works). > > * React could be a good candidate but it requires more library as it's just > a ui framework, even the REST calls need separated library. It could be used > with plain javascript instead of JSX and classes but not straightforward, and > it's more verbose. > > * Ember is used in yarnui2 but the main strength of the ember is the CLI > which couldn't be used for the simplified approach easily. I think ember is > best with the A.) option > * Angular 1 is a good candidate (but not so fancy). In case of angular 1 > the component based approach should be used (in that case later it could be > easier to migrate to angular 2 or react) > * The mainstream side of Angular 2 uses typescript, it could work with > plain JS but it could require additional knowledge, most of the tutorials and > documentation shows the typescript approach. > I suggest to use angular 1 or react. Maybe angular is easier to use as don't > need to emulate JSX with function calls, simple HTML templates could be used. > 3. Backend > I would prefer the approach of the existing namenode ui where the backend is > just the jmx endpoint. To keep it as simple as possible I suggest to try to > avoid dedicated REST backend if possible. Later we can use REST api of > SCM/KSM if they will be implemented. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12005) Ozone: Web interface for SCM
[ https://issues.apache.org/jira/browse/HDFS-12005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056660#comment-16056660 ] Anu Engineer commented on HDFS-12005: - [~elek] Thanks for filing this JIRA. I am fine with any approach you choose. As you suggested let us stick to the jmx endpoint for now. It works well for namenode UI. I am fine with any framework that you would choose. One thing is that we might have to design the UI and then make sure all the needed counters are present in the SCM/KSM. > Ozone: Web interface for SCM > > > Key: HDFS-12005 > URL: https://issues.apache.org/jira/browse/HDFS-12005 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton > > This is a propsal about how a web interface could be implemented for SCM (and > later for KSM) similar to the namenode ui. > 1. JS framework > There are three big option here. > A.) One is to use a full featured web framework with all the webpack/npm > minify/uglify magic. Build time the webpack/npm scripts should be run and the > result will be added to the jar file. > B.) It could be simplified if the generated minified/uglified js files are > added to the project on commit time. It requires an additional step for every > new patch (to generate the new minified javascripts) but doesn't require > additional JS build tools during the build. > C.) The third option is to make it as simple as possible similar to the > current namenode ui which uses javascript but every dependency is commited > (without JS minify/uglify and other preprocessing). > I prefer to the third one as: > * I have seen a lot of problems during frequent builds od older tez-ui > versions (bower version mismatch, npm version mismatch, npm transitive > dependency problems, proxy problem with older versions). All they could be > fixed but requires additional JS/NPM magic/knowledge. Without additional npm > build step the hdfs projects build could be kept more simple. > * The complexity of the planned SCM/KSM ui (hopefully it will remain simple) > doesn't require more sophisticated model. (Eg. we don't need JS require as we > need only a few controllers) > * HDFS developers mostly backend developers and not JS developers > 2. Frameworks > The big advantages of a more modern JS framework is the simplified > programming model (for example with two way databinding) I suggest to use a > more modern framework (not just jquery) which supports plain js (not just > ECMA2015/2016/typescript) and just include the required js files in the > projects (similar to the included bootstrap or as the existing namenode ui > works). > > * React could be a good candidate but it requires more library as it's just > a ui framework, even the REST calls need separated library. It could be used > with plain javascript instead of JSX and classes but not straightforward, and > it's more verbose. > > * Ember is used in yarnui2 but the main strength of the ember is the CLI > which couldn't be used for the simplified approach easily. I think ember is > best with the A.) option > * Angular 1 is a good candidate (but not so fancy). In case of angular 1 > the component based approach should be used (in that case later it could be > easier to migrate to angular 2 or react) > * The mainstream side of Angular 2 uses typescript, it could work with > plain JS but it could require additional knowledge, most of the tutorials and > documentation shows the typescript approach. > I suggest to use angular 1 or react. Maybe angular is easier to use as don't > need to emulate JSX with function calls, simple HTML templates could be used. > 3. Backend > I would prefer the approach of the existing namenode ui where the backend is > just the jmx endpoint. To keep it as simple as possible I suggest to try to > avoid dedicated REST backend if possible. Later we can use REST api of > SCM/KSM if they will be implemented. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
