[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[
https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045174#comment-17045174
]
Hadoop QA commented on YARN-6566:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red} YARN-6566 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6566 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25581/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> add a property for a hadoop job to identified the full hive HQL script text
> in hadoop web view in multi-users environment
> -
>
> Key: YARN-6566
> URL: https://issues.apache.org/jira/browse/YARN-6566
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client, resourcemanager, webapp, yarn
>Affects Versions: 2.6.0
> Environment: centos 6.4 64bit
>Reporter: liuzhenhua
>Priority: Major
> Labels: patch
> Fix For: 2.6.0
>
> Attachments: YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch,
> YARN-6566.4.patch, YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch,
> application-page.bmp, applications-page.bmp
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When I tuning the hive HQL in multi-users environment,I can not get the full
> SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try
> to set the SQL text in hadoop job's jobname property,I realize it is going to
> damage the structure of hadoop applications web view,so I add a property to
> hadoop job,which named "jobdescription",When a hive HQL be submitted,the full
> HQL text was assigned to the property,so i can identified the HQL coveniently
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999657#comment-15999657 ] liuzhenhua commented on YARN-6566: -- [~vrushalic] Thanks > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999822#comment-15999822 ] liuzhenhua commented on YARN-6566: -- I upload 7 attach patches for solving the improvement,if my code was adopted, I will submit the hive's code modifications simultaneously,In addition,I also upload two image renderings named application-page.bmp and applications-page.bmp,show hive HQL script text in hadoop web view > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, > YARN-6566.4.patch, YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001887#comment-16001887 ] liuzhenhua commented on YARN-6566: -- [~templedf] Thanks,The patches which I submit is only for hadoop part,if it is committed,I will submit the code for hive part. > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002006#comment-16002006 ] Haibo Chen commented on YARN-6566: -- Do you think that you can reuse application tags for this purpose? > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002160#comment-16002160 ] liuzhenhua commented on YARN-6566: -- [~haibochen] Thanks first, I tried to use the job's 'name' property,but when the hive HQL is large,it damaged the applications's web view layout, so I add the property 'jobdescription',represent more job's detail information which show in the single application's web page,like the attach renderings application-page.bmp > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002179#comment-16002179 ] liuzhenhua commented on YARN-6566: -- in my opinion when submit a job in hive,hive filled the job's name property with abbreviation HQL text,In order not to destroy the applications's web view layout. sometimes the full large HQL text is not not suitable to be show in the application list web page. In multi-users environment if I want to tuning the others' HQL i must get the full HQL script,but it is difficult > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002918#comment-16002918 ] Haibo Chen commented on YARN-6566: -- Can you please elaborate on how the web UI layout is damaged (Maybe a screenshot)? We may be able to fix that layout issue. > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002941#comment-16002941 ] liuzhenhua commented on YARN-6566: -- for example,if i set a large HQL script text to job's 'name' property,the layout of web page like attach file applications-page.bmp will be damage > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004175#comment-16004175 ] Wilfred Spiegelenburg commented on YARN-6566: - Looking at the feature I think we have two possible problems: - From a security point I do not think that this is a good idea. The Hive SQL query could have sensitive information contained in it. There could be names or details in multiple parts of the query that you do not want to expose. This feature could thus lead to information disclosure that you can not control. - Second problem I see here is that the query info could be large which has a performance impact. Not just for the UI but also for the state store. Queries can be hundreds of kilobytes and that detail would need to be persisted in the state store. There have been a number of jiras to implemented and currently being worked on to decrease the size of the object that is stored in the state store and this would negate a lot of that work. > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004315#comment-16004315 ] liuzhenhua commented on YARN-6566: -- [~Wilfred Spiegelenburg] Thanks for your advice,I think you are right,but sometimes the requirements were really exist,Could you add the feature and make it be configuarble in hive HQL or spark SQL on Yarn? it is convenient for cluster administrator to tuning the others' HQL.If not,when we upgrade the hadoop next time,we must add the feature in our cluster manually again > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009962#comment-16009962 ] liuzhenhua commented on YARN-6566: -- [~wilfreds] Could you add the feature in hadoop and then I make the feature be configurable in Hive?Give the security and memory questions to user,let them choose enable or disable the feature by themselves > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
