[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818516#comment-13818516 ] Eugene Koifman commented on HIVE-5627: -- I believe the string that Pig understands is '-useHCatalog' , i.e. that what can be placed in 'arg' parameter. The webhcat doc has it in all lower case Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818585#comment-13818585 ] Lefty Leverenz commented on HIVE-5627: -- Good catch. Got it right in 'arg' description but wrong in 'usehcatalog' description. Fixed now: [https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Pig]. bq. *arg* Set a program argument. If -useHCatalog is included, then *usehcatalog* is interpreted as true (Hive 0.13.0 and later). bq. *usehcatalog* Specify that the submitted job uses HCatalog and therefore needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. (See HIVE-5133.) This parameter will be introduced in Hive 0.13.0. It can also be set to true by including -useHCatalog in the *arg* parameter. ... (When Hive 0.13.0 is released, I intend to change will be introduced to is introduced for the pig and mapreduce/jar calls -- it's on the TODO list.) Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818610#comment-13818610 ] Eugene Koifman commented on HIVE-5627: -- +1 Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818071#comment-13818071 ] Lefty Leverenz commented on HIVE-5627: -- Done, please review here: * [https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+MapReduceJar] * [https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Pig] I assumed the default was 'false' for both calls. Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818239#comment-13818239 ] Eugene Koifman commented on HIVE-5627: -- For the Pig call I would add the following: If '-useHCatalog' is included in the 'arg' parameter, then 'usehcatalog' is interpreted as true. Othewrwise it looks fine. Thanks! Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818252#comment-13818252 ] Lefty Leverenz commented on HIVE-5627: -- Done. Thanks for the quick review. Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13817874#comment-13817874 ] Lefty Leverenz commented on HIVE-5627: -- In a recent download, ../hive/hcatalog/templeton/Server.java has javadoc comments for this parameter for mapreduce/jar and pig, but not for hive (and it isn't in the hive code either). Also, there are several errors and omissions in the javadocs and code. The user parameter name is usehcatalog which corresponds to the internal usesHcatalog: * mapreduce/jar javadoc says usesHcatalog for the user parameter (incorrect) ** its javadoc omits several params ** also has enablelog in the code -- not in javadoc * pig javadoc says usehcatalog (correct) ** javadoc omits several params ** also has enablelog in the code -- not in javadoc ** but MapString, Object omits usehcatalog * hive javadoc doesn't have the parameter (nor is it in the code) ** but it has javadoc for enablelog ** javadoc has all params but calls 'files' srcFile and 'arg' hiveArgs ** omits arg from MapString, Object Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13817919#comment-13817919 ] Eugene Koifman commented on HIVE-5627: -- [~le...@hortonworks.com] I'm not sure if I understand your comment. The purpose of this ticket is to document that mapreduce/jar and pig REST calls now have additional 'usehcatalog' parameter as well as webhcat-default.xml properties that are used in conjunction with 'usehcatalog'. hive doesn't support this parameter. What the internal variable name is in Java is not relevant to end users. The names in FormParam(usehcatalog) is what the REST call expects in the query string. Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818011#comment-13818011 ] Lefty Leverenz commented on HIVE-5627: -- Missed the update removing hive from the list, sorry. The rest of my comment just points out inconsistencies to fix in the javadocs (internal variable names should not appear there) and omissions in the MapString, Object code that might or might not need fixing. Plus a question of whether enablelog needs to be documented, but that's just something I noticed while researching usehcatalog so it doesn't belong in this ticket. Considering how many inconsistencies I found in these three calls, the other calls should probably be checked for accuracy too. Does the information about shipping the Hive tar to the target node only apply to mapreduce/jar and pig when usehcatalog is true, or is it more general information? Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818044#comment-13818044 ] Lefty Leverenz commented on HIVE-5627: -- Correction -- pig javadoc also gives the internal variable name instead of the parameter name: * @param usesHcatalog ... Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5627) Document 'usehcatalog' parameter on WebHCat calls
[ https://issues.apache.org/jira/browse/HIVE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13818050#comment-13818050 ] Lefty Leverenz commented on HIVE-5627: -- bq. Does the information about shipping the Hive tar to the target node only apply to mapreduce/jar and pig when usehcatalog is true, or is it more general information? Nevermind, the answer was staring me in the face. Document 'usehcatalog' parameter on WebHCat calls - Key: HIVE-5627 URL: https://issues.apache.org/jira/browse/HIVE-5627 Project: Hive Issue Type: Sub-task Components: Documentation, WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz Fix For: 0.13.0 The following REST calls in WebHCat: 1. mapreduce/jar 2. pig now support an additional parameter 'usehcatalog'. This is a mechanism for the caller to tell WebHCat that the submitted job uses HCat, and thus needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. The JavaDoc on corresponding methods in org.apache.hive.hcatalog.templeton.Server describe this parameter. Additionally, if templeton.hive.archive, templeton.hive.home and templeton.hcat.home are defined in webhcat-site.xml (documented in webhcat-default.xml) then WebHCat will ship the Hive tar to the target node where the job actually runs. This means that Hive doesn't need to be installed on every node in the Hadoop cluster. (This part was added in HIVE-5547). This is independent of security, but improves manageability. This should be added to the sections in https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference that correspond to these methods. -- This message was sent by Atlassian JIRA (v6.1#6144)
