[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914528#action_12914528 ] Philip Zeyliger commented on HIVE-1530: --- +1. I'm a big fan of this change. We've repeatedly had customers using an old or weird hive-default or non-existent hive-default, and that's caused quite tricky to debug issues. > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.7.0 > > Attachments: HIVE-1530.1.patch.txt > > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914419#action_12914419 ] HBase Review Board commented on HIVE-1530: -- Message from: "Carl Steinbach" --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/902/ --- Review request for Hive Developers. Summary --- HIVE-1530.1.patch.txt: * Move conf/hive-default.xml to common/resources/hive-default.xml and modify the build so that this gets included in hive-common-xxx.jar * Copy contents of conf/hive-default.xml to conf/hive-site.xml.template * Modify HiveConf so that it logs an INFO level message with the location of the hive-default.xml and hive-site.xml files which were loaded. This addresses bug HIVE-1530. http://issues.apache.org/jira/browse/HIVE-1530 Diffs - build.xml 4b345b5 common/build.xml d9ac07e common/resources/hive-default.xml PRE-CREATION common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 47b7518 conf/hive-default.xml 1465317 conf/hive-site.xml.template PRE-CREATION Diff: http://review.cloudera.org/r/902/diff Testing --- Thanks, Carl > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.7.0 > > Attachments: HIVE-1530.1.patch.txt > > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898266#action_12898266 ] Edward Capriolo commented on HIVE-1530: --- @Joydeep +1 > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898118#action_12898118 ] Joydeep Sen Sarma commented on HIVE-1530: - don't dis-allow hive.* options not specified in HiveConf. reason is that hive is extensible at various points via custom code and those have access to config object and installs may want to set variables specific to their plugins etc. (we shouldn't be in the business of telling them what not to name them) > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898080#action_12898080 ] Joydeep Sen Sarma commented on HIVE-1530: - ok - that makes sense. leave a hive-site.xml.sample and hive-log4j.properties.example in the conf/. i agree with Ed's point about how difficult it is to figure out hadoop config variables now and hadoop is worse off for it. commands are nice - but having a template is better. it's easy to clone an example file and append/modify the default description to add site specific notes. and one can grep. we could autogenerate the hive-site.xml.sample from config variable metadata in the source code. that would keep us in sync with code. > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[
https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898040#action_12898040
]
Carl Steinbach commented on HIVE-1530:
--
@Ed: In my opinion the ideal solution is to get rid of the hive-default.xml
file entirely and rely on the default values that appear in HiveConf. We can
add a {{describe property }} command that prints out a description of the
property, and also add checks that protect the {{hive.*}} configuration
property namespace (i.e. you can't set a {{hive.*}} property unless it is
defined in HiveConf). Another advantage of this approach is that we don't have
to worry about hive-default.xml falling out of sync with HiveConf, e.g. a user
upgrades to a new version of Hive but continues to use an older copy of
hive-default.xml.
> Include hive-default.xml and hive-log4j.properties in hive-common JAR
> -
>
> Key: HIVE-1530
> URL: https://issues.apache.org/jira/browse/HIVE-1530
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Configuration
>Reporter: Carl Steinbach
>
> hive-common-*.jar should include hive-default.xml and hive-log4j.properties,
> and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The
> hive-default.xml file that currently sits in the conf/ directory should be
> removed.
> Motivations for this change:
> * We explicitly tell users that they should never modify hive-default.xml yet
> give them the opportunity to do so by placing the file in the conf dir.
> * Many users are familiar with the Hadoop configuration mechanism that does
> not require *-default.xml files to be present in the HADOOP_CONF_DIR, and
> assume that the same is true for HIVE_CONF_DIR.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898038#action_12898038 ] Carl Steinbach commented on HIVE-1530: -- bq. but users may want to modify the log4.properties files. how would do they do that in the new arrangement? Hive uses a classloader to get the hive-log4j and hive-exec-log4j property resources. If a user wants to override the log4j properties that are bundled with the JAR they only need to make sure that their copy appears first on the CLASSPATH. > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898030#action_12898030 ] Edward Capriolo commented on HIVE-1530: --- I like the default xml. Hive has many undocumented options, new ones are being added often. Are end users going to know which jar the default.xml are in? Users want to extracting a jar just to get the conf out of it to read the description of the setting. As for what hadoop does...I personally find it annoying to have navigate to hadoop/src/mapred/mapred-default.xml or to hadoop/src/hdfs/hdfs-default.xml to figure out what options I have for settings. So i do not really thing we should just do it to be like hadoop it it makes peoples life harder. If anything please keep it as hive-site.xml.sample. > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897993#action_12897993 ] Joydeep Sen Sarma commented on HIVE-1530: - removing the .xml files makes sense. but users may want to modify the log4.properties files. how would do they do that in the new arrangement? > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
