> On June 3, 2013, 9:03 p.m., Ashutosh Chauhan wrote: > > ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryImpl.java, line 86 > > <https://reviews.apache.org/r/11029/diff/2/?file=290954#file290954line86> > > > > In case of incorrect config, should this throw an exception instead of > > silent return, otherwise there will be errors later when something is tried > > to be written in history file.
Errors will not be there later, as it does not write if the histStream has not been initialized. I don't think we should fail the query just because hive history logging failed. This is also current behavior in hive. - Thejas ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11029/#review21352 ----------------------------------------------------------- On Aug. 10, 2013, 4:24 p.m., Thejas Nair wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11029/ > ----------------------------------------------------------- > > (Updated Aug. 10, 2013, 4:24 p.m.) > > > Review request for hive. > > > Bugs: HIVE-4513 > https://issues.apache.org/jira/browse/HIVE-4513 > > > Repository: hive-git > > > Description > ------- > > HiveHistory log files (hive_job_log_hive_*.txt files) store information about > hive query such as query string, plan , counters and MR job progress > information. > > There is no mechanism to delete these files and as a result they get > accumulated over time, using up lot of disk space. > I don't think this is used by most people, so I think it would better to turn > this off by default. Jobtracker logs already capture most of this > information, though it is not as structured as history logs. > > The change : > A new config parameter hive.session.history.enabled controls if the > history-log is enabled. By default it is set to false. > SessionState initializes the HiveHIstory object. When this config is set to > false, it creates a Proxy object that does not do anything. I did this > instead of having SessionState return null, because that would add null > checks in too many places. This keeps the code cleaner and avoids possibility > of NPE. > As the proxy only works against interfaces, i created a HiveHistory > interface, moved the implementation to HiveHistoryImpl. static functions were > moved to HiveHistoryUtil . > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 83f337b > conf/hive-default.xml.template 0a6e433 > hbase-handler/src/test/templates/TestHBaseCliDriver.vm c59e882 > ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java 97436c5 > ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryImpl.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryProxyHandler.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryUtil.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryViewer.java > fdd56db > ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java ab369f0 > ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java a783303 > ql/src/test/templates/TestCliDriver.vm a6ae6c3 > > Diff: https://reviews.apache.org/r/11029/diff/ > > > Testing > ------- > > > Thanks, > > Thejas Nair > >
