> On Dec. 4, 2015, 7:46 a.m., Lenni Kuff wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, > > line 414 > > <https://reviews.apache.org/r/40948/diff/1/?file=1153548#file1153548line414> > > > > Would it make sense to use a cached-gauge for these operations? > > > > https://dropwizard.github.io/metrics/3.1.0/manual/core/#cached-gauges
Currently I am doing the same effect using the volatile variables, they are not actually updated until the timer thread kicks in (see below comment). I am thinking to keep those, instead of relying on the Codahale version, so that it could be used even if metrics is disabled, these variables may be useful for example by webserver or other display tools. > On Dec. 4, 2015, 7:46 a.m., Lenni Kuff wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, > > line 5777 > > <https://reviews.apache.org/r/40948/diff/1/?file=1153548#file1153548line5777> > > > > How expensive is this? What are your thoughts on doing this nce at > > startup, then increment/decrementing as individual objects are > > added/removed? I actually thought about this. I dont think I can do a startup count and then increment/decrement due to the case of multiple HMS (like load-balance mode). There are some notification offered by HCatalog that could potentially fit into this, but then user needs to setup HCatalog and JMS and I wanted to keep this just in HMS layer. I thought the background thread is the best tradeoff, it will not always be accurate but the interval is configurable and a slow query should not matter as much with a long enough timing. And the new query should not be that bad, it is a simple count query. - Szehon ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40948/#review108948 ----------------------------------------------------------- On Dec. 4, 2015, 10:35 p.m., Szehon Ho wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/40948/ > ----------------------------------------------------------- > > (Updated Dec. 4, 2015, 10:35 p.m.) > > > Review request for hive. > > > Bugs: HIVE-12499 > https://issues.apache.org/jira/browse/HIVE-12499 > > > Repository: hive-git > > > Description > ------- > > Add separate timer thread that polls for count of database, table, partition > entries to publish as metrics, the period is configurable. Delay in getting > exact number should be ok as this is for monitoring. > > Implemented for HBase and DB metastores. > > > Diffs > ----- > > > common/src/java/org/apache/hadoop/hive/common/metrics/common/MetricsConstant.java > 95e2bcf > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 4d881ba > common/src/test/org/apache/hadoop/hive/common/metrics/MetricsTestUtils.java > fd420f7 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreMetrics.java > f571c7c > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/hbase/TestHBaseMetastoreMetrics.java > PRE-CREATION > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > 00602e1 > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java > 1c0ab6d > metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 5b36b03 > > metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java > 2fb3e8f > metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java > 98e6c75 > > metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java > 9a1d159 > > metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java > 8dde0af > > Diff: https://reviews.apache.org/r/40948/diff/ > > > Testing > ------- > > Added unit tests for HBase and Db metastores. > > > Thanks, > > Szehon Ho > >
