> On Sept. 3, 2015, 12:39 a.m., Alejandro Fernandez wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java, > > line 67 > > <https://reviews.apache.org/r/38070/diff/2/?file=1062696#file1062696line67> > > > > Why would initializeSubsystem be called from multiple threads?
In response to profiling session yourkit comaplined about a deadlock in this call, could be from upper YARN layers. - Sid ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38070/#review97578 ----------------------------------------------------------- On Sept. 2, 2015, 10:17 p.m., Sid Wagle wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38070/ > ----------------------------------------------------------- > > (Updated Sept. 2, 2015, 10:17 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Mahadev Konar, > Sumit Mohanty, and Srimanth Gunturi. > > > Bugs: AMBARI-12983 > https://issues.apache.org/jira/browse/AMBARI-12983 > > > Repository: ambari > > > Description > ------- > > Use GroupBy aggregators wherever possible. > > Query plan: > > 0: jdbc:phoenix:localhost:61181:/hbase> explain select SUM(METRIC_SUM), > SUM(METRIC_COUNT), MAX(METRIC_MAX), MIN(METRIC_MIN), METRIC_NAME, HOSTNAME, > APP_ID from METRIC_RECORD WHERE SERVER_TIME > 1440106830000 AND SERVER_TIME < > 1440106950000 GROUP BY METRIC_NAME, HOSTNAME, APP_ID; > +------------+ > | PLAN | > +------------+ > | CLIENT PARALLEL 1-WAY FULL SCAN OVER METRIC_RECORD | > | SERVER FILTER BY (SERVER_TIME > 1440106830000 AND SERVER_TIME < > 1440106950000) | > | SERVER AGGREGATE INTO DISTINCT ROWS BY [METRIC_NAME, HOSTNAME, APP_ID] | > | CLIENT MERGE SORT | > +------------+ > > Scan depends on number of Regions, so it would by lets say 4 way parallel > scan with 4 Regions. > Next effort is to come up with a good split policy for the precision table. > > > Diffs > ----- > > > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector > bdf00b5 > ambari-metrics/ambari-metrics-timelineservice/pom.xml d994d52 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryClientService.java > 8a37a57 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > c615804 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > d4f919e > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java > 37fb088 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java > 642fcfe > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/Condition.java > e0cb3d0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java > 462c8d9 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/EmptyCondition.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > ef0f4ce > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/SplitByMetricNamesCondition.java > c8b8709 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java > 13fa348 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java > b480b7a > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 06e153b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > ef6fc58 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml > 2d11412 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > c716bea > > Diff: https://reviews.apache.org/r/38070/diff/ > > > Testing > ------- > > All ams unit test pass. > > > Thanks, > > Sid Wagle > >
