[GitHub] accumulo pull request #245: ACCUMULO-4620 Consolidate all Maven modules into...

[ctubbsii]

Github user ctubbsii commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/245#discussion_r110495402
  
--- Diff: pom.xml ---
@@ -80,24 +80,24 @@
 ${maven.min-version}
   
   
-assemble
-core
-docs
-fate
-iterator-test-harness
-maven-plugin
-minicluster
-proxy
-server/base
-server/gc
-server/master
-server/monitor
-server/native
-server/tracer
-server/tserver
-shell
-start
-test
+modules/assemble
+modules/core
+modules/docs
+modules/fate
+modules/iterator-test-harness
+modules/maven-plugin
+modules/minicluster
+modules/proxy
+modules/base
+modules/gc
+modules/master
+modules/monitor
+modules/native
+modules/tracer
+modules/tserver
+modules/shell
+modules/start
+modules/test
--- End diff --

This changes the sort order. These aren't sorted any more.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #246: ACCUMULO-4620 Consolidate all Maven modules into...

[mikewalch]

GitHub user mikewalch opened a pull request:

https://github.com/apache/accumulo/pull/246

ACCUMULO-4620 Consolidate all Maven modules into one directory



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mikewalch/accumulo modules-dir-1.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/accumulo/pull/246.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #246


commit 2afde60009aa705c29d7bcf670427c7177eaab28
Author: Mike Walch 
Date:   2017-04-07T19:50:49Z

ACCUMULO-4620 Consolidate all Maven modules into one directory




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #245: ACCUMULO-4620 Consolidate all Maven modules into...

[mikewalch]

GitHub user mikewalch opened a pull request:

https://github.com/apache/accumulo/pull/245

ACCUMULO-4620 Consolidate all Maven modules into one directory

This PR is for master.  I am going to create PRs for 1.8 & 1.7 branches to 
make merging easier in the future.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mikewalch/accumulo modules-dir-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/accumulo/pull/245.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #245


commit d781bbec75738723b94c94f3043a96e4520e3246
Author: Mike Walch 
Date:   2017-04-07T16:14:59Z

ACCUMULO-4620 Consolidate all Maven modules into one directory




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on the issue:

https://github.com/apache/accumulo/pull/242
  
@milleruntime Thanks for the feedback! I'll check the JS stuff, I liked the 
ideas you mentioned.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110476772
  
--- Diff: server/monitor/src/main/resources/resources/overview.js ---
@@ -0,0 +1,293 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements.  See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License.  You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates overview initial tables
+ */
+$(document).ready(function() {
+  createMasterTable();
+  createZKTable();
+  refreshOverview();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshOverview() {
+  $.ajaxSetup({
+async: false
+  });
+  getMaster();
+  getZK();
+  getIngestRate();
+  getScanEntries();
+  getIngestByteRate();
+  getQueryByteRate();
+  getLoadAverage();
+  getLookups();
+  getMinorCompactions();
+  getMajorCompactions();
+  getIndexCacheHitRate();
+  getDataCacheHitRate();
+  $.ajaxSetup({
+async: true
+  });
+  refreshMasterTable();
+  refreshZKTable();
+  makePlots();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+  refreshOverview();
+}
+
+/**
+ * Refreshes the master table
+ */
+function refreshMasterTable() {
+  var data = sessionStorage.master === undefined ?
+  [] : JSON.parse(sessionStorage.master);
+
+  $('#master tr td:first').hide();
+  $('#master tr td').hide();
+
+  // If the master is down, show the first row, otherwise refresh old 
values
+  if (data.length === 0 || data.master === 'No Masters running') {
+$('#master tr td:first').show();
+  } else {
+$('#master tr td:not(:first)').show();
+var table = $('#master td.right');
+
+table.eq(0).html(bigNumberForQuantity(data.tables));
+table.eq(1).html(bigNumberForQuantity(data.totalTabletServers));
+table.eq(2).html(bigNumberForQuantity(data.deadTabletServersCount));
+table.eq(3).html(bigNumberForQuantity(data.tablets));
+table.eq(4).html(bigNumberForQuantity(data.numentries));
+table.eq(5).html(bigNumberForQuantity(data.lookups));
+table.eq(6).html(timeDuration(data.uptime));
+  }
+}
+
+/**
+ * Generates the master table
+ */
+function createMasterTable() {
+  var items = [];
+  items.push('Accumulo' +
+  'Master');
+
+  items.push('' +
+  'Master is Down');
+
+  items.push('Tables' +
+  '');
+
+  items.push('Tablet' +
+  'Servers');
+
+  items.push('Dead' +
+  'TabletServers');
+
+  items.push('Tablets');
+  items.push('Entries');
+  items.push('Lookups');
+  items.push('Uptime');
+
+  $('', {
+html: items.join(''),
+class: 'table table-bordered table-striped table-condensed'
+  }).appendTo('#master');
+}
+
+/**
+ * Refresh the zookeeper table
+ */
+function refreshZKTable() {
+  var data = sessionStorage.zk === undefined ?
+  [] : JSON.parse(sessionStorage.zk);
+
+  $('#zookeeper tr td:first').hide();
+  $('#zookeeper tr:gt(2)').remove();
+
+  if (data.length === 0 || data.zkServers.length === 0) {
+$('#zookeeper tr td:first').show();
+  } else {
+var items = [];
+$.each(data.zkServers, function(key, val) {
+  if (val.clients >= 0) {
+items.push('' + val.server + '');
+items.push('' + val.mode + '');
+items.push('' + val.clients + '');
+  } else {
+items.push('' + val.server + '');
+items.push('Down');
+items.push('');
+  }
+});
+$('', {
+  html: items.join('')
+}).appendTo('#zookeeper table');
+  }
+}
+
+/**
+ * Generates the zookeeper table
+ */
+function createZKTable() {
+  var items = [];
+  items.push('Zookeeper');
+  items.push('ServerModeClients');
+  items.push('No Zookeepers');
+  $('', {
+html: items.join(''),
+

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110476674
  
--- Diff: server/monitor/src/main/resources/resources/master.js ---
@@ -0,0 +1,261 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements.  See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License.  You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates master initial table
+ */
+$(document).ready(function() {
+  createHeader();
+  doBanner('masterBanner', 'danger', 'Master Server Not Running');
+  refreshMaster();
+
+  // Create tooltip for table column information
+  $(document).tooltip();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshMaster() {
+  $.ajaxSetup({
+async: false
+  });
+  getMaster();
+  getRecoveryList();
+  $.ajaxSetup({
+async: true
+  });
+  refreshMasterTable();
+  recoveryList();
+}
+
+
+/*
+ * The tables refresh function will do this functionality
+ * If tables are removed from Master, uncomment this function
+ */
+/**
+ * Used to redraw the page
+ */
+/*function refresh() {
+  refreshMaster();
+}*/
+
+/**
+ * Creates recovery list table
+ */
+function recoveryList() {
+  /*
+   * Get the recovery value obtained earlier,
+   * if it doesn't exists, create an empty array
+   */
+  var data = sessionStorage.recoveryList === undefined ?
+  [] : JSON.parse(sessionStorage.recoveryList);
+
+  $('#recoveryList tr').remove();
+
+  // If there is no recovery list data, hide the table
+  if (data.length === 0 || data.recoveryList.length === 0) {
+$('#recoveryList').hide();
+  } else {
+$('#recoveryList').show();
+
+var caption = [];
+
+caption.push('LogRecovery');
+caption.push('Some tablets were 
unloaded' +
+' in an unsafe manner. Write-ahead logs are being' +
+' recovered.');
+
+$('', {
+  html: caption.join('')
+}).appendTo('#recoveryList');
+
+var items = [];
+
+/*
+ * Create the header for the recovery list table
+ * Adds the columns, add sortTable function on click,
+ * if the column has a description, add title taken from the global.js
+ */
+items.push('' +
+'Server');
+items.push('Log');
+items.push('Time');
+items.push('Copy/Sort');
+
+$('', {
+  html: items.join('')
+}).appendTo('#recoveryList');
+
+// Creates the table for the recovery list
+$.each(data.recoveryList, function(key, val) {
+  var items = [];
+  items.push('' +
+  val.server + '');
+  items.push('' + 
val.log +
+  '');
+  var date = new Date(parseInt(val.time));
+  date = date.toLocaleString().split(' ').join('');
+  items.push('' + date 
+
+  '');
+  items.push('' +
+  val.copySort + '');
+
+  $('', {
+html: items.join('')
+  }).appendTo('#recoveryList');
+});
+  }
+}
+
+/**
+ * Generates the master table
+ */
+function refreshMasterTable() {
+  // Gets the master status
+  var status = JSON.parse(sessionStorage.status).masterStatus;
+
+  // Hide the banner and the master table
+  $('#masterBanner').hide();
+  $('#masterStatus tr:gt(0)').remove();
+  $('#masterStatus').hide();
+
+  // If master status is error, show banner, otherwise, create master table
+  if (status === 'ERROR') {
+$('#masterBanner').show();
+  } else {
+$('#masterStatus').show();
+var data = JSON.parse(sessionStorage.master);
+var items = [];
+items.push('' + data.master + '');
+
+items.push('' + data.onlineTabletServers + '');
+
+items.push('' + data.totalTabletServers + '');
+
+var date = new Date(parseInt(data.lastGC));
+date = date.toLocaleString().split(' ').join('');

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110476591
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java
 ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+  // Variable names become JSON keys
+  private TabletStats total, historical;
+
+  /**
+   * Generates tserver summary
+   *
+   * @return tserver summary
+   */
+  @GET
+  public TabletServers getTserverSummary() {
+MasterMonitorInfo mmi = Monitor.getMmi();
+if (null == mmi) {
+  throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+}
+
+TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+for (TabletServerStatus status : mmi.tServerInfo) {
+  tserverInfo.addTablet(new TabletServer(status));
+}
+
+tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+return tserverInfo;
+  }
+
+  /**
+   * REST call to clear dead servers from list
+   *
+   * @param server
+   *  Dead server to clear
+   */
+  @POST
+  @Consumes(MediaType.TEXT_PLAIN)
+  public void clearDeadServer(@QueryParam("server") String server) throws 
Exception {
+DeadServerList obit = new 
DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + 
Constants.ZDEADTSERVERS);
+obit.delete(server);
+  }
+
+  /**
+   * Generates a recovery tserver list
+   *
+   * @return Recovery tserver list
+   */
+  @Path("recovery")
+  @GET
+  public Map>> getTserverRecovery() {
--- End diff --

Yes, it has to return a maps of maps (in this case), since the first map 
uses the Key for JSON (this is another way of 

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110468835
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java
 ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import 
org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+  /**
+   * Generates a trace summary
+   *
+   * @param minutes
+   *  Range of minutes to filter traces
+   * @return Trace summary in specified range
+   */
+  @Path("summary/{minutes}")
+  @GET
+  public RecentTracesList getTraces(@DefaultValue("10") 
@PathParam("minutes") int minutes) throws Exception {
+
+RecentTracesList recentTraces = new RecentTracesList();
+
+Pair pair = getScanner();
+final Scanner scanner = pair.getFirst();
+if (scanner == null) {
+  return recentTraces;
+}
+
+Range range = getRangeForTrace(minutes);
+scanner.setRange(range);
+
+final Map summary = new TreeMap<>();
+if (null != pair.getSecond()) {
+  pair.getSecond().doAs(new PrivilegedAction() {
--- End diff --

Why do you need privilege escalation in this class?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110466878
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+  private static final TabletServerStatus NO_STATUS = new 
TabletServerStatus();
+
+  /**
+   * Generates a table list based on the namespace
+   *
+   * @param namespace
+   *  Namespace used to filter the tables
+   * @return Table list
+   */
+  private TablesList generateTables(String namespace) {
+SortedMap namespaces = 
Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+TablesList tableNamespace = new TablesList();
+
+/*
+ * Add the tables that have the selected namespace Asterisk = All 
namespaces Hyphen = Default namespace
+ */
+for (String key : namespaces.keySet()) {
+  if (namespace.equals("*") || namespace.equals(key) || 
(key.equals("") && namespace.equals("-"))) {
+tableNamespace.addTable(new TableNamespace(key));
+  }
+}
+
+return generateTables(tableNamespace);
+  }
+
+  /**
+   * Generates a table list based on the list of namespaces
+   *
+   * @param tableNamespace
+   *  Namespace list
+   * @return Table list
+   */
+  private TablesList generateTables(TablesList tableNamespace) {
+Instance inst = Monitor.getContext().getInstance();
+Map tidToNameMap = Tables.getIdToNameMap(inst);
+SortedMap tableStats = new TreeMap<>();
+
+if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+  for (Entry te : 
Monitor.getMmi().tableMap.entrySet())
+tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, 
te.getKey()), te.getValue());
+Map compactingByTable = 
TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+TableManager tableManager = TableManager.getInstance();
+List tables = new ArrayList<>();
+
+// Add tables 

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110473250
  
--- Diff: server/monitor/src/main/resources/resources/master.js ---
@@ -0,0 +1,261 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements.  See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License.  You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates master initial table
+ */
+$(document).ready(function() {
+  createHeader();
+  doBanner('masterBanner', 'danger', 'Master Server Not Running');
+  refreshMaster();
+
+  // Create tooltip for table column information
+  $(document).tooltip();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshMaster() {
+  $.ajaxSetup({
+async: false
+  });
+  getMaster();
+  getRecoveryList();
+  $.ajaxSetup({
+async: true
+  });
+  refreshMasterTable();
+  recoveryList();
+}
+
+
+/*
+ * The tables refresh function will do this functionality
+ * If tables are removed from Master, uncomment this function
+ */
+/**
+ * Used to redraw the page
+ */
+/*function refresh() {
+  refreshMaster();
+}*/
+
+/**
+ * Creates recovery list table
+ */
+function recoveryList() {
+  /*
+   * Get the recovery value obtained earlier,
+   * if it doesn't exists, create an empty array
+   */
+  var data = sessionStorage.recoveryList === undefined ?
+  [] : JSON.parse(sessionStorage.recoveryList);
+
+  $('#recoveryList tr').remove();
+
+  // If there is no recovery list data, hide the table
+  if (data.length === 0 || data.recoveryList.length === 0) {
+$('#recoveryList').hide();
+  } else {
+$('#recoveryList').show();
+
+var caption = [];
+
+caption.push('LogRecovery');
+caption.push('Some tablets were 
unloaded' +
+' in an unsafe manner. Write-ahead logs are being' +
+' recovered.');
+
+$('', {
+  html: caption.join('')
+}).appendTo('#recoveryList');
+
+var items = [];
+
+/*
+ * Create the header for the recovery list table
+ * Adds the columns, add sortTable function on click,
+ * if the column has a description, add title taken from the global.js
+ */
+items.push('' +
+'Server');
+items.push('Log');
+items.push('Time');
+items.push('Copy/Sort');
+
+$('', {
+  html: items.join('')
+}).appendTo('#recoveryList');
+
+// Creates the table for the recovery list
+$.each(data.recoveryList, function(key, val) {
+  var items = [];
+  items.push('' +
+  val.server + '');
+  items.push('' + 
val.log +
+  '');
+  var date = new Date(parseInt(val.time));
+  date = date.toLocaleString().split(' ').join('');
+  items.push('' + date 
+
+  '');
+  items.push('' +
+  val.copySort + '');
+
+  $('', {
+html: items.join('')
+  }).appendTo('#recoveryList');
+});
+  }
+}
+
+/**
+ * Generates the master table
+ */
+function refreshMasterTable() {
+  // Gets the master status
+  var status = JSON.parse(sessionStorage.status).masterStatus;
+
+  // Hide the banner and the master table
+  $('#masterBanner').hide();
+  $('#masterStatus tr:gt(0)').remove();
+  $('#masterStatus').hide();
+
+  // If master status is error, show banner, otherwise, create master table
+  if (status === 'ERROR') {
+$('#masterBanner').show();
+  } else {
+$('#masterStatus').show();
+var data = JSON.parse(sessionStorage.master);
+var items = [];
+items.push('' + data.master + '');
+
+items.push('' + data.onlineTabletServers + '');
+
+items.push('' + data.totalTabletServers + '');
+
+var date = new Date(parseInt(data.lastGC));
+date = date.toLocaleString().split(' 

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110471346
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerInformation.java
 ---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tables.CompactionsList;
+import org.apache.accumulo.monitor.rest.tables.CompactionsTypes;
+import org.apache.accumulo.monitor.rest.trace.RecoveryStatusInformation;
+import org.apache.accumulo.server.util.TableInfoUtil;
+
+/**
+ *
+ * Generates tserver information
+ *
+ * @since 2.0.0
+ *
+ */
+public class TabletServerInformation {
+
+  // Variable names become JSON keys
+  @XmlAttribute(name = "id")
+  public String server;
+
+  public String hostname;
+  public long lastContact;
+  public double osload;
+
+  public CompactionsTypes compactions;
+
+  public int tablets;
+  public double ingest, query, ingestMB, queryMB;
+  public Integer scans; // For backwards compatibility, has same 
information as scansRunning
+  public Double scansessions;
+  public Double scanssessions; // For backwards compatibility
+  public long holdtime;
+
+  // New variables
+
+  public String ip;
+  private Integer scansRunning, scansQueued, minorRunning, minorQueued, 
majorRunning, majorQueued;
+  private CompactionsList scansCompacting, major, minor; // if scans is 
removed, change scansCompacting to scans
+  public long entries, lookups, indexCacheHits, indexCacheRequests, 
dataCacheHits, dataCacheRequests;
--- End diff --

More style


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110470413
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/ServerShuttingDownInformation.java
 ---
@@ -14,25 +14,32 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.accumulo.monitor.util.celltypes;
+package org.apache.accumulo.monitor.rest.tservers;
 
-import java.io.Serializable;
-import java.util.Comparator;
+import javax.xml.bind.annotation.XmlAttribute;
 
-public abstract class CellType implements Comparator, Serializable {
-
-  private static final long serialVersionUID = 1L;
-  private boolean sortable = true;
-
-  abstract public String alignment();
+/**
+ *
+ * Generates a tserver shutting down
+ *
+ * @since 2.0.0
+ *
+ */
+public class ServerShuttingDownInformation {
--- End diff --

+1 Much better name


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110467050
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/RecentTracesInformation.java
 ---
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+
+/**
+ *
+ * Generates a recent trace
+ *
+ * @since 2.0.0
+ *
+ */
+public class RecentTracesInformation {
+
+  // Variable names become JSON keys
+  public String type;
+  public Long avg;
+
+  public int total = 0;
+
+  public long min = Long.MAX_VALUE, max = Long.MIN_VALUE;
+  private long totalMS = 0l;
+  public long histogram[] = new long[] {0l, 0l, 0l, 0l, 0l, 0l};
--- End diff --

Nit: Capitol L is easier to read


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110473929
  
--- Diff: server/monitor/src/main/resources/resources/problems.js ---
@@ -0,0 +1,232 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements.  See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License.  You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshProblems() {
+  $.ajaxSetup({
+async: false
+  });
+  getProblems();
+  $.ajaxSetup({
+async: true
+  });
+  refreshProblemSummaryTable();
+  refreshProblemDetailsTable();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+  refreshProblems();
+}
+
+/**
+ * Makes REST POST call to clear the problem summary table
+ *
+ * @param {string} tableID Table ID to clear
+ */
+function clearTableProblemsTable(tableID) {
+  clearTableProblems(tableID);
+  refreshProblems();
+  refreshNavBar();
+}
+
+/**
+ * Makes REST POST call to clear the problem details table
+ *
+ * @param {string} table Table ID to clear
+ * @param {string} resource Resource of problem
+ * @param {string} type Type of problem
+ */
+function clearDetailsProblemsTable(table, resource, type) {
+  clearDetailsProblems(table, resource, type);
+  refreshProblems();
+  refreshNavBar();
+}
+
+/**
+ * Generates the problem summary table
+ */
+function refreshProblemSummaryTable() {
+  clearTable('problemSummary');
+  var data = sessionStorage.problemSummary === undefined ?
+  [] : JSON.parse(sessionStorage.problemSummary);
+
+  if (data.length === 0 || Object.keys(data.problemSummary).length === 0) {
+var items = [];
+items.push('Empty');
+$('', {
+  html: items.join('')
+}).appendTo('#problemSummary');
+  } else {
+$.each(data.problemSummary, function(key, val) {
+  var items = [];
+  items.push('' + val.tableName +
+  '');
+
+  items.push('' + bigNumberForQuantity(val.fileRead) 
+
+  '');
+
+  items.push('' + 
bigNumberForQuantity(val.fileWrite) +
+  '');
+
+  items.push('' + 
bigNumberForQuantity(val.tableLoad) +
+  '');
+  items.push('clear ALL ' + val.tableName +
+  ' problems');
+
+  $('', {
+html: items.join('')
+  }).appendTo('#problemSummary');
+});
+  }
+}
+
+/**
+ * Generates the problem details table
+ */
+function refreshProblemDetailsTable() {
+  clearTable('problemDetails');
+  var data = sessionStorage.problemDetails === undefined ?
+  [] : JSON.parse(sessionStorage.problemDetails);
+
+  if (data.length === 0 || Object.keys(data.problemDetails).length === 0) {
+var items = [];
+items.push('Empty');
+$('', {
+  html: items.join('')
+}).appendTo('#problemDetails');
+  } else {
+$.each(data.problemDetails, function(key, val) {
+  var items = [];
+  // Filters the details problems for the selected tableID
+  if (tableID === val.tableID || tableID === '') {
+items.push('' + val.tableName +
+'');
+
+items.push('' +
+val.type + '');
+
+items.push('' +
+val.server + '');
+
+var date = new Date(val.time);
+items.push('' +
+date.toLocaleString() + '');
+
+items.push('' +
+val.resource + '');
+
+items.push('' 
+
+val.exception + '');
+
+items.push('clear this problem');
--- End diff --

Another spot where a simple function that returns a table cell will help a 
lot


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is 

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110473784
  
--- Diff: server/monitor/src/main/resources/resources/overview.js ---
@@ -0,0 +1,293 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements.  See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License.  You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates overview initial tables
+ */
+$(document).ready(function() {
+  createMasterTable();
+  createZKTable();
+  refreshOverview();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshOverview() {
+  $.ajaxSetup({
+async: false
+  });
+  getMaster();
+  getZK();
+  getIngestRate();
+  getScanEntries();
+  getIngestByteRate();
+  getQueryByteRate();
+  getLoadAverage();
+  getLookups();
+  getMinorCompactions();
+  getMajorCompactions();
+  getIndexCacheHitRate();
+  getDataCacheHitRate();
+  $.ajaxSetup({
+async: true
+  });
+  refreshMasterTable();
+  refreshZKTable();
+  makePlots();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+  refreshOverview();
+}
+
+/**
+ * Refreshes the master table
+ */
+function refreshMasterTable() {
+  var data = sessionStorage.master === undefined ?
+  [] : JSON.parse(sessionStorage.master);
+
+  $('#master tr td:first').hide();
+  $('#master tr td').hide();
+
+  // If the master is down, show the first row, otherwise refresh old 
values
+  if (data.length === 0 || data.master === 'No Masters running') {
+$('#master tr td:first').show();
+  } else {
+$('#master tr td:not(:first)').show();
+var table = $('#master td.right');
+
+table.eq(0).html(bigNumberForQuantity(data.tables));
+table.eq(1).html(bigNumberForQuantity(data.totalTabletServers));
+table.eq(2).html(bigNumberForQuantity(data.deadTabletServersCount));
+table.eq(3).html(bigNumberForQuantity(data.tablets));
+table.eq(4).html(bigNumberForQuantity(data.numentries));
+table.eq(5).html(bigNumberForQuantity(data.lookups));
+table.eq(6).html(timeDuration(data.uptime));
+  }
+}
+
+/**
+ * Generates the master table
+ */
+function createMasterTable() {
+  var items = [];
+  items.push('Accumulo' +
+  'Master');
+
+  items.push('' +
+  'Master is Down');
+
+  items.push('Tables' +
+  '');
+
+  items.push('Tablet' +
+  'Servers');
+
+  items.push('Dead' +
+  'TabletServers');
+
+  items.push('Tablets');
+  items.push('Entries');
+  items.push('Lookups');
+  items.push('Uptime');
+
+  $('', {
+html: items.join(''),
+class: 'table table-bordered table-striped table-condensed'
+  }).appendTo('#master');
+}
+
+/**
+ * Refresh the zookeeper table
+ */
+function refreshZKTable() {
+  var data = sessionStorage.zk === undefined ?
+  [] : JSON.parse(sessionStorage.zk);
+
+  $('#zookeeper tr td:first').hide();
+  $('#zookeeper tr:gt(2)').remove();
+
+  if (data.length === 0 || data.zkServers.length === 0) {
+$('#zookeeper tr td:first').show();
+  } else {
+var items = [];
+$.each(data.zkServers, function(key, val) {
+  if (val.clients >= 0) {
+items.push('' + val.server + '');
+items.push('' + val.mode + '');
+items.push('' + val.clients + '');
+  } else {
+items.push('' + val.server + '');
+items.push('Down');
+items.push('');
+  }
+});
+$('', {
+  html: items.join('')
+}).appendTo('#zookeeper table');
+  }
+}
+
+/**
+ * Generates the zookeeper table
+ */
+function createZKTable() {
+  var items = [];
+  items.push('Zookeeper');
+  items.push('ServerModeClients');
+  items.push('No Zookeepers');
+  $('', {
+html: items.join(''),

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110469952
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java
 ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import 
org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+  /**
+   * Generates a trace summary
+   *
+   * @param minutes
+   *  Range of minutes to filter traces
+   * @return Trace summary in specified range
+   */
+  @Path("summary/{minutes}")
+  @GET
+  public RecentTracesList getTraces(@DefaultValue("10") 
@PathParam("minutes") int minutes) throws Exception {
+
+RecentTracesList recentTraces = new RecentTracesList();
+
+Pair pair = getScanner();
+final Scanner scanner = pair.getFirst();
+if (scanner == null) {
+  return recentTraces;
+}
+
+Range range = getRangeForTrace(minutes);
+scanner.setRange(range);
+
+final Map summary = new TreeMap<>();
+if (null != pair.getSecond()) {
+  pair.getSecond().doAs(new PrivilegedAction() {
+@Override
+public Void run() {
+  parseSpans(scanner, summary);
+  return null;
+}
+  });
+} else {
+  parseSpans(scanner, summary);
+}
+
+// Adds the traces to the list
+for (Entry entry : summary.entrySet()) 
{
+  RecentTracesInformation stat = entry.getValue();
+  

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110471868
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java
 ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+  // Variable names become JSON keys
+  @XmlAttribute
--- End diff --

@joshelser I checked out the code, in all cases it is used in both a 
regular REST call and for the XML summary (which the old monitor had them as an 
attribute instead of a value)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110470726
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLResource.java 
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.monitor.rest.tables.TablesResource;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Responsible for generating an XML summary of the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class XMLResource {
+
+  /**
+   * Generates an XML summary of the Monitor
+   *
+   * @return XML summary
+   */
+  @GET
+  public XMLInformation getXMLInformation() {
+
+MasterMonitorInfo mmi = Monitor.getMmi();
+if (null == mmi) {
+  throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+}
+
+// Add Monitor information
+XMLInformation xml = new XMLInformation(mmi.tServerInfo.size(), new 
MasterResource().getTables(), new TablesResource().getTables());
+
+// Add tserver information
+for (TabletServerStatus status : mmi.tServerInfo) {
+  xml.addTablet(new TabletServer(status));
--- End diff --

@joshelser I'm not sure I understand what is the confusion, is it just for 
the XMLResource class? Or is it anywhere where TabletServer exists? If it is 
the second one, keep in mind that none of the objects of the REST call are the 
actual instances, I created new classes specifically for the REST (in order to 
store variables that will serve as JSON keys). If it is something else, could 
you elaborate?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110468167
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLInformation.java
 ---
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.xml.bind.annotation.XmlRootElement;
+
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerList;
+import org.apache.accumulo.monitor.rest.master.MasterInformation;
+import org.apache.accumulo.monitor.rest.tables.TableInformation;
+import org.apache.accumulo.monitor.rest.tables.TableInformationList;
+import org.apache.accumulo.monitor.rest.tables.TableNamespace;
+import org.apache.accumulo.monitor.rest.tables.TablesList;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServers;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerList;
+import org.apache.accumulo.monitor.rest.tservers.ServersShuttingDown;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Generate XML summary of Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@XmlRootElement(name = "stats")
+public class XMLInformation {
+
+  // Variable names become JSON keys
+  public List servers;
+
+  public String masterGoalState, masterState;
+
+  public BadTabletServers badTabletServers;
+  public ServersShuttingDown tabletServersShuttingDown;
+  public Integer unassignedTablets;
+  public DeadServerList deadTabletServers;
+
+  public DeadLoggerList deadLoggers;
+
+  public TableInformationList tables;
+
+  public Totals totals;
+
+  public XMLInformation() {
+servers = new ArrayList<>();
+  }
+
+  /**
+   * Stores Monitor information as XML
+   *
+   * @param size
+   *  Number of tservers
+   * @param info
+   *  Master information
+   * @param tablesList
+   *  Table list
+   */
+  public XMLInformation(int size, MasterInformation info, TablesList 
tablesList) {
+this.servers = new ArrayList<>(size);
+
+this.masterGoalState = info.masterGoalState;
+this.masterState = info.masterState;
+
+this.badTabletServers = info.badTabletServers;
+this.tabletServersShuttingDown = info.tabletServersShuttingDown;
+this.unassignedTablets = info.unassignedTablets;
+this.deadTabletServers = info.deadTabletServers;
+this.deadLoggers = info.deadLoggers;
+
+getTableInformationList(tablesList.tables);
+
+this.totals = new Totals(info.ingestrate, info.queryrate, 
info.numentries);
+  }
+
+  /**
+   * Adds a new tablet to the XML
+   *
+   * @param tablet
+   *  Tablet to add
+   */
+  public void addTablet(TabletServer tablet) {
--- End diff --

@joshelser not sure what you mean by this, you mean changing addTablet to 
addTabletServer?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110466768
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java
 ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+  private static final TabletServerStatus NO_STATUS = new 
TabletServerStatus();
+
+  /**
+   * Generates a table list based on the namespace
+   *
+   * @param namespace
+   *  Namespace used to filter the tables
+   * @return Table list
+   */
+  private TablesList generateTables(String namespace) {
+SortedMap namespaces = 
Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+TablesList tableNamespace = new TablesList();
+
+/*
+ * Add the tables that have the selected namespace Asterisk = All 
namespaces Hyphen = Default namespace
+ */
+for (String key : namespaces.keySet()) {
+  if (namespace.equals("*") || namespace.equals(key) || 
(key.equals("") && namespace.equals("-"))) {
+tableNamespace.addTable(new TableNamespace(key));
+  }
+}
+
+return generateTables(tableNamespace);
+  }
+
+  /**
+   * Generates a table list based on the list of namespaces
+   *
+   * @param tableNamespace
+   *  Namespace list
+   * @return Table list
+   */
+  private TablesList generateTables(TablesList tableNamespace) {
+Instance inst = Monitor.getContext().getInstance();
+Map tidToNameMap = Tables.getIdToNameMap(inst);
+SortedMap tableStats = new TreeMap<>();
+
+if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+  for (Entry te : 
Monitor.getMmi().tableMap.entrySet())
+tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, 
te.getKey()), te.getValue());
+Map compactingByTable = 
TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+TableManager tableManager = TableManager.getInstance();
+List tables = new ArrayList<>();
+
+// Add tables 

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110465485
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java
 ---
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import org.apache.accumulo.core.master.thrift.TableInfo;
+
+/**
+ *
+ * Generates table information as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+public class TableInformation {
+
+  // Variable names become JSON keys
+  public String tablename, tableId, tableState;
+
+  public int tablets, onlineTablets;
+  public long recs, recsInMemory;
+
+  public double ingest, ingestByteRate, query, queryByteRate;
+
+  public CompactionsList majorCompactions, minorCompactions, scans;
+
+  private int queuedMajorCompactions, runningMajorCompactions, 
queuedMinorCompactions, runningMinorCompactions, queuedScans, runningScans;
--- End diff --

Yup, currently working on a fix for these, will be pushing soon (in it 
there's other fixes from the rest of the suggestions from the PR (not all of 
them though))


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110465276
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/status/StatusResource.java
 ---
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.status;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.master.thrift.DeadServer;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.monitor.DedupedLogEvent;
+import org.apache.accumulo.server.monitor.LogService;
+import org.apache.log4j.Level;
+
+/**
+ *
+ * Generates the status for master, gc, and tservers as well as log and 
problem reports
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/status")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class StatusResource {
+
+  /**
+   * Generates the JSON object with the status
+   *
+   * @return Status report
+   */
+  @GET
+  public StatusInformation getTables() {
+
+StatusInformation status;
+String masterStatus;
+String gcStatus;
+String tServerStatus = "ERROR";
--- End diff --

Didn't think of that, thanks for the suggestion!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110464940
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java
 ---
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import org.apache.accumulo.core.master.thrift.TableInfo;
+
+/**
+ *
+ * Generates table information as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+public class TableInformation {
+
+  // Variable names become JSON keys
+  public String tablename, tableId, tableState;
+
+  public int tablets, onlineTablets;
+  public long recs, recsInMemory;
+
+  public double ingest, ingestByteRate, query, queryByteRate;
+
+  public CompactionsList majorCompactions, minorCompactions, scans;
+
+  private int queuedMajorCompactions, runningMajorCompactions, 
queuedMinorCompactions, runningMinorCompactions, queuedScans, runningScans;
--- End diff --

More code style to fix


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110464998
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java
 ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --

I think my LogEvent would only work for the REST API, I would need to check 
it out and see if it would though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110464603
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/status/StatusResource.java
 ---
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.status;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.master.thrift.DeadServer;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.monitor.DedupedLogEvent;
+import org.apache.accumulo.server.monitor.LogService;
+import org.apache.log4j.Level;
+
+/**
+ *
+ * Generates the status for master, gc, and tservers as well as log and 
problem reports
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/status")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class StatusResource {
+
+  /**
+   * Generates the JSON object with the status
+   *
+   * @return Status report
+   */
+  @GET
+  public StatusInformation getTables() {
+
+StatusInformation status;
+String masterStatus;
+String gcStatus;
+String tServerStatus = "ERROR";
--- End diff --

You could create an enum for the Status values.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110456625
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java
 ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --

Think we could replace DedupedLogEvent with your new LogEvent?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110454857
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java
 ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --

The variables from the DedupedLogEvent are different than the ones I have 
on LogEvents (which I need to make the JSON for the REST call).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[milleruntime]

Github user milleruntime commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110453818
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java
 ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --

How come you didn't extend DedupedLogEvent?  It looks like 
LogResource.getRecentLogs() is doing conversion between the objects.  You might 
be able to save some code there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #244: Discuss ACCUMULO-3079: collapsing the iterator s...

[milleruntime]

GitHub user milleruntime opened a pull request:

https://github.com/apache/accumulo/pull/244

Discuss ACCUMULO-3079: collapsing the iterator stack to improve performance

Applied the changes from the patch submitted by Adam Fuchs on 
[ACCUMULO-3079](https://issues.apache.org/jira/browse/ACCUMULO-3079) to master. 
This PR is to bring these changes back into discussion.  Created JMH benchmark 
tests 
[here](https://github.com/milleruntime/jmh-test/blob/master/src/main/java/org/sample/MyBenchmark.java).
  I tried to isolate the tests to only the system iterators acting on data in 
memory.  

I have attached the results from the benchmark tests. 

[jmh-accumulo-benchmark-results.txt](https://github.com/apache/accumulo/files/906333/jmh-accumulo-benchmark-results.txt)

The JMH tests can be built and run from 
[here](https://github.com/milleruntime/jmh-test). After cloning the repo, 
simply run:
`mvn clean install`
`java -jar target/benchmarks.jar -i 100 -f 1`

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/milleruntime/accumulo ACCUMULO-3079-all

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/accumulo/pull/244.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #244


commit 747bae1c737b9df2bbd3c7a834a536696d9e304a
Author: Mike Miller 
Date:   2017-03-31T16:43:45Z

ACCUMULO-3079: collapsed the iterator stack to improve performance




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[lstav]

Github user lstav commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110439061
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/rest/Totals.java ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+/**
+ *
+ * Generates the totals for XML summary
+ *
+ * @since 2.0.0
+ *
+ */
+public class Totals {
+
+  // Variable names become JSON keys
+  public double ingestrate, queryrate, diskrate = 0.0;
--- End diff --

I am looking at the code and I believe that the reason these are not 
Camel-cased is that on the original Monitor's XML response they are not 
Camel-cased, and as such I kept it that way for backwards compatibility. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[mikewalch]

Github user mikewalch commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110189876
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/view/Indexes.java ---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.view;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.glassfish.jersey.server.mvc.Template;
+
+/**
+ *
+ * Index is responsible of specifying Monitor paths and setting the 
templates for the HTML code
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces(MediaType.TEXT_HTML)
+public class Indexes {
--- End diff --

The name of this class could be better.  Could be `WebApp` `WebViews` or 
`WebResources`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[mikewalch]

Github user mikewalch commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110258980
  
--- Diff: 
server/monitor/src/main/java/org/apache/accumulo/monitor/view/Indexes.java ---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.view;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.glassfish.jersey.server.mvc.Template;
+
+/**
+ *
+ * Index is responsible of specifying Monitor paths and setting the 
templates for the HTML code
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces(MediaType.TEXT_HTML)
+public class Indexes {
+
+  private Map getModel() {
+
+Map model = new HashMap<>();
+model.put("version", Constants.VERSION);
+model.put("instance_name", Monitor.cachedInstanceName.get());
+model.put("instance_id", 
Monitor.getContext().getInstance().getInstanceID());
+return model;
+  }
+
+  /**
+   * Returns the overview template
+   *
+   * @return Overview model
+   */
+  @GET
+  @Template(name = "/index.ftl")
--- End diff --

Could rename `index.ftl` to `base.ftl` or `default.ftl` as it is the base 
template that is used to build all views.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI

[mikewalch]

Github user mikewalch commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/242#discussion_r110252701
  
--- Diff: assemble/pom.xml ---
@@ -195,6 +235,86 @@
   true
 
 
+  org.freemarker
+  freemarker
+
+
+  org.glassfish.hk2
+  hk2-api
+
+
+  org.glassfish.hk2
+  hk2-locator
+
+
+  org.glassfish.hk2
+  hk2-utils
+
+
+  org.glassfish.hk2
+  osgi-resource-locator
+
+
+  org.glassfish.hk2.external
+  aopalliance-repackaged
+
+
+  org.glassfish.hk2.external
+  javax.inject
+
+
+  org.glassfish.jersey.bundles.repackaged
+  jersey-guava
+
+
+  org.glassfish.jersey.containers
+  jersey-container-jetty-http
+
+
+  org.glassfish.jersey.containers
+  jersey-container-servlet
+
+
+  org.glassfish.jersey.containers
+  jersey-container-servlet-core
+
+
+  org.glassfish.jersey.core
+  jersey-client
+
+
+  org.glassfish.jersey.core
--- End diff --

Adding to @joshelser comment below, I think some of these dependencies are 
transitive (like `jersey-common`) and don't need to be explicitly included.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #240: ACCUMULO-4618 Updates from additional code revie...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/accumulo/pull/240


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] accumulo pull request #240: ACCUMULO-4618 Updates from additional code revie...

[mikewalch]

Github user mikewalch commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/240#discussion_r110412136
  
--- Diff: maven-plugin/pom.xml ---
@@ -68,6 +68,7 @@
 
   xml-apis
   xml-apis
+  provided
--- End diff --

Nice. I will fix this and merge. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---