That would be pretty sweet, actually. Potentially parallel to what you
want to do, Supun, but cool nonetheless.
I could see a lot of benefit by having some process that could emulate
the output from a non-trivially-sized Accumulo cluster on a single box.
On 4/22/13 10:43 AM, Eric Newton wrote:
You could mock the stats collection.
-Eric
On Mon, Apr 22, 2013 at 10:41 AM, David Medinets
<[email protected]>wrote:
The average developer probably can't access a large cluster with hundred of
nodes. Is there a way to simulate this?
On Mon, Apr 22, 2013 at 9:05 AM, Eric Newton <[email protected]>
wrote:
Another thing to consider is scale. On large clusters (many hundreds of
nodes), more data is not helpful for visualization. Instead, summaries,
averages and outliers are important.
For example, if one node is consistently slow, it is better to know that
than to see one graph with low numbers in a sea of graphs.
If the monitor collects information using JMX, collection time for each
node would be a good thing to know, too.
-Eric
On Sun, Apr 21, 2013 at 10:00 PM, Josh Elser <[email protected]>
wrote:
Supun,
Yup, very much so. Having a way to consume any and all metrics via JMX
would simplify things for any consumers (internal or external).
On 04/21/2013 02:15 PM, Supun Kamburugamuva wrote:
Hi Josh,
Thanks for the suggestions. I'll incorporate these to the proposal.
Another area I would like to work is on JMX. There is a Jira that says
to
replace the Monitor calls from Thrift to JMX (Accumulo 694). Do you
think
this is a good addition to the Monitor?
Thanks,
Supun..
On Sun, Apr 21, 2013 at 1:45 PM, Josh Elser <[email protected]>
wrote:
Supun,
Looks good! Can I make some suggestions/comments?
For: "Per table plots: ACCUMULO-594", I'd also like to see minor
compactions, major compactions, index cache hit rate, and data cache
hit
rate per table (same graphs that are displayed system-wide when you
visit
http://${MONITOR_HOST}:50095/.
For "Per tablet [server] plots", it would be neat if you could also
extract some general statistics like top N least performing, top N
highest
performing, etc. tablet servers. Ideally, this could correlate with
servers
that may be having problems :).
Do you see these proposed changes as being sufficient for 3-4 months
of
40hrs/week work? If you plan to really dig into these changes
(perhaps
reworking components of the monitor itself), I could perhaps see
this.
Do
you have any ideas for more lofty goals that you could pursue as
well?
I
don't want you/us to get one month into things and see you complete
everything we initially planned to accomplish :)
- Josh
On 04/21/2013 10:37 AM, Supun Kamburugamuva wrote:
Hi all,
I would like to start writing the proposal for the GSoc. I've put
together
some initial high level goals of the project. Please let me know
what
I
can
improve.
Per table plots: Accumulo 594
---------------------
The goal of this is to display plots that explains the various
activtities
that happens per table. When we go to the tables page of the monitor
and
go
to a specific table it displays some information in a table format.
We
can
argument this information by showing graphs for
1. Ingest entries
2. Ingest data size
3. Scan entries
4. Scan data size
Per tablet plots
----------------------
Same as in the table plots we can display information regarding
tablet
servers in the tablet server page. The plots will display the same
information as table plots considering data per tablet server.
Trace Visualization: Accumulo 1198
----------------------------
Since we are displaying graphs about each tablet and each table we
can
add
major and minor compaction graph to each table and each tablet.
Or other option is to display this in a single graph in overview
page
with
different graph lines for different tables and tablets.
Server type information : Accumulo 807
------------------------------****---
For displaying this informations we can add a new page and display
the
information as a table. The table should specify the network address
of
the
server, server type, weather it is active or in-active etc.
Thanks,
Supun...
