Hi Nishani, Thanks for your interesting in phoenix!
For tracing, the first thing I would recommend doing is reading the tracing documentation: http://phoenix.apache.org/tracing.html A careful reading there would then lead you to look at org.apache.phoenix.trace.PhoenixTableMetricsWriterIT for running a unit test that actually looks at the traces being written. >From there you can look into the org.apache.phoenix.trace.TraceReader which is actually reading any traces out from the table. Admittedly, some of the schema is a little bit opaque, but you can take a crack at it if you read through org.apache.phoenix.trace.PhoenixMetricsSink, specifically #createTable() and #putMetrics(). The key to understand is that there are generic annotations and tags that can be appended to a trace - specifically, onto a span of the trace - at any point in its life. To manage that the only known column for the tags and annotations, respectively, is a count column. Then with the count column we can build the names of the dynamic columns <http://phoenix.apache.org/dynamic_columns.html> for each annotation and tag. For instance, say there is a tag count column tags.count with value 1 and annotations.count with value 2. Then we know there are also the dynamic columns: tags.t0, annotations.a0, annotations.a1. Also see the HTace documentation <https://github.com/cloudera/htrace> for info on the version of HTrace we are using - we could upgrade, but no one has had a chance to validate that. BTW, looking at the documentation, its a little out of date for tracing - since we removed support for Hadoop2 we removed some of the indirection so classes like PhoenixTableMetricsWriter got renamed. If you are interested, we would also love if you wanted to help keep our docs up to date with what you find! Hope this helps, Jesse (the guy who wrote tracing for phoenix) On Fri, Mar 20, 2015 at 7:13 PM, Ayola Jayamaha <raphaelan...@gmail.com> wrote: > Hi All, > > As mentioned earlier I completed my research on how existing database > systems visualize tracing information. (Mysql, Postgres etc.) > > Please give me some feedback on it? Or if you have any questions regard to > that or any unclear part I'd be glad to help you out. > > I also successfully built and ran phoenix and went through > SYSTEM.TRACING_STATS > table and tried to understand it's schema. I would like to write a blog > post and share my knowledge with the community. > > Can you also mention some of the attributes and resources where I can > further get a good idea on the tracing information before I try to come up > with ways to visualize them? > > I'm writing my proposal for GSoC 2015 and shall share it as soon as I > finish. Hoping you would be generous in giving out feedback. > > Thanks. > > BR, > Nishani > > On Fri, Mar 20, 2015 at 2:24 PM, Nishani (JIRA) <j...@apache.org> wrote: > > > > > [ > > > https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370997#comment-14370997 > > ] > > > > Nishani edited comment on PHOENIX-1118 at 3/20/15 8:54 AM: > > ------------------------------------------------------------ > > > > Hi, > > > > I have gone through some resources regarding visualization of trace info > > on database servers (Postgres/MySQL, MSSQL). > > > > The DTrace function in MySQL is a profiler that creates flame graphs. [1] > > A tool for visualizations of a database system [2] > > Generates images with help of GraphViz for > > Some data modeling tools [4] > > DBVisualizer Tool [5] > > Data Visualization Application where data is input from a database or a > > file and exported as image or a customized template [6] > > Visualizing Postgres [7] > > A Research Paper on Modeling Trace Information[8] > > A Tool for visualizing Trace Info [9] > > > > Given above is a brief summary of the details I found. Hope it would help > > me to better understand the formats and values of the input traces and > > different available methods of graphically representing them. > > > > Based on them I hope to come up with the most suitable method of Visually > > representing the said data. > > > > Regarding the installation I presume that the HBase version that is > needed > > to run Phoenix for visualizing trace info can be as Standalone and that > > cluster view is not needed(Distributed Version). > > > > Thank you. > > BR, > > Nishani > > > > [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html > > [2] http://docs.cartodb.com/cartodb-editor.html#visualizations > > [3] > > https://bitbucket.org/zzzeek/sqlalchemy/wiki/UsageRecipes/SchemaDisplay > > [4] http://www.databaseanswers.org/modelling_tools.htm > > [5] http://www.dbvis.com/ > > [6] https://sites.google.com/site/datavisualizationapplication/ > > [7] http://www.pgcon.org/2013/schedule/track/Performance/588.en.html > > [8] http://link.springer.com/chapter/10.1007%2F978-3-642-40787-1_13 > > [9] > > > http://www.upscene.com/documentation/fbtm3/index.html?qsg_trace_data_visualization.htm > > > > > > > > was (Author: nishani): > > Hi, > > > > I have gone through some resources regarding visualization of trace info > > on database servers (Postgres/MySQL, MSSQL). > > > > The DTrace function in MySQL is a profiler that creates flame graphs. [1] > > A tool for visualizations of a database system [2] > > Generates images with help of GraphViz for > > Some data modeling tools [4] > > DBVisualizer Tool [5] > > Data Visualization Application where data is input from a database or a > > file and exported as image or a customized template [6] > > Visualizing Postgres [7] > > A Research Paper on Modeling Trace Information[8] > > A Tool for visualizing Trace Info > > > > Given above is a brief summary of the details I found. Hope it would help > > me to better understand the formats and values of the input traces and > > different available methods of graphically representing them. > > > > Based on them I hope to come up with the most suitable method of Visually > > representing the said data. > > > > Regarding the installation I presume that the HBase version that is > needed > > to run Phoenix for visualizing trace info can be as Standalone and that > > cluster view is not needed(Distributed Version). > > > > Thank you. > > BR, > > Nishani > > > > [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html > > [2] http://docs.cartodb.com/cartodb-editor.html#visualizations > > [3] > > https://bitbucket.org/zzzeek/sqlalchemy/wiki/UsageRecipes/SchemaDisplay > > [4] http://www.databaseanswers.org/modelling_tools.htm > > [5] http://www.dbvis.com/ > > [6] https://sites.google.com/site/datavisualizationapplication/ > > [7] http://www.pgcon.org/2013/schedule/track/Performance/588.en.html > > [8] http://link.springer.com/chapter/10.1007%2F978-3-642-40787-1_13 > > [9] > > > http://www.upscene.com/documentation/fbtm3/index.html?qsg_trace_data_visualization.htm > > > > > > > Provide a tool for visualizing Phoenix tracing information > > > ---------------------------------------------------------- > > > > > > Key: PHOENIX-1118 > > > URL: > https://issues.apache.org/jira/browse/PHOENIX-1118 > > > Project: Phoenix > > > Issue Type: Sub-task > > > Reporter: James Taylor > > > Assignee: Nishani > > > Labels: Java, SQL, Visualization, gsoc2015, mentor > > > > > > Currently there's no means of visualizing the trace information > provided > > by Phoenix. We should provide some simple charting over our metrics > tables. > > Take a look at the following JIRA for sample queries: > > > https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151 > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.3.4#6332) > > > > > > -- > Best Regards, > Ayola Jayamaha >
