[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-1118:
--
Fix Version/s: (was: 4.7.0)

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, Screenshot-loading-trace-list.png, m1-mockUI-tracedistribution.png, 
> m1-mockUI-tracetimeline.png, screenshot of tracing timeline.png, screenshot 
> of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
> The tool need to provide visualization over SYSTEM.TRACING_STATS table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated PHOENIX-1118:
---
Fix Version/s: (was: 4.6.0)
   4.7.0

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Fix For: 4.7.0
>
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, Screenshot-loading-trace-list.png, m1-mockUI-tracedistribution.png, 
> m1-mockUI-tracetimeline.png, screenshot of tracing timeline.png, screenshot 
> of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
> The tool need to provide visualization over SYSTEM.TRACING_STATS table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nick Dimiduk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated PHOENIX-1118:
--
Fix Version/s: 4.6.0

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Fix For: 4.6.0
>
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, Screenshot-loading-trace-list.png, m1-mockUI-tracedistribution.png, 
> m1-mockUI-tracetimeline.png, screenshot of tracing timeline.png, screenshot 
> of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
> The tool need to provide visualization over SYSTEM.TRACING_STATS table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Description: 
Currently there's no means of visualizing the trace information provided by 
Phoenix. We should provide some simple charting over our metrics tables. Take a 
look at the following JIRA for sample queries: 
https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
The tool need to provide visualization over SYSTEM.TRACING_STATS table.

  was:Currently there's no means of visualizing the trace information provided 
by Phoenix. We should provide some simple charting over our metrics tables. 
Take a look at the following JIRA for sample queries: 
https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151


> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, Screenshot-loading-trace-list.png, m1-mockUI-tracedistribution.png, 
> m1-mockUI-tracetimeline.png, screenshot of tracing timeline.png, screenshot 
> of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
> The tool need to provide visualization over SYSTEM.TRACING_STATS table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-1118:
--
Issue Type: New Feature  (was: Sub-task)
Parent: (was: PHOENIX-1121)

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, Screenshot-loading-trace-list.png, m1-mockUI-tracedistribution.png, 
> m1-mockUI-tracetimeline.png, screenshot of tracing timeline.png, screenshot 
> of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: Screenshot-loading-trace-list.png

Attached here is the tace list retrieved via the JDBC conector from the 
SYSTEM.TRACING_STATS table. The PR is updated. 

You can start the web app from the command given below.
java -jar target/phoenix-tracing-webapp-4.4.1-HBase-0.98-SNAPSHOT-runnable.jar 
-Dphoenix.traceserver.http.port=8898

A view similar to screenshot will be seen from the Phoenix tracelist.
http://localhost:8898/webapp/#/list

Make sure you enable tracing before you attempt above steps.

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, Screenshot-loading-trace-list.png, m1-mockUI-tracedistribution.png, 
> m1-mockUI-tracetimeline.png, screenshot of tracing timeline.png, screenshot 
> of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

I have created the PR[1] with no conflicts.
>From the SYSTEM.TRACING_STATS table description column is there a mapping
to get the corresponding sql query?
Thanks.

[1] https://github.com/apache/phoenix/pull/96

On Thu, Jul 2, 2015 at 11:50 PM, Ayola Jayamaha 
wrote:

> Hi, All
>
> I have send PR[1], where I have added my work. (PR is In-progress and
> status of my task also update in PR too)
>
> [1] https://github.com/apache/phoenix/pull/95
>
> On Tue, Jun 30, 2015 at 12:41 PM, Ayola Jayamaha 
> wrote:
>
>> Hi Nick,
>>
>> Regard to the implementation the user will search for a particular query.
>> The query will be converted into a statement similar to the description
>> column by GenerateStatementService. This will be rendered as a service. And
>> this service is using a StatementFactory which is responsible for the
>> conversion. The Factory is written as extendable as the Phoenix queries may
>> change.
>> I have started working on milestone 2. It includes the above description
>> found in here
>> .
>> The last two commits are for the service and factory. Implementation will
>> happen as described above and its still on progress.
>>
>> Thanks,
>> Nishani.
>>
>>
>>
>> On Tue, Jun 30, 2015 at 3:33 AM, Nick Dimiduk 
>> wrote:
>>
>>> Hi Nishani,
>>>
>>> Any progress on your module for including the SQL query in the UI?
>>>
>>> Thanks,
>>> Nick
>>>
>>> On Wed, Jun 24, 2015 at 11:46 AM, Ayola Jayamaha >> > wrote:
>>>
 Hi All,
 I'll create a javascript module in angular to solve this issue and
 share.

 Thanks,
 Nishani.

 On Thu, Jun 25, 2015 at 12:09 AM, James Taylor 
 wrote:

> Yes, exactly right.
>
> On Wed, Jun 24, 2015 at 11:35 AM, Ayola Jayamaha <
> raphaelan...@gmail.com> wrote:
> > Hi All,
> >
> > Now it is clear. We can create a statement from the user's input
> query to
> > the format in the description column and filter out the possible
> root spans
> > of the traces of the query. Then by selecting the traces which have
> their
> > parent ids equal to span id of the root span we can get all the
> traces
> > relevant to the query.
> > We can find the total duration for a particular statement.
> Interesting
> > statements/traces can viewed as a timeline.
> > Is this method alright?
> >
> > Thanks,
> > Nishani
> >
> >
> > On Wed, Jun 24, 2015 at 11:21 PM, James Taylor <
> jamestay...@apache.org>
> > wrote:
> >
> >> Yep, Jesse's right - the query is in the description column of the
> >> root span of the trace. We'll need to include this in the trace UI,
> >> otherwise the user won't have the context they need to know what
> >> they're looking at. If there's something missing from the way we're
> >> capturing, we can fix it.
> >> Thanks,
> >> James
> >>
> >> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates <
> jesse.k.ya...@gmail.com>
> >> wrote:
> >> > There was some discussion (maybe internal to salesforce?) around
> how to
> >> > include the query in the trace. I think the simplest we came up
> with was
> >> > just adding it to the trace metadata (as an annotation?) and then
> you can
> >> > pull it out later since you know the key it was stored as
> >> >
> >> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha <
> raphaelan...@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi James,
> >> >>
> >> >> I find it difficult to come up with a method to include the SQL
> >> statements
> >> >> with the traces. But it is possible to filter out the traces for
> a
> >> >> particular table for a given time period.
> >> >>
> >> >> Aggregating over the time spent for each SQL statement is
> possible. With
> >> >> the relationship between parent and span ids it is possible to
> >> >> differentiate between traces belonging to each query.
> >> >>
> >> >> Thanks,
> >> >> Nishani
> >> >>
> >> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor <
> jamestay...@apache.org>
> >> >> wrote:
> >> >>
> >> >> > Hi Nishani,
> >> >> > I think this is a good start. One important part is tying this
> back to
> >> >> > something to which the user can relate - namely the SQL
> statement that
> >> >> > was
> >> >> > executed. Would it be possible to include the string of the
> statement?
> >> >> >
> >> >> > Another interesting angle would be to group by the statement
> and
> >> >> > aggregate
> >> >> > the overall time spent to get an idea of the "top N queries"
> over a
> >> >> > given
> >> >> > time range. Then drilling into those to see the traces.
> >> >> >
> >> >> > Thanks,
> >> >> > James
> >> >> >
> >> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
> >> >>

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi, All

I have send PR[1], where I have added my work. (PR is In-progress and
status of my task also update in PR too)

[1] https://github.com/apache/phoenix/pull/95

On Tue, Jun 30, 2015 at 12:41 PM, Ayola Jayamaha 
wrote:

> Hi Nick,
>
> Regard to the implementation the user will search for a particular query.
> The query will be converted into a statement similar to the description
> column by GenerateStatementService. This will be rendered as a service. And
> this service is using a StatementFactory which is responsible for the
> conversion. The Factory is written as extendable as the Phoenix queries may
> change.
> I have started working on milestone 2. It includes the above description
> found in here
> .
> The last two commits are for the service and factory. Implementation will
> happen as described above and its still on progress.
>
> Thanks,
> Nishani.
>
>
>
> On Tue, Jun 30, 2015 at 3:33 AM, Nick Dimiduk  wrote:
>
>> Hi Nishani,
>>
>> Any progress on your module for including the SQL query in the UI?
>>
>> Thanks,
>> Nick
>>
>> On Wed, Jun 24, 2015 at 11:46 AM, Ayola Jayamaha 
>> wrote:
>>
>>> Hi All,
>>> I'll create a javascript module in angular to solve this issue and share.
>>>
>>> Thanks,
>>> Nishani.
>>>
>>> On Thu, Jun 25, 2015 at 12:09 AM, James Taylor 
>>> wrote:
>>>
 Yes, exactly right.

 On Wed, Jun 24, 2015 at 11:35 AM, Ayola Jayamaha <
 raphaelan...@gmail.com> wrote:
 > Hi All,
 >
 > Now it is clear. We can create a statement from the user's input
 query to
 > the format in the description column and filter out the possible root
 spans
 > of the traces of the query. Then by selecting the traces which have
 their
 > parent ids equal to span id of the root span we can get all the traces
 > relevant to the query.
 > We can find the total duration for a particular statement. Interesting
 > statements/traces can viewed as a timeline.
 > Is this method alright?
 >
 > Thanks,
 > Nishani
 >
 >
 > On Wed, Jun 24, 2015 at 11:21 PM, James Taylor <
 jamestay...@apache.org>
 > wrote:
 >
 >> Yep, Jesse's right - the query is in the description column of the
 >> root span of the trace. We'll need to include this in the trace UI,
 >> otherwise the user won't have the context they need to know what
 >> they're looking at. If there's something missing from the way we're
 >> capturing, we can fix it.
 >> Thanks,
 >> James
 >>
 >> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates <
 jesse.k.ya...@gmail.com>
 >> wrote:
 >> > There was some discussion (maybe internal to salesforce?) around
 how to
 >> > include the query in the trace. I think the simplest we came up
 with was
 >> > just adding it to the trace metadata (as an annotation?) and then
 you can
 >> > pull it out later since you know the key it was stored as
 >> >
 >> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha <
 raphaelan...@gmail.com>
 >> > wrote:
 >> >>
 >> >> Hi James,
 >> >>
 >> >> I find it difficult to come up with a method to include the SQL
 >> statements
 >> >> with the traces. But it is possible to filter out the traces for a
 >> >> particular table for a given time period.
 >> >>
 >> >> Aggregating over the time spent for each SQL statement is
 possible. With
 >> >> the relationship between parent and span ids it is possible to
 >> >> differentiate between traces belonging to each query.
 >> >>
 >> >> Thanks,
 >> >> Nishani
 >> >>
 >> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor <
 jamestay...@apache.org>
 >> >> wrote:
 >> >>
 >> >> > Hi Nishani,
 >> >> > I think this is a good start. One important part is tying this
 back to
 >> >> > something to which the user can relate - namely the SQL
 statement that
 >> >> > was
 >> >> > executed. Would it be possible to include the string of the
 statement?
 >> >> >
 >> >> > Another interesting angle would be to group by the statement and
 >> >> > aggregate
 >> >> > the overall time spent to get an idea of the "top N queries"
 over a
 >> >> > given
 >> >> > time range. Then drilling into those to see the traces.
 >> >> >
 >> >> > Thanks,
 >> >> > James
 >> >> >
 >> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
 >> >> > 
 >> >> > wrote:
 >> >> >
 >> >> >> Hi All,
 >> >> >>
 >> >> >> Milestone-1 can be found in my git repo[1].
 >> >> >> Features :
 >> >> >>
 >> >> >>- Adding tracing to a timeline using sample json
 >> >> >>- Comparing two or more traces on the timeline
 >> >> >>- Visualizing the trace distribution across the time axis
 >> >> >>- Removing a trace from the list of traces represented on

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi Nick,

Regard to the implementation the user will search for a particular query.
The query will be converted into a statement similar to the description
column by GenerateStatementService. This will be rendered as a service. And
this service is using a StatementFactory which is responsible for the
conversion. The Factory is written as extendable as the Phoenix queries may
change.
I have started working on milestone 2. It includes the above description
found in here
.  The
last two commits are for the service and factory. Implementation will
happen as described above and its still on progress.

Thanks,
Nishani.



On Tue, Jun 30, 2015 at 3:33 AM, Nick Dimiduk  wrote:

> Hi Nishani,
>
> Any progress on your module for including the SQL query in the UI?
>
> Thanks,
> Nick
>
> On Wed, Jun 24, 2015 at 11:46 AM, Ayola Jayamaha 
> wrote:
>
>> Hi All,
>> I'll create a javascript module in angular to solve this issue and share.
>>
>> Thanks,
>> Nishani.
>>
>> On Thu, Jun 25, 2015 at 12:09 AM, James Taylor 
>> wrote:
>>
>>> Yes, exactly right.
>>>
>>> On Wed, Jun 24, 2015 at 11:35 AM, Ayola Jayamaha 
>>> wrote:
>>> > Hi All,
>>> >
>>> > Now it is clear. We can create a statement from the user's input query
>>> to
>>> > the format in the description column and filter out the possible root
>>> spans
>>> > of the traces of the query. Then by selecting the traces which have
>>> their
>>> > parent ids equal to span id of the root span we can get all the traces
>>> > relevant to the query.
>>> > We can find the total duration for a particular statement. Interesting
>>> > statements/traces can viewed as a timeline.
>>> > Is this method alright?
>>> >
>>> > Thanks,
>>> > Nishani
>>> >
>>> >
>>> > On Wed, Jun 24, 2015 at 11:21 PM, James Taylor >> >
>>> > wrote:
>>> >
>>> >> Yep, Jesse's right - the query is in the description column of the
>>> >> root span of the trace. We'll need to include this in the trace UI,
>>> >> otherwise the user won't have the context they need to know what
>>> >> they're looking at. If there's something missing from the way we're
>>> >> capturing, we can fix it.
>>> >> Thanks,
>>> >> James
>>> >>
>>> >> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates >> >
>>> >> wrote:
>>> >> > There was some discussion (maybe internal to salesforce?) around
>>> how to
>>> >> > include the query in the trace. I think the simplest we came up
>>> with was
>>> >> > just adding it to the trace metadata (as an annotation?) and then
>>> you can
>>> >> > pull it out later since you know the key it was stored as
>>> >> >
>>> >> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha <
>>> raphaelan...@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Hi James,
>>> >> >>
>>> >> >> I find it difficult to come up with a method to include the SQL
>>> >> statements
>>> >> >> with the traces. But it is possible to filter out the traces for a
>>> >> >> particular table for a given time period.
>>> >> >>
>>> >> >> Aggregating over the time spent for each SQL statement is
>>> possible. With
>>> >> >> the relationship between parent and span ids it is possible to
>>> >> >> differentiate between traces belonging to each query.
>>> >> >>
>>> >> >> Thanks,
>>> >> >> Nishani
>>> >> >>
>>> >> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor <
>>> jamestay...@apache.org>
>>> >> >> wrote:
>>> >> >>
>>> >> >> > Hi Nishani,
>>> >> >> > I think this is a good start. One important part is tying this
>>> back to
>>> >> >> > something to which the user can relate - namely the SQL
>>> statement that
>>> >> >> > was
>>> >> >> > executed. Would it be possible to include the string of the
>>> statement?
>>> >> >> >
>>> >> >> > Another interesting angle would be to group by the statement and
>>> >> >> > aggregate
>>> >> >> > the overall time spent to get an idea of the "top N queries"
>>> over a
>>> >> >> > given
>>> >> >> > time range. Then drilling into those to see the traces.
>>> >> >> >
>>> >> >> > Thanks,
>>> >> >> > James
>>> >> >> >
>>> >> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
>>> >> >> > 
>>> >> >> > wrote:
>>> >> >> >
>>> >> >> >> Hi All,
>>> >> >> >>
>>> >> >> >> Milestone-1 can be found in my git repo[1].
>>> >> >> >> Features :
>>> >> >> >>
>>> >> >> >>- Adding tracing to a timeline using sample json
>>> >> >> >>- Comparing two or more traces on the timeline
>>> >> >> >>- Visualizing the trace distribution across the time axis
>>> >> >> >>- Removing a trace from the list of traces represented on the
>>> >> chart
>>> >> >> >>- Listing the tracing information on a table
>>> >> >>
>>> >> >> >>
>>> >> >> >> Any feedback will be appreciated.
>>> >> >> >> Thanks.
>>> >> >> >>
>>> >> >> >>  [1]
>>> https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>>> >> >> >>
>>> >> >> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha
>>> >> >> >> 
>>> >> >> >> wrote:
>>> >> >> >>
>>> >> >> >>>
>>> >> >> >>> Hi All,
>>> >> >> >>>
>>> >> >> >>> You ca

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nick Dimiduk]

Hi Nishani,

Any progress on your module for including the SQL query in the UI?

Thanks,
Nick

On Wed, Jun 24, 2015 at 11:46 AM, Ayola Jayamaha 
wrote:

> Hi All,
> I'll create a javascript module in angular to solve this issue and share.
>
> Thanks,
> Nishani.
>
> On Thu, Jun 25, 2015 at 12:09 AM, James Taylor 
> wrote:
>
>> Yes, exactly right.
>>
>> On Wed, Jun 24, 2015 at 11:35 AM, Ayola Jayamaha 
>> wrote:
>> > Hi All,
>> >
>> > Now it is clear. We can create a statement from the user's input query
>> to
>> > the format in the description column and filter out the possible root
>> spans
>> > of the traces of the query. Then by selecting the traces which have
>> their
>> > parent ids equal to span id of the root span we can get all the traces
>> > relevant to the query.
>> > We can find the total duration for a particular statement. Interesting
>> > statements/traces can viewed as a timeline.
>> > Is this method alright?
>> >
>> > Thanks,
>> > Nishani
>> >
>> >
>> > On Wed, Jun 24, 2015 at 11:21 PM, James Taylor 
>> > wrote:
>> >
>> >> Yep, Jesse's right - the query is in the description column of the
>> >> root span of the trace. We'll need to include this in the trace UI,
>> >> otherwise the user won't have the context they need to know what
>> >> they're looking at. If there's something missing from the way we're
>> >> capturing, we can fix it.
>> >> Thanks,
>> >> James
>> >>
>> >> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates 
>> >> wrote:
>> >> > There was some discussion (maybe internal to salesforce?) around how
>> to
>> >> > include the query in the trace. I think the simplest we came up with
>> was
>> >> > just adding it to the trace metadata (as an annotation?) and then
>> you can
>> >> > pull it out later since you know the key it was stored as
>> >> >
>> >> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha <
>> raphaelan...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi James,
>> >> >>
>> >> >> I find it difficult to come up with a method to include the SQL
>> >> statements
>> >> >> with the traces. But it is possible to filter out the traces for a
>> >> >> particular table for a given time period.
>> >> >>
>> >> >> Aggregating over the time spent for each SQL statement is possible.
>> With
>> >> >> the relationship between parent and span ids it is possible to
>> >> >> differentiate between traces belonging to each query.
>> >> >>
>> >> >> Thanks,
>> >> >> Nishani
>> >> >>
>> >> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor <
>> jamestay...@apache.org>
>> >> >> wrote:
>> >> >>
>> >> >> > Hi Nishani,
>> >> >> > I think this is a good start. One important part is tying this
>> back to
>> >> >> > something to which the user can relate - namely the SQL statement
>> that
>> >> >> > was
>> >> >> > executed. Would it be possible to include the string of the
>> statement?
>> >> >> >
>> >> >> > Another interesting angle would be to group by the statement and
>> >> >> > aggregate
>> >> >> > the overall time spent to get an idea of the "top N queries" over
>> a
>> >> >> > given
>> >> >> > time range. Then drilling into those to see the traces.
>> >> >> >
>> >> >> > Thanks,
>> >> >> > James
>> >> >> >
>> >> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
>> >> >> > 
>> >> >> > wrote:
>> >> >> >
>> >> >> >> Hi All,
>> >> >> >>
>> >> >> >> Milestone-1 can be found in my git repo[1].
>> >> >> >> Features :
>> >> >> >>
>> >> >> >>- Adding tracing to a timeline using sample json
>> >> >> >>- Comparing two or more traces on the timeline
>> >> >> >>- Visualizing the trace distribution across the time axis
>> >> >> >>- Removing a trace from the list of traces represented on the
>> >> chart
>> >> >> >>- Listing the tracing information on a table
>> >> >>
>> >> >> >>
>> >> >> >> Any feedback will be appreciated.
>> >> >> >> Thanks.
>> >> >> >>
>> >> >> >>  [1]
>> https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> >> >> >>
>> >> >> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha
>> >> >> >> 
>> >> >> >> wrote:
>> >> >> >>
>> >> >> >>>
>> >> >> >>> Hi All,
>> >> >> >>>
>> >> >> >>> You can find milestone-1 in my git repo. This is the working
>> >> >> >>> branch[1].
>> >> >> >>> It has not been binded to backend yet. But the visualization of
>> >> traces
>> >> >> >>> can
>> >> >> >>> be seen from the code.
>> >> >> >>> Traces can be selected from table/time period and shown on the
>> >> >> >>> timeline
>> >> >> >>> as [2]. The parameters could be entered as TableName, StartTime,
>> >> >> >>> EndTime
>> >> >> >>> and the traces would be listed down. The user can select the
>> traces
>> >> as
>> >> >> >>> his
>> >> >> >>> preference and view their timelines. Is the procedure ok?
>> >> >> >>> The start time of different traces could be visualized by
>> bringing
>> >> >> >>> them
>> >> >> >>> up to a same time reference for comparison.
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> [1]
>> https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> >> >> >>> [2]
>> >> >> >>

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,
I'll create a javascript module in angular to solve this issue and share.

Thanks,
Nishani.

On Thu, Jun 25, 2015 at 12:09 AM, James Taylor 
wrote:

> Yes, exactly right.
>
> On Wed, Jun 24, 2015 at 11:35 AM, Ayola Jayamaha 
> wrote:
> > Hi All,
> >
> > Now it is clear. We can create a statement from the user's input query to
> > the format in the description column and filter out the possible root
> spans
> > of the traces of the query. Then by selecting the traces which have their
> > parent ids equal to span id of the root span we can get all the traces
> > relevant to the query.
> > We can find the total duration for a particular statement. Interesting
> > statements/traces can viewed as a timeline.
> > Is this method alright?
> >
> > Thanks,
> > Nishani
> >
> >
> > On Wed, Jun 24, 2015 at 11:21 PM, James Taylor 
> > wrote:
> >
> >> Yep, Jesse's right - the query is in the description column of the
> >> root span of the trace. We'll need to include this in the trace UI,
> >> otherwise the user won't have the context they need to know what
> >> they're looking at. If there's something missing from the way we're
> >> capturing, we can fix it.
> >> Thanks,
> >> James
> >>
> >> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates 
> >> wrote:
> >> > There was some discussion (maybe internal to salesforce?) around how
> to
> >> > include the query in the trace. I think the simplest we came up with
> was
> >> > just adding it to the trace metadata (as an annotation?) and then you
> can
> >> > pull it out later since you know the key it was stored as
> >> >
> >> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha <
> raphaelan...@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi James,
> >> >>
> >> >> I find it difficult to come up with a method to include the SQL
> >> statements
> >> >> with the traces. But it is possible to filter out the traces for a
> >> >> particular table for a given time period.
> >> >>
> >> >> Aggregating over the time spent for each SQL statement is possible.
> With
> >> >> the relationship between parent and span ids it is possible to
> >> >> differentiate between traces belonging to each query.
> >> >>
> >> >> Thanks,
> >> >> Nishani
> >> >>
> >> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor <
> jamestay...@apache.org>
> >> >> wrote:
> >> >>
> >> >> > Hi Nishani,
> >> >> > I think this is a good start. One important part is tying this
> back to
> >> >> > something to which the user can relate - namely the SQL statement
> that
> >> >> > was
> >> >> > executed. Would it be possible to include the string of the
> statement?
> >> >> >
> >> >> > Another interesting angle would be to group by the statement and
> >> >> > aggregate
> >> >> > the overall time spent to get an idea of the "top N queries" over a
> >> >> > given
> >> >> > time range. Then drilling into those to see the traces.
> >> >> >
> >> >> > Thanks,
> >> >> > James
> >> >> >
> >> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
> >> >> > 
> >> >> > wrote:
> >> >> >
> >> >> >> Hi All,
> >> >> >>
> >> >> >> Milestone-1 can be found in my git repo[1].
> >> >> >> Features :
> >> >> >>
> >> >> >>- Adding tracing to a timeline using sample json
> >> >> >>- Comparing two or more traces on the timeline
> >> >> >>- Visualizing the trace distribution across the time axis
> >> >> >>- Removing a trace from the list of traces represented on the
> >> chart
> >> >> >>- Listing the tracing information on a table
> >> >>
> >> >> >>
> >> >> >> Any feedback will be appreciated.
> >> >> >> Thanks.
> >> >> >>
> >> >> >>  [1]
> https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> >> >> >>
> >> >> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha
> >> >> >> 
> >> >> >> wrote:
> >> >> >>
> >> >> >>>
> >> >> >>> Hi All,
> >> >> >>>
> >> >> >>> You can find milestone-1 in my git repo. This is the working
> >> >> >>> branch[1].
> >> >> >>> It has not been binded to backend yet. But the visualization of
> >> traces
> >> >> >>> can
> >> >> >>> be seen from the code.
> >> >> >>> Traces can be selected from table/time period and shown on the
> >> >> >>> timeline
> >> >> >>> as [2]. The parameters could be entered as TableName, StartTime,
> >> >> >>> EndTime
> >> >> >>> and the traces would be listed down. The user can select the
> traces
> >> as
> >> >> >>> his
> >> >> >>> preference and view their timelines. Is the procedure ok?
> >> >> >>> The start time of different traces could be visualized by
> bringing
> >> >> >>> them
> >> >> >>> up to a same time reference for comparison.
> >> >> >>>
> >> >> >>>
> >> >> >>> [1]
> https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> >> >> >>> [2]
> >> >> >>>
> >> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
> >> >> >>>
> >> >> >>>
> >> >> >>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha
> >> >> >>>  >> >> >>> > wrote:
> >> >> >>>
> >> >>  Hi All,
> >> >> 
> >> >>  Attached here are the table schema and data for the join que

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor]

Yes, exactly right.

On Wed, Jun 24, 2015 at 11:35 AM, Ayola Jayamaha  wrote:
> Hi All,
>
> Now it is clear. We can create a statement from the user's input query to
> the format in the description column and filter out the possible root spans
> of the traces of the query. Then by selecting the traces which have their
> parent ids equal to span id of the root span we can get all the traces
> relevant to the query.
> We can find the total duration for a particular statement. Interesting
> statements/traces can viewed as a timeline.
> Is this method alright?
>
> Thanks,
> Nishani
>
>
> On Wed, Jun 24, 2015 at 11:21 PM, James Taylor 
> wrote:
>
>> Yep, Jesse's right - the query is in the description column of the
>> root span of the trace. We'll need to include this in the trace UI,
>> otherwise the user won't have the context they need to know what
>> they're looking at. If there's something missing from the way we're
>> capturing, we can fix it.
>> Thanks,
>> James
>>
>> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates 
>> wrote:
>> > There was some discussion (maybe internal to salesforce?) around how to
>> > include the query in the trace. I think the simplest we came up with was
>> > just adding it to the trace metadata (as an annotation?) and then you can
>> > pull it out later since you know the key it was stored as
>> >
>> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha 
>> > wrote:
>> >>
>> >> Hi James,
>> >>
>> >> I find it difficult to come up with a method to include the SQL
>> statements
>> >> with the traces. But it is possible to filter out the traces for a
>> >> particular table for a given time period.
>> >>
>> >> Aggregating over the time spent for each SQL statement is possible. With
>> >> the relationship between parent and span ids it is possible to
>> >> differentiate between traces belonging to each query.
>> >>
>> >> Thanks,
>> >> Nishani
>> >>
>> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor 
>> >> wrote:
>> >>
>> >> > Hi Nishani,
>> >> > I think this is a good start. One important part is tying this back to
>> >> > something to which the user can relate - namely the SQL statement that
>> >> > was
>> >> > executed. Would it be possible to include the string of the statement?
>> >> >
>> >> > Another interesting angle would be to group by the statement and
>> >> > aggregate
>> >> > the overall time spent to get an idea of the "top N queries" over a
>> >> > given
>> >> > time range. Then drilling into those to see the traces.
>> >> >
>> >> > Thanks,
>> >> > James
>> >> >
>> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
>> >> > 
>> >> > wrote:
>> >> >
>> >> >> Hi All,
>> >> >>
>> >> >> Milestone-1 can be found in my git repo[1].
>> >> >> Features :
>> >> >>
>> >> >>- Adding tracing to a timeline using sample json
>> >> >>- Comparing two or more traces on the timeline
>> >> >>- Visualizing the trace distribution across the time axis
>> >> >>- Removing a trace from the list of traces represented on the
>> chart
>> >> >>- Listing the tracing information on a table
>> >>
>> >> >>
>> >> >> Any feedback will be appreciated.
>> >> >> Thanks.
>> >> >>
>> >> >>  [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> >> >>
>> >> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha
>> >> >> 
>> >> >> wrote:
>> >> >>
>> >> >>>
>> >> >>> Hi All,
>> >> >>>
>> >> >>> You can find milestone-1 in my git repo. This is the working
>> >> >>> branch[1].
>> >> >>> It has not been binded to backend yet. But the visualization of
>> traces
>> >> >>> can
>> >> >>> be seen from the code.
>> >> >>> Traces can be selected from table/time period and shown on the
>> >> >>> timeline
>> >> >>> as [2]. The parameters could be entered as TableName, StartTime,
>> >> >>> EndTime
>> >> >>> and the traces would be listed down. The user can select the traces
>> as
>> >> >>> his
>> >> >>> preference and view their timelines. Is the procedure ok?
>> >> >>> The start time of different traces could be visualized by bringing
>> >> >>> them
>> >> >>> up to a same time reference for comparison.
>> >> >>>
>> >> >>>
>> >> >>> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> >> >>> [2]
>> >> >>>
>> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
>> >> >>>
>> >> >>>
>> >> >>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha
>> >> >>> > >> >>> > wrote:
>> >> >>>
>> >>  Hi All,
>> >> 
>> >>  Attached here are the table schema and data for the join query I
>> >>  executed.
>> >> 
>> >>  ./psql.py localhost:2181/hbase ../examples/school/school.sql
>> >>  ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
>> >>  ../examples/school/MARKS.csv ../examples/school/school_queries.sql
>> >> 
>> >>  Above is the command I executed. But the last query
>> >> 
>> >>  SELECT M.GRADE
>> >>  FROM MARKS AS M
>> >>  INNER JOIN SUBJECTS AS S
>> >>  ON M.SUBJECT_ID = S.SUBJECT_ID;
>> >> 
>> >> 

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

Now it is clear. We can create a statement from the user's input query to
the format in the description column and filter out the possible root spans
of the traces of the query. Then by selecting the traces which have their
parent ids equal to span id of the root span we can get all the traces
relevant to the query.
We can find the total duration for a particular statement. Interesting
statements/traces can viewed as a timeline.
Is this method alright?

Thanks,
Nishani


On Wed, Jun 24, 2015 at 11:21 PM, James Taylor 
wrote:

> Yep, Jesse's right - the query is in the description column of the
> root span of the trace. We'll need to include this in the trace UI,
> otherwise the user won't have the context they need to know what
> they're looking at. If there's something missing from the way we're
> capturing, we can fix it.
> Thanks,
> James
>
> On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates 
> wrote:
> > There was some discussion (maybe internal to salesforce?) around how to
> > include the query in the trace. I think the simplest we came up with was
> > just adding it to the trace metadata (as an annotation?) and then you can
> > pull it out later since you know the key it was stored as
> >
> > On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha 
> > wrote:
> >>
> >> Hi James,
> >>
> >> I find it difficult to come up with a method to include the SQL
> statements
> >> with the traces. But it is possible to filter out the traces for a
> >> particular table for a given time period.
> >>
> >> Aggregating over the time spent for each SQL statement is possible. With
> >> the relationship between parent and span ids it is possible to
> >> differentiate between traces belonging to each query.
> >>
> >> Thanks,
> >> Nishani
> >>
> >> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor 
> >> wrote:
> >>
> >> > Hi Nishani,
> >> > I think this is a good start. One important part is tying this back to
> >> > something to which the user can relate - namely the SQL statement that
> >> > was
> >> > executed. Would it be possible to include the string of the statement?
> >> >
> >> > Another interesting angle would be to group by the statement and
> >> > aggregate
> >> > the overall time spent to get an idea of the "top N queries" over a
> >> > given
> >> > time range. Then drilling into those to see the traces.
> >> >
> >> > Thanks,
> >> > James
> >> >
> >> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
> >> > 
> >> > wrote:
> >> >
> >> >> Hi All,
> >> >>
> >> >> Milestone-1 can be found in my git repo[1].
> >> >> Features :
> >> >>
> >> >>- Adding tracing to a timeline using sample json
> >> >>- Comparing two or more traces on the timeline
> >> >>- Visualizing the trace distribution across the time axis
> >> >>- Removing a trace from the list of traces represented on the
> chart
> >> >>- Listing the tracing information on a table
> >>
> >> >>
> >> >> Any feedback will be appreciated.
> >> >> Thanks.
> >> >>
> >> >>  [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> >> >>
> >> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha
> >> >> 
> >> >> wrote:
> >> >>
> >> >>>
> >> >>> Hi All,
> >> >>>
> >> >>> You can find milestone-1 in my git repo. This is the working
> >> >>> branch[1].
> >> >>> It has not been binded to backend yet. But the visualization of
> traces
> >> >>> can
> >> >>> be seen from the code.
> >> >>> Traces can be selected from table/time period and shown on the
> >> >>> timeline
> >> >>> as [2]. The parameters could be entered as TableName, StartTime,
> >> >>> EndTime
> >> >>> and the traces would be listed down. The user can select the traces
> as
> >> >>> his
> >> >>> preference and view their timelines. Is the procedure ok?
> >> >>> The start time of different traces could be visualized by bringing
> >> >>> them
> >> >>> up to a same time reference for comparison.
> >> >>>
> >> >>>
> >> >>> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> >> >>> [2]
> >> >>>
> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
> >> >>>
> >> >>>
> >> >>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha
> >> >>>  >> >>> > wrote:
> >> >>>
> >>  Hi All,
> >> 
> >>  Attached here are the table schema and data for the join query I
> >>  executed.
> >> 
> >>  ./psql.py localhost:2181/hbase ../examples/school/school.sql
> >>  ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
> >>  ../examples/school/MARKS.csv ../examples/school/school_queries.sql
> >> 
> >>  Above is the command I executed. But the last query
> >> 
> >>  SELECT M.GRADE
> >>  FROM MARKS AS M
> >>  INNER JOIN SUBJECTS AS S
> >>  ON M.SUBJECT_ID = S.SUBJECT_ID;
> >> 
> >>  doesn't give any results and when I check for the traces
> >>  corresponding
> >>  the inner join query I couldn't find them.
> >> 
> >>  What might be the issue?
> >> 
> >>  Thanks.
> >> 
> >> 
> >> >

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor]

Yep, Jesse's right - the query is in the description column of the
root span of the trace. We'll need to include this in the trace UI,
otherwise the user won't have the context they need to know what
they're looking at. If there's something missing from the way we're
capturing, we can fix it.
Thanks,
James

On Wed, Jun 24, 2015 at 9:09 AM, Jesse Yates  wrote:
> There was some discussion (maybe internal to salesforce?) around how to
> include the query in the trace. I think the simplest we came up with was
> just adding it to the trace metadata (as an annotation?) and then you can
> pull it out later since you know the key it was stored as
>
> On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha 
> wrote:
>>
>> Hi James,
>>
>> I find it difficult to come up with a method to include the SQL statements
>> with the traces. But it is possible to filter out the traces for a
>> particular table for a given time period.
>>
>> Aggregating over the time spent for each SQL statement is possible. With
>> the relationship between parent and span ids it is possible to
>> differentiate between traces belonging to each query.
>>
>> Thanks,
>> Nishani
>>
>> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor 
>> wrote:
>>
>> > Hi Nishani,
>> > I think this is a good start. One important part is tying this back to
>> > something to which the user can relate - namely the SQL statement that
>> > was
>> > executed. Would it be possible to include the string of the statement?
>> >
>> > Another interesting angle would be to group by the statement and
>> > aggregate
>> > the overall time spent to get an idea of the "top N queries" over a
>> > given
>> > time range. Then drilling into those to see the traces.
>> >
>> > Thanks,
>> > James
>> >
>> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha
>> > 
>> > wrote:
>> >
>> >> Hi All,
>> >>
>> >> Milestone-1 can be found in my git repo[1].
>> >> Features :
>> >>
>> >>- Adding tracing to a timeline using sample json
>> >>- Comparing two or more traces on the timeline
>> >>- Visualizing the trace distribution across the time axis
>> >>- Removing a trace from the list of traces represented on the chart
>> >>- Listing the tracing information on a table
>>
>> >>
>> >> Any feedback will be appreciated.
>> >> Thanks.
>> >>
>> >>  [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> >>
>> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha
>> >> 
>> >> wrote:
>> >>
>> >>>
>> >>> Hi All,
>> >>>
>> >>> You can find milestone-1 in my git repo. This is the working
>> >>> branch[1].
>> >>> It has not been binded to backend yet. But the visualization of traces
>> >>> can
>> >>> be seen from the code.
>> >>> Traces can be selected from table/time period and shown on the
>> >>> timeline
>> >>> as [2]. The parameters could be entered as TableName, StartTime,
>> >>> EndTime
>> >>> and the traces would be listed down. The user can select the traces as
>> >>> his
>> >>> preference and view their timelines. Is the procedure ok?
>> >>> The start time of different traces could be visualized by bringing
>> >>> them
>> >>> up to a same time reference for comparison.
>> >>>
>> >>>
>> >>> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> >>> [2]
>> >>> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
>> >>>
>> >>>
>> >>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha
>> >>> > >>> > wrote:
>> >>>
>>  Hi All,
>> 
>>  Attached here are the table schema and data for the join query I
>>  executed.
>> 
>>  ./psql.py localhost:2181/hbase ../examples/school/school.sql
>>  ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
>>  ../examples/school/MARKS.csv ../examples/school/school_queries.sql
>> 
>>  Above is the command I executed. But the last query
>> 
>>  SELECT M.GRADE
>>  FROM MARKS AS M
>>  INNER JOIN SUBJECTS AS S
>>  ON M.SUBJECT_ID = S.SUBJECT_ID;
>> 
>>  doesn't give any results and when I check for the traces
>>  corresponding
>>  the inner join query I couldn't find them.
>> 
>>  What might be the issue?
>> 
>>  Thanks.
>> 
>> 
>>   school.zip
>> 
>>  
>>
>> 
>> 
>>  On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha
>>  >  > wrote:
>> 
>> > Hi All,
>> >
>> > On the explain plan to show which part of the code is run where a
>> > graph is shown[1]. Default chart will be a Pie chart and I'm planing
>> > to use
>> > few more chat types so user can pick his choice. If any node
>> > responding
>> > slowly. Phoenix database administrator can exam the node and examin
>> > what
>> > are queries run on a particular time.
>> >
>> > I have run few examples on secondary indexes[4] and I got sample
>> > data
>> > and it can be used for the milestone1(end of this week). I

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Jesse Yates]

There was some discussion (maybe internal to salesforce?) around how to
include the query in the trace. I think the simplest we came up with was
just adding it to the trace metadata (as an annotation?) and then you can
pull it out later since you know the key it was stored as

On Wed, Jun 24, 2015 at 9:05 AM Ayola Jayamaha 
wrote:

> Hi James,
>
> I find it difficult to come up with a method to include the SQL statements
> with the traces. But it is possible to filter out the traces for a
> particular table for a given time period.
>
> Aggregating over the time spent for each SQL statement is possible. With
> the relationship between parent and span ids it is possible to
> differentiate between traces belonging to each query.
>
> Thanks,
> Nishani
>
> On Wed, Jun 24, 2015 at 12:11 PM, James Taylor 
> wrote:
>
> > Hi Nishani,
> > I think this is a good start. One important part is tying this back to
> > something to which the user can relate - namely the SQL statement that
> was
> > executed. Would it be possible to include the string of the statement?
> >
> > Another interesting angle would be to group by the statement and
> aggregate
> > the overall time spent to get an idea of the "top N queries" over a given
> > time range. Then drilling into those to see the traces.
> >
> > Thanks,
> > James
> >
> > On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha  >
> > wrote:
> >
> >> Hi All,
> >>
> >> Milestone-1 can be found in my git repo[1].
> >> Features :
> >>
> >>- Adding tracing to a timeline using sample json
> >>- Comparing two or more traces on the timeline
> >>- Visualizing the trace distribution across the time axis
> >>- Removing a trace from the list of traces represented on the chart
> >>- Listing the tracing information on a table
> >>
> >> Any feedback will be appreciated.
> >> Thanks.
> >>
> >>  [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> >>
> >> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha <
> raphaelan...@gmail.com>
> >> wrote:
> >>
> >>>
> >>> Hi All,
> >>>
> >>> You can find milestone-1 in my git repo. This is the working branch[1].
> >>> It has not been binded to backend yet. But the visualization of traces
> can
> >>> be seen from the code.
> >>> Traces can be selected from table/time period and shown on the timeline
> >>> as [2]. The parameters could be entered as TableName, StartTime,
> EndTime
> >>> and the traces would be listed down. The user can select the traces as
> his
> >>> preference and view their timelines. Is the procedure ok?
> >>> The start time of different traces could be visualized by bringing them
> >>> up to a same time reference for comparison.
> >>>
> >>>
> >>> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> >>> [2]
> >>> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
> >>>
> >>>
> >>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha <
> raphaelan...@gmail.com
> >>> > wrote:
> >>>
>  Hi All,
> 
>  Attached here are the table schema and data for the join query I
>  executed.
> 
>  ./psql.py localhost:2181/hbase ../examples/school/school.sql
>  ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
>  ../examples/school/MARKS.csv ../examples/school/school_queries.sql
> 
>  Above is the command I executed. But the last query
> 
>  SELECT M.GRADE
>  FROM MARKS AS M
>  INNER JOIN SUBJECTS AS S
>  ON M.SUBJECT_ID = S.SUBJECT_ID;
> 
>  doesn't give any results and when I check for the traces corresponding
>  the inner join query I couldn't find them.
> 
>  What might be the issue?
> 
>  Thanks.
> 
>  ​​​
>   school.zip
>  <
> https://drive.google.com/file/d/0Bxpj3lSPvr6WdW15bUc0YkdYemc/edit?usp=drive_web
> >
>  ​
> 
>  On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha <
> raphaelan...@gmail.com
>  > wrote:
> 
> > Hi All,
> >
> > On the explain plan to show which part of the code is run where a
> > graph is shown[1]. Default chart will be a Pie chart and I'm planing
> to use
> > few more chat types so user can pick his choice. If any node
> responding
> > slowly. Phoenix database administrator can exam the node and examin
> what
> > are queries run on a particular time.
> >
> > I have run few examples on secondary indexes[4] and I got sample data
> > and it can be used for the milestone1(end of this week). It is shown
> with
> > timesliding capabilities. Trace segments are shown in a timeline.[2]
> >
> > Does filters mean 'where' like logic statements? The database admin
> > can track the duration for a particular trace from timeline
> visualization
> > so he can use the filters effectively (best order of the filters) in
> a
> > query to get a quick respond.
> >
> > I tried the join query and it didn't give any results or
> corresponding
> > traces. This is the reference I followe

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi James,

I find it difficult to come up with a method to include the SQL statements
with the traces. But it is possible to filter out the traces for a
particular table for a given time period.

Aggregating over the time spent for each SQL statement is possible. With
the relationship between parent and span ids it is possible to
differentiate between traces belonging to each query.

Thanks,
Nishani

On Wed, Jun 24, 2015 at 12:11 PM, James Taylor 
wrote:

> Hi Nishani,
> I think this is a good start. One important part is tying this back to
> something to which the user can relate - namely the SQL statement that was
> executed. Would it be possible to include the string of the statement?
>
> Another interesting angle would be to group by the statement and aggregate
> the overall time spent to get an idea of the "top N queries" over a given
> time range. Then drilling into those to see the traces.
>
> Thanks,
> James
>
> On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha 
> wrote:
>
>> Hi All,
>>
>> Milestone-1 can be found in my git repo[1].
>> Features :
>>
>>- Adding tracing to a timeline using sample json
>>- Comparing two or more traces on the timeline
>>- Visualizing the trace distribution across the time axis
>>- Removing a trace from the list of traces represented on the chart
>>- Listing the tracing information on a table
>>
>> Any feedback will be appreciated.
>> Thanks.
>>
>>  [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>>
>> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha 
>> wrote:
>>
>>>
>>> Hi All,
>>>
>>> You can find milestone-1 in my git repo. This is the working branch[1].
>>> It has not been binded to backend yet. But the visualization of traces can
>>> be seen from the code.
>>> Traces can be selected from table/time period and shown on the timeline
>>> as [2]. The parameters could be entered as TableName, StartTime, EndTime
>>> and the traces would be listed down. The user can select the traces as his
>>> preference and view their timelines. Is the procedure ok?
>>> The start time of different traces could be visualized by bringing them
>>> up to a same time reference for comparison.
>>>
>>>
>>> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>>> [2]
>>> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
>>>
>>>
>>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha >> > wrote:
>>>
 Hi All,

 Attached here are the table schema and data for the join query I
 executed.

 ./psql.py localhost:2181/hbase ../examples/school/school.sql
 ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
 ../examples/school/MARKS.csv ../examples/school/school_queries.sql

 Above is the command I executed. But the last query

 SELECT M.GRADE
 FROM MARKS AS M
 INNER JOIN SUBJECTS AS S
 ON M.SUBJECT_ID = S.SUBJECT_ID;

 doesn't give any results and when I check for the traces corresponding
 the inner join query I couldn't find them.

 What might be the issue?

 Thanks.

 ​​​
  school.zip
 
 ​

 On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha >>> > wrote:

> Hi All,
>
> On the explain plan to show which part of the code is run where a
> graph is shown[1]. Default chart will be a Pie chart and I'm planing to 
> use
> few more chat types so user can pick his choice. If any node responding
> slowly. Phoenix database administrator can exam the node and examin what
> are queries run on a particular time.
>
> I have run few examples on secondary indexes[4] and I got sample data
> and it can be used for the milestone1(end of this week). It is shown with
> timesliding capabilities. Trace segments are shown in a timeline.[2]
>
> Does filters mean 'where' like logic statements? The database admin
> can track the duration for a particular trace from timeline visualization
> so he can use the filters effectively (best order of the filters) in a
> query to get a quick respond.
>
> I tried the join query and it didn't give any results or corresponding
> traces. This is the reference I followed [3]. Is there any more steps to
> follow?
>
> To visualize the tracing details I looked through few charting
> libraries and I will give the comparison details over them.
> Please feel free to give the feedback on the mock uis.
>
> Thanks.
>
> [1]
> https://issues.apache.org/jira/secure/attachment/12739498/m1-mockUI-tracedistribution.png
> [2]
> https://issues.apache.org/jira/secure/attachment/12739499/m1-mockUI-tracetimeline.png
> [3] https://phoenix.apache.org/joins.html
> [4]
> http://ayolajayamaha.blogspot.com/2015/06/tracing-data-secondary-indixes.html
>
> On Thu, Jun 11, 2015 at 11:39 AM, Ayola Jay

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor]

Hi Nishani,
I think this is a good start. One important part is tying this back to
something to which the user can relate - namely the SQL statement that was
executed. Would it be possible to include the string of the statement?

Another interesting angle would be to group by the statement and aggregate
the overall time spent to get an idea of the "top N queries" over a given
time range. Then drilling into those to see the traces.

Thanks,
James

On Fri, Jun 19, 2015 at 11:01 AM, Ayola Jayamaha 
wrote:

> Hi All,
>
> Milestone-1 can be found in my git repo[1].
> Features :
>
>- Adding tracing to a timeline using sample json
>- Comparing two or more traces on the timeline
>- Visualizing the trace distribution across the time axis
>- Removing a trace from the list of traces represented on the chart
>- Listing the tracing information on a table
>
> Any feedback will be appreciated.
> Thanks.
>
>  [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>
> On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha 
> wrote:
>
>>
>> Hi All,
>>
>> You can find milestone-1 in my git repo. This is the working branch[1].
>> It has not been binded to backend yet. But the visualization of traces can
>> be seen from the code.
>> Traces can be selected from table/time period and shown on the timeline
>> as [2]. The parameters could be entered as TableName, StartTime, EndTime
>> and the traces would be listed down. The user can select the traces as his
>> preference and view their timelines. Is the procedure ok?
>> The start time of different traces could be visualized by bringing them
>> up to a same time reference for comparison.
>>
>>
>> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
>> [2]
>> https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
>>
>>
>> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha 
>> wrote:
>>
>>> Hi All,
>>>
>>> Attached here are the table schema and data for the join query I
>>> executed.
>>>
>>> ./psql.py localhost:2181/hbase ../examples/school/school.sql
>>> ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
>>> ../examples/school/MARKS.csv ../examples/school/school_queries.sql
>>>
>>> Above is the command I executed. But the last query
>>>
>>> SELECT M.GRADE
>>> FROM MARKS AS M
>>> INNER JOIN SUBJECTS AS S
>>> ON M.SUBJECT_ID = S.SUBJECT_ID;
>>>
>>> doesn't give any results and when I check for the traces corresponding
>>> the inner join query I couldn't find them.
>>>
>>> What might be the issue?
>>>
>>> Thanks.
>>>
>>> ​​​
>>>  school.zip
>>> 
>>> ​
>>>
>>> On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha 
>>> wrote:
>>>
 Hi All,

 On the explain plan to show which part of the code is run where a graph
 is shown[1]. Default chart will be a Pie chart and I'm planing to use few
 more chat types so user can pick his choice. If any node responding slowly.
 Phoenix database administrator can exam the node and examin what are
 queries run on a particular time.

 I have run few examples on secondary indexes[4] and I got sample data
 and it can be used for the milestone1(end of this week). It is shown with
 timesliding capabilities. Trace segments are shown in a timeline.[2]

 Does filters mean 'where' like logic statements? The database admin can
 track the duration for a particular trace from timeline visualization so he
 can use the filters effectively (best order of the filters) in a query to
 get a quick respond.

 I tried the join query and it didn't give any results or corresponding
 traces. This is the reference I followed [3]. Is there any more steps to
 follow?

 To visualize the tracing details I looked through few charting
 libraries and I will give the comparison details over them.
 Please feel free to give the feedback on the mock uis.

 Thanks.

 [1]
 https://issues.apache.org/jira/secure/attachment/12739498/m1-mockUI-tracedistribution.png
 [2]
 https://issues.apache.org/jira/secure/attachment/12739499/m1-mockUI-tracetimeline.png
 [3] https://phoenix.apache.org/joins.html
 [4]
 http://ayolajayamaha.blogspot.com/2015/06/tracing-data-secondary-indixes.html

 On Thu, Jun 11, 2015 at 11:39 AM, Ayola Jayamaha <
 raphaelan...@gmail.com> wrote:

> Yes. It  was a bit confusing :-). But it was useful to get a good idea
> on the use cases.
> Thanks.
>
> On Wed, Jun 10, 2015 at 11:57 PM, James Taylor  > wrote:
>
>> Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
>> it helped).
>>
>> On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha <
>> raphaelan...@gmail.com> wrote:
>> > Hi James,
>> >
>> > Thanks a lot for the lengthy and descriptive reply. I am currently
>> looking
>> > through UI components and char

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

Milestone-1 can be found in my git repo[1].
Features :

   - Adding tracing to a timeline using sample json
   - Comparing two or more traces on the timeline
   - Visualizing the trace distribution across the time axis
   - Removing a trace from the list of traces represented on the chart
   - Listing the tracing information on a table

Any feedback will be appreciated.
Thanks.

 [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1

On Wed, Jun 17, 2015 at 11:35 PM, Ayola Jayamaha 
wrote:

>
> Hi All,
>
> You can find milestone-1 in my git repo. This is the working branch[1]. It
> has not been binded to backend yet. But the visualization of traces can be
> seen from the code.
> Traces can be selected from table/time period and shown on the timeline as
> [2]. The parameters could be entered as TableName, StartTime, EndTime and
> the traces would be listed down. The user can select the traces as his
> preference and view their timelines. Is the procedure ok?
> The start time of different traces could be visualized by bringing them up
> to a same time reference for comparison.
>
>
> [1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
> [2] https://issues.apache.org/jira/secure/attachment/12740161/timeline.png
>
>
> On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha 
> wrote:
>
>> Hi All,
>>
>> Attached here are the table schema and data for the join query I
>> executed.
>>
>> ./psql.py localhost:2181/hbase ../examples/school/school.sql
>> ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
>> ../examples/school/MARKS.csv ../examples/school/school_queries.sql
>>
>> Above is the command I executed. But the last query
>>
>> SELECT M.GRADE
>> FROM MARKS AS M
>> INNER JOIN SUBJECTS AS S
>> ON M.SUBJECT_ID = S.SUBJECT_ID;
>>
>> doesn't give any results and when I check for the traces corresponding
>> the inner join query I couldn't find them.
>>
>> What might be the issue?
>>
>> Thanks.
>>
>> ​​​
>>  school.zip
>> 
>> ​
>>
>> On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha 
>> wrote:
>>
>>> Hi All,
>>>
>>> On the explain plan to show which part of the code is run where a graph
>>> is shown[1]. Default chart will be a Pie chart and I'm planing to use few
>>> more chat types so user can pick his choice. If any node responding slowly.
>>> Phoenix database administrator can exam the node and examin what are
>>> queries run on a particular time.
>>>
>>> I have run few examples on secondary indexes[4] and I got sample data
>>> and it can be used for the milestone1(end of this week). It is shown with
>>> timesliding capabilities. Trace segments are shown in a timeline.[2]
>>>
>>> Does filters mean 'where' like logic statements? The database admin can
>>> track the duration for a particular trace from timeline visualization so he
>>> can use the filters effectively (best order of the filters) in a query to
>>> get a quick respond.
>>>
>>> I tried the join query and it didn't give any results or corresponding
>>> traces. This is the reference I followed [3]. Is there any more steps to
>>> follow?
>>>
>>> To visualize the tracing details I looked through few charting libraries
>>> and I will give the comparison details over them.
>>> Please feel free to give the feedback on the mock uis.
>>>
>>> Thanks.
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/attachment/12739498/m1-mockUI-tracedistribution.png
>>> [2]
>>> https://issues.apache.org/jira/secure/attachment/12739499/m1-mockUI-tracetimeline.png
>>> [3] https://phoenix.apache.org/joins.html
>>> [4]
>>> http://ayolajayamaha.blogspot.com/2015/06/tracing-data-secondary-indixes.html
>>>
>>> On Thu, Jun 11, 2015 at 11:39 AM, Ayola Jayamaha >> > wrote:
>>>
 Yes. It  was a bit confusing :-). But it was useful to get a good idea
 on the use cases.
 Thanks.

 On Wed, Jun 10, 2015 at 11:57 PM, James Taylor 
 wrote:

> Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
> it helped).
>
> On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha <
> raphaelan...@gmail.com> wrote:
> > Hi James,
> >
> > Thanks a lot for the lengthy and descriptive reply. I am currently
> looking
> > through UI components and charting libraries that can be used for
> the UI. I
> > refered [1] with regard to your explaination and came up with some
> mock ups
> > which I will share soon.
> >
> > Thanks,
> > Nishani
> >
> > [1] https://phoenix.apache.org/language/#index_hint
> > [2]
> >
> https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table
> >
> > On Tue, Jun 9, 2015 at 11:39 PM, James Taylor <
> jamestay...@apache.org>
> > wrote:
> >
> >> Hi Nishani,
> >> I'd recommend focusing on higher level use cases. From the user's
> >> point of view, they're executing a query and for some reason it's

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: screenshot of tracing timeline.png

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, m1-mockUI-tracedistribution.png, m1-mockUI-tracetimeline.png, 
> screenshot of tracing timeline.png, screenshot of tracing web app.png, 
> timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

You can find milestone-1 in my git repo. This is the working branch[1]. It
has not been binded to backend yet. But the visualization of traces can be
seen from the code.
Traces can be selected from table/time period and shown on the timeline as
[2]. The parameters could be entered as TableName, StartTime, EndTime and
the traces would be listed down. The user can select the traces as his
preference and view their timelines. Is the procedure ok?
The start time of different traces could be visualized by bringing them up
to a same time reference for comparison.


[1] https://github.com/AyolaJayamaha/TracingWebApp/tree/milestone-1
[2] https://issues.apache.org/jira/secure/attachment/12740161/timeline.png


On Tue, Jun 16, 2015 at 11:10 AM, Ayola Jayamaha 
wrote:

> Hi All,
>
> Attached here are the table schema and data for the join query I executed.
>
> ./psql.py localhost:2181/hbase ../examples/school/school.sql
> ../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
> ../examples/school/MARKS.csv ../examples/school/school_queries.sql
>
> Above is the command I executed. But the last query
>
> SELECT M.GRADE
> FROM MARKS AS M
> INNER JOIN SUBJECTS AS S
> ON M.SUBJECT_ID = S.SUBJECT_ID;
>
> doesn't give any results and when I check for the traces corresponding the
> inner join query I couldn't find them.
>
> What might be the issue?
>
> Thanks.
>
> ​​​
>  school.zip
> 
> ​
>
> On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha 
> wrote:
>
>> Hi All,
>>
>> On the explain plan to show which part of the code is run where a graph
>> is shown[1]. Default chart will be a Pie chart and I'm planing to use few
>> more chat types so user can pick his choice. If any node responding slowly.
>> Phoenix database administrator can exam the node and examin what are
>> queries run on a particular time.
>>
>> I have run few examples on secondary indexes[4] and I got sample data and
>> it can be used for the milestone1(end of this week). It is shown with
>> timesliding capabilities. Trace segments are shown in a timeline.[2]
>>
>> Does filters mean 'where' like logic statements? The database admin can
>> track the duration for a particular trace from timeline visualization so he
>> can use the filters effectively (best order of the filters) in a query to
>> get a quick respond.
>>
>> I tried the join query and it didn't give any results or corresponding
>> traces. This is the reference I followed [3]. Is there any more steps to
>> follow?
>>
>> To visualize the tracing details I looked through few charting libraries
>> and I will give the comparison details over them.
>> Please feel free to give the feedback on the mock uis.
>>
>> Thanks.
>>
>> [1]
>> https://issues.apache.org/jira/secure/attachment/12739498/m1-mockUI-tracedistribution.png
>> [2]
>> https://issues.apache.org/jira/secure/attachment/12739499/m1-mockUI-tracetimeline.png
>> [3] https://phoenix.apache.org/joins.html
>> [4]
>> http://ayolajayamaha.blogspot.com/2015/06/tracing-data-secondary-indixes.html
>>
>> On Thu, Jun 11, 2015 at 11:39 AM, Ayola Jayamaha 
>> wrote:
>>
>>> Yes. It  was a bit confusing :-). But it was useful to get a good idea
>>> on the use cases.
>>> Thanks.
>>>
>>> On Wed, Jun 10, 2015 at 11:57 PM, James Taylor 
>>> wrote:
>>>
 Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
 it helped).

 On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha <
 raphaelan...@gmail.com> wrote:
 > Hi James,
 >
 > Thanks a lot for the lengthy and descriptive reply. I am currently
 looking
 > through UI components and charting libraries that can be used for the
 UI. I
 > refered [1] with regard to your explaination and came up with some
 mock ups
 > which I will share soon.
 >
 > Thanks,
 > Nishani
 >
 > [1] https://phoenix.apache.org/language/#index_hint
 > [2]
 >
 https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table
 >
 > On Tue, Jun 9, 2015 at 11:39 PM, James Taylor >>> >
 > wrote:
 >
 >> Hi Nishani,
 >> I'd recommend focusing on higher level use cases. From the user's
 >> point of view, they're executing a query and for some reason it's
 >> slower than they expect. How do they figure out why?
 >>
 >> They might first do an EXPLAIN on their query to see how Phoenix is
 >> executing it. Which parts are run where? Are secondary indexes being
 >> used as expected? Are filters being pushed down as expected? A better
 >> way to visualize the explain plan might be a good thing for you to
 >> start with.
 >>
 >> Second, assuming the explain plan looks good, they'll want to turn on
 >> tracing so that they can get runtime information on which parts of
 >> their query are taking the longest.
 >>
 >> Maybe more than one Phoenix table is involved - how will you display
>>>

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: timeline.png

Tracing visualization with timeline charts

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, m1-mockUI-tracedistribution.png, m1-mockUI-tracetimeline.png, 
> screenshot of tracing web app.png, timeline.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

Attached here are the table schema and data for the join query I executed.

./psql.py localhost:2181/hbase ../examples/school/school.sql
../examples/school/STUDENTS.csv ../examples/school/SUBJECTS.csv
../examples/school/MARKS.csv ../examples/school/school_queries.sql

Above is the command I executed. But the last query

SELECT M.GRADE
FROM MARKS AS M
INNER JOIN SUBJECTS AS S
ON M.SUBJECT_ID = S.SUBJECT_ID;

doesn't give any results and when I check for the traces corresponding the
inner join query I couldn't find them.

What might be the issue?

Thanks.

​​​
 school.zip

​

On Sun, Jun 14, 2015 at 9:58 PM, Ayola Jayamaha 
wrote:

> Hi All,
>
> On the explain plan to show which part of the code is run where a graph is
> shown[1]. Default chart will be a Pie chart and I'm planing to use few more
> chat types so user can pick his choice. If any node responding slowly.
> Phoenix database administrator can exam the node and examin what are
> queries run on a particular time.
>
> I have run few examples on secondary indexes[4] and I got sample data and
> it can be used for the milestone1(end of this week). It is shown with
> timesliding capabilities. Trace segments are shown in a timeline.[2]
>
> Does filters mean 'where' like logic statements? The database admin can
> track the duration for a particular trace from timeline visualization so he
> can use the filters effectively (best order of the filters) in a query to
> get a quick respond.
>
> I tried the join query and it didn't give any results or corresponding
> traces. This is the reference I followed [3]. Is there any more steps to
> follow?
>
> To visualize the tracing details I looked through few charting libraries
> and I will give the comparison details over them.
> Please feel free to give the feedback on the mock uis.
>
> Thanks.
>
> [1]
> https://issues.apache.org/jira/secure/attachment/12739498/m1-mockUI-tracedistribution.png
> [2]
> https://issues.apache.org/jira/secure/attachment/12739499/m1-mockUI-tracetimeline.png
> [3] https://phoenix.apache.org/joins.html
> [4]
> http://ayolajayamaha.blogspot.com/2015/06/tracing-data-secondary-indixes.html
>
> On Thu, Jun 11, 2015 at 11:39 AM, Ayola Jayamaha 
> wrote:
>
>> Yes. It  was a bit confusing :-). But it was useful to get a good idea on
>> the use cases.
>> Thanks.
>>
>> On Wed, Jun 10, 2015 at 11:57 PM, James Taylor 
>> wrote:
>>
>>> Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
>>> it helped).
>>>
>>> On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha 
>>> wrote:
>>> > Hi James,
>>> >
>>> > Thanks a lot for the lengthy and descriptive reply. I am currently
>>> looking
>>> > through UI components and charting libraries that can be used for the
>>> UI. I
>>> > refered [1] with regard to your explaination and came up with some
>>> mock ups
>>> > which I will share soon.
>>> >
>>> > Thanks,
>>> > Nishani
>>> >
>>> > [1] https://phoenix.apache.org/language/#index_hint
>>> > [2]
>>> >
>>> https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table
>>> >
>>> > On Tue, Jun 9, 2015 at 11:39 PM, James Taylor 
>>> > wrote:
>>> >
>>> >> Hi Nishani,
>>> >> I'd recommend focusing on higher level use cases. From the user's
>>> >> point of view, they're executing a query and for some reason it's
>>> >> slower than they expect. How do they figure out why?
>>> >>
>>> >> They might first do an EXPLAIN on their query to see how Phoenix is
>>> >> executing it. Which parts are run where? Are secondary indexes being
>>> >> used as expected? Are filters being pushed down as expected? A better
>>> >> way to visualize the explain plan might be a good thing for you to
>>> >> start with.
>>> >>
>>> >> Second, assuming the explain plan looks good, they'll want to turn on
>>> >> tracing so that they can get runtime information on which parts of
>>> >> their query are taking the longest.
>>> >>
>>> >> Maybe more than one Phoenix table is involved - how will you display
>>> >> the tracing information across multiple tables for a query that does a
>>> >> join? Maybe you can punt on this first pass, and focus on single table
>>> >> queries. A related use case would be a DML statement that's executed
>>> >> and taking longer than expected. Let's say that the table being
>>> >> updated has one or more secondary indexes that are also updating the
>>> >> index tables. Seeing the entire picture of both the table writes plus
>>> >> the index writes on the same graph would be great.
>>> >>
>>> >> For the single-table query user case, what does the distribution of
>>> >> time look like across all the region servers participating in the
>>> >> query? Maybe some kind of graph that shows quickly if one region
>>> >> server is taking much more time than the others. Perhaps that's an
>>> >> indication that the table statistics need to be re-run, as there may
>>> >> be skew that's developed such that one of the thre

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

On the explain plan to show which part of the code is run where a graph is
shown[1]. Default chart will be a Pie chart and I'm planing to use few more
chat types so user can pick his choice. If any node responding slowly.
Phoenix database administrator can exam the node and examin what are
queries run on a particular time.

I have run few examples on secondary indexes[4] and I got sample data and
it can be used for the milestone1(end of this week). It is shown with
timesliding capabilities. Trace segments are shown in a timeline.[2]

Does filters mean 'where' like logic statements? The database admin can
track the duration for a particular trace from timeline visualization so he
can use the filters effectively (best order of the filters) in a query to
get a quick respond.

I tried the join query and it didn't give any results or corresponding
traces. This is the reference I followed [3]. Is there any more steps to
follow?

To visualize the tracing details I looked through few charting libraries
and I will give the comparison details over them.
Please feel free to give the feedback on the mock uis.

Thanks.

[1]
https://issues.apache.org/jira/secure/attachment/12739498/m1-mockUI-tracedistribution.png
[2]
https://issues.apache.org/jira/secure/attachment/12739499/m1-mockUI-tracetimeline.png
[3] https://phoenix.apache.org/joins.html
[4]
http://ayolajayamaha.blogspot.com/2015/06/tracing-data-secondary-indixes.html

On Thu, Jun 11, 2015 at 11:39 AM, Ayola Jayamaha 
wrote:

> Yes. It  was a bit confusing :-). But it was useful to get a good idea on
> the use cases.
> Thanks.
>
> On Wed, Jun 10, 2015 at 11:57 PM, James Taylor 
> wrote:
>
>> Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
>> it helped).
>>
>> On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha 
>> wrote:
>> > Hi James,
>> >
>> > Thanks a lot for the lengthy and descriptive reply. I am currently
>> looking
>> > through UI components and charting libraries that can be used for the
>> UI. I
>> > refered [1] with regard to your explaination and came up with some mock
>> ups
>> > which I will share soon.
>> >
>> > Thanks,
>> > Nishani
>> >
>> > [1] https://phoenix.apache.org/language/#index_hint
>> > [2]
>> >
>> https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table
>> >
>> > On Tue, Jun 9, 2015 at 11:39 PM, James Taylor 
>> > wrote:
>> >
>> >> Hi Nishani,
>> >> I'd recommend focusing on higher level use cases. From the user's
>> >> point of view, they're executing a query and for some reason it's
>> >> slower than they expect. How do they figure out why?
>> >>
>> >> They might first do an EXPLAIN on their query to see how Phoenix is
>> >> executing it. Which parts are run where? Are secondary indexes being
>> >> used as expected? Are filters being pushed down as expected? A better
>> >> way to visualize the explain plan might be a good thing for you to
>> >> start with.
>> >>
>> >> Second, assuming the explain plan looks good, they'll want to turn on
>> >> tracing so that they can get runtime information on which parts of
>> >> their query are taking the longest.
>> >>
>> >> Maybe more than one Phoenix table is involved - how will you display
>> >> the tracing information across multiple tables for a query that does a
>> >> join? Maybe you can punt on this first pass, and focus on single table
>> >> queries. A related use case would be a DML statement that's executed
>> >> and taking longer than expected. Let's say that the table being
>> >> updated has one or more secondary indexes that are also updating the
>> >> index tables. Seeing the entire picture of both the table writes plus
>> >> the index writes on the same graph would be great.
>> >>
>> >> For the single-table query user case, what does the distribution of
>> >> time look like across all the region servers participating in the
>> >> query? Maybe some kind of graph that shows quickly if one region
>> >> server is taking much more time than the others. Perhaps that's an
>> >> indication that the table statistics need to be re-run, as there may
>> >> be skew that's developed such that one of the threads is handling more
>> >> data than it should. Or perhaps there's an issue with that particular
>> >> region server. Was there something else going on at the same time on
>> >> that region server, like a background compaction/split process? If
>> >> that information is available in the trace table (not sure), it would
>> >> be very cool to be able to superimpose that on top of the query trace
>> >> graph.
>> >>
>> >> Another test might be to run a query over a different table and see if
>> >> the same region server shows up again as being slow. So superimposing
>> >> the query trace graphs of multiple queries might give the user some
>> >> insight.
>> >>
>> >> IMHO, this is the kind of angle you should come at this from.
>> >>
>> >> Thanks,
>> >> James
>> >>
>> >> On Mon, Jun 8, 2015 at 4:12 AM, Ayola Jayamaha > >
>> >> wrote:
>> >> > Hi All,
>>

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: m1-mockUI-tracetimeline.png
m1-mockUI-tracedistribution.png

Mock UI for milestone1
1. Understanding the tracing distribution over the nodes
2. Visualizing tracing details over a time period

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, m1-mockUI-tracedistribution.png, m1-mockUI-tracetimeline.png, 
> screenshot of tracing web app.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Yes. It  was a bit confusing :-). But it was useful to get a good idea on
the use cases.
Thanks.

On Wed, Jun 10, 2015 at 11:57 PM, James Taylor 
wrote:

> Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
> it helped).
>
> On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha 
> wrote:
> > Hi James,
> >
> > Thanks a lot for the lengthy and descriptive reply. I am currently
> looking
> > through UI components and charting libraries that can be used for the
> UI. I
> > refered [1] with regard to your explaination and came up with some mock
> ups
> > which I will share soon.
> >
> > Thanks,
> > Nishani
> >
> > [1] https://phoenix.apache.org/language/#index_hint
> > [2]
> >
> https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table
> >
> > On Tue, Jun 9, 2015 at 11:39 PM, James Taylor 
> > wrote:
> >
> >> Hi Nishani,
> >> I'd recommend focusing on higher level use cases. From the user's
> >> point of view, they're executing a query and for some reason it's
> >> slower than they expect. How do they figure out why?
> >>
> >> They might first do an EXPLAIN on their query to see how Phoenix is
> >> executing it. Which parts are run where? Are secondary indexes being
> >> used as expected? Are filters being pushed down as expected? A better
> >> way to visualize the explain plan might be a good thing for you to
> >> start with.
> >>
> >> Second, assuming the explain plan looks good, they'll want to turn on
> >> tracing so that they can get runtime information on which parts of
> >> their query are taking the longest.
> >>
> >> Maybe more than one Phoenix table is involved - how will you display
> >> the tracing information across multiple tables for a query that does a
> >> join? Maybe you can punt on this first pass, and focus on single table
> >> queries. A related use case would be a DML statement that's executed
> >> and taking longer than expected. Let's say that the table being
> >> updated has one or more secondary indexes that are also updating the
> >> index tables. Seeing the entire picture of both the table writes plus
> >> the index writes on the same graph would be great.
> >>
> >> For the single-table query user case, what does the distribution of
> >> time look like across all the region servers participating in the
> >> query? Maybe some kind of graph that shows quickly if one region
> >> server is taking much more time than the others. Perhaps that's an
> >> indication that the table statistics need to be re-run, as there may
> >> be skew that's developed such that one of the threads is handling more
> >> data than it should. Or perhaps there's an issue with that particular
> >> region server. Was there something else going on at the same time on
> >> that region server, like a background compaction/split process? If
> >> that information is available in the trace table (not sure), it would
> >> be very cool to be able to superimpose that on top of the query trace
> >> graph.
> >>
> >> Another test might be to run a query over a different table and see if
> >> the same region server shows up again as being slow. So superimposing
> >> the query trace graphs of multiple queries might give the user some
> >> insight.
> >>
> >> IMHO, this is the kind of angle you should come at this from.
> >>
> >> Thanks,
> >> James
> >>
> >> On Mon, Jun 8, 2015 at 4:12 AM, Ayola Jayamaha 
> >> wrote:
> >> > Hi All,
> >> >
> >> > Basically what type of use cases are you expecting or performing at
> the
> >> > moment with regard to tracing? For example these are the use cases I'm
> >> > planing.
> >> > 1. Searching by parent id / trace id / description (regx search)
> >> > 2. Grouping and ordering the tracing information by time period.
> >> > 3. Counting the trace count per day / hour.
> >> > 4. Comparing and distinguishing  two sets of tracing.
> >> > Thanks.
> >> >
> >> >
> >> > On Mon, Jun 8, 2015 at 4:00 PM, Nishani (JIRA) 
> wrote:
> >> >
> >> >>
> >> >>  [
> >> >>
> >>
> https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> >> ]
> >> >>
> >> >> Nishani  updated PHOENIX-1118:
> >> >> --
> >> >> Attachment: Screenshot of dependency tree.png
> >> >>
> >> >> Attaching the dependency tree on tracing.
> >> >> Pull request can be found here.
> >> >> https://github.com/AyolaJayamaha/TracingWebApp/pull/1
> >> >>
> >> >> > Provide a tool for visualizing Phoenix tracing information
> >> >> > --
> >> >> >
> >> >> > Key: PHOENIX-1118
> >> >> > URL:
> >> https://issues.apache.org/jira/browse/PHOENIX-1118
> >> >> > Project: Phoenix
> >> >> >  Issue Type: Sub-task
> >> >> >Reporter: James Taylor
> >> >> >Assignee: Nishani
> >> >> >  Labels: Java, SQL, Visualization, gsoc2015, mentor
> >> >> > Attachments: MockUp1-TimeSlider.png,
> >> Mock

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor]

Excellent, Nishani (and you forgot to say "rambling" :-), but I'm glad
it helped).

On Wed, Jun 10, 2015 at 11:16 AM, Ayola Jayamaha  wrote:
> Hi James,
>
> Thanks a lot for the lengthy and descriptive reply. I am currently looking
> through UI components and charting libraries that can be used for the UI. I
> refered [1] with regard to your explaination and came up with some mock ups
> which I will share soon.
>
> Thanks,
> Nishani
>
> [1] https://phoenix.apache.org/language/#index_hint
> [2]
> https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table
>
> On Tue, Jun 9, 2015 at 11:39 PM, James Taylor 
> wrote:
>
>> Hi Nishani,
>> I'd recommend focusing on higher level use cases. From the user's
>> point of view, they're executing a query and for some reason it's
>> slower than they expect. How do they figure out why?
>>
>> They might first do an EXPLAIN on their query to see how Phoenix is
>> executing it. Which parts are run where? Are secondary indexes being
>> used as expected? Are filters being pushed down as expected? A better
>> way to visualize the explain plan might be a good thing for you to
>> start with.
>>
>> Second, assuming the explain plan looks good, they'll want to turn on
>> tracing so that they can get runtime information on which parts of
>> their query are taking the longest.
>>
>> Maybe more than one Phoenix table is involved - how will you display
>> the tracing information across multiple tables for a query that does a
>> join? Maybe you can punt on this first pass, and focus on single table
>> queries. A related use case would be a DML statement that's executed
>> and taking longer than expected. Let's say that the table being
>> updated has one or more secondary indexes that are also updating the
>> index tables. Seeing the entire picture of both the table writes plus
>> the index writes on the same graph would be great.
>>
>> For the single-table query user case, what does the distribution of
>> time look like across all the region servers participating in the
>> query? Maybe some kind of graph that shows quickly if one region
>> server is taking much more time than the others. Perhaps that's an
>> indication that the table statistics need to be re-run, as there may
>> be skew that's developed such that one of the threads is handling more
>> data than it should. Or perhaps there's an issue with that particular
>> region server. Was there something else going on at the same time on
>> that region server, like a background compaction/split process? If
>> that information is available in the trace table (not sure), it would
>> be very cool to be able to superimpose that on top of the query trace
>> graph.
>>
>> Another test might be to run a query over a different table and see if
>> the same region server shows up again as being slow. So superimposing
>> the query trace graphs of multiple queries might give the user some
>> insight.
>>
>> IMHO, this is the kind of angle you should come at this from.
>>
>> Thanks,
>> James
>>
>> On Mon, Jun 8, 2015 at 4:12 AM, Ayola Jayamaha 
>> wrote:
>> > Hi All,
>> >
>> > Basically what type of use cases are you expecting or performing at the
>> > moment with regard to tracing? For example these are the use cases I'm
>> > planing.
>> > 1. Searching by parent id / trace id / description (regx search)
>> > 2. Grouping and ordering the tracing information by time period.
>> > 3. Counting the trace count per day / hour.
>> > 4. Comparing and distinguishing  two sets of tracing.
>> > Thanks.
>> >
>> >
>> > On Mon, Jun 8, 2015 at 4:00 PM, Nishani (JIRA)  wrote:
>> >
>> >>
>> >>  [
>> >>
>> https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> >> ]
>> >>
>> >> Nishani  updated PHOENIX-1118:
>> >> --
>> >> Attachment: Screenshot of dependency tree.png
>> >>
>> >> Attaching the dependency tree on tracing.
>> >> Pull request can be found here.
>> >> https://github.com/AyolaJayamaha/TracingWebApp/pull/1
>> >>
>> >> > Provide a tool for visualizing Phoenix tracing information
>> >> > --
>> >> >
>> >> > Key: PHOENIX-1118
>> >> > URL:
>> https://issues.apache.org/jira/browse/PHOENIX-1118
>> >> > Project: Phoenix
>> >> >  Issue Type: Sub-task
>> >> >Reporter: James Taylor
>> >> >Assignee: Nishani
>> >> >  Labels: Java, SQL, Visualization, gsoc2015, mentor
>> >> > Attachments: MockUp1-TimeSlider.png,
>> MockUp2-AdvanceSearch.png,
>> >> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of
>> >> dependency tree.png, screenshot of tracing web app.png
>> >> >
>> >> >
>> >> > Currently there's no means of visualizing the trace information
>> provided
>> >> by Phoenix. We should provide some simple charting over our metrics
>> tables.
>> >> Take a look at the following JIRA for sampl

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi James,

Thanks a lot for the lengthy and descriptive reply. I am currently looking
through UI components and charting libraries that can be used for the UI. I
refered [1] with regard to your explaination and came up with some mock ups
which I will share soon.

Thanks,
Nishani

[1] https://phoenix.apache.org/language/#index_hint
[2]
https://phoenix.apache.org/faq.html#How_do_I_create_Secondary_Index_on_a_table

On Tue, Jun 9, 2015 at 11:39 PM, James Taylor 
wrote:

> Hi Nishani,
> I'd recommend focusing on higher level use cases. From the user's
> point of view, they're executing a query and for some reason it's
> slower than they expect. How do they figure out why?
>
> They might first do an EXPLAIN on their query to see how Phoenix is
> executing it. Which parts are run where? Are secondary indexes being
> used as expected? Are filters being pushed down as expected? A better
> way to visualize the explain plan might be a good thing for you to
> start with.
>
> Second, assuming the explain plan looks good, they'll want to turn on
> tracing so that they can get runtime information on which parts of
> their query are taking the longest.
>
> Maybe more than one Phoenix table is involved - how will you display
> the tracing information across multiple tables for a query that does a
> join? Maybe you can punt on this first pass, and focus on single table
> queries. A related use case would be a DML statement that's executed
> and taking longer than expected. Let's say that the table being
> updated has one or more secondary indexes that are also updating the
> index tables. Seeing the entire picture of both the table writes plus
> the index writes on the same graph would be great.
>
> For the single-table query user case, what does the distribution of
> time look like across all the region servers participating in the
> query? Maybe some kind of graph that shows quickly if one region
> server is taking much more time than the others. Perhaps that's an
> indication that the table statistics need to be re-run, as there may
> be skew that's developed such that one of the threads is handling more
> data than it should. Or perhaps there's an issue with that particular
> region server. Was there something else going on at the same time on
> that region server, like a background compaction/split process? If
> that information is available in the trace table (not sure), it would
> be very cool to be able to superimpose that on top of the query trace
> graph.
>
> Another test might be to run a query over a different table and see if
> the same region server shows up again as being slow. So superimposing
> the query trace graphs of multiple queries might give the user some
> insight.
>
> IMHO, this is the kind of angle you should come at this from.
>
> Thanks,
> James
>
> On Mon, Jun 8, 2015 at 4:12 AM, Ayola Jayamaha 
> wrote:
> > Hi All,
> >
> > Basically what type of use cases are you expecting or performing at the
> > moment with regard to tracing? For example these are the use cases I'm
> > planing.
> > 1. Searching by parent id / trace id / description (regx search)
> > 2. Grouping and ordering the tracing information by time period.
> > 3. Counting the trace count per day / hour.
> > 4. Comparing and distinguishing  two sets of tracing.
> > Thanks.
> >
> >
> > On Mon, Jun 8, 2015 at 4:00 PM, Nishani (JIRA)  wrote:
> >
> >>
> >>  [
> >>
> https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> ]
> >>
> >> Nishani  updated PHOENIX-1118:
> >> --
> >> Attachment: Screenshot of dependency tree.png
> >>
> >> Attaching the dependency tree on tracing.
> >> Pull request can be found here.
> >> https://github.com/AyolaJayamaha/TracingWebApp/pull/1
> >>
> >> > Provide a tool for visualizing Phoenix tracing information
> >> > --
> >> >
> >> > Key: PHOENIX-1118
> >> > URL:
> https://issues.apache.org/jira/browse/PHOENIX-1118
> >> > Project: Phoenix
> >> >  Issue Type: Sub-task
> >> >Reporter: James Taylor
> >> >Assignee: Nishani
> >> >  Labels: Java, SQL, Visualization, gsoc2015, mentor
> >> > Attachments: MockUp1-TimeSlider.png,
> MockUp2-AdvanceSearch.png,
> >> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of
> >> dependency tree.png, screenshot of tracing web app.png
> >> >
> >> >
> >> > Currently there's no means of visualizing the trace information
> provided
> >> by Phoenix. We should provide some simple charting over our metrics
> tables.
> >> Take a look at the following JIRA for sample queries:
> >>
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
> >>
> >>
> >>
> >> --
> >> This message was sent by Atlassian JIRA
> >> (v6.3.4#6332)

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor]

Hi Nishani,
I'd recommend focusing on higher level use cases. From the user's
point of view, they're executing a query and for some reason it's
slower than they expect. How do they figure out why?

They might first do an EXPLAIN on their query to see how Phoenix is
executing it. Which parts are run where? Are secondary indexes being
used as expected? Are filters being pushed down as expected? A better
way to visualize the explain plan might be a good thing for you to
start with.

Second, assuming the explain plan looks good, they'll want to turn on
tracing so that they can get runtime information on which parts of
their query are taking the longest.

Maybe more than one Phoenix table is involved - how will you display
the tracing information across multiple tables for a query that does a
join? Maybe you can punt on this first pass, and focus on single table
queries. A related use case would be a DML statement that's executed
and taking longer than expected. Let's say that the table being
updated has one or more secondary indexes that are also updating the
index tables. Seeing the entire picture of both the table writes plus
the index writes on the same graph would be great.

For the single-table query user case, what does the distribution of
time look like across all the region servers participating in the
query? Maybe some kind of graph that shows quickly if one region
server is taking much more time than the others. Perhaps that's an
indication that the table statistics need to be re-run, as there may
be skew that's developed such that one of the threads is handling more
data than it should. Or perhaps there's an issue with that particular
region server. Was there something else going on at the same time on
that region server, like a background compaction/split process? If
that information is available in the trace table (not sure), it would
be very cool to be able to superimpose that on top of the query trace
graph.

Another test might be to run a query over a different table and see if
the same region server shows up again as being slow. So superimposing
the query trace graphs of multiple queries might give the user some
insight.

IMHO, this is the kind of angle you should come at this from.

Thanks,
James

On Mon, Jun 8, 2015 at 4:12 AM, Ayola Jayamaha  wrote:
> Hi All,
>
> Basically what type of use cases are you expecting or performing at the
> moment with regard to tracing? For example these are the use cases I'm
> planing.
> 1. Searching by parent id / trace id / description (regx search)
> 2. Grouping and ordering the tracing information by time period.
> 3. Counting the trace count per day / hour.
> 4. Comparing and distinguishing  two sets of tracing.
> Thanks.
>
>
> On Mon, Jun 8, 2015 at 4:00 PM, Nishani (JIRA)  wrote:
>
>>
>>  [
>> https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> ]
>>
>> Nishani  updated PHOENIX-1118:
>> --
>> Attachment: Screenshot of dependency tree.png
>>
>> Attaching the dependency tree on tracing.
>> Pull request can be found here.
>> https://github.com/AyolaJayamaha/TracingWebApp/pull/1
>>
>> > Provide a tool for visualizing Phoenix tracing information
>> > --
>> >
>> > Key: PHOENIX-1118
>> > URL: https://issues.apache.org/jira/browse/PHOENIX-1118
>> > Project: Phoenix
>> >  Issue Type: Sub-task
>> >Reporter: James Taylor
>> >Assignee: Nishani
>> >  Labels: Java, SQL, Visualization, gsoc2015, mentor
>> > Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png,
>> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of
>> dependency tree.png, screenshot of tracing web app.png
>> >
>> >
>> > Currently there's no means of visualizing the trace information provided
>> by Phoenix. We should provide some simple charting over our metrics tables.
>> Take a look at the following JIRA for sample queries:
>> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)
>>
>
>
>
> --
> Best Regards,
> Nishani Jayamaha
> http://ayolajayamaha.blogspot.com/

Re: [jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Ayola Jayamaha]

Hi All,

Basically what type of use cases are you expecting or performing at the
moment with regard to tracing? For example these are the use cases I'm
planing.
1. Searching by parent id / trace id / description (regx search)
2. Grouping and ordering the tracing information by time period.
3. Counting the trace count per day / hour.
4. Comparing and distinguishing  two sets of tracing.
Thanks.


On Mon, Jun 8, 2015 at 4:00 PM, Nishani (JIRA)  wrote:

>
>  [
> https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Nishani  updated PHOENIX-1118:
> --
> Attachment: Screenshot of dependency tree.png
>
> Attaching the dependency tree on tracing.
> Pull request can be found here.
> https://github.com/AyolaJayamaha/TracingWebApp/pull/1
>
> > Provide a tool for visualizing Phoenix tracing information
> > --
> >
> > Key: PHOENIX-1118
> > URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> > Project: Phoenix
> >  Issue Type: Sub-task
> >Reporter: James Taylor
> >Assignee: Nishani
> >  Labels: Java, SQL, Visualization, gsoc2015, mentor
> > Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png,
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of
> dependency tree.png, screenshot of tracing web app.png
> >
> >
> > Currently there's no means of visualizing the trace information provided
> by Phoenix. We should provide some simple charting over our metrics tables.
> Take a look at the following JIRA for sample queries:
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>



-- 
Best Regards,
Nishani Jayamaha
http://ayolajayamaha.blogspot.com/

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: Screenshot of dependency tree.png

Attaching the dependency tree on tracing.
Pull request can be found here.
https://github.com/AyolaJayamaha/TracingWebApp/pull/1

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, Screenshot of dependency 
> tree.png, screenshot of tracing web app.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: screenshot of tracing web app.png

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png, screenshot of tracing 
> web app.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[Nishani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1118:
--
Attachment: MockUp4-FlameGraph.png
MockUp3-PatternDetector.png
MockUp2-AdvanceSearch.png
MockUp1-TimeSlider.png

Attaching the Mock Up UIs for the Dashboard of Phoenix Tracing Information.

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
> Attachments: MockUp1-TimeSlider.png, MockUp2-AdvanceSearch.png, 
> MockUp3-PatternDetector.png, MockUp4-FlameGraph.png
>
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-1118:
--
Assignee: Nishani 

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-1118:
--
Labels: Java SQL Visualization gsoc2015 mentor  (was: )

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>  Labels: Java, SQL, Visualization, gsoc2015, mentor
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-1118) Provide a tool for visualizing Phoenix tracing information

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-1118:
--
Summary: Provide a tool for visualizing Phoenix tracing information  (was: 
Provide a simple charting facility on top of Phoenix tracing metrics table)

> Provide a tool for visualizing Phoenix tracing information
> --
>
> Key: PHOENIX-1118
> URL: https://issues.apache.org/jira/browse/PHOENIX-1118
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>
> Currently there's no means of visualizing the trace information provided by 
> Phoenix. We should provide some simple charting over our metrics tables. Take 
> a look at the following JIRA for sample queries: 
> https://issues.apache.org/jira/browse/PHOENIX-1115?focusedCommentId=14323151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14323151



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)