Re: Introducing Graft: A debugging and testing tool for Giraph algorithms

[Semih Salihoglu]

Hi Eugene,

These are very interesting ideas. We designed Graft with the assumption
that users would visualize only a very very small part of the graph (e.g. a
single vertex or a single vertex and its neighbors). This has a few
advantages: (1) Because very few number of vertices are logged Graft's
overhead is very small. (2) Visualization is simple. In order to become a
complete GUI for large-graphs, we'd need to figure out how users would want
to visualize large-graphs. We think that's a very interesting question but
we don't know the answer. Probably not in a node-link format as even at a
scale of a few hundred vertices, meaningful visualization in a node-link
graph becomes very difficult.

Can you elaborate on what you mean by the logging being terse?

Best,

semih

Re: Introducing Graft: A debugging and testing tool for Giraph algorithms

[Eugene Koontz]

The docs on github.comI noticed that it's logging stuff like the 
adjacency list; that's useful. I think there's a lot  of potential to 
go beyond being just a debugger and could become a complete GUI for 
Giraph itself.


Sorry, I meant to say that the docs on 
https://github.com/semihsalihoglu/graft/wiki/ are good, and the logging 
is good also, though a bit terse, more logging would be helpful to 
understand how the server is reacting to user input.


-Eugene

Re: Introducing Graft: A debugging and testing tool for Giraph algorithms

[Eugene Koontz]

Graft looks really cool! Initially I just got it running and played 
around with the GUI. I am looking forward to going through 
https://github.com/semihsalihoglu/graft/wiki/Advanced-Examples, but 
haven't got that far yet.


The docs on github.comI noticed that it's logging stuff like the 
adjacency list; that's useful. I think there's a lot  of potential to go 
beyond being just a debugger and could become a complete GUI for Giraph 
itself.


Imagine being able to browse graphs in HDFS through a GUI, run 
computations on them, share results via hrefs to other people, etc..


-Eugene


On 6/6/14, 1:51 PM, Vijayakumar Ramdoss wrote:

Thanks Semih, I will give a try

Sent from my iPhone

On Jun 5, 2014, at 11:38 AM, Eugene Koontz > wrote:



Interesting; thanks Semih! I will try Graft out over the weekend.



On Jun 4, 2014, at 11:00 AM, Semih Salihoglu > wrote:



Hi Giraph Users,

I wanted to introduce to you Graft, a project that some of us at 
Stanford have built over the last quarter. If you are a Giraph user, 
who ran into an annoying bug in which the code was throwing an 
exception or resulting in incorrect looking  messages or vertex 
values (e.g. NaNs or NullPointerExceptions) and you had put in 
println statements into your compute() functions, and then inspect 
logs of Hadoop workers for debugging purposes, you should read on. 
You might find Graft very useful.


In a nutshell, Graft is based on the idea of /capturing /the 
contexts under which a bug becomes noticeable (an exception is 
thrown or an incorrect message is sent, or a vertex is assigned an 
incorrect value) programmatically. The captured contexts can then be 
/visualized/ through a GUI. The contexts that a user thinks could be 
helpful for catching the bug can then be /reproduced/ in a 
compilable program and the user can then use his/her favorite IDE's 
debugger to do step-by-step debugging into the context. For example, 
when a vertex /v/ throws an exception, the user can reproduce the 
context under which /v/ throws the exception and then use (say) 
Eclipse to do step-by-step debugging to see exactly what lines were 
executed that resulted in the exception being thrown.


On the testing side, Graft makes it easier to generate unit and 
end-to-end tests by letting users curate small graphs through its 
GUI's testing mode, and then generates code snippets which can be 
copied and pasted into a JUnit test.


The project is still under development but interested users can 
start using it. We have a wiki with documentation and instructions 
on how to install and use Graft: 
https://github.com/semihsalihoglu/graft/wiki. Since the project is 
under development, we'd highly appreciate users to start using it 
and giving us direction on how to make it more useful. Our emails 
are on the documentation page. We also encourage interested 
developers to contribute to it if there are requested features that 
we don't get to very quickly.


Just a small note: Graft works for the Giraph at trunk: 
https://github.com/apache/giraph/tree/trunk. We do not support 
earlier version. In particular your programs need to be written by 
extending Computation and optionally the Master class, instead of 
the older Vertex class.


Best,

semih

Re: Introducing Graft: A debugging and testing tool for Giraph algorithms

[Vijayakumar Ramdoss]

Thanks Semih, I will give a try

Sent from my iPhone

> On Jun 5, 2014, at 11:38 AM, Eugene Koontz  wrote:
> 
> Interesting; thanks Semih! I will try Graft out over the weekend.
> 
> 
> 
>> On Jun 4, 2014, at 11:00 AM, Semih Salihoglu  wrote:
>> 
>> Hi Giraph Users,
>> 
>> I wanted to introduce to you Graft, a project that some of us at Stanford 
>> have built over the last quarter. If you are a Giraph user, who ran into an 
>> annoying bug in which the code was throwing an exception or resulting in 
>> incorrect looking messages or vertex values (e.g. NaNs or 
>> NullPointerExceptions) and you had put in println statements into your 
>> compute() functions, and then inspect logs of Hadoop workers for debugging 
>> purposes, you should read on. You might find Graft very useful.
>> 
>> In a nutshell, Graft is based on the idea of capturing the contexts under 
>> which a bug becomes noticeable (an exception is thrown or an incorrect 
>> message is sent, or a vertex is assigned an incorrect value) 
>> programmatically. The captured contexts can then be visualized through a 
>> GUI. The contexts that a user thinks could be helpful for catching the bug 
>> can then be reproduced in a compilable program and the user can then use 
>> his/her favorite IDE's debugger to do step-by-step debugging into the 
>> context. For example, when a vertex v throws an exception, the user can 
>> reproduce the context under which v throws the exception and then use (say) 
>> Eclipse to do step-by-step debugging to see exactly what lines were executed 
>> that resulted in the exception being thrown.
>> 
>> On the testing side, Graft makes it easier to generate unit and end-to-end 
>> tests by letting users curate small graphs through its GUI's testing mode, 
>> and then generates code snippets which can be copied and pasted into a JUnit 
>> test. 
>> 
>> The project is still under development but interested users can start using 
>> it. We have a wiki with documentation and instructions on how to install and 
>> use Graft: https://github.com/semihsalihoglu/graft/wiki. Since the project 
>> is under development, we'd highly appreciate users to start using it and 
>> giving us direction on how to make it more useful. Our emails are on the 
>> documentation page. We also encourage interested developers to contribute to 
>> it if there are requested features that we don't get to very quickly. 
>> 
>> Just a small note: Graft works for the Giraph at trunk: 
>> https://github.com/apache/giraph/tree/trunk. We do not support earlier 
>> version. In particular your programs need to be written by extending 
>> Computation and optionally the Master class, instead of the older Vertex 
>> class.
>> 
>> Best,
>> 
>> semih

Re: Introducing Graft: A debugging and testing tool for Giraph algorithms

[Eugene Koontz]

Interesting; thanks Semih! I will try Graft out over the weekend.



> On Jun 4, 2014, at 11:00 AM, Semih Salihoglu  wrote:
> 
> Hi Giraph Users,
> 
> I wanted to introduce to you Graft, a project that some of us at Stanford 
> have built over the last quarter. If you are a Giraph user, who ran into an 
> annoying bug in which the code was throwing an exception or resulting in 
> incorrect looking messages or vertex values (e.g. NaNs or 
> NullPointerExceptions) and you had put in println statements into your 
> compute() functions, and then inspect logs of Hadoop workers for debugging 
> purposes, you should read on. You might find Graft very useful.
> 
> In a nutshell, Graft is based on the idea of capturing the contexts under 
> which a bug becomes noticeable (an exception is thrown or an incorrect 
> message is sent, or a vertex is assigned an incorrect value) 
> programmatically. The captured contexts can then be visualized through a GUI. 
> The contexts that a user thinks could be helpful for catching the bug can 
> then be reproduced in a compilable program and the user can then use his/her 
> favorite IDE's debugger to do step-by-step debugging into the context. For 
> example, when a vertex v throws an exception, the user can reproduce the 
> context under which v throws the exception and then use (say) Eclipse to do 
> step-by-step debugging to see exactly what lines were executed that resulted 
> in the exception being thrown.
> 
> On the testing side, Graft makes it easier to generate unit and end-to-end 
> tests by letting users curate small graphs through its GUI's testing mode, 
> and then generates code snippets which can be copied and pasted into a JUnit 
> test. 
> 
> The project is still under development but interested users can start using 
> it. We have a wiki with documentation and instructions on how to install and 
> use Graft: https://github.com/semihsalihoglu/graft/wiki. Since the project is 
> under development, we'd highly appreciate users to start using it and giving 
> us direction on how to make it more useful. Our emails are on the 
> documentation page. We also encourage interested developers to contribute to 
> it if there are requested features that we don't get to very quickly. 
> 
> Just a small note: Graft works for the Giraph at trunk: 
> https://github.com/apache/giraph/tree/trunk. We do not support earlier 
> version. In particular your programs need to be written by extending 
> Computation and optionally the Master class, instead of the older Vertex 
> class.
> 
> Best,
> 
> semih

Re: Introducing Graft: A debugging and testing tool for Giraph algorithms

[Avery Ching]

I'm seen this work demoed.  It's awesome, especially for applications 
that are not very predictable.


Avery

On 6/4/14, 11:00 AM, Semih Salihoglu wrote:


Hi Giraph Users,

I wanted to introduce to you Graft, a project that some of us at 
Stanford have built over the last quarter. If you are a Giraph user, 
who ran into an annoying bug in which the code was throwing an 
exception or resulting in incorrect looking messages or vertex values 
(e.g. NaNs or NullPointerExceptions) and you had put in println 
statements into your compute() functions, and then inspect logs of 
Hadoop workers for debugging purposes, you should read on. You might 
find Graft very useful.


In a nutshell, Graft is based on the idea of /capturing /the contexts 
under which a bug becomes noticeable (an exception is thrown or an 
incorrect message is sent, or a vertex is assigned an incorrect value) 
programmatically. The captured contexts can then be /visualized/ 
through a GUI. The contexts that a user thinks could be helpful for 
catching the bug can then be /reproduced/ in a compilable program and 
the user can then use his/her favorite IDE's debugger to do 
step-by-step debugging into the context. For example, when a vertex 
/v/ throws an exception, the user can reproduce the context under 
which /v/ throws the exception and then use (say) Eclipse to do 
step-by-step debugging to see exactly what lines were executed that 
resulted in the exception being thrown.


On the testing side, Graft makes it easier to generate unit and 
end-to-end tests by letting users curate small graphs through its 
GUI's testing mode, and then generates code snippets which can be 
copied and pasted into a JUnit test.


The project is still under development but interested users can start 
using it. We have a wiki with documentation and instructions on how to 
install and use Graft: https://github.com/semihsalihoglu/graft/wiki. 
Since the project is under development, we'd highly appreciate users 
to start using it and giving us direction on how to make it more 
useful. Our emails are on the documentation page. We also encourage 
interested developers to contribute to it if there are requested 
features that we don't get to very quickly.


Just a small note: Graft works for the Giraph at trunk: 
https://github.com/apache/giraph/tree/trunk. We do not support earlier 
version. In particular your programs need to be written by extending 
Computation and optionally the Master class, instead of the older 
Vertex class.


Best,

semih

Introducing Graft: A debugging and testing tool for Giraph algorithms

[Semih Salihoglu]

Hi Giraph Users,

I wanted to introduce to you Graft, a project that some of us at Stanford
have built over the last quarter. If you are a Giraph user, who ran into an
annoying bug in which the code was throwing an exception or resulting in
incorrect looking messages or vertex values (e.g. NaNs or
NullPointerExceptions) and you had put in println statements into your
compute() functions, and then inspect logs of Hadoop workers for debugging
purposes, you should read on. You might find Graft very useful.

In a nutshell, Graft is based on the idea of *capturing *the contexts under
which a bug becomes noticeable (an exception is thrown or an incorrect
message is sent, or a vertex is assigned an incorrect value)
programmatically. The captured contexts can then be *visualized* through a
GUI. The contexts that a user thinks could be helpful for catching the bug
can then be *reproduced* in a compilable program and the user can then use
his/her favorite IDE's debugger to do step-by-step debugging into the
context. For example, when a vertex *v* throws an exception, the user can
reproduce the context under which *v* throws the exception and then use
(say) Eclipse to do step-by-step debugging to see exactly what lines were
executed that resulted in the exception being thrown.

On the testing side, Graft makes it easier to generate unit and end-to-end
tests by letting users curate small graphs through its GUI's testing mode,
and then generates code snippets which can be copied and pasted into a
JUnit test.

The project is still under development but interested users can start using
it. We have a wiki with documentation and instructions on how to install
and use Graft: https://github.com/semihsalihoglu/graft/wiki. Since the
project is under development, we'd highly appreciate users to start using
it and giving us direction on how to make it more useful. Our emails are on
the documentation page. We also encourage interested developers to
contribute to it if there are requested features that we don't get to very
quickly.

Just a small note: Graft works for the Giraph at trunk:
https://github.com/apache/giraph/tree/trunk. We do not support earlier
version. In particular your programs need to be written by extending
Computation and optionally the Master class, instead of the older Vertex
class.

Best,

semih