Re: InMemoryVertex Format(s)

2015-06-04 Thread Sergey Edunov
Hi Khaled, Unfortunately, there is no official way to read data between supersteps right now. You can read all the data in the beginning and just keep it in memory. And you can write output between supersteps. But reading between supersteps is not supported yet. It has been in our TODO list for a

Re: InMemoryVertex Format(s)

2015-06-04 Thread Khaled Ammar
Hi Sergey, Thank you for your clarification. I noticed Igor email the other day, sounds exciting. I am looking forward to it. *One more question please*, What is the best approach to load data between supersteps? I understand that it is possible to create a FileSystem object, open a file, read

Re: InMemoryVertex Format(s)

2015-06-03 Thread Khaled Ammar
Thank you Sergey, This is exactly what I am looking for. I would like to run multiple computation classes following each other, such that each computation class will execute until convergence. I think may be the GiraphJob class may help. Can I use setComputationClass to change the computation

Re: InMemoryVertex Format(s)

2015-06-03 Thread Sergey Edunov
Hi Khaled, As far as I know, InMemory input and output formats are only used in test cases. Can you elaborate more on why do you want to use I/O formats? You can use different computation classes within one application and you don't need to do I/O between them. All intermediate results can be

InMemoryVertex Format(s)

2015-06-02 Thread Khaled Ammar
Hi all, There are InMemory input and output format for giraph. These could be useful when a specific computation should be executed until convergence and then another computation is needed. Instead of writing intermediate results to HDFS and read it again, InMemoryVertex format sounds very