The question now would be can it be done in streaming fashion? Are you talking about the union of two streaming dataframes and then constructing a graphframe (also during streaming) ?
On Sat, Jul 14, 2018 at 8:07 AM, Jörn Franke <jornfra...@gmail.com> wrote: > For your use case one might indeed be able to work simply with incremental > graph updates. However they are not straight forward in Spark. You can > union the new Data with the existing dataframes that represent your graph > and create from that a new graph frame. > > However I am not sure if this will fully fulfill your requirement for > incremental graph updates. > > On 14. Jul 2018, at 15:59, kant kodali <kanth...@gmail.com> wrote: > > "You want to update incrementally an existing graph and run incrementally > a graph algorithm suitable for this - you have to implement yourself as > far as I am aware" > > I want to update the graph incrementally and want to run some graph > queries similar to Cypher like give me all the vertices that are connected > by a specific set of edges and so on. Don't really intend to run graph > algorithms like ConnectedComponents or anything else at this point but of > course, it's great to have. > > If we were to do this myself should I extend the GraphFrame? any > suggestions? > > > On Sun, Apr 29, 2018 at 3:24 AM, Jörn Franke <jornfra...@gmail.com> wrote: > >> What is the use case you are trying to solve? >> You want to load graph data from a streaming window in separate graphs - >> possible but requires probably a lot of memory. >> You want to update an existing graph with new streaming data and then >> fully rerun an algorithms -> look at Janusgraph >> You want to update incrementally an existing graph and run incrementally >> a graph algorithm suitable for this - you have to implement yourself as far >> as I am aware >> >> > On 29. Apr 2018, at 11:43, kant kodali <kanth...@gmail.com> wrote: >> > >> > Do GraphFrames support streaming? >> > >
