[
https://issues.apache.org/jira/browse/OOZIE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168306#comment-16168306
]
Robert Kanter commented on OOZIE-2406:
--------------------------------------
I left some specific comments on RB. A few additional things:
# I think we can safely remove Jung and the Commons-Collections fork given that
you got the graphviz version. It still produces an image, and I think it's
reasonable to assume nobody is somehow parsing those images, so it's okay if we
change the look of the images by using a different library. One of the goals
of this JIRA was to remove the jung library and the Commons-Collections fork
because they're super old, abandoned, and potentially security liabilities.
# Have you done any performance testing on the graphviz library? The Jung
library was slow and ate up a ton of memory, which is why we limited the number
of nodes it could process. If graphviz is way better, we could increase this
number to something higher; though I think we should still have _some_ limit
just to be safe. Another thing to check is what happens if Oozie gets a lot of
requests for images at the same time - can the graphviz library handle that
well?
# Can we make the "end" circle a darker color? It's sort of hard to see in the
image you attached.
# I just thought of an interesting idea: it might be a good idea to create a
cache for the generated images. Some kind of LRU cache in memory and/or just
writing them to the local disk. We'd only do this for completed jobs (because
the images will change as the job runs). This is more of a nice to have, which
may or may not even be necessary, and could be a lot of work, so I think this
should be a separate JIRA. Can you file a JIRA to look into this idea?
# It might be a neat improvement to add a way to return the dot file (I assume
there's a way to do that from the graphviz library?). We have to keep the
current API returning an image for compatibility
({{job-3?show=graph\[&show-kill=true\]}}), but we could add a new parameter
like {{{{job-3?show=graph\[&show-kill=true\]\[&format=dot\]}} (the default
being {{&format=png}}). This would allow the caller to use the dot file in
various ways (for example, perhaps Hue might use this).
# Can you file a new JIRA for adding support for getting the image (whether
that's png or dot format, or both) to the Oozie CLI? It's currently only
supported in the REST API.
> Completely rewrite GraphGenerator code
> --------------------------------------
>
> Key: OOZIE-2406
> URL: https://issues.apache.org/jira/browse/OOZIE-2406
> Project: Oozie
> Issue Type: New Feature
> Affects Versions: 4.2.0
> Reporter: Robert Kanter
> Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: graphviz-many-nodes.png, jung-many-nodes.png,
> OOZIE-2406.004.patch, OOZIE-2406.004.patch, OOZIE-2406.005.patch
>
>
> The Web UI currently generates a graph of the workflow DAG as a png image
> that it can show to the user ({{GraphGenerator}} class). Unfortunately,
> there are a number of downsides to the current implementation:
> # The image is generated server-side, which doesn't scale well and eats up
> lots of memory
> #- To help combat this issue, we only generate graphs for workflows with less
> than 25 nodes and we disabled the refresh button in the UI
> # It's slow
> # It uses a library ({{net.sf.jung}} from http://jung.sourceforge.net), which
> hasn't been updated since 2010.
> #- This library also has a dependency on a fork of Commons-Collections
> ({{net.sourceforge.collections}} from
> http://sourceforge.net/projects/collections/), which similarly hasn't been
> updated since 2010. This is a problem because we can't update
> Commons-Collections when there are security concerns such as COLLECTIONS-580
> (though Oozie is *not* susceptible to this attack).
> It would be good to do a complete rewrite, using a different library and
> remove Jung and the Commons-Collections fork. Whatever we choose, it should
> an svg which will draw the image on the browser.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
