Re: Auto-Organize Layout

[Elli Schwarz]

Joe,
What I'm referring to as far as emphasizing "important" processors is that 
there are many places in my workflow where I do a PutFile for logging purposes, 
and I have one for success and a separate one for failure. So for many of my 
main processors, there are two connectors to PutFiles. This makes the workflow 
look very cluttered, and when doing a demo it is difficult to see the main path 
that flowfiles are taking without explanation, when I'd like it to be 
intuitive. In fact, there are some cases that I really want to add PutFiles for 
logging but since it will make the workflow look so cluttered I don't do it. 
Maybe this problem can be solved by an option to route all failures or even 
successes from some or all processors to do a PutFile to a specific folder? 
Maybe we can minimize some processor such as these PutFiles to a small icon 
instead of the whole box to save screen space? I often don't care about 
processor stats for these processors anyway, but maybe they'd be displayed on 
mouse hover.

Furthermore, in some of my workflows I have my main tasks that deal with the 
actual processing of the incoming data, and other "side tasks" that I do with 
the data like collect special metrics,  storing some metadata in a special 
database, sending status emails, etc. I handle this now with Processor Groups, 
and that helps, but I find it a bit unwieldy to create many processor groups 
that only have two or three processors in them (and then the processor group 
needs the input/output ports, further complicated an otherwise simple 
workflow). There are also cases where for some failures, I have a ControlRate 
processor and then retry the flowfile after a certain period of time - this is 
not a "main" part of the workflow, but it clutters it and it's not intuitive to 
see what's happening. I think I'd like to solve this problem by just being able 
to align selected processors that I consider more important, and having the 
others off to a side.

As far as accidentally dragging processors is concerned, sometimes I intend to 
the pan screen and end up moving a processor or moving a label that I used to 
highlight a section of my flow. For demos, it would be nice to lock the entire 
page layout. Maybe this can be accomplished with a button on the top right 
corner to enable/disable layout changes or disable adding new processors, and 
only allow viewing processor properties - a sort of "read only" mode.
Thanks for taking my feedback into consideration. I still find Nifi incredibly 
useful for handling my complicated workflows and appreciate your work in 
developing it!
-Elli



 

On Tuesday, January 19, 2016 5:03 PM, Joe Witt  wrote:
 
 

 Elli

"it is sometimes too easy to mis-align processors by dragging them accidentally"
  Great point  I must admit I too do that.  Often in really important
demos.  I've gotten good at making jokes about it.  Probably should
have gotten good at submitting a JIRA :-)

I'd like to understand more about your other idea for emphasizing
processors which are more important.  I can understand the idea I
think but I'm worried about how we could make the user experience
worth the effort for the person signaling the emphasis to be of use
for the people consuming that detail.

Thanks
Joe

On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess  wrote:
> +1 for "snap to grid" feature
>
> Sent from my iPhone
>
>> On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
>>
>> Maybe not exactly "auto-layout" but I would back a notion of having the
>> components snap to a coarser grain grid than what we currently have.
>> Sometimes I care a lot about having everything line up in the graph
>> horizontally and vertically, and it always takes a long time to achieve
>> this.
>>
>> I could see this being achieved by snapping the component to the same spot
>> horizontally as the component above it when you move it underneath another
>> component.  Or some magical "auto snap" button that does its best to align
>> everything with its nearest neighbors.
>>
>>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:
>>>
>>> I like your idea Rob, that would help with lining up relationships too
>>> (straight lines).
>>>
>>> On Matt's note, I don't think there should be a "standard" either, although
>>> best practices are always out there.
>>>
>>> On Matt's note of putting failures up above processes, we do that too.
>>> Totally depends on who made the flow first.  Sometimes, people don't even
>>> follow a convention in the same flow.xml file.
>>>
>>> For these reasons, I'd recommend alternate views to the flow.
>>>
>>> We have a couple projects that just allow you to rearrange a node-based
>>> graph, based on your preference, hierarchy, circular, pyramid, etc.
>>>
>>> Applying this to NiFi, having a couple different default auto-layout
>>> options that you can swap your current view to, but NOT change the original
>>> flow, 

Re: Auto-Organize Layout

[Elli Schwarz]

Joe, 

Responses to your bullets: 1) I wasn't aware of the details of how provenance 
works. After reading about it, it seems quite powerful and it may help for some 
cases. I especially like the ability to replay files and track timing of 
events. However, there are some problems I think I've experienced in regards to 
provenance: we have thousands of flowfiles going through daily. We like to keep 
the flowfiles from certain processors for several days, but if we keep them 
from all processors we will quickly run out of disk space. When I first 
upgraded to 0.3.0, it had provenance on by default (I think that was a bug, 
since in 0.2.0 it was off by default) and I quickly and unexpectedly ran out of 
disk space on my production server. Furthermore, there are many cases where we 
store a collection of flowfile content in a directory and we grep looking for 
certain strings to debug something - can you do something similar to a grep on 
content through provenance? Can you select which processors you want to store 
flow files for so not to fill up the disk with flowfile content from processors 
we don't care as much about? If the answer to these questions could be yes, 
then I think provenance would work for the kind of tracking I've done with 
PutFile and LogAttributes.
2) For emphasizing the main parts of a flow, I think the ability to align 
selected processors would help. I would select the main processors, line them 
up horizontally, and maybe it would automatically move other processors above 
or below. Also, maybe I didn't fully explain my processor minimization idea - I 
would like an option on a processor to "show icon only" so instead of a full 
processor box with all the stats and details, it would look something like a 
Processor Group port looks - no stats, just text and some icons indicating 
started/stopped/disabled. (It should still look differently than a port does so 
no one gets confused).
Of course, I look forward to hearing what others have to say about these ideas!
-Elli

On Wednesday, January 20, 2016 4:49 PM, Joe Witt  wrote:
 
 

 Elli,

Ok great feedback.  I'll categorize these in the following ways (if
you disagree please share)

1) How best to handle log/debug sort of flows

One thing to consider is that with our provenance and content archival
it eliminated the need for many uses of
LogAttribute and for PutFile for these cases.  Something to consider.

2) How best to de-emphasize 'special case' handling of flows or parts
of a flow which are necessary but not the primary logic thread

I 'see' the idea but not sure what a good user experience for it would
be.  Anyone have visuals/UX concept in mind for this?

3) Modes of edit / Lock-out

Makes sense.  This has been asked for before.  Basically allow the
user to express that they want to go into an edit mode.  How do others
feel?

Thanks
Joe

On Wed, Jan 20, 2016 at 4:00 PM, Elli Schwarz
 wrote:
> Joe,
> What I'm referring to as far as emphasizing "important" processors is that 
> there are many places in my workflow where I do a PutFile for logging 
> purposes, and I have one for success and a separate one for failure. So for 
> many of my main processors, there are two connectors to PutFiles. This makes 
> the workflow look very cluttered, and when doing a demo it is difficult to 
> see the main path that flowfiles are taking without explanation, when I'd 
> like it to be intuitive. In fact, there are some cases that I really want to 
> add PutFiles for logging but since it will make the workflow look so 
> cluttered I don't do it. Maybe this problem can be solved by an option to 
> route all failures or even successes from some or all processors to do a 
> PutFile to a specific folder? Maybe we can minimize some processor such as 
> these PutFiles to a small icon instead of the whole box to save screen space? 
> I often don't care about processor stats for these processors anyway, but 
> maybe they'd be displayed on mouse hover.
>
> Furthermore, in some of my workflows I have my main tasks that deal with the 
> actual processing of the incoming data, and other "side tasks" that I do with 
> the data like collect special metrics,  storing some metadata in a special 
> database, sending status emails, etc. I handle this now with Processor 
> Groups, and that helps, but I find it a bit unwieldy to create many processor 
> groups that only have two or three processors in them (and then the processor 
> group needs the input/output ports, further complicated an otherwise simple 
> workflow). There are also cases where for some failures, I have a ControlRate 
> processor and then retry the flowfile after a certain period of time - this 
> is not a "main" part of the workflow, but it clutters it and it's not 
> intuitive to see what's happening. I think I'd like to solve this problem by 
> just being able to align selected processors that I consider more important, 
> and 

Re: Auto-Organize Layout

[Mike Drob]

Re: Emphasising important processors, is there a resize feature to make
some boxes on the canvas bigger/smaller than others? Then the main flow
could be "follow the big boxes"

Or! Allow them to auto-size based on volume. That would be neat.

On Wed, Jan 20, 2016 at 3:49 PM, Joe Witt  wrote:

> Elli,
>
> Ok great feedback.  I'll categorize these in the following ways (if
> you disagree please share)
>
> 1) How best to handle log/debug sort of flows
>
> One thing to consider is that with our provenance and content archival
> it eliminated the need for many uses of
> LogAttribute and for PutFile for these cases.  Something to consider.
>
> 2) How best to de-emphasize 'special case' handling of flows or parts
> of a flow which are necessary but not the primary logic thread
>
> I 'see' the idea but not sure what a good user experience for it would
> be.  Anyone have visuals/UX concept in mind for this?
>
> 3) Modes of edit / Lock-out
>
> Makes sense.  This has been asked for before.  Basically allow the
> user to express that they want to go into an edit mode.  How do others
> feel?
>
> Thanks
> Joe
>
> On Wed, Jan 20, 2016 at 4:00 PM, Elli Schwarz
>  wrote:
> > Joe,
> > What I'm referring to as far as emphasizing "important" processors is
> that there are many places in my workflow where I do a PutFile for logging
> purposes, and I have one for success and a separate one for failure. So for
> many of my main processors, there are two connectors to PutFiles. This
> makes the workflow look very cluttered, and when doing a demo it is
> difficult to see the main path that flowfiles are taking without
> explanation, when I'd like it to be intuitive. In fact, there are some
> cases that I really want to add PutFiles for logging but since it will make
> the workflow look so cluttered I don't do it. Maybe this problem can be
> solved by an option to route all failures or even successes from some or
> all processors to do a PutFile to a specific folder? Maybe we can minimize
> some processor such as these PutFiles to a small icon instead of the whole
> box to save screen space? I often don't care about processor stats for
> these processors anyway, but maybe they'd be displayed on mouse hover.
> >
> > Furthermore, in some of my workflows I have my main tasks that deal with
> the actual processing of the incoming data, and other "side tasks" that I
> do with the data like collect special metrics,  storing some metadata in a
> special database, sending status emails, etc. I handle this now with
> Processor Groups, and that helps, but I find it a bit unwieldy to create
> many processor groups that only have two or three processors in them (and
> then the processor group needs the input/output ports, further complicated
> an otherwise simple workflow). There are also cases where for some
> failures, I have a ControlRate processor and then retry the flowfile after
> a certain period of time - this is not a "main" part of the workflow, but
> it clutters it and it's not intuitive to see what's happening. I think I'd
> like to solve this problem by just being able to align selected processors
> that I consider more important, and having the others off to a side.
> >
> > As far as accidentally dragging processors is concerned, sometimes I
> intend to the pan screen and end up moving a processor or moving a label
> that I used to highlight a section of my flow. For demos, it would be nice
> to lock the entire page layout. Maybe this can be accomplished with a
> button on the top right corner to enable/disable layout changes or disable
> adding new processors, and only allow viewing processor properties - a sort
> of "read only" mode.
> > Thanks for taking my feedback into consideration. I still find Nifi
> incredibly useful for handling my complicated workflows and appreciate your
> work in developing it!
> > -Elli
> >
> >
> >
> >
> >
> > On Tuesday, January 19, 2016 5:03 PM, Joe Witt 
> wrote:
> >
> >
> >
> >  Elli
> >
> > "it is sometimes too easy to mis-align processors by dragging them
> accidentally"
> >   Great point  I must admit I too do that.  Often in really important
> > demos.  I've gotten good at making jokes about it.  Probably should
> > have gotten good at submitting a JIRA :-)
> >
> > I'd like to understand more about your other idea for emphasizing
> > processors which are more important.  I can understand the idea I
> > think but I'm worried about how we could make the user experience
> > worth the effort for the person signaling the emphasis to be of use
> > for the people consuming that detail.
> >
> > Thanks
> > Joe
> >
> > On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess 
> wrote:
> >> +1 for "snap to grid" feature
> >>
> >> Sent from my iPhone
> >>
> >>> On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
> >>>
> >>> Maybe not exactly "auto-layout" but I would back a notion of having the
> >>> 

Re: Auto-Organize Layout

[dan bress]

Maybe not exactly "auto-layout" but I would back a notion of having the
components snap to a coarser grain grid than what we currently have.
Sometimes I care a lot about having everything line up in the graph
horizontally and vertically, and it always takes a long time to achieve
this.

I could see this being achieved by snapping the component to the same spot
horizontally as the component above it when you move it underneath another
component.  Or some magical "auto snap" button that does its best to align
everything with its nearest neighbors.

On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:

> I like your idea Rob, that would help with lining up relationships too
> (straight lines).
>
> On Matt's note, I don't think there should be a "standard" either, although
> best practices are always out there.
>
> On Matt's note of putting failures up above processes, we do that too.
> Totally depends on who made the flow first.  Sometimes, people don't even
> follow a convention in the same flow.xml file.
>
> For these reasons, I'd recommend alternate views to the flow.
>
> We have a couple projects that just allow you to rearrange a node-based
> graph, based on your preference, hierarchy, circular, pyramid, etc.
>
> Applying this to NiFi, having a couple different default auto-layout
> options that you can swap your current view to, but NOT change the original
> flow, would be nice.
>
> It would let you walk into someone else's, potentially large, dataflow and
> have a familiar way to view the flow.
>
> Ryan
>
>
> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
>
> > I agree with Matt's points. I was just replying with something similar
> > basically saying I think trying to set a standard would not be
> > well-received.
> >
> > I believe what could be more useful are layout tools that would help
> users
> > place components to help achieve their preferred layouts. For example,
> the
> > ability to align (left, right, center) components
> > or horizontally/vertically distribute components evenly. Other features
> > such as snap-to and/or smart-guides could make it easier for users to
> > follow their organization's best practices when designing a flow.
> >
> > Rob
> >
> > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
> matt.clarke@gmail.com
> > >
> > wrote:
> >
> > > Ryan,
> > >
> > >   Setting a standard is a difficult thing to do.  The
> complexity
> > > that can exist in many flows would make enforcing a standard difficult.
> > The
> > > first example you provide of success to points right while failures
> point
> > > up is not recommended. It would be better to have failures point down
> > since
> > > it is common to put labels over processor(s). Any relationships
> pointing
> > up
> > > would pass through these labels making both the relationship box and
> the
> > > label hard to read.  It is often coomon to see flows designed with a
> > > combination of left to right and top to bottom design.
> > >
> > > Matt
> > >
> > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> > > wrote:
> > >
> > > > Hi Rob,
> > > > Yea we did, it was at the end of the meeting.
> > > >
> > > > I think it would be useful to have a couple default type views
> that
> > > > help standardize flow layout across the community.
> > > >
> > > > For example, when we organize processors left-to-right, failure
> > > > relationships always point up, and success always point right.
> > > > Alternatively, when we organize processors up-and-down, failure
> > > > relationships always point left, and successes always point down.
> > > >
> > > > Of course, in some of these scenarios there are processors that
> > have
> > > > more than 1 success relationship, but that's when a good layout
> library
> > > > would come into play.
> > > >
> > > > What do you think?
> > > >
> > > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran 
> wrote:
> > > >
> > > > > Ryan - I think we spoke briefly (at a very high level) about this
> at
> > a
> > > > > prior meetup. What alternate views did you have in mind, and in
> what
> > > ways
> > > > > do you think they'd be useful?
> > > > >
> > > > > Rob
> > > > >
> > > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H <
> > rhendrickson.w...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > It'd be pretty awesome if NiFi provided the ability to
> > auto-organize
> > > a
> > > > > > layout.
> > > > > >
> > > > > > Maybe even just a auto-organized layout that doesn't change the
> > > > flow.xml,
> > > > > > just an alternate view.
> > > > > >
> > > > > > Looking at these demos here: http://js.cytoscape.org/#demos
> > > > > >
> > > > > > Ryan
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Auto-Organize Layout

[Matt Gilman]

+1 to adding capabilities that will allow users to build the desired layout
more efficiently. Allowing them to more easily align components,
snap-to-grid, lock, and bend connections would all fall into this category.

I'm a little indifferent to supporting an auto-layout feature. I've tried a
number of different options for creating an optimum layout of directed
graphs and the results are typically sub-par for this purpose. Some of the
layouts being described are very useful when analyzing a dataset and
learning about the relationships between the elements. However, I'm not
sure how applicable they would be for visualizing a data flow. Building a
simple top-down left-right auto layout for a cyclic graph would also yield
poor results in my opinion. Additionally, supporting a view that is not
representative of the underlying flow.xml is a departure from our current
design model.

In short, my preference would be to provide the tools for the users to
build the layout they want rather than provide auto layouts that may not be
helpful.

Matt

On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess  wrote:

> +1 for "snap to grid" feature
>
> Sent from my iPhone
>
> > On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
> >
> > Maybe not exactly "auto-layout" but I would back a notion of having the
> > components snap to a coarser grain grid than what we currently have.
> > Sometimes I care a lot about having everything line up in the graph
> > horizontally and vertically, and it always takes a long time to achieve
> > this.
> >
> > I could see this being achieved by snapping the component to the same
> spot
> > horizontally as the component above it when you move it underneath
> another
> > component.  Or some magical "auto snap" button that does its best to
> align
> > everything with its nearest neighbors.
> >
> >> On Tue, Jan 19, 2016 at 12:37 PM Ryan H 
> wrote:
> >>
> >> I like your idea Rob, that would help with lining up relationships too
> >> (straight lines).
> >>
> >> On Matt's note, I don't think there should be a "standard" either,
> although
> >> best practices are always out there.
> >>
> >> On Matt's note of putting failures up above processes, we do that too.
> >> Totally depends on who made the flow first.  Sometimes, people don't
> even
> >> follow a convention in the same flow.xml file.
> >>
> >> For these reasons, I'd recommend alternate views to the flow.
> >>
> >> We have a couple projects that just allow you to rearrange a node-based
> >> graph, based on your preference, hierarchy, circular, pyramid, etc.
> >>
> >> Applying this to NiFi, having a couple different default auto-layout
> >> options that you can swap your current view to, but NOT change the
> original
> >> flow, would be nice.
> >>
> >> It would let you walk into someone else's, potentially large, dataflow
> and
> >> have a familiar way to view the flow.
> >>
> >> Ryan
> >>
> >>
> >>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
> >>>
> >>> I agree with Matt's points. I was just replying with something similar
> >>> basically saying I think trying to set a standard would not be
> >>> well-received.
> >>>
> >>> I believe what could be more useful are layout tools that would help
> >> users
> >>> place components to help achieve their preferred layouts. For example,
> >> the
> >>> ability to align (left, right, center) components
> >>> or horizontally/vertically distribute components evenly. Other features
> >>> such as snap-to and/or smart-guides could make it easier for users to
> >>> follow their organization's best practices when designing a flow.
> >>>
> >>> Rob
> >>>
> >>> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
> >> matt.clarke@gmail.com
> >>> wrote:
> >>>
>  Ryan,
> 
>   Setting a standard is a difficult thing to do.  The
> >> complexity
>  that can exist in many flows would make enforcing a standard
> difficult.
> >>> The
>  first example you provide of success to points right while failures
> >> point
>  up is not recommended. It would be better to have failures point down
> >>> since
>  it is common to put labels over processor(s). Any relationships
> >> pointing
> >>> up
>  would pass through these labels making both the relationship box and
> >> the
>  label hard to read.  It is often coomon to see flows designed with a
>  combination of left to right and top to bottom design.
> 
>  Matt
> 
>  On Tue, Jan 19, 2016 at 12:07 PM, Ryan H  >
>  wrote:
> 
> > Hi Rob,
> >Yea we did, it was at the end of the meeting.
> >
> >I think it would be useful to have a couple default type views
> >> that
> > help standardize flow layout across the community.
> >
> >For example, when we organize processors left-to-right, failure
> > relationships always point up, and success always point right.
> 

Re: Auto-Organize Layout

[Ryan H]

I like your idea Rob, that would help with lining up relationships too
(straight lines).

On Matt's note, I don't think there should be a "standard" either, although
best practices are always out there.

On Matt's note of putting failures up above processes, we do that too.
Totally depends on who made the flow first.  Sometimes, people don't even
follow a convention in the same flow.xml file.

For these reasons, I'd recommend alternate views to the flow.

We have a couple projects that just allow you to rearrange a node-based
graph, based on your preference, hierarchy, circular, pyramid, etc.

Applying this to NiFi, having a couple different default auto-layout
options that you can swap your current view to, but NOT change the original
flow, would be nice.

It would let you walk into someone else's, potentially large, dataflow and
have a familiar way to view the flow.

Ryan


On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:

> I agree with Matt's points. I was just replying with something similar
> basically saying I think trying to set a standard would not be
> well-received.
>
> I believe what could be more useful are layout tools that would help users
> place components to help achieve their preferred layouts. For example, the
> ability to align (left, right, center) components
> or horizontally/vertically distribute components evenly. Other features
> such as snap-to and/or smart-guides could make it easier for users to
> follow their organization's best practices when designing a flow.
>
> Rob
>
> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke  >
> wrote:
>
> > Ryan,
> >
> >   Setting a standard is a difficult thing to do.  The complexity
> > that can exist in many flows would make enforcing a standard difficult.
> The
> > first example you provide of success to points right while failures point
> > up is not recommended. It would be better to have failures point down
> since
> > it is common to put labels over processor(s). Any relationships pointing
> up
> > would pass through these labels making both the relationship box and the
> > label hard to read.  It is often coomon to see flows designed with a
> > combination of left to right and top to bottom design.
> >
> > Matt
> >
> > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> > wrote:
> >
> > > Hi Rob,
> > > Yea we did, it was at the end of the meeting.
> > >
> > > I think it would be useful to have a couple default type views that
> > > help standardize flow layout across the community.
> > >
> > > For example, when we organize processors left-to-right, failure
> > > relationships always point up, and success always point right.
> > > Alternatively, when we organize processors up-and-down, failure
> > > relationships always point left, and successes always point down.
> > >
> > > Of course, in some of these scenarios there are processors that
> have
> > > more than 1 success relationship, but that's when a good layout library
> > > would come into play.
> > >
> > > What do you think?
> > >
> > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran  wrote:
> > >
> > > > Ryan - I think we spoke briefly (at a very high level) about this at
> a
> > > > prior meetup. What alternate views did you have in mind, and in what
> > ways
> > > > do you think they'd be useful?
> > > >
> > > > Rob
> > > >
> > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H <
> rhendrickson.w...@gmail.com>
> > > > wrote:
> > > >
> > > > > It'd be pretty awesome if NiFi provided the ability to
> auto-organize
> > a
> > > > > layout.
> > > > >
> > > > > Maybe even just a auto-organized layout that doesn't change the
> > > flow.xml,
> > > > > just an alternate view.
> > > > >
> > > > > Looking at these demos here: http://js.cytoscape.org/#demos
> > > > >
> > > > > Ryan
> > > > >
> > > >
> > >
> >
>

Re: Auto-Organize Layout

[Matt Burgess]

+1 for "snap to grid" feature

Sent from my iPhone

> On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
> 
> Maybe not exactly "auto-layout" but I would back a notion of having the
> components snap to a coarser grain grid than what we currently have.
> Sometimes I care a lot about having everything line up in the graph
> horizontally and vertically, and it always takes a long time to achieve
> this.
> 
> I could see this being achieved by snapping the component to the same spot
> horizontally as the component above it when you move it underneath another
> component.  Or some magical "auto snap" button that does its best to align
> everything with its nearest neighbors.
> 
>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:
>> 
>> I like your idea Rob, that would help with lining up relationships too
>> (straight lines).
>> 
>> On Matt's note, I don't think there should be a "standard" either, although
>> best practices are always out there.
>> 
>> On Matt's note of putting failures up above processes, we do that too.
>> Totally depends on who made the flow first.  Sometimes, people don't even
>> follow a convention in the same flow.xml file.
>> 
>> For these reasons, I'd recommend alternate views to the flow.
>> 
>> We have a couple projects that just allow you to rearrange a node-based
>> graph, based on your preference, hierarchy, circular, pyramid, etc.
>> 
>> Applying this to NiFi, having a couple different default auto-layout
>> options that you can swap your current view to, but NOT change the original
>> flow, would be nice.
>> 
>> It would let you walk into someone else's, potentially large, dataflow and
>> have a familiar way to view the flow.
>> 
>> Ryan
>> 
>> 
>>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
>>> 
>>> I agree with Matt's points. I was just replying with something similar
>>> basically saying I think trying to set a standard would not be
>>> well-received.
>>> 
>>> I believe what could be more useful are layout tools that would help
>> users
>>> place components to help achieve their preferred layouts. For example,
>> the
>>> ability to align (left, right, center) components
>>> or horizontally/vertically distribute components evenly. Other features
>>> such as snap-to and/or smart-guides could make it easier for users to
>>> follow their organization's best practices when designing a flow.
>>> 
>>> Rob
>>> 
>>> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
>> matt.clarke@gmail.com
>>> wrote:
>>> 
 Ryan,
 
  Setting a standard is a difficult thing to do.  The
>> complexity
 that can exist in many flows would make enforcing a standard difficult.
>>> The
 first example you provide of success to points right while failures
>> point
 up is not recommended. It would be better to have failures point down
>>> since
 it is common to put labels over processor(s). Any relationships
>> pointing
>>> up
 would pass through these labels making both the relationship box and
>> the
 label hard to read.  It is often coomon to see flows designed with a
 combination of left to right and top to bottom design.
 
 Matt
 
 On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
 wrote:
 
> Hi Rob,
>Yea we did, it was at the end of the meeting.
> 
>I think it would be useful to have a couple default type views
>> that
> help standardize flow layout across the community.
> 
>For example, when we organize processors left-to-right, failure
> relationships always point up, and success always point right.
>Alternatively, when we organize processors up-and-down, failure
> relationships always point left, and successes always point down.
> 
>Of course, in some of these scenarios there are processors that
>>> have
> more than 1 success relationship, but that's when a good layout
>> library
> would come into play.
> 
>What do you think?
> 
> On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran 
>> wrote:
> 
>> Ryan - I think we spoke briefly (at a very high level) about this
>> at
>>> a
>> prior meetup. What alternate views did you have in mind, and in
>> what
 ways
>> do you think they'd be useful?
>> 
>> Rob
>> 
>> On Tue, Jan 19, 2016 at 10:56 AM, Ryan H <
>>> rhendrickson.w...@gmail.com>
>> wrote:
>> 
>>> It'd be pretty awesome if NiFi provided the ability to
>>> auto-organize
 a
>>> layout.
>>> 
>>> Maybe even just a auto-organized layout that doesn't change the
> flow.xml,
>>> just an alternate view.
>>> 
>>> Looking at these demos here: http://js.cytoscape.org/#demos
>>> 
>>> Ryan
>> 

Re: Auto-Organize Layout

[Joe Witt]

Elli

"it is sometimes too easy to mis-align processors by dragging them accidentally"
  Great point  I must admit I too do that.  Often in really important
demos.  I've gotten good at making jokes about it.  Probably should
have gotten good at submitting a JIRA :-)

I'd like to understand more about your other idea for emphasizing
processors which are more important.  I can understand the idea I
think but I'm worried about how we could make the user experience
worth the effort for the person signaling the emphasis to be of use
for the people consuming that detail.

Thanks
Joe

On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess  wrote:
> +1 for "snap to grid" feature
>
> Sent from my iPhone
>
>> On Jan 19, 2016, at 4:20 PM, dan bress  wrote:
>>
>> Maybe not exactly "auto-layout" but I would back a notion of having the
>> components snap to a coarser grain grid than what we currently have.
>> Sometimes I care a lot about having everything line up in the graph
>> horizontally and vertically, and it always takes a long time to achieve
>> this.
>>
>> I could see this being achieved by snapping the component to the same spot
>> horizontally as the component above it when you move it underneath another
>> component.  Or some magical "auto snap" button that does its best to align
>> everything with its nearest neighbors.
>>
>>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:
>>>
>>> I like your idea Rob, that would help with lining up relationships too
>>> (straight lines).
>>>
>>> On Matt's note, I don't think there should be a "standard" either, although
>>> best practices are always out there.
>>>
>>> On Matt's note of putting failures up above processes, we do that too.
>>> Totally depends on who made the flow first.  Sometimes, people don't even
>>> follow a convention in the same flow.xml file.
>>>
>>> For these reasons, I'd recommend alternate views to the flow.
>>>
>>> We have a couple projects that just allow you to rearrange a node-based
>>> graph, based on your preference, hierarchy, circular, pyramid, etc.
>>>
>>> Applying this to NiFi, having a couple different default auto-layout
>>> options that you can swap your current view to, but NOT change the original
>>> flow, would be nice.
>>>
>>> It would let you walk into someone else's, potentially large, dataflow and
>>> have a familiar way to view the flow.
>>>
>>> Ryan
>>>
>>>
 On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:

 I agree with Matt's points. I was just replying with something similar
 basically saying I think trying to set a standard would not be
 well-received.

 I believe what could be more useful are layout tools that would help
>>> users
 place components to help achieve their preferred layouts. For example,
>>> the
 ability to align (left, right, center) components
 or horizontally/vertically distribute components evenly. Other features
 such as snap-to and/or smart-guides could make it easier for users to
 follow their organization's best practices when designing a flow.

 Rob

 On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
>>> matt.clarke@gmail.com
 wrote:

> Ryan,
>
>  Setting a standard is a difficult thing to do.  The
>>> complexity
> that can exist in many flows would make enforcing a standard difficult.
 The
> first example you provide of success to points right while failures
>>> point
> up is not recommended. It would be better to have failures point down
 since
> it is common to put labels over processor(s). Any relationships
>>> pointing
 up
> would pass through these labels making both the relationship box and
>>> the
> label hard to read.  It is often coomon to see flows designed with a
> combination of left to right and top to bottom design.
>
> Matt
>
> On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> wrote:
>
>> Hi Rob,
>>Yea we did, it was at the end of the meeting.
>>
>>I think it would be useful to have a couple default type views
>>> that
>> help standardize flow layout across the community.
>>
>>For example, when we organize processors left-to-right, failure
>> relationships always point up, and success always point right.
>>Alternatively, when we organize processors up-and-down, failure
>> relationships always point left, and successes always point down.
>>
>>Of course, in some of these scenarios there are processors that
 have
>> more than 1 success relationship, but that's when a good layout
>>> library
>> would come into play.
>>
>>What do you think?
>>
>> On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran 
>>> wrote:
>>
>>> Ryan - I think we spoke briefly (at a very high level) about this
>>> at
 a
>>> prior 

Re: Auto-Organize Layout

[Elli Schwarz]

I also think that some way to help layout the flow would be very useful. One of 
the hardest parts of creating an easy to read workflow is the designing a good 
layout, and in many cases I simply want to be able to select a few processors 
and line them up (horizontally or vertically). Also, it is sometimes too easy 
to mis-align processors by dragging them accidentally. It would be nice if 
there would be a way to "lock" the layout of a group of selected processors so 
they can't be accidentally moved. Also, the ability to "undo" a layout change 
or save a layout (ie, just save the position of a group of selected processors) 
so that we can revert to it would also be helpful.

Another thing that could be helpful is a way to emphasize the more important 
processors, ie, the ones integral to the flow, as opposed to the many "PutFile" 
processors I have that are simply for logging (although I realize that 
sometimes a PutFile can be an integral part of a flow). I know I can select a 
group of processors and change the color, and also create a label to highlight 
a certain group of processors, but it might be nice to be able to make selected 
processors appear larger on the screen than others, to emphasize their 
importance. It is sometimes hard to follow the trail of a flow simply because 
there are many connections, so a way to highlight the "main" path might be 
useful.

-Elli


On Tuesday, January 19, 2016 4:20 PM, dan bress  wrote:
 
 

 Maybe not exactly "auto-layout" but I would back a notion of having the
components snap to a coarser grain grid than what we currently have.
Sometimes I care a lot about having everything line up in the graph
horizontally and vertically, and it always takes a long time to achieve
this.

I could see this being achieved by snapping the component to the same spot
horizontally as the component above it when you move it underneath another
component.  Or some magical "auto snap" button that does its best to align
everything with its nearest neighbors.

On Tue, Jan 19, 2016 at 12:37 PM Ryan H  wrote:

> I like your idea Rob, that would help with lining up relationships too
> (straight lines).
>
> On Matt's note, I don't think there should be a "standard" either, although
> best practices are always out there.
>
> On Matt's note of putting failures up above processes, we do that too.
> Totally depends on who made the flow first.  Sometimes, people don't even
> follow a convention in the same flow.xml file.
>
> For these reasons, I'd recommend alternate views to the flow.
>
> We have a couple projects that just allow you to rearrange a node-based
> graph, based on your preference, hierarchy, circular, pyramid, etc.
>
> Applying this to NiFi, having a couple different default auto-layout
> options that you can swap your current view to, but NOT change the original
> flow, would be nice.
>
> It would let you walk into someone else's, potentially large, dataflow and
> have a familiar way to view the flow.
>
> Ryan
>
>
> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran  wrote:
>
> > I agree with Matt's points. I was just replying with something similar
> > basically saying I think trying to set a standard would not be
> > well-received.
> >
> > I believe what could be more useful are layout tools that would help
> users
> > place components to help achieve their preferred layouts. For example,
> the
> > ability to align (left, right, center) components
> > or horizontally/vertically distribute components evenly. Other features
> > such as snap-to and/or smart-guides could make it easier for users to
> > follow their organization's best practices when designing a flow.
> >
> > Rob
> >
> > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke <
> matt.clarke@gmail.com
> > >
> > wrote:
> >
> > > Ryan,
> > >
> > >          Setting a standard is a difficult thing to do.  The
> complexity
> > > that can exist in many flows would make enforcing a standard difficult.
> > The
> > > first example you provide of success to points right while failures
> point
> > > up is not recommended. It would be better to have failures point down
> > since
> > > it is common to put labels over processor(s). Any relationships
> pointing
> > up
> > > would pass through these labels making both the relationship box and
> the
> > > label hard to read.  It is often coomon to see flows designed with a
> > > combination of left to right and top to bottom design.
> > >
> > > Matt
> > >
> > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> > > wrote:
> > >
> > > > Hi Rob,
> > > >    Yea we did, it was at the end of the meeting.
> > > >
> > > >    I think it would be useful to have a couple default type views
> that
> > > > help standardize flow layout across the community.
> > > >
> > > >    For example, when we organize processors left-to-right, failure
> > > > relationships always point up, and success always point right.
> > > 

Re: Auto-Organize Layout

[Rob Moran]

I agree with Matt's points. I was just replying with something similar
basically saying I think trying to set a standard would not be
well-received.

I believe what could be more useful are layout tools that would help users
place components to help achieve their preferred layouts. For example, the
ability to align (left, right, center) components
or horizontally/vertically distribute components evenly. Other features
such as snap-to and/or smart-guides could make it easier for users to
follow their organization's best practices when designing a flow.

Rob

On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke 
wrote:

> Ryan,
>
>   Setting a standard is a difficult thing to do.  The complexity
> that can exist in many flows would make enforcing a standard difficult. The
> first example you provide of success to points right while failures point
> up is not recommended. It would be better to have failures point down since
> it is common to put labels over processor(s). Any relationships pointing up
> would pass through these labels making both the relationship box and the
> label hard to read.  It is often coomon to see flows designed with a
> combination of left to right and top to bottom design.
>
> Matt
>
> On Tue, Jan 19, 2016 at 12:07 PM, Ryan H 
> wrote:
>
> > Hi Rob,
> > Yea we did, it was at the end of the meeting.
> >
> > I think it would be useful to have a couple default type views that
> > help standardize flow layout across the community.
> >
> > For example, when we organize processors left-to-right, failure
> > relationships always point up, and success always point right.
> > Alternatively, when we organize processors up-and-down, failure
> > relationships always point left, and successes always point down.
> >
> > Of course, in some of these scenarios there are processors that have
> > more than 1 success relationship, but that's when a good layout library
> > would come into play.
> >
> > What do you think?
> >
> > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran  wrote:
> >
> > > Ryan - I think we spoke briefly (at a very high level) about this at a
> > > prior meetup. What alternate views did you have in mind, and in what
> ways
> > > do you think they'd be useful?
> > >
> > > Rob
> > >
> > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H 
> > > wrote:
> > >
> > > > It'd be pretty awesome if NiFi provided the ability to auto-organize
> a
> > > > layout.
> > > >
> > > > Maybe even just a auto-organized layout that doesn't change the
> > flow.xml,
> > > > just an alternate view.
> > > >
> > > > Looking at these demos here: http://js.cytoscape.org/#demos
> > > >
> > > > Ryan
> > > >
> > >
> >
>