Re: Auto-Organize Layout
Joe, What I'm referring to as far as emphasizing "important" processors is that there are many places in my workflow where I do a PutFile for logging purposes, and I have one for success and a separate one for failure. So for many of my main processors, there are two connectors to PutFiles. This makes the workflow look very cluttered, and when doing a demo it is difficult to see the main path that flowfiles are taking without explanation, when I'd like it to be intuitive. In fact, there are some cases that I really want to add PutFiles for logging but since it will make the workflow look so cluttered I don't do it. Maybe this problem can be solved by an option to route all failures or even successes from some or all processors to do a PutFile to a specific folder? Maybe we can minimize some processor such as these PutFiles to a small icon instead of the whole box to save screen space? I often don't care about processor stats for these processors anyway, but maybe they'd be displayed on mouse hover. Furthermore, in some of my workflows I have my main tasks that deal with the actual processing of the incoming data, and other "side tasks" that I do with the data like collect special metrics, storing some metadata in a special database, sending status emails, etc. I handle this now with Processor Groups, and that helps, but I find it a bit unwieldy to create many processor groups that only have two or three processors in them (and then the processor group needs the input/output ports, further complicated an otherwise simple workflow). There are also cases where for some failures, I have a ControlRate processor and then retry the flowfile after a certain period of time - this is not a "main" part of the workflow, but it clutters it and it's not intuitive to see what's happening. I think I'd like to solve this problem by just being able to align selected processors that I consider more important, and having the others off to a side. As far as accidentally dragging processors is concerned, sometimes I intend to the pan screen and end up moving a processor or moving a label that I used to highlight a section of my flow. For demos, it would be nice to lock the entire page layout. Maybe this can be accomplished with a button on the top right corner to enable/disable layout changes or disable adding new processors, and only allow viewing processor properties - a sort of "read only" mode. Thanks for taking my feedback into consideration. I still find Nifi incredibly useful for handling my complicated workflows and appreciate your work in developing it! -Elli On Tuesday, January 19, 2016 5:03 PM, Joe Wittwrote: Elli "it is sometimes too easy to mis-align processors by dragging them accidentally" Great point I must admit I too do that. Often in really important demos. I've gotten good at making jokes about it. Probably should have gotten good at submitting a JIRA :-) I'd like to understand more about your other idea for emphasizing processors which are more important. I can understand the idea I think but I'm worried about how we could make the user experience worth the effort for the person signaling the emphasis to be of use for the people consuming that detail. Thanks Joe On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess wrote: > +1 for "snap to grid" feature > > Sent from my iPhone > >> On Jan 19, 2016, at 4:20 PM, dan bress wrote: >> >> Maybe not exactly "auto-layout" but I would back a notion of having the >> components snap to a coarser grain grid than what we currently have. >> Sometimes I care a lot about having everything line up in the graph >> horizontally and vertically, and it always takes a long time to achieve >> this. >> >> I could see this being achieved by snapping the component to the same spot >> horizontally as the component above it when you move it underneath another >> component. Or some magical "auto snap" button that does its best to align >> everything with its nearest neighbors. >> >>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H wrote: >>> >>> I like your idea Rob, that would help with lining up relationships too >>> (straight lines). >>> >>> On Matt's note, I don't think there should be a "standard" either, although >>> best practices are always out there. >>> >>> On Matt's note of putting failures up above processes, we do that too. >>> Totally depends on who made the flow first. Sometimes, people don't even >>> follow a convention in the same flow.xml file. >>> >>> For these reasons, I'd recommend alternate views to the flow. >>> >>> We have a couple projects that just allow you to rearrange a node-based >>> graph, based on your preference, hierarchy, circular, pyramid, etc. >>> >>> Applying this to NiFi, having a couple different default auto-layout >>> options that you can swap your current view to, but NOT change the original >>> flow,
Re: Auto-Organize Layout
Joe, Responses to your bullets: 1) I wasn't aware of the details of how provenance works. After reading about it, it seems quite powerful and it may help for some cases. I especially like the ability to replay files and track timing of events. However, there are some problems I think I've experienced in regards to provenance: we have thousands of flowfiles going through daily. We like to keep the flowfiles from certain processors for several days, but if we keep them from all processors we will quickly run out of disk space. When I first upgraded to 0.3.0, it had provenance on by default (I think that was a bug, since in 0.2.0 it was off by default) and I quickly and unexpectedly ran out of disk space on my production server. Furthermore, there are many cases where we store a collection of flowfile content in a directory and we grep looking for certain strings to debug something - can you do something similar to a grep on content through provenance? Can you select which processors you want to store flow files for so not to fill up the disk with flowfile content from processors we don't care as much about? If the answer to these questions could be yes, then I think provenance would work for the kind of tracking I've done with PutFile and LogAttributes. 2) For emphasizing the main parts of a flow, I think the ability to align selected processors would help. I would select the main processors, line them up horizontally, and maybe it would automatically move other processors above or below. Also, maybe I didn't fully explain my processor minimization idea - I would like an option on a processor to "show icon only" so instead of a full processor box with all the stats and details, it would look something like a Processor Group port looks - no stats, just text and some icons indicating started/stopped/disabled. (It should still look differently than a port does so no one gets confused). Of course, I look forward to hearing what others have to say about these ideas! -Elli On Wednesday, January 20, 2016 4:49 PM, Joe Wittwrote: Elli, Ok great feedback. I'll categorize these in the following ways (if you disagree please share) 1) How best to handle log/debug sort of flows One thing to consider is that with our provenance and content archival it eliminated the need for many uses of LogAttribute and for PutFile for these cases. Something to consider. 2) How best to de-emphasize 'special case' handling of flows or parts of a flow which are necessary but not the primary logic thread I 'see' the idea but not sure what a good user experience for it would be. Anyone have visuals/UX concept in mind for this? 3) Modes of edit / Lock-out Makes sense. This has been asked for before. Basically allow the user to express that they want to go into an edit mode. How do others feel? Thanks Joe On Wed, Jan 20, 2016 at 4:00 PM, Elli Schwarz wrote: > Joe, > What I'm referring to as far as emphasizing "important" processors is that > there are many places in my workflow where I do a PutFile for logging > purposes, and I have one for success and a separate one for failure. So for > many of my main processors, there are two connectors to PutFiles. This makes > the workflow look very cluttered, and when doing a demo it is difficult to > see the main path that flowfiles are taking without explanation, when I'd > like it to be intuitive. In fact, there are some cases that I really want to > add PutFiles for logging but since it will make the workflow look so > cluttered I don't do it. Maybe this problem can be solved by an option to > route all failures or even successes from some or all processors to do a > PutFile to a specific folder? Maybe we can minimize some processor such as > these PutFiles to a small icon instead of the whole box to save screen space? > I often don't care about processor stats for these processors anyway, but > maybe they'd be displayed on mouse hover. > > Furthermore, in some of my workflows I have my main tasks that deal with the > actual processing of the incoming data, and other "side tasks" that I do with > the data like collect special metrics, storing some metadata in a special > database, sending status emails, etc. I handle this now with Processor > Groups, and that helps, but I find it a bit unwieldy to create many processor > groups that only have two or three processors in them (and then the processor > group needs the input/output ports, further complicated an otherwise simple > workflow). There are also cases where for some failures, I have a ControlRate > processor and then retry the flowfile after a certain period of time - this > is not a "main" part of the workflow, but it clutters it and it's not > intuitive to see what's happening. I think I'd like to solve this problem by > just being able to align selected processors that I consider more important, > and
Re: Auto-Organize Layout
Re: Emphasising important processors, is there a resize feature to make some boxes on the canvas bigger/smaller than others? Then the main flow could be "follow the big boxes" Or! Allow them to auto-size based on volume. That would be neat. On Wed, Jan 20, 2016 at 3:49 PM, Joe Wittwrote: > Elli, > > Ok great feedback. I'll categorize these in the following ways (if > you disagree please share) > > 1) How best to handle log/debug sort of flows > > One thing to consider is that with our provenance and content archival > it eliminated the need for many uses of > LogAttribute and for PutFile for these cases. Something to consider. > > 2) How best to de-emphasize 'special case' handling of flows or parts > of a flow which are necessary but not the primary logic thread > > I 'see' the idea but not sure what a good user experience for it would > be. Anyone have visuals/UX concept in mind for this? > > 3) Modes of edit / Lock-out > > Makes sense. This has been asked for before. Basically allow the > user to express that they want to go into an edit mode. How do others > feel? > > Thanks > Joe > > On Wed, Jan 20, 2016 at 4:00 PM, Elli Schwarz > wrote: > > Joe, > > What I'm referring to as far as emphasizing "important" processors is > that there are many places in my workflow where I do a PutFile for logging > purposes, and I have one for success and a separate one for failure. So for > many of my main processors, there are two connectors to PutFiles. This > makes the workflow look very cluttered, and when doing a demo it is > difficult to see the main path that flowfiles are taking without > explanation, when I'd like it to be intuitive. In fact, there are some > cases that I really want to add PutFiles for logging but since it will make > the workflow look so cluttered I don't do it. Maybe this problem can be > solved by an option to route all failures or even successes from some or > all processors to do a PutFile to a specific folder? Maybe we can minimize > some processor such as these PutFiles to a small icon instead of the whole > box to save screen space? I often don't care about processor stats for > these processors anyway, but maybe they'd be displayed on mouse hover. > > > > Furthermore, in some of my workflows I have my main tasks that deal with > the actual processing of the incoming data, and other "side tasks" that I > do with the data like collect special metrics, storing some metadata in a > special database, sending status emails, etc. I handle this now with > Processor Groups, and that helps, but I find it a bit unwieldy to create > many processor groups that only have two or three processors in them (and > then the processor group needs the input/output ports, further complicated > an otherwise simple workflow). There are also cases where for some > failures, I have a ControlRate processor and then retry the flowfile after > a certain period of time - this is not a "main" part of the workflow, but > it clutters it and it's not intuitive to see what's happening. I think I'd > like to solve this problem by just being able to align selected processors > that I consider more important, and having the others off to a side. > > > > As far as accidentally dragging processors is concerned, sometimes I > intend to the pan screen and end up moving a processor or moving a label > that I used to highlight a section of my flow. For demos, it would be nice > to lock the entire page layout. Maybe this can be accomplished with a > button on the top right corner to enable/disable layout changes or disable > adding new processors, and only allow viewing processor properties - a sort > of "read only" mode. > > Thanks for taking my feedback into consideration. I still find Nifi > incredibly useful for handling my complicated workflows and appreciate your > work in developing it! > > -Elli > > > > > > > > > > > > On Tuesday, January 19, 2016 5:03 PM, Joe Witt > wrote: > > > > > > > > Elli > > > > "it is sometimes too easy to mis-align processors by dragging them > accidentally" > > Great point I must admit I too do that. Often in really important > > demos. I've gotten good at making jokes about it. Probably should > > have gotten good at submitting a JIRA :-) > > > > I'd like to understand more about your other idea for emphasizing > > processors which are more important. I can understand the idea I > > think but I'm worried about how we could make the user experience > > worth the effort for the person signaling the emphasis to be of use > > for the people consuming that detail. > > > > Thanks > > Joe > > > > On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgess > wrote: > >> +1 for "snap to grid" feature > >> > >> Sent from my iPhone > >> > >>> On Jan 19, 2016, at 4:20 PM, dan bress wrote: > >>> > >>> Maybe not exactly "auto-layout" but I would back a notion of having the > >>>
Re: Auto-Organize Layout
Maybe not exactly "auto-layout" but I would back a notion of having the components snap to a coarser grain grid than what we currently have. Sometimes I care a lot about having everything line up in the graph horizontally and vertically, and it always takes a long time to achieve this. I could see this being achieved by snapping the component to the same spot horizontally as the component above it when you move it underneath another component. Or some magical "auto snap" button that does its best to align everything with its nearest neighbors. On Tue, Jan 19, 2016 at 12:37 PM Ryan Hwrote: > I like your idea Rob, that would help with lining up relationships too > (straight lines). > > On Matt's note, I don't think there should be a "standard" either, although > best practices are always out there. > > On Matt's note of putting failures up above processes, we do that too. > Totally depends on who made the flow first. Sometimes, people don't even > follow a convention in the same flow.xml file. > > For these reasons, I'd recommend alternate views to the flow. > > We have a couple projects that just allow you to rearrange a node-based > graph, based on your preference, hierarchy, circular, pyramid, etc. > > Applying this to NiFi, having a couple different default auto-layout > options that you can swap your current view to, but NOT change the original > flow, would be nice. > > It would let you walk into someone else's, potentially large, dataflow and > have a familiar way to view the flow. > > Ryan > > > On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran wrote: > > > I agree with Matt's points. I was just replying with something similar > > basically saying I think trying to set a standard would not be > > well-received. > > > > I believe what could be more useful are layout tools that would help > users > > place components to help achieve their preferred layouts. For example, > the > > ability to align (left, right, center) components > > or horizontally/vertically distribute components evenly. Other features > > such as snap-to and/or smart-guides could make it easier for users to > > follow their organization's best practices when designing a flow. > > > > Rob > > > > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke < > matt.clarke@gmail.com > > > > > wrote: > > > > > Ryan, > > > > > > Setting a standard is a difficult thing to do. The > complexity > > > that can exist in many flows would make enforcing a standard difficult. > > The > > > first example you provide of success to points right while failures > point > > > up is not recommended. It would be better to have failures point down > > since > > > it is common to put labels over processor(s). Any relationships > pointing > > up > > > would pass through these labels making both the relationship box and > the > > > label hard to read. It is often coomon to see flows designed with a > > > combination of left to right and top to bottom design. > > > > > > Matt > > > > > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H > > > wrote: > > > > > > > Hi Rob, > > > > Yea we did, it was at the end of the meeting. > > > > > > > > I think it would be useful to have a couple default type views > that > > > > help standardize flow layout across the community. > > > > > > > > For example, when we organize processors left-to-right, failure > > > > relationships always point up, and success always point right. > > > > Alternatively, when we organize processors up-and-down, failure > > > > relationships always point left, and successes always point down. > > > > > > > > Of course, in some of these scenarios there are processors that > > have > > > > more than 1 success relationship, but that's when a good layout > library > > > > would come into play. > > > > > > > > What do you think? > > > > > > > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran > wrote: > > > > > > > > > Ryan - I think we spoke briefly (at a very high level) about this > at > > a > > > > > prior meetup. What alternate views did you have in mind, and in > what > > > ways > > > > > do you think they'd be useful? > > > > > > > > > > Rob > > > > > > > > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H < > > rhendrickson.w...@gmail.com> > > > > > wrote: > > > > > > > > > > > It'd be pretty awesome if NiFi provided the ability to > > auto-organize > > > a > > > > > > layout. > > > > > > > > > > > > Maybe even just a auto-organized layout that doesn't change the > > > > flow.xml, > > > > > > just an alternate view. > > > > > > > > > > > > Looking at these demos here: http://js.cytoscape.org/#demos > > > > > > > > > > > > Ryan > > > > > > > > > > > > > > > > > > > > >
Re: Auto-Organize Layout
+1 to adding capabilities that will allow users to build the desired layout more efficiently. Allowing them to more easily align components, snap-to-grid, lock, and bend connections would all fall into this category. I'm a little indifferent to supporting an auto-layout feature. I've tried a number of different options for creating an optimum layout of directed graphs and the results are typically sub-par for this purpose. Some of the layouts being described are very useful when analyzing a dataset and learning about the relationships between the elements. However, I'm not sure how applicable they would be for visualizing a data flow. Building a simple top-down left-right auto layout for a cyclic graph would also yield poor results in my opinion. Additionally, supporting a view that is not representative of the underlying flow.xml is a departure from our current design model. In short, my preference would be to provide the tools for the users to build the layout they want rather than provide auto layouts that may not be helpful. Matt On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgesswrote: > +1 for "snap to grid" feature > > Sent from my iPhone > > > On Jan 19, 2016, at 4:20 PM, dan bress wrote: > > > > Maybe not exactly "auto-layout" but I would back a notion of having the > > components snap to a coarser grain grid than what we currently have. > > Sometimes I care a lot about having everything line up in the graph > > horizontally and vertically, and it always takes a long time to achieve > > this. > > > > I could see this being achieved by snapping the component to the same > spot > > horizontally as the component above it when you move it underneath > another > > component. Or some magical "auto snap" button that does its best to > align > > everything with its nearest neighbors. > > > >> On Tue, Jan 19, 2016 at 12:37 PM Ryan H > wrote: > >> > >> I like your idea Rob, that would help with lining up relationships too > >> (straight lines). > >> > >> On Matt's note, I don't think there should be a "standard" either, > although > >> best practices are always out there. > >> > >> On Matt's note of putting failures up above processes, we do that too. > >> Totally depends on who made the flow first. Sometimes, people don't > even > >> follow a convention in the same flow.xml file. > >> > >> For these reasons, I'd recommend alternate views to the flow. > >> > >> We have a couple projects that just allow you to rearrange a node-based > >> graph, based on your preference, hierarchy, circular, pyramid, etc. > >> > >> Applying this to NiFi, having a couple different default auto-layout > >> options that you can swap your current view to, but NOT change the > original > >> flow, would be nice. > >> > >> It would let you walk into someone else's, potentially large, dataflow > and > >> have a familiar way to view the flow. > >> > >> Ryan > >> > >> > >>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran wrote: > >>> > >>> I agree with Matt's points. I was just replying with something similar > >>> basically saying I think trying to set a standard would not be > >>> well-received. > >>> > >>> I believe what could be more useful are layout tools that would help > >> users > >>> place components to help achieve their preferred layouts. For example, > >> the > >>> ability to align (left, right, center) components > >>> or horizontally/vertically distribute components evenly. Other features > >>> such as snap-to and/or smart-guides could make it easier for users to > >>> follow their organization's best practices when designing a flow. > >>> > >>> Rob > >>> > >>> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke < > >> matt.clarke@gmail.com > >>> wrote: > >>> > Ryan, > > Setting a standard is a difficult thing to do. The > >> complexity > that can exist in many flows would make enforcing a standard > difficult. > >>> The > first example you provide of success to points right while failures > >> point > up is not recommended. It would be better to have failures point down > >>> since > it is common to put labels over processor(s). Any relationships > >> pointing > >>> up > would pass through these labels making both the relationship box and > >> the > label hard to read. It is often coomon to see flows designed with a > combination of left to right and top to bottom design. > > Matt > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H > > wrote: > > > Hi Rob, > >Yea we did, it was at the end of the meeting. > > > >I think it would be useful to have a couple default type views > >> that > > help standardize flow layout across the community. > > > >For example, when we organize processors left-to-right, failure > > relationships always point up, and success always point right. >
Re: Auto-Organize Layout
I like your idea Rob, that would help with lining up relationships too (straight lines). On Matt's note, I don't think there should be a "standard" either, although best practices are always out there. On Matt's note of putting failures up above processes, we do that too. Totally depends on who made the flow first. Sometimes, people don't even follow a convention in the same flow.xml file. For these reasons, I'd recommend alternate views to the flow. We have a couple projects that just allow you to rearrange a node-based graph, based on your preference, hierarchy, circular, pyramid, etc. Applying this to NiFi, having a couple different default auto-layout options that you can swap your current view to, but NOT change the original flow, would be nice. It would let you walk into someone else's, potentially large, dataflow and have a familiar way to view the flow. Ryan On Tue, Jan 19, 2016 at 2:03 PM, Rob Moranwrote: > I agree with Matt's points. I was just replying with something similar > basically saying I think trying to set a standard would not be > well-received. > > I believe what could be more useful are layout tools that would help users > place components to help achieve their preferred layouts. For example, the > ability to align (left, right, center) components > or horizontally/vertically distribute components evenly. Other features > such as snap-to and/or smart-guides could make it easier for users to > follow their organization's best practices when designing a flow. > > Rob > > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke > > wrote: > > > Ryan, > > > > Setting a standard is a difficult thing to do. The complexity > > that can exist in many flows would make enforcing a standard difficult. > The > > first example you provide of success to points right while failures point > > up is not recommended. It would be better to have failures point down > since > > it is common to put labels over processor(s). Any relationships pointing > up > > would pass through these labels making both the relationship box and the > > label hard to read. It is often coomon to see flows designed with a > > combination of left to right and top to bottom design. > > > > Matt > > > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H > > wrote: > > > > > Hi Rob, > > > Yea we did, it was at the end of the meeting. > > > > > > I think it would be useful to have a couple default type views that > > > help standardize flow layout across the community. > > > > > > For example, when we organize processors left-to-right, failure > > > relationships always point up, and success always point right. > > > Alternatively, when we organize processors up-and-down, failure > > > relationships always point left, and successes always point down. > > > > > > Of course, in some of these scenarios there are processors that > have > > > more than 1 success relationship, but that's when a good layout library > > > would come into play. > > > > > > What do you think? > > > > > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran wrote: > > > > > > > Ryan - I think we spoke briefly (at a very high level) about this at > a > > > > prior meetup. What alternate views did you have in mind, and in what > > ways > > > > do you think they'd be useful? > > > > > > > > Rob > > > > > > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H < > rhendrickson.w...@gmail.com> > > > > wrote: > > > > > > > > > It'd be pretty awesome if NiFi provided the ability to > auto-organize > > a > > > > > layout. > > > > > > > > > > Maybe even just a auto-organized layout that doesn't change the > > > flow.xml, > > > > > just an alternate view. > > > > > > > > > > Looking at these demos here: http://js.cytoscape.org/#demos > > > > > > > > > > Ryan > > > > > > > > > > > > > > >
Re: Auto-Organize Layout
+1 for "snap to grid" feature Sent from my iPhone > On Jan 19, 2016, at 4:20 PM, dan bresswrote: > > Maybe not exactly "auto-layout" but I would back a notion of having the > components snap to a coarser grain grid than what we currently have. > Sometimes I care a lot about having everything line up in the graph > horizontally and vertically, and it always takes a long time to achieve > this. > > I could see this being achieved by snapping the component to the same spot > horizontally as the component above it when you move it underneath another > component. Or some magical "auto snap" button that does its best to align > everything with its nearest neighbors. > >> On Tue, Jan 19, 2016 at 12:37 PM Ryan H wrote: >> >> I like your idea Rob, that would help with lining up relationships too >> (straight lines). >> >> On Matt's note, I don't think there should be a "standard" either, although >> best practices are always out there. >> >> On Matt's note of putting failures up above processes, we do that too. >> Totally depends on who made the flow first. Sometimes, people don't even >> follow a convention in the same flow.xml file. >> >> For these reasons, I'd recommend alternate views to the flow. >> >> We have a couple projects that just allow you to rearrange a node-based >> graph, based on your preference, hierarchy, circular, pyramid, etc. >> >> Applying this to NiFi, having a couple different default auto-layout >> options that you can swap your current view to, but NOT change the original >> flow, would be nice. >> >> It would let you walk into someone else's, potentially large, dataflow and >> have a familiar way to view the flow. >> >> Ryan >> >> >>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran wrote: >>> >>> I agree with Matt's points. I was just replying with something similar >>> basically saying I think trying to set a standard would not be >>> well-received. >>> >>> I believe what could be more useful are layout tools that would help >> users >>> place components to help achieve their preferred layouts. For example, >> the >>> ability to align (left, right, center) components >>> or horizontally/vertically distribute components evenly. Other features >>> such as snap-to and/or smart-guides could make it easier for users to >>> follow their organization's best practices when designing a flow. >>> >>> Rob >>> >>> On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke < >> matt.clarke@gmail.com >>> wrote: >>> Ryan, Setting a standard is a difficult thing to do. The >> complexity that can exist in many flows would make enforcing a standard difficult. >>> The first example you provide of success to points right while failures >> point up is not recommended. It would be better to have failures point down >>> since it is common to put labels over processor(s). Any relationships >> pointing >>> up would pass through these labels making both the relationship box and >> the label hard to read. It is often coomon to see flows designed with a combination of left to right and top to bottom design. Matt On Tue, Jan 19, 2016 at 12:07 PM, Ryan H wrote: > Hi Rob, >Yea we did, it was at the end of the meeting. > >I think it would be useful to have a couple default type views >> that > help standardize flow layout across the community. > >For example, when we organize processors left-to-right, failure > relationships always point up, and success always point right. >Alternatively, when we organize processors up-and-down, failure > relationships always point left, and successes always point down. > >Of course, in some of these scenarios there are processors that >>> have > more than 1 success relationship, but that's when a good layout >> library > would come into play. > >What do you think? > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran >> wrote: > >> Ryan - I think we spoke briefly (at a very high level) about this >> at >>> a >> prior meetup. What alternate views did you have in mind, and in >> what ways >> do you think they'd be useful? >> >> Rob >> >> On Tue, Jan 19, 2016 at 10:56 AM, Ryan H < >>> rhendrickson.w...@gmail.com> >> wrote: >> >>> It'd be pretty awesome if NiFi provided the ability to >>> auto-organize a >>> layout. >>> >>> Maybe even just a auto-organized layout that doesn't change the > flow.xml, >>> just an alternate view. >>> >>> Looking at these demos here: http://js.cytoscape.org/#demos >>> >>> Ryan >>
Re: Auto-Organize Layout
Elli "it is sometimes too easy to mis-align processors by dragging them accidentally" Great point I must admit I too do that. Often in really important demos. I've gotten good at making jokes about it. Probably should have gotten good at submitting a JIRA :-) I'd like to understand more about your other idea for emphasizing processors which are more important. I can understand the idea I think but I'm worried about how we could make the user experience worth the effort for the person signaling the emphasis to be of use for the people consuming that detail. Thanks Joe On Tue, Jan 19, 2016 at 4:46 PM, Matt Burgesswrote: > +1 for "snap to grid" feature > > Sent from my iPhone > >> On Jan 19, 2016, at 4:20 PM, dan bress wrote: >> >> Maybe not exactly "auto-layout" but I would back a notion of having the >> components snap to a coarser grain grid than what we currently have. >> Sometimes I care a lot about having everything line up in the graph >> horizontally and vertically, and it always takes a long time to achieve >> this. >> >> I could see this being achieved by snapping the component to the same spot >> horizontally as the component above it when you move it underneath another >> component. Or some magical "auto snap" button that does its best to align >> everything with its nearest neighbors. >> >>> On Tue, Jan 19, 2016 at 12:37 PM Ryan H wrote: >>> >>> I like your idea Rob, that would help with lining up relationships too >>> (straight lines). >>> >>> On Matt's note, I don't think there should be a "standard" either, although >>> best practices are always out there. >>> >>> On Matt's note of putting failures up above processes, we do that too. >>> Totally depends on who made the flow first. Sometimes, people don't even >>> follow a convention in the same flow.xml file. >>> >>> For these reasons, I'd recommend alternate views to the flow. >>> >>> We have a couple projects that just allow you to rearrange a node-based >>> graph, based on your preference, hierarchy, circular, pyramid, etc. >>> >>> Applying this to NiFi, having a couple different default auto-layout >>> options that you can swap your current view to, but NOT change the original >>> flow, would be nice. >>> >>> It would let you walk into someone else's, potentially large, dataflow and >>> have a familiar way to view the flow. >>> >>> Ryan >>> >>> On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran wrote: I agree with Matt's points. I was just replying with something similar basically saying I think trying to set a standard would not be well-received. I believe what could be more useful are layout tools that would help >>> users place components to help achieve their preferred layouts. For example, >>> the ability to align (left, right, center) components or horizontally/vertically distribute components evenly. Other features such as snap-to and/or smart-guides could make it easier for users to follow their organization's best practices when designing a flow. Rob On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke < >>> matt.clarke@gmail.com wrote: > Ryan, > > Setting a standard is a difficult thing to do. The >>> complexity > that can exist in many flows would make enforcing a standard difficult. The > first example you provide of success to points right while failures >>> point > up is not recommended. It would be better to have failures point down since > it is common to put labels over processor(s). Any relationships >>> pointing up > would pass through these labels making both the relationship box and >>> the > label hard to read. It is often coomon to see flows designed with a > combination of left to right and top to bottom design. > > Matt > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H > wrote: > >> Hi Rob, >>Yea we did, it was at the end of the meeting. >> >>I think it would be useful to have a couple default type views >>> that >> help standardize flow layout across the community. >> >>For example, when we organize processors left-to-right, failure >> relationships always point up, and success always point right. >>Alternatively, when we organize processors up-and-down, failure >> relationships always point left, and successes always point down. >> >>Of course, in some of these scenarios there are processors that have >> more than 1 success relationship, but that's when a good layout >>> library >> would come into play. >> >>What do you think? >> >> On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran >>> wrote: >> >>> Ryan - I think we spoke briefly (at a very high level) about this >>> at a >>> prior
Re: Auto-Organize Layout
I also think that some way to help layout the flow would be very useful. One of the hardest parts of creating an easy to read workflow is the designing a good layout, and in many cases I simply want to be able to select a few processors and line them up (horizontally or vertically). Also, it is sometimes too easy to mis-align processors by dragging them accidentally. It would be nice if there would be a way to "lock" the layout of a group of selected processors so they can't be accidentally moved. Also, the ability to "undo" a layout change or save a layout (ie, just save the position of a group of selected processors) so that we can revert to it would also be helpful. Another thing that could be helpful is a way to emphasize the more important processors, ie, the ones integral to the flow, as opposed to the many "PutFile" processors I have that are simply for logging (although I realize that sometimes a PutFile can be an integral part of a flow). I know I can select a group of processors and change the color, and also create a label to highlight a certain group of processors, but it might be nice to be able to make selected processors appear larger on the screen than others, to emphasize their importance. It is sometimes hard to follow the trail of a flow simply because there are many connections, so a way to highlight the "main" path might be useful. -Elli On Tuesday, January 19, 2016 4:20 PM, dan bresswrote: Maybe not exactly "auto-layout" but I would back a notion of having the components snap to a coarser grain grid than what we currently have. Sometimes I care a lot about having everything line up in the graph horizontally and vertically, and it always takes a long time to achieve this. I could see this being achieved by snapping the component to the same spot horizontally as the component above it when you move it underneath another component. Or some magical "auto snap" button that does its best to align everything with its nearest neighbors. On Tue, Jan 19, 2016 at 12:37 PM Ryan H wrote: > I like your idea Rob, that would help with lining up relationships too > (straight lines). > > On Matt's note, I don't think there should be a "standard" either, although > best practices are always out there. > > On Matt's note of putting failures up above processes, we do that too. > Totally depends on who made the flow first. Sometimes, people don't even > follow a convention in the same flow.xml file. > > For these reasons, I'd recommend alternate views to the flow. > > We have a couple projects that just allow you to rearrange a node-based > graph, based on your preference, hierarchy, circular, pyramid, etc. > > Applying this to NiFi, having a couple different default auto-layout > options that you can swap your current view to, but NOT change the original > flow, would be nice. > > It would let you walk into someone else's, potentially large, dataflow and > have a familiar way to view the flow. > > Ryan > > > On Tue, Jan 19, 2016 at 2:03 PM, Rob Moran wrote: > > > I agree with Matt's points. I was just replying with something similar > > basically saying I think trying to set a standard would not be > > well-received. > > > > I believe what could be more useful are layout tools that would help > users > > place components to help achieve their preferred layouts. For example, > the > > ability to align (left, right, center) components > > or horizontally/vertically distribute components evenly. Other features > > such as snap-to and/or smart-guides could make it easier for users to > > follow their organization's best practices when designing a flow. > > > > Rob > > > > On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarke < > matt.clarke@gmail.com > > > > > wrote: > > > > > Ryan, > > > > > > Setting a standard is a difficult thing to do. The > complexity > > > that can exist in many flows would make enforcing a standard difficult. > > The > > > first example you provide of success to points right while failures > point > > > up is not recommended. It would be better to have failures point down > > since > > > it is common to put labels over processor(s). Any relationships > pointing > > up > > > would pass through these labels making both the relationship box and > the > > > label hard to read. It is often coomon to see flows designed with a > > > combination of left to right and top to bottom design. > > > > > > Matt > > > > > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H > > > wrote: > > > > > > > Hi Rob, > > > > Yea we did, it was at the end of the meeting. > > > > > > > > I think it would be useful to have a couple default type views > that > > > > help standardize flow layout across the community. > > > > > > > > For example, when we organize processors left-to-right, failure > > > > relationships always point up, and success always point right. > > >
Re: Auto-Organize Layout
I agree with Matt's points. I was just replying with something similar basically saying I think trying to set a standard would not be well-received. I believe what could be more useful are layout tools that would help users place components to help achieve their preferred layouts. For example, the ability to align (left, right, center) components or horizontally/vertically distribute components evenly. Other features such as snap-to and/or smart-guides could make it easier for users to follow their organization's best practices when designing a flow. Rob On Tue, Jan 19, 2016 at 1:49 PM, Matthew Clarkewrote: > Ryan, > > Setting a standard is a difficult thing to do. The complexity > that can exist in many flows would make enforcing a standard difficult. The > first example you provide of success to points right while failures point > up is not recommended. It would be better to have failures point down since > it is common to put labels over processor(s). Any relationships pointing up > would pass through these labels making both the relationship box and the > label hard to read. It is often coomon to see flows designed with a > combination of left to right and top to bottom design. > > Matt > > On Tue, Jan 19, 2016 at 12:07 PM, Ryan H > wrote: > > > Hi Rob, > > Yea we did, it was at the end of the meeting. > > > > I think it would be useful to have a couple default type views that > > help standardize flow layout across the community. > > > > For example, when we organize processors left-to-right, failure > > relationships always point up, and success always point right. > > Alternatively, when we organize processors up-and-down, failure > > relationships always point left, and successes always point down. > > > > Of course, in some of these scenarios there are processors that have > > more than 1 success relationship, but that's when a good layout library > > would come into play. > > > > What do you think? > > > > On Tue, Jan 19, 2016 at 11:10 AM, Rob Moran wrote: > > > > > Ryan - I think we spoke briefly (at a very high level) about this at a > > > prior meetup. What alternate views did you have in mind, and in what > ways > > > do you think they'd be useful? > > > > > > Rob > > > > > > On Tue, Jan 19, 2016 at 10:56 AM, Ryan H > > > wrote: > > > > > > > It'd be pretty awesome if NiFi provided the ability to auto-organize > a > > > > layout. > > > > > > > > Maybe even just a auto-organized layout that doesn't change the > > flow.xml, > > > > just an alternate view. > > > > > > > > Looking at these demos here: http://js.cytoscape.org/#demos > > > > > > > > Ryan > > > > > > > > > >