Re: Default termination for relationships

[Joe Skora]

Mark and JoeW's comments make sense to me.  I think the default should be
the safest for the user and should follow the Principle of Least
Astonishment[1].  In a flow building UI it's natural to connect things
together, so not connecting is the special case.

That said, as Mark pointed out there are some cases where the processor
provides a relationship for testing, auditing, etc. and does not require a
connection.  In those cases the Relationship class' new AutoTerminate
property should eliminate the need to set AutoTerminate on a relationship
that's usually not going to be connected.  After that is propagated out to
the processors as appropriate I think this will be less of a problem.

[1] https://en.wikipedia.org/wiki/Principle_of_least_astonishment

On Mon, Apr 25, 2016 at 8:30 AM, Joe Witt  wrote:

> I agree Mark.  We should have the user make a simple explicit
> expression of intent to terminate data but we should not by default
> silently do so.  I can see how during early phases of building a brand
> new flow or experimenting with NiFi one could feel like they'd rather
> it just terminated data by default but I think that preference shifts
> over time.
>
> On Mon, Apr 25, 2016 at 8:23 AM, Mark Payne  wrote:
> > Manish,
> >
> > I think changing the default to auto terminate would be a rather
> dangerous
> > move. When
> > a user is creating a dataflow, it would be very easy to overlook one of
> the
> > relationships
> > on a Processor and forget to configure it. If it were configured to
> > auto-terminate, the data
> > that is routed to that relationship would be deleted, resulting in
> > unintentional data loss.
> >
> > This is why we chose to force the user to explicitly acknowledge that
> they
> > want a Relationship
> > to be auto-terminated. As Pierre noted, there is a ticket [1] that was
> > resolved that allows the
> > developer to auto-terminate a specific Relationship. However, this
> should be
> > used sparingly,
> > for cases where it is expected that the data routed to that relationship
> > will not be needed most
> > of the time.
> >
> > An example of where this may come into play is the 'original'
> Relationship
> > of the
> > MergeContent Processor. There are a few cases where we want the original
> > FlowFiles that
> > were merged together, so the 'original' Relationship does exist, but in
> most
> > cases we only
> > want the merged FlowFile. So auto-terminating the 'original'
> Relationship by
> > default may be
> > a reasonable thing to do.
> >
> > Do others have thoughts on the issue?
> >
> > Thanks
> > -Mark
> >
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-1003
> >
> >
> >
> > On Apr 24, 2016, at 6:03 AM, Pierre Villard  >
> > wrote:
> >
> > Hi,
> >
> > I may be wrong, but I think this is something that has been already
> > discussed for 1.0 branch. However, I had a quick look on JIRA and didn't
> > find anything...
> > Otherwise there is NIFI-1003 to allow a default auto-termination for some
> > relationships.
> >
> > Pierre
> >
> >
> > 2016-04-24 11:00 GMT+02:00 Oleg Zhurakousky <
> ozhurakou...@hortonworks.com>:
> >>
> >> Manish
> >>
> >> That is an interesting suggestion. I personally don't see issues with it
> >> and generally do believe it would improve user experience, but
> interested
> >> what others think.
> >>
> >> Cheers
> >> Oleg
> >>
> >> Sent from my iPhone
> >>
> >> On Apr 24, 2016, at 08:46, Manish Gupta 8  wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> Does it make sense to keep all the out-flowing relationships
> >> auto-terminated by default when a new processor is dragged in? When user
> >> connects the processor and specify a relationship, only the selected one
> >> becomes non-terminating.
> >>
> >>
> >>
> >> I think this will be good from usability point of view.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Manish
> >>
> >>
> >>
> >>
> >
> >
> >
>

Re: Default termination for relationships

[Joe Witt]

I agree Mark.  We should have the user make a simple explicit
expression of intent to terminate data but we should not by default
silently do so.  I can see how during early phases of building a brand
new flow or experimenting with NiFi one could feel like they'd rather
it just terminated data by default but I think that preference shifts
over time.

On Mon, Apr 25, 2016 at 8:23 AM, Mark Payne  wrote:
> Manish,
>
> I think changing the default to auto terminate would be a rather dangerous
> move. When
> a user is creating a dataflow, it would be very easy to overlook one of the
> relationships
> on a Processor and forget to configure it. If it were configured to
> auto-terminate, the data
> that is routed to that relationship would be deleted, resulting in
> unintentional data loss.
>
> This is why we chose to force the user to explicitly acknowledge that they
> want a Relationship
> to be auto-terminated. As Pierre noted, there is a ticket [1] that was
> resolved that allows the
> developer to auto-terminate a specific Relationship. However, this should be
> used sparingly,
> for cases where it is expected that the data routed to that relationship
> will not be needed most
> of the time.
>
> An example of where this may come into play is the 'original' Relationship
> of the
> MergeContent Processor. There are a few cases where we want the original
> FlowFiles that
> were merged together, so the 'original' Relationship does exist, but in most
> cases we only
> want the merged FlowFile. So auto-terminating the 'original' Relationship by
> default may be
> a reasonable thing to do.
>
> Do others have thoughts on the issue?
>
> Thanks
> -Mark
>
>
> [1] https://issues.apache.org/jira/browse/NIFI-1003
>
>
>
> On Apr 24, 2016, at 6:03 AM, Pierre Villard 
> wrote:
>
> Hi,
>
> I may be wrong, but I think this is something that has been already
> discussed for 1.0 branch. However, I had a quick look on JIRA and didn't
> find anything...
> Otherwise there is NIFI-1003 to allow a default auto-termination for some
> relationships.
>
> Pierre
>
>
> 2016-04-24 11:00 GMT+02:00 Oleg Zhurakousky :
>>
>> Manish
>>
>> That is an interesting suggestion. I personally don't see issues with it
>> and generally do believe it would improve user experience, but interested
>> what others think.
>>
>> Cheers
>> Oleg
>>
>> Sent from my iPhone
>>
>> On Apr 24, 2016, at 08:46, Manish Gupta 8  wrote:
>>
>> Hi,
>>
>>
>>
>> Does it make sense to keep all the out-flowing relationships
>> auto-terminated by default when a new processor is dragged in? When user
>> connects the processor and specify a relationship, only the selected one
>> becomes non-terminating.
>>
>>
>>
>> I think this will be good from usability point of view.
>>
>>
>>
>> Thanks,
>>
>> Manish
>>
>>
>>
>>
>
>
>

Re: Default termination for relationships

[Mark Payne]

Manish,

I think changing the default to auto terminate would be a rather dangerous 
move. When
a user is creating a dataflow, it would be very easy to overlook one of the 
relationships
on a Processor and forget to configure it. If it were configured to 
auto-terminate, the data
that is routed to that relationship would be deleted, resulting in 
unintentional data loss.

This is why we chose to force the user to explicitly acknowledge that they want 
a Relationship
to be auto-terminated. As Pierre noted, there is a ticket [1] that was resolved 
that allows the
developer to auto-terminate a specific Relationship. However, this should be 
used sparingly,
for cases where it is expected that the data routed to that relationship will 
not be needed most
of the time.

An example of where this may come into play is the 'original' Relationship of 
the
MergeContent Processor. There are a few cases where we want the original 
FlowFiles that
were merged together, so the 'original' Relationship does exist, but in most 
cases we only
want the merged FlowFile. So auto-terminating the 'original' Relationship by 
default may be
a reasonable thing to do.

Do others have thoughts on the issue?

Thanks
-Mark


[1] https://issues.apache.org/jira/browse/NIFI-1003 




> On Apr 24, 2016, at 6:03 AM, Pierre Villard  
> wrote:
> 
> Hi,
> 
> I may be wrong, but I think this is something that has been already discussed 
> for 1.0 branch. However, I had a quick look on JIRA and didn't find 
> anything...
> Otherwise there is NIFI-1003 
>  to allow a default 
> auto-termination for some relationships.
> 
> Pierre
> 
> 
> 2016-04-24 11:00 GMT+02:00 Oleg Zhurakousky  >:
> Manish
> 
> That is an interesting suggestion. I personally don't see issues with it and 
> generally do believe it would improve user experience, but interested what 
> others think.
> 
> Cheers 
> Oleg 
> 
> Sent from my iPhone
> 
> On Apr 24, 2016, at 08:46, Manish Gupta 8  > wrote:
> 
>> Hi,
>> 
>>  
>> 
>> Does it make sense to keep all the out-flowing relationships auto-terminated 
>> by default when a new processor is dragged in? When user connects the 
>> processor and specify a relationship, only the selected one becomes 
>> non-terminating.
>> 
>>  
>> 
>> I think this will be good from usability point of view.
>> 
>>  
>> 
>> Thanks,
>> 
>> Manish
>> 
>>  
>> 
>>  
>> 
> 

Re: Default termination for relationships

[Pierre Villard]

Hi,

I may be wrong, but I think this is something that has been already
discussed for 1.0 branch. However, I had a quick look on JIRA and didn't
find anything...
Otherwise there is NIFI-1003
 to allow a default
auto-termination for some relationships.

Pierre


2016-04-24 11:00 GMT+02:00 Oleg Zhurakousky :

> Manish
>
> That is an interesting suggestion. I personally don't see issues with it
> and generally do believe it would improve user experience, but interested
> what others think.
>
> Cheers
> Oleg
>
> Sent from my iPhone
>
> On Apr 24, 2016, at 08:46, Manish Gupta 8  wrote:
>
> Hi,
>
>
>
> Does it make sense to keep all the out-flowing relationships
> auto-terminated by default when a new processor is dragged in? When user
> connects the processor and specify a relationship, only the selected one
> becomes non-terminating.
>
>
>
> I think this will be good from usability point of view.
>
>
>
> Thanks,
>
> Manish
>
>
>
>
>
>

Re: Default termination for relationships

[Oleg Zhurakousky]

Manish

That is an interesting suggestion. I personally don't see issues with it and 
generally do believe it would improve user experience, but interested what 
others think.

Cheers
Oleg

Sent from my iPhone

On Apr 24, 2016, at 08:46, Manish Gupta 8 
> wrote:

Hi,

Does it make sense to keep all the out-flowing relationships auto-terminated by 
default when a new processor is dragged in? When user connects the processor 
and specify a relationship, only the selected one becomes non-terminating.

I think this will be good from usability point of view.

Thanks,
Manish

Default termination for relationships

[Manish Gupta 8]

Hi,

Does it make sense to keep all the out-flowing relationships auto-terminated by 
default when a new processor is dragged in? When user connects the processor 
and specify a relationship, only the selected one becomes non-terminating.

I think this will be good from usability point of view.

Thanks,
Manish