Re: [EXT] Re: Primary Only Content Migration

2018-06-08 Thread Joe Witt
t; >> > this is an important step to take soon. If you want to start >> >> > sketching out design ideas that would be awesome. >> >> > >> >> > Thanks >> >> > On Thu, Jun 7, 2018 at 6:11 PM Peter Wicks (pwicks) < >> pwi...

Re: [EXT] Re: Primary Only Content Migration

2018-06-08 Thread Brandon DeVries
gt; >> Joe, > >> >> > >> >> I agree it is a lot of work, which is why I was thinking of starting > >> with a processor that could do some of these operations before looking > >> further. If the processor could move flowfile's between nodes in th

Re: [EXT] Re: Primary Only Content Migration

2018-06-07 Thread Koji Kawamura
Original Message- >> From: Joe Witt [mailto:joe.w...@gmail.com] >> Sent: Thursday, June 7, 2018 3:34 PM >> To: dev@nifi.apache.org >> Subject: Re: [EXT] Re: Primary Only Content Migration >> >> Peter, >> >> It isn't a pattern that is

Re: [EXT] Re: Primary Only Content Migration

2018-06-07 Thread Joe Witt
> > -Original Message- > From: Joe Witt [mailto:joe.w...@gmail.com] > Sent: Thursday, June 7, 2018 3:34 PM > To: dev@nifi.apache.org > Subject: Re: [EXT] Re: Primary Only Content Migration > > Peter, > > It isn't a pattern that is well supported now in a cluster

RE: [EXT] Re: Primary Only Content Migration

2018-06-07 Thread Peter Wicks (pwicks)
: Joe Witt [mailto:joe.w...@gmail.com] Sent: Thursday, June 7, 2018 3:34 PM To: dev@nifi.apache.org Subject: Re: [EXT] Re: Primary Only Content Migration Peter, It isn't a pattern that is well supported now in a cluster context. What is needed are automatically load balanced connections

Re: [EXT] Re: Primary Only Content Migration

2018-06-07 Thread Joe Witt
so we send them to the primary node before continuing > processing. > > --Peter > > -Original Message- > From: Bryan Bende [mailto:bbe...@gmail.com] > Sent: Thursday, June 7, 2018 12:47 PM > To: dev@nifi.apache.org > Subject: [EXT] Re: Primary Only Content Migra

RE: [EXT] Re: Primary Only Content Migration

2018-06-07 Thread Peter Wicks (pwicks)
[mailto:bbe...@gmail.com] Sent: Thursday, June 7, 2018 12:47 PM To: dev@nifi.apache.org Subject: [EXT] Re: Primary Only Content Migration Peter, There really shouldn't be any non-source processors scheduled for primary node only. We may even want to consider preventing that option when

Re: Primary Only Content Migration

2018-06-07 Thread Bryan Bende
Peter, There really shouldn't be any non-source processors scheduled for primary node only. We may even want to consider preventing that option when the processor has an incoming connection to avoid creating any confusion. As long as you set source processors to primary node only then everything

Primary Only Content Migration

2018-06-07 Thread Peter Wicks (pwicks)
I'm sure many of you have the same situation, a flow that runs on a cluster, and at some point merges back down to a primary only processor; your files sit there in the queue with nowhere to go... We've used the work around of having a remote processor group that loops the data back to the