Re: Nifi-1325 - Enhancing Nifi AWS S3 for cross account access - Refactoring Nifi-AWS Processor credentials to use credentials provider

2016-01-08 Thread M Singh
Hi Aldrin:
The unfortunate things is that AWSCredentialsProvider does not inherit from 
AWSCredentials interface. 
As far as I can see, the provider interface is much more flexible and provides 
everything with we/anyone can need.  As we can see, the creds based 
constructors (AmazonS3/SQS/SNSClients) internally create a static creds 
provider instance.  If we support both the creds and creds provider based 
arguments, it could also confusing and error prone for developers extending the 
class.
Even if we have an adapter how would the subclass of AbstractAWSProcessor call 
createClient allow the two arguments (creds and creds provider) to work 
seemlessly. Let me know if you have any other thoughts/paths I can investigate.
Thanks for the feedback and I am learning a lot with this experience.
Mans 

On Friday, January 8, 2016 11:49 AM, Aldrin Piri  
wrote:
 

 Mans,

Thanks for sticking with this and continuing to see things through, the
community certainly appreciates it as these are very popular processors and
this functionality will help a wide base of users.

I am poking around a bit more and thinking we might be able to work
something out with a class that adapts an AWSCredentialsProvider to
AWSCredentials.  The AWSProvider interface is just composition of
AWSCredentials with a refresh method.  Need to mull things over a bit, and
dig through the associated libraries to understand how these are typically
used, but this feels like it could be another avenue to consider and where
I am directing my attention at the moment.

--aldrin

On Fri, Jan 8, 2016 at 9:58 AM, M Singh 
wrote:

> Just one more note Joe (as mentioned in the Jira Ticket)
> From what I can see, we cannot just deprecate createClient method in the
> AbstractAWSProcessor which uses the AWSCredentials argument, since the
> subclasses AbstractS3/SNS/SQSProcessor call that to create the respective
> clients.  We will have to change the argument to AWSCredentialProvider.
> If I can assist with any other investigation, please let me know.
> Thanks again.
>
>    On Friday, January 8, 2016 5:31 AM, M Singh
>  wrote:
>
>
>  Thanks Joe.
> If you think that we can accept the change to creds provider, I will work
> on making all the components in nifi aws processors to be consistent.  I
> think using the creds provider interface is the way to go since it is more
> flexible and at this moment we just have 3 aws processors to migrate.
> Looking forward to hearing from you/anyone else for advice/feedback.
> Mans
>
>    On Friday, January 8, 2016 5:18 AM, Joe Witt 
> wrote:
>
>
>  Mans,
>
> I am working to put out a proposed roadmap and then probably won't be
> very responsive until later tonight.  Will try to help then if no one
> else has had a chance to.
>
> That said I see what you mean in terms of a breaking change in the
> processor implementation as far as anyone else that has extended it.
> There have been some discussions recently about this and I think the
> plan is to start annotating everything with the audience and stability
> of a given bit of code.  Processors are not meant to be locked down
> APIs.  So, for now, given that it has been ambiguous to the community
> the best course is to probably just deprecate a given method if it
> cannot be safely repurposed and then use a new one which does meet the
> need in the event the controller service is supplied.  This last
> statement though is not based on me having looked at the code in any
> detail yet.
>
> Thanks
> Joe
>
> On Fri, Jan 8, 2016 at 8:08 AM, M Singh 
> wrote:
> > Hi Joe:
> > I have not worked with the controller interface and aws processors so
> perhaps you can help me understand it .
> > From what I can see (as mentioned in
> https://issues.apache.org/jira/browse/NIFI-1325):  currently, the Nifi
> AbstractAWSProcessor has a method
> > protected abstract ClientType createClient(final ProcessContext context,
> final AWSCredentials credentials, final ClientConfiguration config);
> > This method is overridden in the AbstractS3/SNS/SQSProcesors to provide
> the respective amazon s3/sns/sqs client using AWSCredentials argument.
> > Here is a snippet from AmazonS3Client:
> >    public AmazonS3Client(AWSCredentials awsCredentials,
> ClientConfiguration clientConfiguration) {
> super(clientConfiguration);        this.awsCredentialsProvider = new
> StaticCredentialsProvider(awsCredentials);        init();    }
> > So, AmazonS3/SNS/SQSClient created with AWSCredentials use
> StaticCredentialsProvider in their implementation.
> > All the AWSCredentials impls are static creds (Anonymous/Properties
> Credentials) except for the STSSessionCredentials which has a refresh
> method but is deprecated in favor of the STSSessionCredentialsProvider
> interface.  AWSCredentials is the interface being used in nifi aws
> processors.
> > The AWSCredentialsProvider 

S3 configuration

2016-01-08 Thread plj
Greetings,

  I'm trying to do a PutS3Object.  When I try to use it I get :
2016-01-08 15:43:17,518 INFO [Timer-Driven Process Thread-6]
com.amazonaws.http.AmazonHttpClient Unable to execute HTTP request:
Connection refused: conjava.net.ConnectException: Connection refused:
connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
~[na:1.8.0_66]
...

Can someone give me a clue as to what I'm doing wrong?  I set the bucket,
access key, secret key, and the region. Everything else I left as the
default.  Is there more I have to do?

thank you,

Patrick Jones



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/S3-configuration-tp6122.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.


Nifi + Oracle redo files?

2016-01-08 Thread Jason . Byers
Hello… I’ve been looking through the documentation and have not found any 
reference to Nifi reading Oracle redo files.

Are you aware of anyone who has done this successfully?  …or pursued a similar 
alternative to directly hitting transactional Oracle tables?

Thanks,

Jason


Re: Nifi + Oracle redo files?

2016-01-08 Thread Joe Witt
Jason,

I'm not aware of anyone reading Oracle redo logs via NiFi directly
yet.  More and more we're interacting with relational databases and
their related logs as a source but some are more challenging than
others to support in an open source capacity.

Our existing SQL oriented processors have been shown to work fine on
certain Oracle versions for pulling or pushing data into tables but
fall short of providing complete change data capture or bulk table
export as-is.  There is definitely more we can and should do there.

Are you aware of any libraries that are open source friendly and
support Oracle's redo logs?

Thanks
Joe

On Fri, Jan 8, 2016 at 4:53 PM, Jason.Byers  wrote:
> Hello… I’ve been looking through the documentation and have not found any 
> reference to Nifi reading Oracle redo files.
>
> Are you aware of anyone who has done this successfully?  …or pursued a 
> similar alternative to directly hitting transactional Oracle tables?
>
> Thanks,
>
> Jason


Re: Nifi-1325 - Enhancing Nifi AWS S3 for cross account access - Refactoring Nifi-AWS Processor credentials to use credentials provider

2016-01-08 Thread Joe Skora
An AWS controller service may be a good idea.

I've been talking to Tony about and working on state management for the S3
Multipart upload, to make sure that aborted uploads get cleaned up.  I'm
keeping it in the processor for now, but if there's other benefit to an AWS
controller service, such as credential management, it may make sense to
move the local and remote state management there too so it will be handled
in one central place regardless outside of any processors.

On Fri, Jan 8, 2016 at 8:31 AM, M Singh 
wrote:

> Thanks Joe.
> If you think that we can accept the change to creds provider, I will work
> on making all the components in nifi aws processors to be consistent.  I
> think using the creds provider interface is the way to go since it is more
> flexible and at this moment we just have 3 aws processors to migrate.
> Looking forward to hearing from you/anyone else for advice/feedback.
> Mans
>
> On Friday, January 8, 2016 5:18 AM, Joe Witt 
> wrote:
>
>
>  Mans,
>
> I am working to put out a proposed roadmap and then probably won't be
> very responsive until later tonight.  Will try to help then if no one
> else has had a chance to.
>
> That said I see what you mean in terms of a breaking change in the
> processor implementation as far as anyone else that has extended it.
> There have been some discussions recently about this and I think the
> plan is to start annotating everything with the audience and stability
> of a given bit of code.  Processors are not meant to be locked down
> APIs.  So, for now, given that it has been ambiguous to the community
> the best course is to probably just deprecate a given method if it
> cannot be safely repurposed and then use a new one which does meet the
> need in the event the controller service is supplied.  This last
> statement though is not based on me having looked at the code in any
> detail yet.
>
> Thanks
> Joe
>
> On Fri, Jan 8, 2016 at 8:08 AM, M Singh 
> wrote:
> > Hi Joe:
> > I have not worked with the controller interface and aws processors so
> perhaps you can help me understand it .
> > From what I can see (as mentioned in
> https://issues.apache.org/jira/browse/NIFI-1325):  currently, the Nifi
> AbstractAWSProcessor has a method
> > protected abstract ClientType createClient(final ProcessContext context,
> final AWSCredentials credentials, final ClientConfiguration config);
> > This method is overridden in the AbstractS3/SNS/SQSProcesors to provide
> the respective amazon s3/sns/sqs client using AWSCredentials argument.
> > Here is a snippet from AmazonS3Client:
> >public AmazonS3Client(AWSCredentials awsCredentials,
> ClientConfiguration clientConfiguration) {
> super(clientConfiguration);this.awsCredentialsProvider = new
> StaticCredentialsProvider(awsCredentials);init();}
> > So, AmazonS3/SNS/SQSClient created with AWSCredentials use
> StaticCredentialsProvider in their implementation.
> > All the AWSCredentials impls are static creds (Anonymous/Properties
> Credentials) except for the STSSessionCredentials which has a refresh
> method but is deprecated in favor of the STSSessionCredentialsProvider
> interface.  AWSCredentials is the interface being used in nifi aws
> processors.
> > The AWSCredentialsProvider interface has a fresh method which all it's
> subclasses implement appropriately - the static ones (like
> PropertyFileCredentialsProvider/StaticCredentialsProvider have a no op for
> refresh method) as follows:
> >public void refresh() {}
> > From what I can see, there is no common interface available for
> AWSCredentials and AWSCredentialsProvider that Nifi's
> AbstractAWSProcessor.createClient can support.  So if we need to use the
> controller interface with creds providers, will will have to change
> AbstractAWSProcessor.createClient to the following
> >
> > protected abstract ClientType createClient(final ProcessContext context,
> final AWSCredentialsProvider credentialsProvider,
> > final ClientConfiguration config);
> > This appears to be a breaking change for the clients who have extended
> the AbstractAWSProcessor.createClient with the AWSCredentials argument
> rather that the AWSCredentialsProvider.
> > So, can you please elaborate on how the AbstractAWSProcessor will be
> able to support both the current impl (ie, invoking aws components with
> creds) and the proposed credentials provider interface ?
> > Thanks
> > Mans
> >
> >On Thursday, January 7, 2016 9:00 PM, Joe Witt 
> wrote:
> >
> >
> >  Mans,
> >
> > It appears to me that there is a path for this not to be a breaking
> > change for the flow.  By creating a controller service to handle the
> > credential provider piece you should be able to just update the
> > processor to support that controller service interface.  If the user
> > sets that controller service then you use that and if they don't then
> > you revert to using 

Re: Nifi-1325 - Enhancing Nifi AWS S3 for cross account access - Refactoring Nifi-AWS Processor credentials to use credentials provider

2016-01-08 Thread Joe Witt
Mans,

I am working to put out a proposed roadmap and then probably won't be
very responsive until later tonight.  Will try to help then if no one
else has had a chance to.

That said I see what you mean in terms of a breaking change in the
processor implementation as far as anyone else that has extended it.
There have been some discussions recently about this and I think the
plan is to start annotating everything with the audience and stability
of a given bit of code.  Processors are not meant to be locked down
APIs.  So, for now, given that it has been ambiguous to the community
the best course is to probably just deprecate a given method if it
cannot be safely repurposed and then use a new one which does meet the
need in the event the controller service is supplied.  This last
statement though is not based on me having looked at the code in any
detail yet.

Thanks
Joe

On Fri, Jan 8, 2016 at 8:08 AM, M Singh  wrote:
> Hi Joe:
> I have not worked with the controller interface and aws processors so perhaps 
> you can help me understand it .
> From what I can see (as mentioned in 
> https://issues.apache.org/jira/browse/NIFI-1325):  currently, the Nifi 
> AbstractAWSProcessor has a method
> protected abstract ClientType createClient(final ProcessContext context, 
> final AWSCredentials credentials, final ClientConfiguration config);
> This method is overridden in the AbstractS3/SNS/SQSProcesors to provide the 
> respective amazon s3/sns/sqs client using AWSCredentials argument.
> Here is a snippet from AmazonS3Client:
> public AmazonS3Client(AWSCredentials awsCredentials, ClientConfiguration 
> clientConfiguration) {super(clientConfiguration);
> this.awsCredentialsProvider = new StaticCredentialsProvider(awsCredentials);  
>   init();}
> So, AmazonS3/SNS/SQSClient created with AWSCredentials use 
> StaticCredentialsProvider in their implementation.
> All the AWSCredentials impls are static creds (Anonymous/Properties 
> Credentials) except for the STSSessionCredentials which has a refresh method 
> but is deprecated in favor of the STSSessionCredentialsProvider interface.  
> AWSCredentials is the interface being used in nifi aws processors.
> The AWSCredentialsProvider interface has a fresh method which all it's 
> subclasses implement appropriately - the static ones (like 
> PropertyFileCredentialsProvider/StaticCredentialsProvider have a no op for 
> refresh method) as follows:
> public void refresh() {}
> From what I can see, there is no common interface available for 
> AWSCredentials and AWSCredentialsProvider that Nifi's 
> AbstractAWSProcessor.createClient can support.  So if we need to use the 
> controller interface with creds providers, will will have to change 
> AbstractAWSProcessor.createClient to the following
>
> protected abstract ClientType createClient(final ProcessContext context, 
> final AWSCredentialsProvider credentialsProvider,
> final ClientConfiguration config);
> This appears to be a breaking change for the clients who have extended the 
> AbstractAWSProcessor.createClient with the AWSCredentials argument rather 
> that the AWSCredentialsProvider.
> So, can you please elaborate on how the AbstractAWSProcessor will be able to 
> support both the current impl (ie, invoking aws components with creds) and 
> the proposed credentials provider interface ?
> Thanks
> Mans
>
> On Thursday, January 7, 2016 9:00 PM, Joe Witt  wrote:
>
>
>  Mans,
>
> It appears to me that there is a path for this not to be a breaking
> change for the flow.  By creating a controller service to handle the
> credential provider piece you should be able to just update the
> processor to support that controller service interface.  If the user
> sets that controller service then you use that and if they don't then
> you revert to using the older properties.  We can mark those
> properties as no longer the preferred model and deprecate them in the
> codebase then when we reach a 1.0 milestone we can remove them.
>
> Thanks
> Joe
>
> On Thu, Jan 7, 2016 at 9:07 PM, M Singh  wrote:
>> Hi:
>> Just wanted to mention that if we go with the creds provider interface it 
>> will be breaking change for nifi aws components as mentioned in 
>> https://issues.apache.org/jira/browse/NIFI-1325.
>> Also, I am considering creating one aws creds provider controller which will 
>> provide creds provider based on property file, basic, anonymous or assume 
>> role session params.
>> Please let me know if there is any additional feedback for me.
>> Thanks again.
>> Mans
>>
>>On Thursday, January 7, 2016 2:56 PM, M Singh 
>>  wrote:
>>
>>
>>  Hi Joe:
>> Based on your feedback I will try to explore the controller interface for 
>> aws creds provider.
>> Thanks for your advice.
>> Mans
>>
>>On Thursday, January 7, 2016 4:15 AM, Joe Witt  wrote:
>>
>>
>>  Mans
>>
>> 

Re: How to remote debug Jetty UI part

2016-01-08 Thread Matt Gilman
Lei,

Are you interested in debugging Jetty or NiFi's UI?

Assuming you're referring to the NiFi UI, you'll need to rebuild because by
default all the JS/CSS files are aggregated and minified. When you rebuild,
you'll need to disable the minify-and-compress maven profile. Assuming your
running from the command line it would look something like this:

$ mvn clean install -P\!minify-and-compress

The ! tells maven to disable the profile while the \ is escaping the ! in
your shell. If your building through an IDE then you likely wouldn't need
the !. The resulting build ends up in
nifi/nifi-assembly/target/nifi--bin/nifi-.

Once you launch NiFi you should be able to use the developer tools in your
browser to debug the client side Javascript that allows most of the UI to
happen.

If you're interested in debugging the server side of things (Jetty or NiFi
REST API), then you'll want to attach the remote debugging as you've
already done and add breakpoints to any Jetty class or the NiFI REST API
which is located here:

https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api

Let me know if you have any more questions.

Matt

On Fri, Jan 8, 2016 at 4:02 AM, 杨磊  wrote:

> Hi,
>
>
> I am a newbie to Nifi community. Although i read Hortonwork's announcement
> on the dataflow months ago, i just realize how amazing the Nifi is.
> However, when i am eager to learn more and more about Nifi, i got some
> trouble to debug it.
> Could you please tell me how to remote debug the Jetty UI part? Or
> introducing the regular way to develop this part to us?
> I have uncommented the remote debugging line on conf/boostrap.conf and
> attached my Intellij on it.
>
>
>
>
> Thanks in advance!
> Lei


Re: Nifi-1325 - Enhancing Nifi AWS S3 for cross account access - Refactoring Nifi-AWS Processor credentials to use credentials provider

2016-01-08 Thread M Singh
Just one more note Joe (as mentioned in the Jira Ticket)
>From what I can see, we cannot just deprecate createClient method in the 
>AbstractAWSProcessor which uses the AWSCredentials argument, since the 
>subclasses AbstractS3/SNS/SQSProcessor call that to create the respective 
>clients.  We will have to change the argument to AWSCredentialProvider.
If I can assist with any other investigation, please let me know.
Thanks again. 

On Friday, January 8, 2016 5:31 AM, M Singh  
wrote:
 

 Thanks Joe.
If you think that we can accept the change to creds provider, I will work on 
making all the components in nifi aws processors to be consistent.  I think 
using the creds provider interface is the way to go since it is more flexible 
and at this moment we just have 3 aws processors to migrate.
Looking forward to hearing from you/anyone else for advice/feedback.
Mans 

    On Friday, January 8, 2016 5:18 AM, Joe Witt  wrote:
 

 Mans,

I am working to put out a proposed roadmap and then probably won't be
very responsive until later tonight.  Will try to help then if no one
else has had a chance to.

That said I see what you mean in terms of a breaking change in the
processor implementation as far as anyone else that has extended it.
There have been some discussions recently about this and I think the
plan is to start annotating everything with the audience and stability
of a given bit of code.  Processors are not meant to be locked down
APIs.  So, for now, given that it has been ambiguous to the community
the best course is to probably just deprecate a given method if it
cannot be safely repurposed and then use a new one which does meet the
need in the event the controller service is supplied.  This last
statement though is not based on me having looked at the code in any
detail yet.

Thanks
Joe

On Fri, Jan 8, 2016 at 8:08 AM, M Singh  wrote:
> Hi Joe:
> I have not worked with the controller interface and aws processors so perhaps 
> you can help me understand it .
> From what I can see (as mentioned in 
> https://issues.apache.org/jira/browse/NIFI-1325):  currently, the Nifi 
> AbstractAWSProcessor has a method
> protected abstract ClientType createClient(final ProcessContext context, 
> final AWSCredentials credentials, final ClientConfiguration config);
> This method is overridden in the AbstractS3/SNS/SQSProcesors to provide the 
> respective amazon s3/sns/sqs client using AWSCredentials argument.
> Here is a snippet from AmazonS3Client:
>    public AmazonS3Client(AWSCredentials awsCredentials, ClientConfiguration 
>clientConfiguration) {        super(clientConfiguration);        
>this.awsCredentialsProvider = new StaticCredentialsProvider(awsCredentials);   
>     init();    }
> So, AmazonS3/SNS/SQSClient created with AWSCredentials use 
> StaticCredentialsProvider in their implementation.
> All the AWSCredentials impls are static creds (Anonymous/Properties 
> Credentials) except for the STSSessionCredentials which has a refresh method 
> but is deprecated in favor of the STSSessionCredentialsProvider interface.  
> AWSCredentials is the interface being used in nifi aws processors.
> The AWSCredentialsProvider interface has a fresh method which all it's 
> subclasses implement appropriately - the static ones (like 
> PropertyFileCredentialsProvider/StaticCredentialsProvider have a no op for 
> refresh method) as follows:
>    public void refresh() {}
> From what I can see, there is no common interface available for 
> AWSCredentials and AWSCredentialsProvider that Nifi's 
> AbstractAWSProcessor.createClient can support.  So if we need to use the 
> controller interface with creds providers, will will have to change 
> AbstractAWSProcessor.createClient to the following
>
> protected abstract ClientType createClient(final ProcessContext context, 
> final AWSCredentialsProvider credentialsProvider,
> final ClientConfiguration config);
> This appears to be a breaking change for the clients who have extended the 
> AbstractAWSProcessor.createClient with the AWSCredentials argument rather 
> that the AWSCredentialsProvider.
> So, can you please elaborate on how the AbstractAWSProcessor will be able to 
> support both the current impl (ie, invoking aws components with creds) and 
> the proposed credentials provider interface ?
> Thanks
> Mans
>
>    On Thursday, January 7, 2016 9:00 PM, Joe Witt  wrote:
>
>
>  Mans,
>
> It appears to me that there is a path for this not to be a breaking
> change for the flow.  By creating a controller service to handle the
> credential provider piece you should be able to just update the
> processor to support that controller service interface.  If the user
> sets that controller service then you use that and if they don't then
> you revert to using the older properties.  We can mark those
> properties as no longer the preferred model and deprecate them in the
> codebase 

Re: [DISCUSS] roadmap for the next 6-12 months

2016-01-08 Thread Tony Kurc
Joe! Looks amazing! Thanks for pulling this together
Hello NiFi Community

Please review the proposed roadmap and suggest things to add, remove,
update, prioritize differently.

Over the past year we have had great contributions that improved the
quality of NiFi and the community.  As we know there are many JIRAs
related to improvements or new features.  However, it can be difficult
for new and existing community members to know when to jump in, how to
help, or when things they are interested in might be able to happen.
So among other things we should provide and maintain a documented
roadmap to help this.

The following suggested timeline follows pretty strictly against our
previously stated objective of having releases roughly every 6 weeks.
We do not need to stay strict to this but a time bound (as opposed to
feature bound) release cycle does help provide a regular cadence to
follow and helps give contributors a sense of when their efforts might
end up in a release.

The items below are largely just focused on some of the major muscle
movements needed at a framework level.  There will of course be many
many more JIRAs with each releas.

NiFi 0.5.0 (Feb 5)

  State Management
https://cwiki.apache.org/confluence/display/NIFI/State+Management
  Kerberos

https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication
  Interactive Queue

https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management
  Scripting Support
https://issues.apache.org/jira/browse/NIFI-210

NiFi 0.6.0 (Mar 18)

  Deterministic Template Export
https://issues.apache.org/jira/browse/NIFI-826
  Schema/Format Editor (JSON to JSON,…?)
Numerous JIRAs related to format/schema conversion.  An example
implementation:

https://github.com/fsauer65/NiFi-Extensions/tree/master/nifi-jsontransform-bundle

NiFi 0.7.0 (Apr 29)

NiFi 1.0.0 (Jun 17)

  HA Cluster Management
https://cwiki.apache.org/confluence/display/NIFI/Clustering+Redesign
  HA Data

https://cwiki.apache.org/confluence/display/NIFI/High+Availability+Processing
  Multi-Tenant Authorization
https://cwiki.apache.org/confluence/display/NIFI/Multi-Tentant+Dataflow
  Improved Authorization API

https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication
NiFi's current authority provider gathers data then makes a decision.
We should instead support PDP/PEP model [4].
  Modernized UX
https://cwiki.apache.org/confluence/display/NIFI/Redesign+User+Interface
  Dependent Properties
https://issues.apache.org/jira/browse/NIFI-1121

NiFi 1.1.0 (Sep?)

  Template/Extension Registry
https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry
  Variable Registry
https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry

The timing of these things is pretty important as far as
interdependencies and how long they'll take to develop and validate.
State management in 050 sets up an important way to have cluster-wide
shared state and introduces zookeeper formally into NiFi which
obviously will be important for the distributed systems behaviors
we'll need as we restructure how clustering works and introduce data
replication for HA data processing.  The 060 and 070 releases are
intentionally quite light to allow for very focused work on a parallel
branch of 1.x which probably should start right on the heels of 050.
The 1.0 release will introduce some very key features, offer a chance
to remove deprecated classes, ensure all APIs are extremely clearly
documented for stability and audience, allow us to introduce UI/UX
changes, and so on.

Now, of course this is simply a proposal and these timings may not
work out.  But this should serve as a good basis for discussion and
consensus forming.

As we progress I plan to update the roadmap [1], product requirements
[2], and feature proposals [3] to reflect these things a bit more
clearly than they are now.

We should also have a discussion on how long we should be committed to
supporting the 0.x line and what that means.  We need to document a
commitment for the community.

[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851850
[2] https://cwiki.apache.org/confluence/display/NIFI/Product+requirements
[3] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals
[4] https://en.wikipedia.org/wiki/Common_Open_Policy_Service

Thanks
Joe