Re: Contributing Apache-2.0 licensed code from a 3rd party into NiFi

[Joe Witt]

Peter,

Broadly speaking there are a few ways code becomes part of the project
as I understand it.

1) An author, an individual contributor, provides a source code
contribution to the Apache project that is an original work of that
author.  This could be a patch attached to a JIRA or as a pull request
via github, etc..  The author of the code is the contributor
themselves and they're doing it as a contribution to the ASF and to
that apache project specifically.  You'd use our standard source
header and it would be ALv2 licensed entirely and can have
dependencies from any of the approved legal [1] licenses of type A or
B.

2) An author, an individual contributor, provides a source code
contribution to the Apache project that is one or the other or a
combination of of a work of the original author and includes copied
code from some other location.  The original work of the author is
just like #1 above.  The part they copy/pasted must be cited in the
nifi source NOTICE consistent with how we handle [2] licensing which
is meant to be consistent with ASF guidance and it must strictly be
source code that is clearly from the approved legal type category A
[3].  For whole class files copied in or for mostly fully copied items
you must keep the headers they had - not just add the standard ASF
ones as per [4]. Or you, if you're the original author/agent/owner of
the code can provide written permission to do so [4].

3) The author or representative of the author can submit code to the
Apache IP Clearance process as called out in [2] with the intent of
moving the work to the Apache project for follow on care and feeding.
This should start with a discuss thread to the community they wish to
transfer the project to and if that goes well result in a VOTE thread
with the project agreeing to take it on.  The code must be ALv2 (i
believe) and at that point would likely have packages changed, headers
changed, etc as part of the IP clearance and actual import process.
I'm sure there can be some variations here but i'm not quite sure to
be honest.  The IP clearance page describes the software grant
agreement process and such in detail towards the bottom as well.

#1 is super common and is the majority all day every day case we can
handle within the project easily.
#2 is less common but clear in handling and can easily be done within
the project.
#3 has never happened in NiFi as of yet but can be achieved if needed.

#1 does not appear to be an option (as far as I can tell) here since
it is already written elsewhere and has different headers/etc..
presumably and is not Copyright of ASF/etc..

#2 still is possible here.  You can just copy paste in relevant code
and we can maintain in nifi land from here forward.  It would require
NOTICE updates to reference the source code and any necessary NOTICE
entries such as Copyrights.  But, it would be forking the code which
is presumably a fork of NiFi so could be a bit strange to manage the
provenance of.  Would you keep maintaining it in the other repo too?
These sorts of things would need to be sorted out.

#3 is possible and requires your organization (presuming they own the
code) to help the community work through the IP clearance process if
the community decides it is a good thing to transfer over.

I am probably wrong on some of this and perhaps even significant bits.
But I think generally the above serves as a decent starting point for
discussion and understanding.

[1] https://www.apache.org/legal/resolved.html
[2] http://incubator.apache.org/ip-clearance/
[3] https://www.apache.org/legal/resolved.html#category-a
[4] https://www.apache.org/legal/src-headers.html

Thanks
Joe

On Tue, Jun 12, 2018 at 3:41 PM, Horváth Péter Gergely
 wrote:
> Hi All,
>
> I would like to make some contribution to NiFi out of a project that was
> released on GitHub under Apache-2.0. NiFi Test [1] would allow a NiFi
> instance to be started programmatically and unit-tests to be written
> against flow files.
>
> The code base has been properly released as open source under the terms of
> Apache-2.0 and now each file has the original licence header, something
> like this;
>
> Copyright (c) 2018 X, All Rights Reserved.
> Licensed under the Apache License, Version 2.0 (the "License");
> ...
>
> Based on my understanding this means that I have the green light to adapt
> and contribute such pieces of codes into NiFi, since the license is the
> same (Apache-2.0), and hence they are compatible.
>
> The question is: once I have a piece of software that was originally
> released by someone else but was adapted to NiFi, what should be the license
> header?
>
> Shall it retain the original copyright lines or should it contain the
> generic NiFi license headers:
>
> Licensed to the Apache Software Foundation (ASF) under one or more
> contributor license agreements.  See the NOTICE file distributed with
> this work for additional information regarding copyright ...
>
>
> I am not sure I understand what is the 

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Scott Aslan]

Aldrin,

Yes we do need INFRA to help us delete the rel/nifi-registry-0.1.0. The
node_modules for the gh-pages branch, however, have been purposely
committed to support the github.io demo 
.

On Tue, Jun 12, 2018 at 10:30 AM Kevin Doran  wrote:

> +1 (non-binding)
>
> Completed all the steps of the helper guide and played with the demo app.
> Verified the LN and Readme. Also verified version strings in the latest RC
> 3.
>
> Nice work on this everyone and thanks for RM'ing this initial release,
> Scott!
>
> Thanks,
> Kevin
>
> On 6/12/18, 10:12, "Matt Gilman"  wrote:
>
> +1 binding
>
> Ran through helper guide. Verified signatures, hashes, build, etc.
> Version
> strings look better with this RC. Provided other feedback regarding
> improvements to the demo app during RC1.
>
> Thanks for RMing this Scott!
>
> Matt
>
> On Tue, Jun 12, 2018 at 8:28 AM, Rob Moran  wrote:
>
> > +1
> >
> > * Stepped through the release guide to verify the sig, hashes, and
> commit
> > ID
> > * Checked for readme, notice, and license in source and target
> > * Checked for the package version update changes
> > * Ran the demo app – seems to be a slight delay initially loading up
> but
> > all looks good
> >
> > Thanks for all your effort on this, Scott!
> >
> >
> > On Tue, Jun 12, 2018 at 2:59 AM Pierre Villard <
> > pierre.villard...@gmail.com>
> > wrote:
> >
> > > +1, binding
> > >
> > > Went through the release helper guide and all looks good to me
> AFAICT.
> > > I merged the PR from Andy and set the fix version to 0.2.0.
> > > (could be a good idea to ask infra for a dedicated JIRA so we
> don't mix
> > > things regarding version numbers?)
> > >
> > > Congrats on this first release and thanks to everyone involved!
> > >
> > > Pierre
> > >
> > > 2018-06-12 1:02 GMT+02:00 Andy LoPresto :
> > >
> > > > Verified the GPG signature (SHA-512 digest, excellent) and
> checksums.
> > > > Ran through the build and deployed, testing the demo app.
> > > > Verified the LICENSE, NOTICE, and README in the source and
> deployable
> > > > application.
> > > > Checked the versioning in package.json and package-lock.json
> (not a
> > > > JS/node expert, but I think it’s right).
> > > >
> > > > (One tiny typo in the README, which I fixed in a PR).
> > > >
> > > > +1, binding
> > > >
> > > > Thanks for your continued work on this release, Scott.
> > > >
> > > >
> > > > Andy LoPresto
> > > > alopre...@apache.org
> > > > *alopresto.apa...@gmail.com *
> > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> EF69
> > > >
> > > > On Jun 11, 2018, at 1:50 PM, Scott Aslan 
> > wrote:
> > > >
> > > > Hello,
> > > >
> > > >
> > > > I am pleased to be calling this vote for the source release of
> Apache
> > > >
> > > > NiFi Flow Design System 0.1.0.
> > > >
> > > >
> > > > The source zip, including signatures, etc. can be found at:
> > > >
> > > >
> https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
> > > >
> > > >
> > > > The Git tag is nifi-fds-0.1.0-RC3
> > > >
> > > > The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184
> > > >
> > > > *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=
> > > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
> > > >  > > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184>*
> > > >
> > > >
> > > > Checksums of nifi-fds-0.1.0-source-release.zip:
> > > >
> > > > SHA1:
> > > >
> > > > ce971d78cc0660299dcee2233bc22fa78e2940af
> > > >
> > > > SHA256:
> > > >
> > > > 7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827
> > > >
> > > > SHA512:
> > > >
> > > > 11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16b
> > > >
> fed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15
> > > >
> > > >
> > > > Release artifacts are signed with the following key:
> > > >
> > > > https://people.apache.org/keys/committer/scottyaslan.asc
> > > >
> > > >
> > > > KEYS file available here:
> > > >
> > > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > >
> > > >
> > > > 7 issues were closed/resolved for this release:
> > > >
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > > > ctId=12316020=12343357
> > > >
> > > >
> > > > Release note highlights can be found here:
> > > >
> > > > https://cwiki.apache.org/confluence/display/NIFI/Release+
> > > > Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
> > > >
> > > >
> > > > The vote will be open for 72 hours.
> > > >

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Kevin Doran]

+1 (non-binding)

Completed all the steps of the helper guide and played with the demo app. 
Verified the LN and Readme. Also verified version strings in the latest RC 3.

Nice work on this everyone and thanks for RM'ing this initial release, Scott!

Thanks,
Kevin

On 6/12/18, 10:12, "Matt Gilman"  wrote:

+1 binding

Ran through helper guide. Verified signatures, hashes, build, etc. Version
strings look better with this RC. Provided other feedback regarding
improvements to the demo app during RC1.

Thanks for RMing this Scott!

Matt

On Tue, Jun 12, 2018 at 8:28 AM, Rob Moran  wrote:

> +1
>
> * Stepped through the release guide to verify the sig, hashes, and commit
> ID
> * Checked for readme, notice, and license in source and target
> * Checked for the package version update changes
> * Ran the demo app – seems to be a slight delay initially loading up but
> all looks good
>
> Thanks for all your effort on this, Scott!
>
>
> On Tue, Jun 12, 2018 at 2:59 AM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > +1, binding
> >
> > Went through the release helper guide and all looks good to me AFAICT.
> > I merged the PR from Andy and set the fix version to 0.2.0.
> > (could be a good idea to ask infra for a dedicated JIRA so we don't mix
> > things regarding version numbers?)
> >
> > Congrats on this first release and thanks to everyone involved!
> >
> > Pierre
> >
> > 2018-06-12 1:02 GMT+02:00 Andy LoPresto :
> >
> > > Verified the GPG signature (SHA-512 digest, excellent) and checksums.
> > > Ran through the build and deployed, testing the demo app.
> > > Verified the LICENSE, NOTICE, and README in the source and deployable
> > > application.
> > > Checked the versioning in package.json and package-lock.json (not a
> > > JS/node expert, but I think it’s right).
> > >
> > > (One tiny typo in the README, which I fixed in a PR).
> > >
> > > +1, binding
> > >
> > > Thanks for your continued work on this release, Scott.
> > >
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > *alopresto.apa...@gmail.com *
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > On Jun 11, 2018, at 1:50 PM, Scott Aslan 
> wrote:
> > >
> > > Hello,
> > >
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > >
> > > NiFi Flow Design System 0.1.0.
> > >
> > >
> > > The source zip, including signatures, etc. can be found at:
> > >
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
> > >
> > >
> > > The Git tag is nifi-fds-0.1.0-RC3
> > >
> > > The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184
> > >
> > > *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=
> > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
> > >  > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184>*
> > >
> > >
> > > Checksums of nifi-fds-0.1.0-source-release.zip:
> > >
> > > SHA1:
> > >
> > > ce971d78cc0660299dcee2233bc22fa78e2940af
> > >
> > > SHA256:
> > >
> > > 7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827
> > >
> > > SHA512:
> > >
> > > 11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16b
> > > fed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15
> > >
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/scottyaslan.asc
> > >
> > >
> > > KEYS file available here:
> > >
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > >
> > > 7 issues were closed/resolved for this release:
> > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > > ctId=12316020=12343357
> > >
> > >
> > > Release note highlights can be found here:
> > >
> > > https://cwiki.apache.org/confluence/display/NIFI/Release+
> > > Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
> > >
> > >
> > > The vote will be open for 72 hours.
> > >
> > >
> > > Please download the release candidate and evaluate the necessary items
> > >
> > > including checking hashes, signatures, build from source, and test.
> > >
> > >
> > > The please vote:
> > >
> > >
> > > [ ] +1 Release this package as nifi-fds-0.1.0
> > >
> > > [ ] +0 no opinion
> > >
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > >
> >
>

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Matt Gilman]

+1 binding

Ran through helper guide. Verified signatures, hashes, build, etc. Version
strings look better with this RC. Provided other feedback regarding
improvements to the demo app during RC1.

Thanks for RMing this Scott!

Matt

On Tue, Jun 12, 2018 at 8:28 AM, Rob Moran  wrote:

> +1
>
> * Stepped through the release guide to verify the sig, hashes, and commit
> ID
> * Checked for readme, notice, and license in source and target
> * Checked for the package version update changes
> * Ran the demo app – seems to be a slight delay initially loading up but
> all looks good
>
> Thanks for all your effort on this, Scott!
>
>
> On Tue, Jun 12, 2018 at 2:59 AM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > +1, binding
> >
> > Went through the release helper guide and all looks good to me AFAICT.
> > I merged the PR from Andy and set the fix version to 0.2.0.
> > (could be a good idea to ask infra for a dedicated JIRA so we don't mix
> > things regarding version numbers?)
> >
> > Congrats on this first release and thanks to everyone involved!
> >
> > Pierre
> >
> > 2018-06-12 1:02 GMT+02:00 Andy LoPresto :
> >
> > > Verified the GPG signature (SHA-512 digest, excellent) and checksums.
> > > Ran through the build and deployed, testing the demo app.
> > > Verified the LICENSE, NOTICE, and README in the source and deployable
> > > application.
> > > Checked the versioning in package.json and package-lock.json (not a
> > > JS/node expert, but I think it’s right).
> > >
> > > (One tiny typo in the README, which I fixed in a PR).
> > >
> > > +1, binding
> > >
> > > Thanks for your continued work on this release, Scott.
> > >
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > *alopresto.apa...@gmail.com *
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > On Jun 11, 2018, at 1:50 PM, Scott Aslan 
> wrote:
> > >
> > > Hello,
> > >
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > >
> > > NiFi Flow Design System 0.1.0.
> > >
> > >
> > > The source zip, including signatures, etc. can be found at:
> > >
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
> > >
> > >
> > > The Git tag is nifi-fds-0.1.0-RC3
> > >
> > > The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184
> > >
> > > *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=
> > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
> > >  > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184>*
> > >
> > >
> > > Checksums of nifi-fds-0.1.0-source-release.zip:
> > >
> > > SHA1:
> > >
> > > ce971d78cc0660299dcee2233bc22fa78e2940af
> > >
> > > SHA256:
> > >
> > > 7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827
> > >
> > > SHA512:
> > >
> > > 11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16b
> > > fed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15
> > >
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/scottyaslan.asc
> > >
> > >
> > > KEYS file available here:
> > >
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > >
> > > 7 issues were closed/resolved for this release:
> > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > > ctId=12316020=12343357
> > >
> > >
> > > Release note highlights can be found here:
> > >
> > > https://cwiki.apache.org/confluence/display/NIFI/Release+
> > > Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
> > >
> > >
> > > The vote will be open for 72 hours.
> > >
> > >
> > > Please download the release candidate and evaluate the necessary items
> > >
> > > including checking hashes, signatures, build from source, and test.
> > >
> > >
> > > The please vote:
> > >
> > >
> > > [ ] +1 Release this package as nifi-fds-0.1.0
> > >
> > > [ ] +0 no opinion
> > >
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > >
> >
>

Re: Unable to install Apachi nifi on my unix box

[Tony Kurc]

Srinivasarao,
I think you attempted to attach an image with the error which did not make
it through to the email list.

Tony

On Tue, Jun 12, 2018 at 10:03 AM, Sivaprasanna 
wrote:

> What error do you get when you run it? Please share it over here. BTW,
> which version of Apache NiFi are you using?
>
> Thanks,
> Sivaprasanna
>
> On Tue, Jun 12, 2018 at 7:24 PM, Srinivasa Rao Potla <
> srinivasarao.po...@infosys.com> wrote:
>
> > Hi Team,
> >
> >
> >
> > I have unable to run the below step , I have downloaded the as per github
> > steps .
> >
> >
> >
> > But unable to start the below step to start the apache nifi
> >
> > ./bin/nifi.sh start
> >
> >
> >
> >
> >
> >
> >
> > Regards,
> >
> > Srinivasarao Potla
> >
> > +91 8886210
> >
> > www.infosys.com
> >
> >
> >
>

Re: Unable to install Apachi nifi on my unix box

[Sivaprasanna]

What error do you get when you run it? Please share it over here. BTW,
which version of Apache NiFi are you using?

Thanks,
Sivaprasanna

On Tue, Jun 12, 2018 at 7:24 PM, Srinivasa Rao Potla <
srinivasarao.po...@infosys.com> wrote:

> Hi Team,
>
>
>
> I have unable to run the below step , I have downloaded the as per github
> steps .
>
>
>
> But unable to start the below step to start the apache nifi
>
> ./bin/nifi.sh start
>
>
>
>
>
>
>
> Regards,
>
> Srinivasarao Potla
>
> +91 8886210
>
> www.infosys.com
>
>
>

Unable to install Apachi nifi on my unix box

[Srinivasa Rao Potla]

Hi Team,

I have unable to run the below step , I have downloaded the as per github steps 
.

But unable to start the below step to start the apache nifi
./bin/nifi.sh start


[cid:image001.png@01D40283.03881BD0]

Regards,
Srinivasarao Potla
+91 8886210
www.infosys.com
[cid:image002.png@01D40283.03881BD0]

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Aldrin Piri]

+1, binding

comments:
* source commit looks correct
* hashes and signature looked good
* build was successful
* LICENSE and NOTICE looks good.  There was a small error referring to
Registry as being ASLv2 which should be ALv2.  A PR has been created to
correct this: https://github.com/apache/nifi-fds/pull/7
* Demo app ran successfully
* I know extremely little about npm.  Not sure if applicable but might be
helpful to list as a pre-requisite if a minimum version is needed.
Otherwise, please disregard.
* When checking out the source repository (not the voted upon distribution)
I noticed that repository is quite large at >80MB.  It seems there are two
items behind this:
  1) a nifi-registry release tag was pushed to this repo.
https://github.com/apache/nifi-fds/tree/rel/nifi-registry-0.1.0
Unfortunately, rel tags are typically protected but we should look to get
this removed from the repository with the help of INFRA (NIFI-5300),
  2) the gh-pages branch had some node modules slip in that were quite
large.  It might make sense to do a filter-branch/squash of some of these
commits as I believe this should not be an actively developed branch such
that we can bring the repo size down and remove the unnecessary binaries
(NIFI-5301)

On Tue, Jun 12, 2018 at 8:28 AM Rob Moran  wrote:

> +1
>
> * Stepped through the release guide to verify the sig, hashes, and commit
> ID
> * Checked for readme, notice, and license in source and target
> * Checked for the package version update changes
> * Ran the demo app – seems to be a slight delay initially loading up but
> all looks good
>
> Thanks for all your effort on this, Scott!
>
>
> On Tue, Jun 12, 2018 at 2:59 AM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > +1, binding
> >
> > Went through the release helper guide and all looks good to me AFAICT.
> > I merged the PR from Andy and set the fix version to 0.2.0.
> > (could be a good idea to ask infra for a dedicated JIRA so we don't mix
> > things regarding version numbers?)
> >
> > Congrats on this first release and thanks to everyone involved!
> >
> > Pierre
> >
> > 2018-06-12 1:02 GMT+02:00 Andy LoPresto :
> >
> > > Verified the GPG signature (SHA-512 digest, excellent) and checksums.
> > > Ran through the build and deployed, testing the demo app.
> > > Verified the LICENSE, NOTICE, and README in the source and deployable
> > > application.
> > > Checked the versioning in package.json and package-lock.json (not a
> > > JS/node expert, but I think it’s right).
> > >
> > > (One tiny typo in the README, which I fixed in a PR).
> > >
> > > +1, binding
> > >
> > > Thanks for your continued work on this release, Scott.
> > >
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > *alopresto.apa...@gmail.com *
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > On Jun 11, 2018, at 1:50 PM, Scott Aslan 
> wrote:
> > >
> > > Hello,
> > >
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > >
> > > NiFi Flow Design System 0.1.0.
> > >
> > >
> > > The source zip, including signatures, etc. can be found at:
> > >
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
> > >
> > >
> > > The Git tag is nifi-fds-0.1.0-RC3
> > >
> > > The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184
> > >
> > > *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=
> > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
> > >  > > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184>*
> > >
> > >
> > > Checksums of nifi-fds-0.1.0-source-release.zip:
> > >
> > > SHA1:
> > >
> > > ce971d78cc0660299dcee2233bc22fa78e2940af
> > >
> > > SHA256:
> > >
> > > 7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827
> > >
> > > SHA512:
> > >
> > > 11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16b
> > > fed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15
> > >
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/scottyaslan.asc
> > >
> > >
> > > KEYS file available here:
> > >
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > >
> > > 7 issues were closed/resolved for this release:
> > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > > ctId=12316020=12343357
> > >
> > >
> > > Release note highlights can be found here:
> > >
> > > https://cwiki.apache.org/confluence/display/NIFI/Release+
> > > Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
> > >
> > >
> > > The vote will be open for 72 hours.
> > >
> > >
> > > Please download the release candidate and evaluate the necessary items
> > >
> > > including checking hashes, signatures, build from source, and test.
> > >
> > >
> > > The please vote:
> > >
> > >
> > > [ ] +1 Release this package as nifi-fds-0.1.0
> > >
> > > [ ] +0 no opinion
> > >
> > > [ ] -1 Do not release this package because...
> > 

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Rob Moran]

+1

* Stepped through the release guide to verify the sig, hashes, and commit ID
* Checked for readme, notice, and license in source and target
* Checked for the package version update changes
* Ran the demo app – seems to be a slight delay initially loading up but
all looks good

Thanks for all your effort on this, Scott!


On Tue, Jun 12, 2018 at 2:59 AM Pierre Villard 
wrote:

> +1, binding
>
> Went through the release helper guide and all looks good to me AFAICT.
> I merged the PR from Andy and set the fix version to 0.2.0.
> (could be a good idea to ask infra for a dedicated JIRA so we don't mix
> things regarding version numbers?)
>
> Congrats on this first release and thanks to everyone involved!
>
> Pierre
>
> 2018-06-12 1:02 GMT+02:00 Andy LoPresto :
>
> > Verified the GPG signature (SHA-512 digest, excellent) and checksums.
> > Ran through the build and deployed, testing the demo app.
> > Verified the LICENSE, NOTICE, and README in the source and deployable
> > application.
> > Checked the versioning in package.json and package-lock.json (not a
> > JS/node expert, but I think it’s right).
> >
> > (One tiny typo in the README, which I fixed in a PR).
> >
> > +1, binding
> >
> > Thanks for your continued work on this release, Scott.
> >
> >
> > Andy LoPresto
> > alopre...@apache.org
> > *alopresto.apa...@gmail.com *
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On Jun 11, 2018, at 1:50 PM, Scott Aslan  wrote:
> >
> > Hello,
> >
> >
> > I am pleased to be calling this vote for the source release of Apache
> >
> > NiFi Flow Design System 0.1.0.
> >
> >
> > The source zip, including signatures, etc. can be found at:
> >
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
> >
> >
> > The Git tag is nifi-fds-0.1.0-RC3
> >
> > The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184
> >
> > *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=
> > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
> >  > commit;h=4f62f3d4a4626f66011289c7f1346c082a932184>*
> >
> >
> > Checksums of nifi-fds-0.1.0-source-release.zip:
> >
> > SHA1:
> >
> > ce971d78cc0660299dcee2233bc22fa78e2940af
> >
> > SHA256:
> >
> > 7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827
> >
> > SHA512:
> >
> > 11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16b
> > fed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15
> >
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/scottyaslan.asc
> >
> >
> > KEYS file available here:
> >
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> >
> > 7 issues were closed/resolved for this release:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> > ctId=12316020=12343357
> >
> >
> > Release note highlights can be found here:
> >
> > https://cwiki.apache.org/confluence/display/NIFI/Release+
> > Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
> >
> >
> > The vote will be open for 72 hours.
> >
> >
> > Please download the release candidate and evaluate the necessary items
> >
> > including checking hashes, signatures, build from source, and test.
> >
> >
> > The please vote:
> >
> >
> > [ ] +1 Release this package as nifi-fds-0.1.0
> >
> > [ ] +0 no opinion
> >
> > [ ] -1 Do not release this package because...
> >
> >
> >
>

Re: LookupService + flowfile attributes

[Mike Thomsen]

I think the only services that won't break if you merge the variables are
the ones that use only mandatory keys. Ones like MongoDBLookupService will
simply take the entire lookup coordinate map and turn it into a query. I
don't think there's an elegant solution to allow the service to pick and
choose in that case.

On Tue, Jun 12, 2018 at 4:43 AM Koji Kawamura 
wrote:

> Hi Mike,
>
> I'm still not sure which is better, separating the variables or
> containing all values.
> I wrote a comment so that we can keep the discussion there.
> https://github.com/apache/nifi/pull/2777#issuecomment-396512384
>
> Thanks,
> Koji
>
> On Tue, Jun 12, 2018 at 1:56 AM, Mike Thomsen 
> wrote:
> > Koji,
> >
> > After reading Mark's comments on GitHub, it occurred to me that the
> MongoDB
> > lookup service and the ES one I have as a PR would be screwed up if we
> take
> > the original approach because they blindly build a query from the total
> > coordinates set. So they'd add flowfile attributes as criteria by
> default.
> > I'll update the PR accordingly and make the new method default to the
> > existing one in all of the lookup services that are already there.
> >
> > On Sat, Jun 9, 2018 at 8:44 AM Mike Thomsen 
> wrote:
> >
> >> https://issues.apache.org/jira/browse/NIFI-5287
> >>
> >> On Sat, Jun 9, 2018 at 1:20 AM Koji Kawamura 
> >> wrote:
> >>
> >>> Thanks Mike for starting the discussion.
> >>>
> >>> Yes, I believe that will make LookupService and Schema access strategy
> >>> much easier, reusable, and useful.
> >>>
> >>> What I was imagined is not adding new method signature, but simply
> >>> copy certain FlowFile attributes into the coordinates map.
> >>> We can add that at LookupRecord.
> >>> Currently LookupAttribute only uses one coordinate value and can be
> >>> left as it is.
> >>>
> >>> Specifically, by adding new processor property, 'Copy FlowFile
> >>> Attributes into Coordinates', where user can define RegularExpression
> >>> to select which attributes to copy.
> >>> I think it's fine to mix FlowFile attributes and values defined as
> >>> dynamic properties into the same coordinates map.
> >>> The put oder should be FlowFile attributes, then dynamic properties,
> >>> so that user can overwrite attribute values when necessary.
> >>>
> >>> Koji
> >>>
> >>>
> >>> On Sat, Jun 9, 2018 at 1:06 AM, Mike Thomsen 
> >>> wrote:
> >>> > On the RestLookupService PR I think Koji mentioned the idea of
> expanding
> >>> > the lookup capability to include flowfile attributes. That sort of
> thing
> >>> > would be immensely useful on two PRs I have already open for lookup
> >>> service
> >>> > changes for ES and Mongo. Koji, add your thoughts, but what I'm
> thinking
> >>> > would be a new PR that adds:
> >>> >
> >>> > T lookup(Map flowfileAttributes, Map
> >>> > coordinates);
> >>> >
> >>> > to the LookupService interface and has the related processors pass in
> >>> the
> >>> > flowfile attribute map. Specifically, it would help make the schema
> >>> access
> >>> > capabilities really usable with lookup services (see
> >>> MongoDBLookupService
> >>> > PR for example; I added a new SchemaRegistryService type for JSON
> >>> sources)
> >>>
> >>
>

Re: Question on Docker module

[Peter Wilcsinszky]

Hi,

following up on my last email I've implemented both changes I was referring
to.

First I've added the the same integration tests, that dockermaven already
uses to cover the dockerhub image as well. Unfortunately for this I had to
use the latest public artifact (1.6.0) since only that is available
publicly. I've added it to the existing pull request which I look forward
to be merged very soon, to get into 1.7.0:
https://github.com/apache/nifi/pull/2747/commits/0c4f9a0ba4b64e258386ef3526c7b44b1f4ba9f7

Sidenote: with the latest snapshot build NiFi seems to bind on all
available interfaces instead of using only the primary by default, as it
did in 1.6.0 which I'm not sure is intentional or not. If not I need to
look deeper what's happening.

Second is what I mentioned as a theoretical option, to host the locally
built snapshot with it's checksum for the Docker build to consume. This way
we can use the dockerhub Dockerfile with locally built NiFi artifacts
easily with all it's advantages, but currently it only works on Docker for
Mac and Docker for Windows 18.03 onwards. I've implemented it using a
simple busybox container that is started before the Docker build and hosts
the content of the target folder locally where the NiFi artifact and
checksum are located:
https://github.com/pepov/nifi/commit/a1d9d8020fb579a50be51e67d98224cc86d985cc

I did not add this to the previous pull request, because I don't want to
hold it back. However it adds very little complexity to the Dockerfile
itself (one extra build arg), instead some tooling around the package and
integration-test phases of the docker profile in the nifi-docker module. I
know it's not ideal that portability is not guaranteed currently (bash,
curl, docker all can have their issues), but I hope it is only temporary
and as more people start to use it, it will converge to a fully
multiplatform solution. Until then the dockermaven modul can stay of course
to be available for legacy usage.

Any feedback would be appreciated!

Cheers,
Peter

On Thu, Jun 7, 2018 at 4:25 PM Peter Wilcsinszky 
wrote:

> Hi,
>
> Disclaimer: I've been working with the nifi-docker module for only a
> couple of weeks now so I'm relatively new to this.
> Unfortunately there's no way I'm aware of right now using a pure
> Dockerfile to implement a build time switch for cases like this where you
> have a a) locally built binary b) remote artifact.
> There is the ADD [1] instruction that combines something like that, but it
> falls short when used with a custom user and a big artifact, like in our
> case. The problem with that is that even if you provide the nifi user with
> --chown flag, docker uses that only to set the ownership of the downloaded
> archive, but then the extraction happens and ownership info will be used
> based on whatever was set in the archive [2].
> Given that, you will need one extra instruction to set the ownership of
> the extracted files and boom, you now have double the size of your image
> [3]. This is why it is wise to do the download and chown in one
> instruction, like in the dockerhub Dockerfile [4]
> But back to scenario a) You have the files built with maven locally, but
> you can't use curl to download it, because it is not hosted anywhere. While
> I'm writing this I can think of a theoretical option to implement something
> to host the file locally for the docker container to consume. With that we
> could avoid the separate maven Dockerfile, but until then we are left with
> the ADD instruction, following with a chown that makes that image almost
> 3G...
>
> For now I'm adding some integration tests for both scenarios [5], but
> would be glad to hear others feedback as well!
>
> [1] https://docs.docker.com/engine/reference/builder/#add
> [2] https://github.com/moby/moby/issues/35525#issuecomment-344908030
> [3] https://github.com/moby/moby/issues/22810#issuecomment-220021580
> [4]
> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/Dockerfile#L45
> [5] https://github.com/apache/nifi/pull/2747
>
> On Thu, Jun 7, 2018 at 3:37 PM Jeremy Dyer  wrote:
>
>> Sivaprasanna - They are very similar but different. Exists to primarily
>> assist with speeding up a developers dev cycle. Its purpose is to allow
>> you
>> to make source code changes to NiFi, build with maven (including docker
>> image), and then run the container with your latest changes. So in a sense
>> you can think of the docker maven module as providing you with the latest
>> from master. The dockerhub module on the other hand is used to build an
>> image for a stable Apache validated release. The current dockerhub build
>> will pull down Apache NiFi 1.6 from the Apache mirrors.
>>
>> TLDR: In cases outside of development you should use the dockerhub one.
>>
>> Thanks,
>> Jeremy Dyer
>>
>> On Thu, Jun 7, 2018 at 9:09 AM, Sivaprasanna 
>> wrote:
>>
>> > Team,
>> >
>> > In the project's nifi-docker
>> >  module, we
>> have
>> 

Re: LookupService + flowfile attributes

[Koji Kawamura]

Hi Mike,

I'm still not sure which is better, separating the variables or
containing all values.
I wrote a comment so that we can keep the discussion there.
https://github.com/apache/nifi/pull/2777#issuecomment-396512384

Thanks,
Koji

On Tue, Jun 12, 2018 at 1:56 AM, Mike Thomsen  wrote:
> Koji,
>
> After reading Mark's comments on GitHub, it occurred to me that the MongoDB
> lookup service and the ES one I have as a PR would be screwed up if we take
> the original approach because they blindly build a query from the total
> coordinates set. So they'd add flowfile attributes as criteria by default.
> I'll update the PR accordingly and make the new method default to the
> existing one in all of the lookup services that are already there.
>
> On Sat, Jun 9, 2018 at 8:44 AM Mike Thomsen  wrote:
>
>> https://issues.apache.org/jira/browse/NIFI-5287
>>
>> On Sat, Jun 9, 2018 at 1:20 AM Koji Kawamura 
>> wrote:
>>
>>> Thanks Mike for starting the discussion.
>>>
>>> Yes, I believe that will make LookupService and Schema access strategy
>>> much easier, reusable, and useful.
>>>
>>> What I was imagined is not adding new method signature, but simply
>>> copy certain FlowFile attributes into the coordinates map.
>>> We can add that at LookupRecord.
>>> Currently LookupAttribute only uses one coordinate value and can be
>>> left as it is.
>>>
>>> Specifically, by adding new processor property, 'Copy FlowFile
>>> Attributes into Coordinates', where user can define RegularExpression
>>> to select which attributes to copy.
>>> I think it's fine to mix FlowFile attributes and values defined as
>>> dynamic properties into the same coordinates map.
>>> The put oder should be FlowFile attributes, then dynamic properties,
>>> so that user can overwrite attribute values when necessary.
>>>
>>> Koji
>>>
>>>
>>> On Sat, Jun 9, 2018 at 1:06 AM, Mike Thomsen 
>>> wrote:
>>> > On the RestLookupService PR I think Koji mentioned the idea of expanding
>>> > the lookup capability to include flowfile attributes. That sort of thing
>>> > would be immensely useful on two PRs I have already open for lookup
>>> service
>>> > changes for ES and Mongo. Koji, add your thoughts, but what I'm thinking
>>> > would be a new PR that adds:
>>> >
>>> > T lookup(Map flowfileAttributes, Map
>>> > coordinates);
>>> >
>>> > to the LookupService interface and has the related processors pass in
>>> the
>>> > flowfile attribute map. Specifically, it would help make the schema
>>> access
>>> > capabilities really usable with lookup services (see
>>> MongoDBLookupService
>>> > PR for example; I added a new SchemaRegistryService type for JSON
>>> sources)
>>>
>>

Re: Custom Processor Bundle Contribution

[Prashanth Venkatesan]

Thanks Pierre for the reply. I will check and follow the procedure.

Regards,
Prashanth V

On Tue, Jun 12, 2018 at 1:01 PM, Pierre Villard  wrote:

> Hi Prashanth,
>
> First step is to create a JIRA describing the new feature you want to
> contribute [1].
> Then you can have a look at the contributor guide [2] to help you submit a
> pull request on Github [2].
>
> Let us know if something is not clear.
>
> [1] https://issues.apache.org/jira/projects/NIFI
> [2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide
>
> Pierre
>
> 2018-06-12 9:23 GMT+02:00 Prashanth Venkatesan  >:
>
> > Hi Team,
> >
> > I created network related custom processor and would like to contribute
> it
> > to NiFi source. I want to know the procedure to contribute to NiFi.
> Please
> > provide me the detailed guidelines..
> >
> > --
> > *With regards,*
> > Prashanth V
> >
>



-- 
*With regards,*
Prashanth V

Fwd: Custom Processor Bundle Contribution

[Prashanth Venkatesan]

Hi Team,

I created network related custom processor and would like to contribute it
to NiFi source. I want to know the procedure to contribute to NiFi. Please
provide me the detailed guidelines..

-- 
*With regards,*
Prashanth V

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Pierre Villard]

+1, binding

Went through the release helper guide and all looks good to me AFAICT.
I merged the PR from Andy and set the fix version to 0.2.0.
(could be a good idea to ask infra for a dedicated JIRA so we don't mix
things regarding version numbers?)

Congrats on this first release and thanks to everyone involved!

Pierre

2018-06-12 1:02 GMT+02:00 Andy LoPresto :

> Verified the GPG signature (SHA-512 digest, excellent) and checksums.
> Ran through the build and deployed, testing the demo app.
> Verified the LICENSE, NOTICE, and README in the source and deployable
> application.
> Checked the versioning in package.json and package-lock.json (not a
> JS/node expert, but I think it’s right).
>
> (One tiny typo in the README, which I fixed in a PR).
>
> +1, binding
>
> Thanks for your continued work on this release, Scott.
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 11, 2018, at 1:50 PM, Scott Aslan  wrote:
>
> Hello,
>
>
> I am pleased to be calling this vote for the source release of Apache
>
> NiFi Flow Design System 0.1.0.
>
>
> The source zip, including signatures, etc. can be found at:
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
>
>
> The Git tag is nifi-fds-0.1.0-RC3
>
> The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184
>
> *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=
> commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
>  commit;h=4f62f3d4a4626f66011289c7f1346c082a932184>*
>
>
> Checksums of nifi-fds-0.1.0-source-release.zip:
>
> SHA1:
>
> ce971d78cc0660299dcee2233bc22fa78e2940af
>
> SHA256:
>
> 7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827
>
> SHA512:
>
> 11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16b
> fed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15
>
>
> Release artifacts are signed with the following key:
>
> https://people.apache.org/keys/committer/scottyaslan.asc
>
>
> KEYS file available here:
>
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
>
> 7 issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> ctId=12316020=12343357
>
>
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+
> Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
>
>
> The vote will be open for 72 hours.
>
>
> Please download the release candidate and evaluate the necessary items
>
> including checking hashes, signatures, build from source, and test.
>
>
> The please vote:
>
>
> [ ] +1 Release this package as nifi-fds-0.1.0
>
> [ ] +0 no opinion
>
> [ ] -1 Do not release this package because...
>
>
>