Re: Default NiFi registry port
Thanks Bryan! -- Jagrut On Wed, Jun 20, 2018 at 12:18 PM, Bryan Bende wrote: > Hello, > > Since the port is configurable and can easily be changed I don't think > we would plan to change it. > > There are also lots of people who are not running NiFi Registry on the > same server as Spark History Server, so I don't think changing it just > for that makes sense. > > Thanks, > > Bryan > > > On Wed, Jun 20, 2018 at 3:02 PM, Jagrut Sharma > wrote: > > Hi - The default NiFi registry port is 18080, which is also the default > > port for Spark History Server UI. Due to this, the startup failed for the > > first time with 'Address already in use' exception. Changing it to 18081 > > resolve the issue. Just wanted to know if this is expected, or should the > > default port be changed in future versions to avoid this conflict. > > > > Thanks. > > -- > > Jagrut > -- Jagrut
Re: Default NiFi registry port
Hello, Since the port is configurable and can easily be changed I don't think we would plan to change it. There are also lots of people who are not running NiFi Registry on the same server as Spark History Server, so I don't think changing it just for that makes sense. Thanks, Bryan On Wed, Jun 20, 2018 at 3:02 PM, Jagrut Sharma wrote: > Hi - The default NiFi registry port is 18080, which is also the default > port for Spark History Server UI. Due to this, the startup failed for the > first time with 'Address already in use' exception. Changing it to 18081 > resolve the issue. Just wanted to know if this is expected, or should the > default port be changed in future versions to avoid this conflict. > > Thanks. > -- > Jagrut
Re: Apache NiFi 1.7.0 RC1 Release Helper Guide
I am working on a script to automate a bunch of this. I just created a work in progress PR if you would like to check it out. https://github.com/apache/nifi/pull/2806 Checking the commit is next thing on my list to automate. On June 20, 2018 at 10:52:23, Bryan Bende (bbe...@gmail.com) wrote: Correct that should all be fine, mainly there shouldn't be any differences in any module/src path. On Wed, Jun 20, 2018 at 10:48 AM, Mike Thomsen wrote: > I took your suggestion and got this: > > Only in nifi: .git > > Only in nifi: .gitignore > > Only in temp/nifi-1.7.0/: DEPENDENCIES > > Only in > nifi/nifi-nar-bundles/nifi-standard-services/nifi-kerberos-credentials-service-api: > .gitignore > > Only in > nifi/nifi-nar-bundles/nifi-standard-services/nifi-kerberos-credentials-service-bundle/nifi-kerberos-credentials-service: > .gitignore > > Only in > nifi/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services: > .gitignore > > > So that looks fine to me because I'd expect those to be excluded from an > official source release that doesn't have the git artifacts (and I don't > think DEPENDENCIES matters either unless I'm missing something) > > On Wed, Jun 20, 2018 at 10:11 AM Bryan Bende wrote: > >> Others may know a better way to do this, but the only way I know to >> truly verify the commit id is something like the following: >> >> git clone https://git-wip-us.apache.org/repos/asf/nifi.git >> git -C nifi checkout >> diff --brief -r > dir from above> >> >> For verifying the RC was branched off the correct git commit id, you >> look at the branch that was used to create the RC... >> >> So looking at the commit from the release email shows the JIRA was >> NIFI-5323 so there should be a branch like NIFI-5323-RC#: >> >> https://github.com/apache/nifi/commits/NIFI-5323-RC1 >> >> The "prepare" commit in there should line up with the commit >> referenced in the vote email, and should also be the commit referenced >> in the release tag: >> >> https://github.com/apache/nifi/commits/nifi-1.7.0-RC1 >> >> >> On Wed, Jun 20, 2018 at 9:59 AM, Kevin Doran >> wrote: >> > >> > >> > >> > >> > >> > >> > >> > Hi Mike, >> > These values are in the VOTE email: >> https://lists.apache.org/thread.html/d8bfef873317c5f681a5deb226d9dd9483aec56a7abc9a72090cb570@ >> >> > Cheers,Kevin >> > >> > >> > >> > >> > >> > >> > On Wed, Jun 20, 2018 at 6:55 AM -0700, "Mike Thomsen" < >> mikerthom...@gmail.com> wrote: >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Do we store these values somewhere in the zip? >> > >> > # Verify the git commit ID is correct >> > >> > # Verify the RC was branched off the correct git commit ID >> > >> > On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: >> > >> >> Hello Apache NiFi community, >> >> >> >> Please find the associated guidance to help those interested in >> >> validating/verifying the release so they can vote. >> >> >> >> # Download latest KEYS file: >> >> https://dist.apache.org/repos/dist/dev/nifi/KEYS >> >> >> >> # Import keys file: >> >> gpg --import KEYS >> >> >> >> # [optional] Clear out local maven artifact repository >> >> >> >> # Pull down nifi-1.7.0 source release artifacts for review: >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 >> >> >> >> # Verify the signature >> >> gpg --verify nifi-1.7.0-source-release.zip.asc >> >> >> >> # Verify the hashes (sha1, sha256, sha512) match the source and what was >> >> provided in the vote email thread >> >> shasum -a 1 nifi-1.7.0-source-release.zip >> >> shasum -a 256 nifi-1.7.0-source-release.zip >> >> shasum -a 512 nifi-1.7.0-source-release.zip >> >> >> >> # Unzip nifi-1.7.0-source-release.zip >> >> >> >> # Verify the build works including release audit tool (RAT) checks >> >> cd nifi-1.7.0 >> >> mvn clean install -Pcontrib-check,include-grpc >> >> >> >> # Verify the contents contain a good README, NOTICE, and LICENSE. >> >> >> >> # Verify the git commit ID is correct >> >> >> >> # Verify the RC was branched off the correct git commit ID >> >> >> >> # Look at the resulting convenience binary as found in >> nifi-assembly/target >> >> >> >> # Make sure the README, NOTICE, and LICENSE are present and correct >> >> >> >> # Run the resulting convenience binary and make sure it works as >> expected >> >> >> >> # Send a response to the vote thread indicating a +1, 0, -1 based on >> your >> >> findings. >> >> >> >> Thank you for your time and effort to validate the release! >> >> Andy LoPresto >> >>
Default NiFi registry port
Hi - The default NiFi registry port is 18080, which is also the default port for Spark History Server UI. Due to this, the startup failed for the first time with 'Address already in use' exception. Changing it to 18081 resolve the issue. Just wanted to know if this is expected, or should the default port be changed in future versions to avoid this conflict. Thanks. -- Jagrut
Re: Adding new data anonymization processor bundle
Andy, You raise a great point about considering the provenance. Unless there's a way to exclude attributes from provenance tracking, I think we'd need to force the issue by not allowing attributes to be an input source for expression language. That's the only way to kinda force people to think "hey, I shouldn't put this here." In my opinion, that's not really something we should allow given the ramifications of people using the feature without reading up on the relevant documentation. On Wed, Jun 20, 2018 at 1:35 PM Andy LoPresto wrote: > Sivaprasanna, > > Thanks for joining this effort. I don’t recall what’s on the existing > Jira, but please be very aware of the challenges in data anonymization and > the various threat models — de-anonymizing data can lead to the leak of > PII, EPHI, PCI data, etc. In some cases, it can even lead to physical > danger against persons. > > There are a number of high impact examples of avoidable scenarios like > this. > > > https://arstechnica.com/tech-policy/2009/09/your-secrets-live-online-in-databases-of-ruin/ > > > https://arstechnica.com/tech-policy/2014/06/poorly-anonymized-logs-reveal-nyc-cab-drivers-detailed-whereabouts/ > > We should use publicly reviewed algorithms, document the risks and known > challenges well, take into consideration provenance and other NiFi-specific > features, and write a good summary of these features if/when they are > introduced. > > Andy LoPresto > alopre...@apache.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 20, 2018, at 10:06, Sivaprasanna > wrote: > > > > Wow.. I dint realize there was a JIRA already. I'm interested and would > be > > happy to contribute my time & efforts on this. > > > >> On Wed, Jun 20, 2018 at 10:34 PM, Matt Burgess > wrote: > >> > >> I think is a great idea, I filed a Jira [1] a while ago in case > >> someone wanted to start working on it (or in case I got a chance). It > >> mentions ARX but any Apache-friendly implementation is of course > >> welcome. I think it should be in its own bundle as it is functionality > >> separate from all our other bundles (and not ubiquitous enough to put > >> in the standard NAR). > >> > >> Glad to hear you're interested in this, please feel free to reach out > >> with any questions and I too would be happy to review any > >> contributions. > >> > >> Thanks, > >> Matt > >> > >> [1] https://issues.apache.org/jira/browse/NIFI-4492 > >> > >> On Wed, Jun 20, 2018 at 12:57 PM Mike Thomsen > >> wrote: > >>> > >>> There's a framework called ARX that could very useful for this. The > only > >>> question you have is how compliant it would be with different sets of > >>> distinct legal requirements for privacy handling. In the absence of > >> strong > >>> legal guidance, I'd say err on the side of complying with health care > >>> regulations because that's where you're likely to find the clearest > >>> guidance and established tools. > >>> > >>> Ping me on any PR you send. > >>> > >>> On Wed, Jun 20, 2018 at 12:49 PM Sivaprasanna < > sivaprasanna...@gmail.com > >>> > >>> wrote: > >>> > With data becoming more critical and substantial to business > >> development, > new stringent regulations & law are getting introduced (GDPR being a > >> recent > example), I've been spending some time lately doing research on data > anonymization and after some hefty thinking, I finally decided to go > >> ahead > with the creation of new processor bundle that has processors like > 'AnonymizeRecord', 'DeanonymizeRecord' (not quite sure about the name > though). Following are my questions: > > - What do you guys think about these proposed processors? > - If the processors are okay to be introduced, are they "standard" > enough to get them added to our 'nifi-standard-bundles' module or > >> is it > better to keep it separated much like others like AWS, Azure > >> bundles, > etc. > > Having said this, I'm very much in the beginning phase with my > >> research and > development efforts so all your inputs & feedback on this one are > >> greatly > appreciated. > > Thanks. > > - > Sivaprasanna > > >> >
Re: [VOTE] Release Apache NiFi 1.7.0
+1 (binding) Verified all artifacts, full build with contrib-check, verified the Hive 3 NAR is not in the assembly unless the include-hive3 profile is activated, also ran through various flows to exercise Hive 3 and PutORC functionality (and their associated Record Readers, Writers, and intermediate processors). On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: > > Hello, > > I am pleased to be calling this vote for the source release of Apache NiFi > nifi-1.7.0. > > The source zip, including signatures, digests, etc. can be found at: > https://repository.apache.org/content/repositories/orgapachenifi-1127 > > and > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0 > > The Git tag is nifi-1.7.0-RC1 > The Git commit ID is 99bcd1f88dc826f857ae4ab33e842110bfc6ce21 > https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=99bcd1f88dc826f857ae4ab33e842110bfc6ce21 > > Checksums of nifi-1.7.0-source-release.zip: > SHA1: 11086ef532bb51462d7e1ac818f6308d4ac62f03 > SHA256: b616f985d486af3d05c04e375f952a4a5678f486017a2211657d5ba03aaaf563 > SHA512: > d81e9c6eb7fc51905d6f6629b25151fc3d8af7a3cd7cbc3aa03be390c0561858d614b62d8379a90fdb736fcf5c1b4832f4e050fdcfcd786e9615a0b5cc1d563d > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/alopresto.asc > > KEYS file available here: > https://dist.apache.org/repos/dist/release/nifi/KEYS > > 194 issues were closed/resolved for this release: > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342979=12316020 > > Release note highlights can be found here: > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.7.0 > > The vote will be open for 72 hours. > Please download the release candidate and evaluate the necessary items > including checking hashes, signatures, build > from source, and test. Then please vote: > > [ ] +1 Release this package as nifi-1.7.0 > [ ] +0 no opinion > [ ] -1 Do not release this package because… > > Andy LoPresto > alopre...@apache.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >
Re: Adding new data anonymization processor bundle
Sivaprasanna, Thanks for joining this effort. I don’t recall what’s on the existing Jira, but please be very aware of the challenges in data anonymization and the various threat models — de-anonymizing data can lead to the leak of PII, EPHI, PCI data, etc. In some cases, it can even lead to physical danger against persons. There are a number of high impact examples of avoidable scenarios like this. https://arstechnica.com/tech-policy/2009/09/your-secrets-live-online-in-databases-of-ruin/ https://arstechnica.com/tech-policy/2014/06/poorly-anonymized-logs-reveal-nyc-cab-drivers-detailed-whereabouts/ We should use publicly reviewed algorithms, document the risks and known challenges well, take into consideration provenance and other NiFi-specific features, and write a good summary of these features if/when they are introduced. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 20, 2018, at 10:06, Sivaprasanna wrote: > > Wow.. I dint realize there was a JIRA already. I'm interested and would be > happy to contribute my time & efforts on this. > >> On Wed, Jun 20, 2018 at 10:34 PM, Matt Burgess wrote: >> >> I think is a great idea, I filed a Jira [1] a while ago in case >> someone wanted to start working on it (or in case I got a chance). It >> mentions ARX but any Apache-friendly implementation is of course >> welcome. I think it should be in its own bundle as it is functionality >> separate from all our other bundles (and not ubiquitous enough to put >> in the standard NAR). >> >> Glad to hear you're interested in this, please feel free to reach out >> with any questions and I too would be happy to review any >> contributions. >> >> Thanks, >> Matt >> >> [1] https://issues.apache.org/jira/browse/NIFI-4492 >> >> On Wed, Jun 20, 2018 at 12:57 PM Mike Thomsen >> wrote: >>> >>> There's a framework called ARX that could very useful for this. The only >>> question you have is how compliant it would be with different sets of >>> distinct legal requirements for privacy handling. In the absence of >> strong >>> legal guidance, I'd say err on the side of complying with health care >>> regulations because that's where you're likely to find the clearest >>> guidance and established tools. >>> >>> Ping me on any PR you send. >>> >>> On Wed, Jun 20, 2018 at 12:49 PM Sivaprasanna >> >>> wrote: >>> With data becoming more critical and substantial to business >> development, new stringent regulations & law are getting introduced (GDPR being a >> recent example), I've been spending some time lately doing research on data anonymization and after some hefty thinking, I finally decided to go >> ahead with the creation of new processor bundle that has processors like 'AnonymizeRecord', 'DeanonymizeRecord' (not quite sure about the name though). Following are my questions: - What do you guys think about these proposed processors? - If the processors are okay to be introduced, are they "standard" enough to get them added to our 'nifi-standard-bundles' module or >> is it better to keep it separated much like others like AWS, Azure >> bundles, etc. Having said this, I'm very much in the beginning phase with my >> research and development efforts so all your inputs & feedback on this one are >> greatly appreciated. Thanks. - Sivaprasanna >>
Re: Adding new data anonymization processor bundle
Wow.. I dint realize there was a JIRA already. I'm interested and would be happy to contribute my time & efforts on this. On Wed, Jun 20, 2018 at 10:34 PM, Matt Burgess wrote: > I think is a great idea, I filed a Jira [1] a while ago in case > someone wanted to start working on it (or in case I got a chance). It > mentions ARX but any Apache-friendly implementation is of course > welcome. I think it should be in its own bundle as it is functionality > separate from all our other bundles (and not ubiquitous enough to put > in the standard NAR). > > Glad to hear you're interested in this, please feel free to reach out > with any questions and I too would be happy to review any > contributions. > > Thanks, > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-4492 > > On Wed, Jun 20, 2018 at 12:57 PM Mike Thomsen > wrote: > > > > There's a framework called ARX that could very useful for this. The only > > question you have is how compliant it would be with different sets of > > distinct legal requirements for privacy handling. In the absence of > strong > > legal guidance, I'd say err on the side of complying with health care > > regulations because that's where you're likely to find the clearest > > guidance and established tools. > > > > Ping me on any PR you send. > > > > On Wed, Jun 20, 2018 at 12:49 PM Sivaprasanna > > > wrote: > > > > > With data becoming more critical and substantial to business > development, > > > new stringent regulations & law are getting introduced (GDPR being a > recent > > > example), I've been spending some time lately doing research on data > > > anonymization and after some hefty thinking, I finally decided to go > ahead > > > with the creation of new processor bundle that has processors like > > > 'AnonymizeRecord', 'DeanonymizeRecord' (not quite sure about the name > > > though). Following are my questions: > > > > > >- What do you guys think about these proposed processors? > > >- If the processors are okay to be introduced, are they "standard" > > >enough to get them added to our 'nifi-standard-bundles' module or > is it > > >better to keep it separated much like others like AWS, Azure > bundles, > > > etc. > > > > > > Having said this, I'm very much in the beginning phase with my > research and > > > development efforts so all your inputs & feedback on this one are > greatly > > > appreciated. > > > > > > Thanks. > > > > > > - > > > Sivaprasanna > > > >
Re: Adding new data anonymization processor bundle
I think is a great idea, I filed a Jira [1] a while ago in case someone wanted to start working on it (or in case I got a chance). It mentions ARX but any Apache-friendly implementation is of course welcome. I think it should be in its own bundle as it is functionality separate from all our other bundles (and not ubiquitous enough to put in the standard NAR). Glad to hear you're interested in this, please feel free to reach out with any questions and I too would be happy to review any contributions. Thanks, Matt [1] https://issues.apache.org/jira/browse/NIFI-4492 On Wed, Jun 20, 2018 at 12:57 PM Mike Thomsen wrote: > > There's a framework called ARX that could very useful for this. The only > question you have is how compliant it would be with different sets of > distinct legal requirements for privacy handling. In the absence of strong > legal guidance, I'd say err on the side of complying with health care > regulations because that's where you're likely to find the clearest > guidance and established tools. > > Ping me on any PR you send. > > On Wed, Jun 20, 2018 at 12:49 PM Sivaprasanna > wrote: > > > With data becoming more critical and substantial to business development, > > new stringent regulations & law are getting introduced (GDPR being a recent > > example), I've been spending some time lately doing research on data > > anonymization and after some hefty thinking, I finally decided to go ahead > > with the creation of new processor bundle that has processors like > > 'AnonymizeRecord', 'DeanonymizeRecord' (not quite sure about the name > > though). Following are my questions: > > > >- What do you guys think about these proposed processors? > >- If the processors are okay to be introduced, are they "standard" > >enough to get them added to our 'nifi-standard-bundles' module or is it > >better to keep it separated much like others like AWS, Azure bundles, > > etc. > > > > Having said this, I'm very much in the beginning phase with my research and > > development efforts so all your inputs & feedback on this one are greatly > > appreciated. > > > > Thanks. > > > > - > > Sivaprasanna > >
Re: Adding new data anonymization processor bundle
There's a framework called ARX that could very useful for this. The only question you have is how compliant it would be with different sets of distinct legal requirements for privacy handling. In the absence of strong legal guidance, I'd say err on the side of complying with health care regulations because that's where you're likely to find the clearest guidance and established tools. Ping me on any PR you send. On Wed, Jun 20, 2018 at 12:49 PM Sivaprasanna wrote: > With data becoming more critical and substantial to business development, > new stringent regulations & law are getting introduced (GDPR being a recent > example), I've been spending some time lately doing research on data > anonymization and after some hefty thinking, I finally decided to go ahead > with the creation of new processor bundle that has processors like > 'AnonymizeRecord', 'DeanonymizeRecord' (not quite sure about the name > though). Following are my questions: > >- What do you guys think about these proposed processors? >- If the processors are okay to be introduced, are they "standard" >enough to get them added to our 'nifi-standard-bundles' module or is it >better to keep it separated much like others like AWS, Azure bundles, > etc. > > Having said this, I'm very much in the beginning phase with my research and > development efforts so all your inputs & feedback on this one are greatly > appreciated. > > Thanks. > > - > Sivaprasanna >
Adding new data anonymization processor bundle
With data becoming more critical and substantial to business development, new stringent regulations & law are getting introduced (GDPR being a recent example), I've been spending some time lately doing research on data anonymization and after some hefty thinking, I finally decided to go ahead with the creation of new processor bundle that has processors like 'AnonymizeRecord', 'DeanonymizeRecord' (not quite sure about the name though). Following are my questions: - What do you guys think about these proposed processors? - If the processors are okay to be introduced, are they "standard" enough to get them added to our 'nifi-standard-bundles' module or is it better to keep it separated much like others like AWS, Azure bundles, etc. Having said this, I'm very much in the beginning phase with my research and development efforts so all your inputs & feedback on this one are greatly appreciated. Thanks. - Sivaprasanna
Re: [VOTE] Release Apache NiFi 1.7.0
+1 binding. Everything seemed to match when I checked the sums, looked at the legal documentation and I tried a deliberately cumbersome Mongo-based flow and it worked just fine for me. Didn't try enabling security. On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: > Hello, > > I am pleased to be calling this vote for the source release of Apache NiFi > nifi-1.7.0. > > The source zip, including signatures, digests, etc. can be found at: > https://repository.apache.org/content/repositories/orgapachenifi-1127 > > and > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0 > > The Git tag is nifi-1.7.0-RC1 > The Git commit ID is 99bcd1f88dc826f857ae4ab33e842110bfc6ce21 > > https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=99bcd1f88dc826f857ae4ab33e842110bfc6ce21 > > Checksums of nifi-1.7.0-source-release.zip: > SHA1: 11086ef532bb51462d7e1ac818f6308d4ac62f03 > SHA256: b616f985d486af3d05c04e375f952a4a5678f486017a2211657d5ba03aaaf563 > SHA512: > d81e9c6eb7fc51905d6f6629b25151fc3d8af7a3cd7cbc3aa03be390c0561858d614b62d8379a90fdb736fcf5c1b4832f4e050fdcfcd786e9615a0b5cc1d563d > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/alopresto.asc > > KEYS file available here: > https://dist.apache.org/repos/dist/release/nifi/KEYS > > 194 issues were closed/resolved for this release: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342979=12316020 > > Release note highlights can be found here: > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.7.0 > > The vote will be open for 72 hours. > Please download the release candidate and evaluate the necessary items > including checking hashes, signatures, build > from source, and test. Then please vote: > > [ ] +1 Release this package as nifi-1.7.0 > [ ] +0 no opinion > [ ] -1 Do not release this package because… > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >
Re: Upgrading NiFi Registry
Thanks Mark and Bryan. I will add a NiFi Registry 0.1 -> 0.2 migration guide to include these steps as part of updating the site with news of the new release. Thanks, Kevin From: Mark Bean Sent: Wednesday, June 20, 2018 8:53:39 AM To: dev@nifi.apache.org Subject: Re: Upgrading NiFi Registry Thanks Bryan. There is actually another step not explicitly mentioned. At least for 0.1.0 -> 0.2.0, I needed to modify the nifi-registry.properties file as well. The 0.2.0 version has new properties/values not in the 0.1.0. And, I had to set the following for the database (using values from 0.1.0). This was only required on the first startup; these properties can be empty values on subsequent startups. nifi.registry.db.directory= nifi.registry.db.url.append= In the future, if using a database location that is external to the installation directory, is nifi.registry.db.url the only property that needs to be modified? On Wed, Jun 20, 2018 at 11:18 AM Bryan Bende wrote: > Mark, > > The database directory and flow storage directory are where all the > data are. By default these are created in the root of NiFi Registry, > so depending how you want to set it up you could move those > directories to the new install, or you could set them up to be > external locations so you don't have to move them every time, or you > could upgrade the lib directory of your current install and leave > everything in place. > > The policies are the same as NiFi... stored in users.xml and > authorizations.xml the conf directory, depending how you configured > everything. So just copying those two files over to the new install. > > -Bryan > > > On Wed, Jun 20, 2018 at 11:13 AM, Mark Bean wrote: > > How does one upgrade the NiFi Registry? > > > > After unpacking the .tar.gz file, how does one get all the flows > registered > > in a previous version of NiFi Registry into the newly installed version? > > And, how does one ensure all the policies transfer as well? > > > > Thanks, > > Mark >
Re: Upgrading NiFi Registry
Thanks Bryan. There is actually another step not explicitly mentioned. At least for 0.1.0 -> 0.2.0, I needed to modify the nifi-registry.properties file as well. The 0.2.0 version has new properties/values not in the 0.1.0. And, I had to set the following for the database (using values from 0.1.0). This was only required on the first startup; these properties can be empty values on subsequent startups. nifi.registry.db.directory= nifi.registry.db.url.append= In the future, if using a database location that is external to the installation directory, is nifi.registry.db.url the only property that needs to be modified? On Wed, Jun 20, 2018 at 11:18 AM Bryan Bende wrote: > Mark, > > The database directory and flow storage directory are where all the > data are. By default these are created in the root of NiFi Registry, > so depending how you want to set it up you could move those > directories to the new install, or you could set them up to be > external locations so you don't have to move them every time, or you > could upgrade the lib directory of your current install and leave > everything in place. > > The policies are the same as NiFi... stored in users.xml and > authorizations.xml the conf directory, depending how you configured > everything. So just copying those two files over to the new install. > > -Bryan > > > On Wed, Jun 20, 2018 at 11:13 AM, Mark Bean wrote: > > How does one upgrade the NiFi Registry? > > > > After unpacking the .tar.gz file, how does one get all the flows > registered > > in a previous version of NiFi Registry into the newly installed version? > > And, how does one ensure all the policies transfer as well? > > > > Thanks, > > Mark >
Re: Apache NiFi 1.7.0 RC1 Release Helper Guide
Correct that should all be fine, mainly there shouldn't be any differences in any module/src path. On Wed, Jun 20, 2018 at 10:48 AM, Mike Thomsen wrote: > I took your suggestion and got this: > > Only in nifi: .git > > Only in nifi: .gitignore > > Only in temp/nifi-1.7.0/: DEPENDENCIES > > Only in > nifi/nifi-nar-bundles/nifi-standard-services/nifi-kerberos-credentials-service-api: > .gitignore > > Only in > nifi/nifi-nar-bundles/nifi-standard-services/nifi-kerberos-credentials-service-bundle/nifi-kerberos-credentials-service: > .gitignore > > Only in > nifi/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services: > .gitignore > > > So that looks fine to me because I'd expect those to be excluded from an > official source release that doesn't have the git artifacts (and I don't > think DEPENDENCIES matters either unless I'm missing something) > > On Wed, Jun 20, 2018 at 10:11 AM Bryan Bende wrote: > >> Others may know a better way to do this, but the only way I know to >> truly verify the commit id is something like the following: >> >> git clone https://git-wip-us.apache.org/repos/asf/nifi.git >> git -C nifi checkout >> diff --brief -r > dir from above> >> >> For verifying the RC was branched off the correct git commit id, you >> look at the branch that was used to create the RC... >> >> So looking at the commit from the release email shows the JIRA was >> NIFI-5323 so there should be a branch like NIFI-5323-RC#: >> >> https://github.com/apache/nifi/commits/NIFI-5323-RC1 >> >> The "prepare" commit in there should line up with the commit >> referenced in the vote email, and should also be the commit referenced >> in the release tag: >> >> https://github.com/apache/nifi/commits/nifi-1.7.0-RC1 >> >> >> On Wed, Jun 20, 2018 at 9:59 AM, Kevin Doran >> wrote: >> > >> > >> > >> > >> > >> > >> > >> > Hi Mike, >> > These values are in the VOTE email: >> https://lists.apache.org/thread.html/d8bfef873317c5f681a5deb226d9dd9483aec56a7abc9a72090cb570@ >> >> > Cheers,Kevin >> > >> > >> > >> > >> > >> > >> > On Wed, Jun 20, 2018 at 6:55 AM -0700, "Mike Thomsen" < >> mikerthom...@gmail.com> wrote: >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Do we store these values somewhere in the zip? >> > >> > # Verify the git commit ID is correct >> > >> > # Verify the RC was branched off the correct git commit ID >> > >> > On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: >> > >> >> Hello Apache NiFi community, >> >> >> >> Please find the associated guidance to help those interested in >> >> validating/verifying the release so they can vote. >> >> >> >> # Download latest KEYS file: >> >> https://dist.apache.org/repos/dist/dev/nifi/KEYS >> >> >> >> # Import keys file: >> >> gpg --import KEYS >> >> >> >> # [optional] Clear out local maven artifact repository >> >> >> >> # Pull down nifi-1.7.0 source release artifacts for review: >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 >> >> wget >> >> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 >> >> >> >> # Verify the signature >> >> gpg --verify nifi-1.7.0-source-release.zip.asc >> >> >> >> # Verify the hashes (sha1, sha256, sha512) match the source and what was >> >> provided in the vote email thread >> >> shasum -a 1 nifi-1.7.0-source-release.zip >> >> shasum -a 256 nifi-1.7.0-source-release.zip >> >> shasum -a 512 nifi-1.7.0-source-release.zip >> >> >> >> # Unzip nifi-1.7.0-source-release.zip >> >> >> >> # Verify the build works including release audit tool (RAT) checks >> >> cd nifi-1.7.0 >> >> mvn clean install -Pcontrib-check,include-grpc >> >> >> >> # Verify the contents contain a good README, NOTICE, and LICENSE. >> >> >> >> # Verify the git commit ID is correct >> >> >> >> # Verify the RC was branched off the correct git commit ID >> >> >> >> # Look at the resulting convenience binary as found in >> nifi-assembly/target >> >> >> >> # Make sure the README, NOTICE, and LICENSE are present and correct >> >> >> >> # Run the resulting convenience binary and make sure it works as >> expected >> >> >> >> # Send a response to the vote thread indicating a +1, 0, -1 based on >> your >> >> findings. >> >> >> >> Thank you for your time and effort to validate the release! >> >> Andy LoPresto >> >> alopre...@apache.org >> >> *alopresto.apa...@gmail.com * >> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> >> >> >> > >> > >> > >> > >> > >>
Re: Apache NiFi 1.7.0 RC1 Release Helper Guide
I took your suggestion and got this: Only in nifi: .git Only in nifi: .gitignore Only in temp/nifi-1.7.0/: DEPENDENCIES Only in nifi/nifi-nar-bundles/nifi-standard-services/nifi-kerberos-credentials-service-api: .gitignore Only in nifi/nifi-nar-bundles/nifi-standard-services/nifi-kerberos-credentials-service-bundle/nifi-kerberos-credentials-service: .gitignore Only in nifi/nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services: .gitignore So that looks fine to me because I'd expect those to be excluded from an official source release that doesn't have the git artifacts (and I don't think DEPENDENCIES matters either unless I'm missing something) On Wed, Jun 20, 2018 at 10:11 AM Bryan Bende wrote: > Others may know a better way to do this, but the only way I know to > truly verify the commit id is something like the following: > > git clone https://git-wip-us.apache.org/repos/asf/nifi.git > git -C nifi checkout > diff --brief -r dir from above> > > For verifying the RC was branched off the correct git commit id, you > look at the branch that was used to create the RC... > > So looking at the commit from the release email shows the JIRA was > NIFI-5323 so there should be a branch like NIFI-5323-RC#: > > https://github.com/apache/nifi/commits/NIFI-5323-RC1 > > The "prepare" commit in there should line up with the commit > referenced in the vote email, and should also be the commit referenced > in the release tag: > > https://github.com/apache/nifi/commits/nifi-1.7.0-RC1 > > > On Wed, Jun 20, 2018 at 9:59 AM, Kevin Doran > wrote: > > > > > > > > > > > > > > > > Hi Mike, > > These values are in the VOTE email: > https://lists.apache.org/thread.html/d8bfef873317c5f681a5deb226d9dd9483aec56a7abc9a72090cb570@ > > > Cheers,Kevin > > > > > > > > > > > > > > On Wed, Jun 20, 2018 at 6:55 AM -0700, "Mike Thomsen" < > mikerthom...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Do we store these values somewhere in the zip? > > > > # Verify the git commit ID is correct > > > > # Verify the RC was branched off the correct git commit ID > > > > On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: > > > >> Hello Apache NiFi community, > >> > >> Please find the associated guidance to help those interested in > >> validating/verifying the release so they can vote. > >> > >> # Download latest KEYS file: > >> https://dist.apache.org/repos/dist/dev/nifi/KEYS > >> > >> # Import keys file: > >> gpg --import KEYS > >> > >> # [optional] Clear out local maven artifact repository > >> > >> # Pull down nifi-1.7.0 source release artifacts for review: > >> wget > >> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip > >> wget > >> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc > >> wget > >> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 > >> wget > >> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 > >> wget > >> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 > >> > >> # Verify the signature > >> gpg --verify nifi-1.7.0-source-release.zip.asc > >> > >> # Verify the hashes (sha1, sha256, sha512) match the source and what was > >> provided in the vote email thread > >> shasum -a 1 nifi-1.7.0-source-release.zip > >> shasum -a 256 nifi-1.7.0-source-release.zip > >> shasum -a 512 nifi-1.7.0-source-release.zip > >> > >> # Unzip nifi-1.7.0-source-release.zip > >> > >> # Verify the build works including release audit tool (RAT) checks > >> cd nifi-1.7.0 > >> mvn clean install -Pcontrib-check,include-grpc > >> > >> # Verify the contents contain a good README, NOTICE, and LICENSE. > >> > >> # Verify the git commit ID is correct > >> > >> # Verify the RC was branched off the correct git commit ID > >> > >> # Look at the resulting convenience binary as found in > nifi-assembly/target > >> > >> # Make sure the README, NOTICE, and LICENSE are present and correct > >> > >> # Run the resulting convenience binary and make sure it works as > expected > >> > >> # Send a response to the vote thread indicating a +1, 0, -1 based on > your > >> findings. > >> > >> Thank you for your time and effort to validate the release! > >> Andy LoPresto > >> alopre...@apache.org > >> *alopresto.apa...@gmail.com * > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> > >> > > > > > > > > > > >
Re: Apache NiFi 1.7.0 RC1 Release Helper Guide
Others may know a better way to do this, but the only way I know to truly verify the commit id is something like the following: git clone https://git-wip-us.apache.org/repos/asf/nifi.git git -C nifi checkout diff --brief -r For verifying the RC was branched off the correct git commit id, you look at the branch that was used to create the RC... So looking at the commit from the release email shows the JIRA was NIFI-5323 so there should be a branch like NIFI-5323-RC#: https://github.com/apache/nifi/commits/NIFI-5323-RC1 The "prepare" commit in there should line up with the commit referenced in the vote email, and should also be the commit referenced in the release tag: https://github.com/apache/nifi/commits/nifi-1.7.0-RC1 On Wed, Jun 20, 2018 at 9:59 AM, Kevin Doran wrote: > > > > > > > > Hi Mike, > These values are in the VOTE > email:https://lists.apache.org/thread.html/d8bfef873317c5f681a5deb226d9dd9483aec56a7abc9a72090cb570@ > Cheers,Kevin > > > > > > > On Wed, Jun 20, 2018 at 6:55 AM -0700, "Mike Thomsen" > wrote: > > > > > > > > > > > Do we store these values somewhere in the zip? > > # Verify the git commit ID is correct > > # Verify the RC was branched off the correct git commit ID > > On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: > >> Hello Apache NiFi community, >> >> Please find the associated guidance to help those interested in >> validating/verifying the release so they can vote. >> >> # Download latest KEYS file: >> https://dist.apache.org/repos/dist/dev/nifi/KEYS >> >> # Import keys file: >> gpg --import KEYS >> >> # [optional] Clear out local maven artifact repository >> >> # Pull down nifi-1.7.0 source release artifacts for review: >> wget >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip >> wget >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc >> wget >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 >> wget >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 >> wget >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 >> >> # Verify the signature >> gpg --verify nifi-1.7.0-source-release.zip.asc >> >> # Verify the hashes (sha1, sha256, sha512) match the source and what was >> provided in the vote email thread >> shasum -a 1 nifi-1.7.0-source-release.zip >> shasum -a 256 nifi-1.7.0-source-release.zip >> shasum -a 512 nifi-1.7.0-source-release.zip >> >> # Unzip nifi-1.7.0-source-release.zip >> >> # Verify the build works including release audit tool (RAT) checks >> cd nifi-1.7.0 >> mvn clean install -Pcontrib-check,include-grpc >> >> # Verify the contents contain a good README, NOTICE, and LICENSE. >> >> # Verify the git commit ID is correct >> >> # Verify the RC was branched off the correct git commit ID >> >> # Look at the resulting convenience binary as found in nifi-assembly/target >> >> # Make sure the README, NOTICE, and LICENSE are present and correct >> >> # Run the resulting convenience binary and make sure it works as expected >> >> # Send a response to the vote thread indicating a +1, 0, -1 based on your >> findings. >> >> Thank you for your time and effort to validate the release! >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com * >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> > > > > >
Re: Securing Nifi registry with nginx Error
> I am working on an ubuntu server. I do not have the possibility to
generate the keychain and to access the graphical interface of nifi
Where did you get the certificates if you are not able to generate the
keychain yourself? It looks like whatever server cert you use for nginx and
for the registry are not part of the same trust chain. Also, as far as I
know, you cannot just proxy the identity of the user identified by nginx to
the registry; its X509 support assumes that it's going to get the user
cert, not just a DN.
If you want some prebuilt certs for testing this or deploying into an
environment that's not meant for production use you can steal some of the
ones I created for these Docker Compose configurations:
https://github.com/MikeThomsen/nifi-docker-compose
Example server DNs are demo.nif, prov.nifi and registry.nifi. Used the TLS
Toolkit for those so should be plug and play if you need them for testing.
On Wed, Jun 20, 2018 at 8:52 AM amira...@gmail.com
wrote:
> I followed this tutorial to set up a secure version of Nifi registry:
> https://community.hortonworks.com/content/kbentry/170966/setting-up-a-secure-apache-nifi-registry.html
>
> I am working on an ubuntu server. I do not have the possibility to
> generate the keychain and to access the graphical interface of nifi I use
> google chrome on my local machine (windows10). So I imported the p12 file
> in my browser. My nginx configuration file is as follows:
>
> upstream container {
> server 172.0.0.2:9000;
> }
> server {
> listen 443 ssl;
> ssl On;
> ssl_certificate /etc/letsencrypt/live/sm/fullchain.pem;
> #/etc/nginx/ssl/fullchain.$
> ssl_certificate_key /etc/letsencrypt/live/sm/privkey.pem;
> #/etc/nginx/ssl/privkey$
>if ($ssl_protocol = "") {
> rewrite ^ https://$host$request_uri? permanent; # optional, to
> force use of$
> }
> root /var/www/html;
> # Add index.php to the list if you are using PHP
> index index.html index.htm index.php;
>server_name workshop1.smart-mobility.alstom.com; # managed by
> Certbot
>
>
> location ~ \.php$ {
>include snippets/fastcgi-php.conf;
> fastcgi_pass unix:/run/php/php7.0-fpm.sock;
> auth_basic "Restricted";
> auth_basic_user_file /etc/nginx/.htpasswd;
> }
> location ~ /\.ht {
> deny all;}
>
> location / {
> # First attempt to serve request as file, then
> # as directory, then fall back to displaying a 404.
> try_files $uri $uri/ =404;
> auth_basic "Restricted";auth_basic_user_file
> /etc/nginx/.htpasswd;
> }
>
> location /nifi-registry-api/ {
>rewrite ^/nifi-registry-api/(.*) /nifi-registry-api/$1 break;
>proxy_pass https://localhost:18443/nifi-registry;
>proxy_http_version 1.1;
>proxy_set_header Upgrade $http_upgrade;
>proxy_set_header Connection 'upgrade';
>proxy_set_header Host $host;
>proxy_cache_bypass $http_upgrade;
> }
>
> location /nifi-registry/ {
> proxy_pass https://localhost:18443/nifi-registry;
> proxy_http_version 1.1;
> proxy_set_header Upgrade $http_upgrade;
> proxy_set_header Connection 'upgrade';
> proxy_set_header Host $host;
> proxy_cache_bypass $http_upgrade;
> proxy_set_header X-ProxyScheme "https";
> proxy_set_header X-ProxyHost $proxy_host;
> proxy_set_header X-ProxiedEntitiesChain "<%{SSL_CLIENT_S_DN}>";
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Scheme $scheme;
> proxy_connect_timeout 1;
>
>} }
>
>
> When I log on to the nifi-registry page I have the following error: 502
> Bad Gateway
>
> can someone help me on this point please I do not find examples
>
> Error log nginx :
>
> *28739 SSL_do_handshake() failed (SSL: error:14094412:SSL
> routines:ssl3_read_bytes:sslv3 alert bad certificate:SSL alert number 42)
> while SSL hands
>
>
>
Re: Apache NiFi 1.7.0 RC1 Release Helper Guide
Hi Mike, These values are in the VOTE email:https://lists.apache.org/thread.html/d8bfef873317c5f681a5deb226d9dd9483aec56a7abc9a72090cb570@ Cheers,Kevin On Wed, Jun 20, 2018 at 6:55 AM -0700, "Mike Thomsen" wrote: Do we store these values somewhere in the zip? # Verify the git commit ID is correct # Verify the RC was branched off the correct git commit ID On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: > Hello Apache NiFi community, > > Please find the associated guidance to help those interested in > validating/verifying the release so they can vote. > > # Download latest KEYS file: > https://dist.apache.org/repos/dist/dev/nifi/KEYS > > # Import keys file: > gpg --import KEYS > > # [optional] Clear out local maven artifact repository > > # Pull down nifi-1.7.0 source release artifacts for review: > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 > > # Verify the signature > gpg --verify nifi-1.7.0-source-release.zip.asc > > # Verify the hashes (sha1, sha256, sha512) match the source and what was > provided in the vote email thread > shasum -a 1 nifi-1.7.0-source-release.zip > shasum -a 256 nifi-1.7.0-source-release.zip > shasum -a 512 nifi-1.7.0-source-release.zip > > # Unzip nifi-1.7.0-source-release.zip > > # Verify the build works including release audit tool (RAT) checks > cd nifi-1.7.0 > mvn clean install -Pcontrib-check,include-grpc > > # Verify the contents contain a good README, NOTICE, and LICENSE. > > # Verify the git commit ID is correct > > # Verify the RC was branched off the correct git commit ID > > # Look at the resulting convenience binary as found in nifi-assembly/target > > # Make sure the README, NOTICE, and LICENSE are present and correct > > # Run the resulting convenience binary and make sure it works as expected > > # Send a response to the vote thread indicating a +1, 0, -1 based on your > findings. > > Thank you for your time and effort to validate the release! > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >
Re: Apache NiFi 1.7.0 RC1 Release Helper Guide
Do we store these values somewhere in the zip? # Verify the git commit ID is correct # Verify the RC was branched off the correct git commit ID On Wed, Jun 20, 2018 at 3:16 AM Andy LoPresto wrote: > Hello Apache NiFi community, > > Please find the associated guidance to help those interested in > validating/verifying the release so they can vote. > > # Download latest KEYS file: > https://dist.apache.org/repos/dist/dev/nifi/KEYS > > # Import keys file: > gpg --import KEYS > > # [optional] Clear out local maven artifact repository > > # Pull down nifi-1.7.0 source release artifacts for review: > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 > wget > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 > > # Verify the signature > gpg --verify nifi-1.7.0-source-release.zip.asc > > # Verify the hashes (sha1, sha256, sha512) match the source and what was > provided in the vote email thread > shasum -a 1 nifi-1.7.0-source-release.zip > shasum -a 256 nifi-1.7.0-source-release.zip > shasum -a 512 nifi-1.7.0-source-release.zip > > # Unzip nifi-1.7.0-source-release.zip > > # Verify the build works including release audit tool (RAT) checks > cd nifi-1.7.0 > mvn clean install -Pcontrib-check,include-grpc > > # Verify the contents contain a good README, NOTICE, and LICENSE. > > # Verify the git commit ID is correct > > # Verify the RC was branched off the correct git commit ID > > # Look at the resulting convenience binary as found in nifi-assembly/target > > # Make sure the README, NOTICE, and LICENSE are present and correct > > # Run the resulting convenience binary and make sure it works as expected > > # Send a response to the vote thread indicating a +1, 0, -1 based on your > findings. > > Thank you for your time and effort to validate the release! > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >
Re: Setting values as System properties
I agree it is not what we would hope for. But I have not found any information to contradict you. The proxy setting we use now for Google Cloud Storage apparently does not apply to Google Cloud PubSub. What user experience would you propose? It seems reasonable that a user might be able to configure the setting globally, if they know that is what they are doing. It would be bad if configuring the properties on one processor set a global proxy setting that had a wide impact. It might be better to simply document that proxy use requires the configuration of environment variables with a global impact. On Tue, Jun 19, 2018 at 6:06 AM Sivaprasanna wrote: > Team, > > As part of NIFI-5133, I started doing some bit of research on Google Cloud > PubSub service and it's support on proxy configuration and came to know > that the service uses 'gRPC' so the proxy configuration is expected to be > at the System properties level. I know this approach is not good and > believe it has to be avoided but wanted to know the community's thoughts on > this. > > Thanks, > Sivaprasanna >
Securing Nifi registry with nginx Error
I followed this tutorial to set up a secure version of Nifi registry:
https://community.hortonworks.com/content/kbentry/170966/setting-up-a-secure-apache-nifi-registry.html
I am working on an ubuntu server. I do not have the possibility to generate the
keychain and to access the graphical interface of nifi I use google chrome on
my local machine (windows10). So I imported the p12 file in my browser. My
nginx configuration file is as follows:
upstream container {
server 172.0.0.2:9000;
}
server {
listen 443 ssl;
ssl On;
ssl_certificate /etc/letsencrypt/live/sm/fullchain.pem;
#/etc/nginx/ssl/fullchain.$
ssl_certificate_key /etc/letsencrypt/live/sm/privkey.pem;
#/etc/nginx/ssl/privkey$
if ($ssl_protocol = "") {
rewrite ^ https://$host$request_uri? permanent; # optional, to force
use of$
}
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.php;
server_name workshop1.smart-mobility.alstom.com; # managed by Certbot
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/.htpasswd;
}
location ~ /\.ht {
deny all;}
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
auth_basic "Restricted";auth_basic_user_file
/etc/nginx/.htpasswd;
}
location /nifi-registry-api/ {
rewrite ^/nifi-registry-api/(.*) /nifi-registry-api/$1 break;
proxy_pass https://localhost:18443/nifi-registry;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
location /nifi-registry/ {
proxy_pass https://localhost:18443/nifi-registry;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-ProxyScheme "https";
proxy_set_header X-ProxyHost $proxy_host;
proxy_set_header X-ProxiedEntitiesChain "<%{SSL_CLIENT_S_DN}>";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 1;
} }
When I log on to the nifi-registry page I have the following error: 502 Bad
Gateway
can someone help me on this point please I do not find examples
Error log nginx :
*28739 SSL_do_handshake() failed (SSL: error:14094412:SSL
routines:ssl3_read_bytes:sslv3 alert bad certificate:SSL alert number 42) while
SSL hands
[VOTE] Release Apache NiFi 1.7.0
Hello, I am pleased to be calling this vote for the source release of Apache NiFi nifi-1.7.0. The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1127 and https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0 The Git tag is nifi-1.7.0-RC1 The Git commit ID is 99bcd1f88dc826f857ae4ab33e842110bfc6ce21 https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=99bcd1f88dc826f857ae4ab33e842110bfc6ce21 Checksums of nifi-1.7.0-source-release.zip: SHA1: 11086ef532bb51462d7e1ac818f6308d4ac62f03 SHA256: b616f985d486af3d05c04e375f952a4a5678f486017a2211657d5ba03aaaf563 SHA512: d81e9c6eb7fc51905d6f6629b25151fc3d8af7a3cd7cbc3aa03be390c0561858d614b62d8379a90fdb736fcf5c1b4832f4e050fdcfcd786e9615a0b5cc1d563d Release artifacts are signed with the following key: https://people.apache.org/keys/committer/alopresto.asc KEYS file available here: https://dist.apache.org/repos/dist/release/nifi/KEYS 194 issues were closed/resolved for this release: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342979=12316020 Release note highlights can be found here: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.7.0 The vote will be open for 72 hours. Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote: [ ] +1 Release this package as nifi-1.7.0 [ ] +0 no opinion [ ] -1 Do not release this package because… Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 signature.asc Description: Message signed with OpenPGP using GPGMail
Apache NiFi 1.7.0 RC1 Release Helper Guide
Hello Apache NiFi community, Please find the associated guidance to help those interested in validating/verifying the release so they can vote. # Download latest KEYS file: https://dist.apache.org/repos/dist/dev/nifi/KEYS # Import keys file: gpg --import KEYS # [optional] Clear out local maven artifact repository # Pull down nifi-1.7.0 source release artifacts for review: wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.asc wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha1 wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha256 wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.7.0/nifi-1.7.0-source-release.zip.sha512 # Verify the signature gpg --verify nifi-1.7.0-source-release.zip.asc # Verify the hashes (sha1, sha256, sha512) match the source and what was provided in the vote email thread shasum -a 1 nifi-1.7.0-source-release.zip shasum -a 256 nifi-1.7.0-source-release.zip shasum -a 512 nifi-1.7.0-source-release.zip # Unzip nifi-1.7.0-source-release.zip # Verify the build works including release audit tool (RAT) checks cd nifi-1.7.0 mvn clean install -Pcontrib-check,include-grpc # Verify the contents contain a good README, NOTICE, and LICENSE. # Verify the git commit ID is correct # Verify the RC was branched off the correct git commit ID # Look at the resulting convenience binary as found in nifi-assembly/target # Make sure the README, NOTICE, and LICENSE are present and correct # Run the resulting convenience binary and make sure it works as expected # Send a response to the vote thread indicating a +1, 0, -1 based on your findings. Thank you for your time and effort to validate the release! Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 signature.asc Description: Message signed with OpenPGP using GPGMail
