Re: NiFi using HiveQL in secured kerberos environment with Zookeeper connection string - failing
Erik, It's been fixed in Hive 3, and even though the Hive 3 components don't come with the distribution of Apache NiFi (due to size concerns), they can be added manually from the Apache repository [1] [2]. The issue was fixed in Hive 2 but NiFi does not have a Hive 2 bundle, so if you are running into NIFI-2575, IMO the solution is to upgrade to Hive 3 and add the NiFi Hive 3 components (the API NAR and the Hive 3 NAR). I'm not sure if there is a vendor-specific distribution that fixes this for Hive 1... Regards, Matt [1] https://repository.apache.org/content/repositories/releases/org/apache/nifi/nifi-hive-services-api-nar/ [2] https://repository.apache.org/content/repositories/releases/org/apache/nifi/nifi-hive3-nar/ On Thu, Jan 16, 2020 at 5:05 PM Erik Anderson wrote: > > I am 100% running into this issue. > > https://issues.apache.org/jira/browse/NIFI-2575 > > Failed to process session due to org.apache.commons.dbcp.SQLNestedException: > Cannot create PoolableConnectionFactory (Could not open client transport for > any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from > ZooKeeper): org.apache.nifi.processor.exception.ProcessException: > org.apache.commons.dbcp.SQLNestedException: Cannot create > PoolableConnectionFactory (Could not open client transport for any of the > Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper) > > I am assuming this hasnt been fixed? Any pointers to fixing it? > > Erik Anderson > Bloomberg
[VOTE] Release Apache NiFi 1.11.0 (rc2)
Hello, I am pleased to be calling this vote for the source release of Apache NiFi nifi-1.11.0. The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1153 The source being voted upon and the convenience binaries can be found at: https://dist.apache.org/repos/dist/dev/nifi/nifi-1.11.0/ The Git tag is nifi-1.11.0-RC2 The Git commit ID is 97bec7ad0f1efe3e28ac9cb7d2364134727cd07b https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=97bec7ad0f1efe3e28ac9cb7d2364134727cd07b Checksums of nifi-1.11.0-source-release.zip: SHA256: f43245e21dba63061a7f5bc8e2590f1242112380083791acff10a07ecf357234 SHA512: 4634981608af35b76a96375e84660fbb75148e67ab1d0b6c192ee01eeb7c9a4cb1ebd7671ad73fa27a8e9bb71f361e32f49214573be5564d781612316cc8f88f Release artifacts are signed with the following key: https://people.apache.org/keys/committer/joewitt.asc KEYS file available here: https://dist.apache.org/repos/dist/release/nifi/KEYS 112 issues were closed/resolved for this release: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12346451 Release note highlights can be found here: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.11.0 The vote will be open for 72 hours. Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote: [ ] +1 Release this package as nifi-1.11.0 [ ] +0 no opinion [ ] -1 Do not release this package because...
NiFi using HiveQL in secured kerberos environment with Zookeeper connection string - failing
I am 100% running into this issue. https://issues.apache.org/jira/browse/NIFI-2575 Failed to process session due to org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper): org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper) I am assuming this hasnt been fixed? Any pointers to fixing it? Erik Anderson Bloomberg
Re: [discuss] Apache NiFi 1.11.0
Restarting RC process now and will send note for 1.11 RC2 once ready. Thanks Joe On Wed, Jan 15, 2020 at 8:55 AM Joe Witt wrote: > All, > > I've initiated the 1.11 RC. > > For any new JIRAs merged please use fix version of 1.12.0 which is now > available. > https://issues.apache.org/jira/browse/NIFI/fixforversion/12346778 > > If we have any issues with RC1 I'll grab those fixes into RC2 by looking > at new commits on master since the 1.11 RC1 branch which would be tagged to > 1.12. > > Thanks > > On Mon, Jan 13, 2020 at 3:21 PM Joe Witt wrote: > >> Team, >> >> Ok we're at the point where we have no remaining items tagged to 1.11 at >> this time. I'll do my best to initiate the RC tomorrow afternoon but if >> not *should* be able to do so Wed. >> >> https://issues.apache.org/jira/projects/NIFI/versions/12346451 >> >> Thanks! >> Joe >> >> On Mon, Jan 13, 2020 at 12:49 PM Joe Witt wrote: >> >>> NIFI-6852: That is now merged and closed. Please note I did have to fix >>> a RAT issue flagging a missing license file. No idea why Travis didn't >>> catch that. >>> >>> I will try to initiate the RC mid week. I wont be looking to pull in >>> additional PRs beyond what we've already had on the list. >>> >>> Thanks >>> Joe >>> >>> On Sun, Jan 12, 2020 at 4:42 PM Shawn Weeks >>> wrote: >>> I've pushed a new version of NIFI-6852 with integration tests. Assuming the build completes can someone review please. Thanks Shawn On 1/12/20, 9:51 AM, "Shawn Weeks" wrote: I'm working on the test cases this morning. I'm planning on being done today if nothing comes up. Thanks On 1/11/20, 6:09 PM, "Mark Bean" wrote: There's an open PR [1] for NIFI-6852. Could really use this in 1.11.0. [1] https://github.com/apache/nifi/pull/3973 On Mon, Jan 6, 2020 at 12:06 PM Joe Witt wrote: > Team, > > We've already seen a good stream of nice fixes, improvements, and some > minor features land on NIFi 1.11.0. It is enough to my eyes to warrant > doing a release to pull these in plus it helps get back our release cadence > mojo. > > I'll volunteer to RM this one as well as it almost makes me feel like a > software developer still. > > Here are things already resolved for 1.11 > > https://issues.apache.org/jira/browse/NIFI-6956?jql=project%20%3D%20NIFI%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%201.11.0 > > Here are things open but tagged to 1.11 > > https://issues.apache.org/jira/browse/NIFI-6982?jql=project%20%3D%20NIFI%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%201.11.0 > > I'l start cleaning/moving as necessary. Please advise if there are major > things not present that we need to consider. But otherwise lets keep the > ball moving forward and get after more regular releases again. > > Thanks > Joe >
Re: [VOTE] Release Apache NiFi 1.11.0 (RC1)
ill take a look at converting that test to an integration test. such timing sensitive tests that are not reliable have no business as unit tests. that said I dont see that impacting osx laptop, linux many cpu desktop, or travis builds. Thanks On Thu, Jan 16, 2020 at 8:09 AM Mike Thomsen wrote: > FYI, I was about to get a -1 for this: > > [INFO] > [ERROR] Failures: > [ERROR] > > org.apache.nifi.processors.standard.TestListFile.testFilterAge(org.apache.nifi.processors.standard.TestListFile) > [ERROR] Run 1: TestListFile.testFilterAge:360 expected:<2> but was:<1> > [ERROR] Run 2: TestListFile.testFilterAge State dump: > > timestamp date from timestamp t0 delta > --- - --- > started at = 1579170547964 2020-01-16T05:29:07.9640 > current time= 1579170568994 2020-01-16T05:29:28.9940 > processor state --- > processed.timestamp = 157917056 2020-01-16T05:29:20.000-8994 > listing.timestamp = 157917056 2020-01-16T05:29:20.000-8994 > input folder contents - >age1.txt = 157917056 2020-01-16T05:29:20.000-8994 >age2.txt = 157917055 2020-01-16T05:29:10.000 -18994 >age3.txt = 157917054 2020-01-16T05:29:00.000 -28994 > output flowfiles -- >age1.txt = 157917056 2020-01-16T05:29:20.000-8994 > REL_SUCCESS count = 1 > > > Would be good to see if someone can replicate first before you bundle a > RC2. Hopefully it's just my laptop. > > On Wed, Jan 15, 2020 at 8:15 PM Joe Witt wrote: > > > I cancelled the vote and dropped the repo. Will get rc2 asap > > > > thanks > > > > On Wed, Jan 15, 2020 at 7:17 PM Mike Thomsen > > wrote: > > > > > Getting a 404 on this URL: > > > https://repository.apache.org/content/repositories/orgapachenifi-1152 > > > > > > On Wed, Jan 15, 2020 at 11:56 AM Joe Witt wrote: > > > > > > > Hello, > > > > > > > > I am pleased to be calling this vote for the source release of Apache > > > NiFi > > > > nifi-1.11.0. > > > > > > > > The source zip, including signatures, digests, etc. can be found at: > > > > > https://repository.apache.org/content/repositories/orgapachenifi-1152 > > > > > > > > The source being voted upon and the convenience binaries can be found > > at: > > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.11.0/ > > > > > > > > The Git tag is nifi-1.11.0-RC1 > > > > The Git commit ID is c5c0dc1a0ad0df306dbd8ee4914e46699109642d > > > > > > > > > > > > > > https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=c5c0dc1a0ad0df306dbd8ee4914e46699109642d > > > > > > > > Checksums of nifi-1.11.0-source-release.zip: > > > > SHA256: > > 8bc7a3d365aea2fcb159b0ad7f26e315760e8170b562effd1a21d5881f146019 > > > > SHA512: > > > > > > > > > > > > > > be39f9d5f0db0dadc3423c4976b018f50f22f9a7c257a1de9ec56fa2f374bf386415d4718fc0c96d450cb1d2c5228ee8bd18f0042afc83b9a9b8a4838f1d400d > > > > > > > > Release artifacts are signed with the following key: > > > > https://people.apache.org/keys/committer/joewitt.asc > > > > > > > > KEYS file available here: > > > > https://dist.apache.org/repos/dist/release/nifi/KEYS > > > > > > > > 106 issues were closed/resolved for this release: > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12346451 > > > > > > > > Release note highlights can be found here: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.11.0 > > > > > > > > The vote will be open for 72 hours. > > > > Please download the release candidate and evaluate the necessary > items > > > > including checking hashes, signatures, build > > > > from source, and test. Then please vote: > > > > > > > > [ ] +1 Release this package as nifi-1.11.0 > > > > [ ] +0 no opinion > > > > [ ] -1 Do not release this package because... > > > > > > > > > >
Re: [VOTE] Release Apache NiFi 1.11.0 (RC1)
FYI, I was about to get a -1 for this: [INFO] [ERROR] Failures: [ERROR] org.apache.nifi.processors.standard.TestListFile.testFilterAge(org.apache.nifi.processors.standard.TestListFile) [ERROR] Run 1: TestListFile.testFilterAge:360 expected:<2> but was:<1> [ERROR] Run 2: TestListFile.testFilterAge State dump: timestamp date from timestamp t0 delta --- - --- started at = 1579170547964 2020-01-16T05:29:07.9640 current time= 1579170568994 2020-01-16T05:29:28.9940 processor state --- processed.timestamp = 157917056 2020-01-16T05:29:20.000-8994 listing.timestamp = 157917056 2020-01-16T05:29:20.000-8994 input folder contents - age1.txt = 157917056 2020-01-16T05:29:20.000-8994 age2.txt = 157917055 2020-01-16T05:29:10.000 -18994 age3.txt = 157917054 2020-01-16T05:29:00.000 -28994 output flowfiles -- age1.txt = 157917056 2020-01-16T05:29:20.000-8994 REL_SUCCESS count = 1 Would be good to see if someone can replicate first before you bundle a RC2. Hopefully it's just my laptop. On Wed, Jan 15, 2020 at 8:15 PM Joe Witt wrote: > I cancelled the vote and dropped the repo. Will get rc2 asap > > thanks > > On Wed, Jan 15, 2020 at 7:17 PM Mike Thomsen > wrote: > > > Getting a 404 on this URL: > > https://repository.apache.org/content/repositories/orgapachenifi-1152 > > > > On Wed, Jan 15, 2020 at 11:56 AM Joe Witt wrote: > > > > > Hello, > > > > > > I am pleased to be calling this vote for the source release of Apache > > NiFi > > > nifi-1.11.0. > > > > > > The source zip, including signatures, digests, etc. can be found at: > > > https://repository.apache.org/content/repositories/orgapachenifi-1152 > > > > > > The source being voted upon and the convenience binaries can be found > at: > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.11.0/ > > > > > > The Git tag is nifi-1.11.0-RC1 > > > The Git commit ID is c5c0dc1a0ad0df306dbd8ee4914e46699109642d > > > > > > > > > https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=c5c0dc1a0ad0df306dbd8ee4914e46699109642d > > > > > > Checksums of nifi-1.11.0-source-release.zip: > > > SHA256: > 8bc7a3d365aea2fcb159b0ad7f26e315760e8170b562effd1a21d5881f146019 > > > SHA512: > > > > > > > > > be39f9d5f0db0dadc3423c4976b018f50f22f9a7c257a1de9ec56fa2f374bf386415d4718fc0c96d450cb1d2c5228ee8bd18f0042afc83b9a9b8a4838f1d400d > > > > > > Release artifacts are signed with the following key: > > > https://people.apache.org/keys/committer/joewitt.asc > > > > > > KEYS file available here: > > > https://dist.apache.org/repos/dist/release/nifi/KEYS > > > > > > 106 issues were closed/resolved for this release: > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12346451 > > > > > > Release note highlights can be found here: > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.11.0 > > > > > > The vote will be open for 72 hours. > > > Please download the release candidate and evaluate the necessary items > > > including checking hashes, signatures, build > > > from source, and test. Then please vote: > > > > > > [ ] +1 Release this package as nifi-1.11.0 > > > [ ] +0 no opinion > > > [ ] -1 Do not release this package because... > > > > > >
