Re: zookeeper connection string question/clarification

[Andy LoPresto]

Thanks Dan. If it works, we can update the MG with that example as well. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Feb 20, 2020, at 3:28 PM, dan young  wrote:
> 
> ok, great thank you. Yes, we're using external zookeeper; 3.5.6.  I'm going 
> to test this change out on a dev cluster real quick.
> 
> On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto  > wrote:
> Sorry, I should have elaborated that I was referencing the link from the MG. 
> I realize you’re using external ZK and this is for embedded. Yes, I believe 
> you will need to change the format of your connection string. 
> 
> 
> Andy LoPresto
> alopre...@apache.org 
> alopresto.apa...@gmail.com 
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Feb 20, 2020, at 3:20 PM, Andy LoPresto > > wrote:
>> 
>> Hi Dan,
>> 
>> I believe the changes you’re looking for are here [1], copied below:
>> 
>> The Zookeeper dependency that NiFi uses for state management and cluster 
>> elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the 
>> zookeeper.properties file format and as a result NiFi users using an 
>> existing embedded zookeeper will need to adjust their existing 
>> zookeeper.properties file accordingly. More details here: 
>> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
>>  
>> .
>> For new deployments of the 1.10.0 release onwards, NiFi will be packaged 
>> with an updated template zookeeper.properties file.
>> To update an existing zookeeper.properties file however, edit the 
>> conf/zookeeper.properties file:
>> Remove the clientPort=2181 line (or whatever your port number may be)
>> Add the client port to the end of the server string eg: 
>> server.1=localhost:2888:3888;2181
>> 
>> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance 
>> 
>> 
>> Andy LoPresto
>> alopre...@apache.org 
>> alopresto.apa...@gmail.com 
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>>> On Feb 20, 2020, at 3:18 PM, dan young >> > wrote:
>>> 
>>> Hello,
>>> 
>>> Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the 
>>> nifi.zookeeper.connect string in the nifi.properties need to change from 
>>> say:
>>> 
>>>  
>>> nifi.zookeeper.connect.string=10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>>> 
>>>  
>>> nifi.zookeeper.connect.string=10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>>> 
>>> Changing the : to ; between the host and client port?
>> 
> 

Re: zookeeper connection string question/clarification

[Bryan Bende]

I think this change only affects the server entries in
zookeeper.properties, so it’s mainly for people with existing embedded
zookeeper setups.

The nifi properties and state management entries didn’t change.

On Thu, Feb 20, 2020 at 6:45 PM Andy LoPresto  wrote:

> Thanks Dan. If it works, we can update the MG with that example as well.
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:28 PM, dan young  wrote:
>
> ok, great thank you. Yes, we're using external zookeeper; 3.5.6.  I'm
> going to test this change out on a dev cluster real quick.
>
> On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto 
> wrote:
>
>> Sorry, I should have elaborated that I was referencing the link from the
>> MG. I realize you’re using external ZK and this is for embedded. Yes, I
>> believe you will need to change the format of your connection string.
>>
>>
>> Andy LoPresto
>> alopre...@apache.org
>> *alopresto.apa...@gmail.com *
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Feb 20, 2020, at 3:20 PM, Andy LoPresto  wrote:
>>
>> Hi Dan,
>>
>> I believe the changes you’re looking for are here [1], copied below:
>>
>>
>>- The Zookeeper dependency that NiFi uses for state management and
>>cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper
>>changed the zookeeper.properties file format and as a result NiFi users
>>using an existing embedded zookeeper will need to adjust their existing
>>zookeeper.properties file accordingly*. More details here:
>>
>> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
>>.
>>For new deployments of the 1.10.0 release onwards, NiFi will be
>>packaged with an updated template zookeeper.properties file.
>>To update an existing zookeeper.properties file however, edit the
>>conf/zookeeper.properties file:
>>   1. Remove the clientPort=2181 line (or whatever your port number
>>   may be)
>>   2. Add the client port to the end of the server string eg:
>>   server.1=localhost:2888:3888;2181
>>
>>
>> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>>
>> Andy LoPresto
>> alopre...@apache.org
>> *alopresto.apa...@gmail.com *
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Feb 20, 2020, at 3:18 PM, dan young  wrote:
>>
>> Hello,
>>
>> Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the
>> nifi.zookeeper.connect string in the nifi.properties need to change from
>> say:
>>
>>  nifi.zookeeper.connect.string=
>> 10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>>
>>  nifi.zookeeper.connect.string=
>> 10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>>
>> Changing the : to ; between the host and client port?
>>
>>
>>
>>
> --
Sent from Gmail Mobile

Re: zookeeper connection string question/clarification

[dan young]

ok, great thank you. Yes, we're using external zookeeper; 3.5.6.  I'm going
to test this change out on a dev cluster real quick.

On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto  wrote:

> Sorry, I should have elaborated that I was referencing the link from the
> MG. I realize you’re using external ZK and this is for embedded. Yes, I
> believe you will need to change the format of your connection string.
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:20 PM, Andy LoPresto  wrote:
>
> Hi Dan,
>
> I believe the changes you’re looking for are here [1], copied below:
>
>
>- The Zookeeper dependency that NiFi uses for state management and
>cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper
>changed the zookeeper.properties file format and as a result NiFi users
>using an existing embedded zookeeper will need to adjust their existing
>zookeeper.properties file accordingly*. More details here:
>
> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
>.
>For new deployments of the 1.10.0 release onwards, NiFi will be
>packaged with an updated template zookeeper.properties file.
>To update an existing zookeeper.properties file however, edit the
>conf/zookeeper.properties file:
>   1. Remove the clientPort=2181 line (or whatever your port number
>   may be)
>   2. Add the client port to the end of the server string eg:
>   server.1=localhost:2888:3888;2181
>
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:18 PM, dan young  wrote:
>
> Hello,
>
> Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the
> nifi.zookeeper.connect string in the nifi.properties need to change from
> say:
>
>  nifi.zookeeper.connect.string=
> 10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>
>  nifi.zookeeper.connect.string=
> 10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>
> Changing the : to ; between the host and client port?
>
>
>
>

Re: zookeeper connection string question/clarification

[dan young]

ok, I tried host;2181 and that doesn't seem to work.  So I think host:2181
is still valid

On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto  wrote:

> Sorry, I should have elaborated that I was referencing the link from the
> MG. I realize you’re using external ZK and this is for embedded. Yes, I
> believe you will need to change the format of your connection string.
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:20 PM, Andy LoPresto  wrote:
>
> Hi Dan,
>
> I believe the changes you’re looking for are here [1], copied below:
>
>
>- The Zookeeper dependency that NiFi uses for state management and
>cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper
>changed the zookeeper.properties file format and as a result NiFi users
>using an existing embedded zookeeper will need to adjust their existing
>zookeeper.properties file accordingly*. More details here:
>
> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
>.
>For new deployments of the 1.10.0 release onwards, NiFi will be
>packaged with an updated template zookeeper.properties file.
>To update an existing zookeeper.properties file however, edit the
>conf/zookeeper.properties file:
>   1. Remove the clientPort=2181 line (or whatever your port number
>   may be)
>   2. Add the client port to the end of the server string eg:
>   server.1=localhost:2888:3888;2181
>
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:18 PM, dan young  wrote:
>
> Hello,
>
> Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the
> nifi.zookeeper.connect string in the nifi.properties need to change from
> say:
>
>  nifi.zookeeper.connect.string=
> 10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>
>  nifi.zookeeper.connect.string=
> 10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>
> Changing the : to ; between the host and client port?
>
>
>
>

Re: zookeeper connection string question/clarification

[Andy LoPresto]

Sorry, I should have elaborated that I was referencing the link from the MG. I 
realize you’re using external ZK and this is for embedded. Yes, I believe you 
will need to change the format of your connection string. 


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Feb 20, 2020, at 3:20 PM, Andy LoPresto  wrote:
> 
> Hi Dan,
> 
> I believe the changes you’re looking for are here [1], copied below:
> 
> The Zookeeper dependency that NiFi uses for state management and cluster 
> elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the 
> zookeeper.properties file format and as a result NiFi users using an existing 
> embedded zookeeper will need to adjust their existing zookeeper.properties 
> file accordingly. More details here: 
> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
>  
> .
> For new deployments of the 1.10.0 release onwards, NiFi will be packaged with 
> an updated template zookeeper.properties file.
> To update an existing zookeeper.properties file however, edit the 
> conf/zookeeper.properties file:
> Remove the clientPort=2181 line (or whatever your port number may be)
> Add the client port to the end of the server string eg: 
> server.1=localhost:2888:3888;2181
> 
> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance 
> 
> 
> Andy LoPresto
> alopre...@apache.org 
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Feb 20, 2020, at 3:18 PM, dan young > > wrote:
>> 
>> Hello,
>> 
>> Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the 
>> nifi.zookeeper.connect string in the nifi.properties need to change from say:
>> 
>>  
>> nifi.zookeeper.connect.string=10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>> 
>>  
>> nifi.zookeeper.connect.string=10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>> 
>> Changing the : to ; between the host and client port?
> 

Re: zookeeper connection string question/clarification

[Andy LoPresto]

Hi Dan,

I believe the changes you’re looking for are here [1], copied below:

The Zookeeper dependency that NiFi uses for state management and cluster 
elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the 
zookeeper.properties file format and as a result NiFi users using an existing 
embedded zookeeper will need to adjust their existing zookeeper.properties file 
accordingly. More details here: 
https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
 
.
For new deployments of the 1.10.0 release onwards, NiFi will be packaged with 
an updated template zookeeper.properties file.
To update an existing zookeeper.properties file however, edit the 
conf/zookeeper.properties file:
Remove the clientPort=2181 line (or whatever your port number may be)
Add the client port to the end of the server string eg: 
server.1=localhost:2888:3888;2181

[1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance 


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Feb 20, 2020, at 3:18 PM, dan young  wrote:
> 
> Hello,
> 
> Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the 
> nifi.zookeeper.connect string in the nifi.properties need to change from say:
> 
>  
> nifi.zookeeper.connect.string=10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
> 
>  
> nifi.zookeeper.connect.string=10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
> 
> Changing the : to ; between the host and client port?

zookeeper connection string question/clarification

[dan young]

Hello,

Using Nifi 1.11.1 in cluster mode with external zookeeper.  Does the
nifi.zookeeper.connect string in the nifi.properties need to change from
say:

 nifi.zookeeper.connect.string=
10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181

 nifi.zookeeper.connect.string=
10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181

Changing the : to ; between the host and client port?

[ANNOUNCE] Apache NiFi 1.11.2 release

[Joe Witt]

Hello

The Apache NiFi team would like to announce the release of Apache NiFi
1.11.2.

Apache NiFi is an easy to use, powerful, and reliable system to process and
distribute
data.  Apache NiFi was made for dataflow.  It supports highly configurable
directed graphs
of data routing, transformation, and system mediation logic.

More details on Apache NiFi can be found here:
https://nifi.apache.org/

The release artifacts can be downloaded from here:
https://nifi.apache.org/download.html

Maven artifacts have been made available and mirrored as per normal ASF
artifact processes.

Issues closed/resolved for this list can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12346972

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.11.2

Thank you
The Apache NiFi team

Re: FW: Execute SQL to MSFT SQLServer - getting error

[Juan Pablo Gardella]

According JDBC documentation driver
,
integratedSecurity is a boolean property, and to authenticate against
SQLServer you have three options for integratedSecurity=true. Kerberos,
NTLM or NativeAuthentication. NativeAuthentication only works in windows
os, so you cannot use it in linux.

Juan

On Thu, 20 Feb 2020 at 14:49, Pierre Villard 
wrote:

> Based on this thread [1], it looks like it's because the DLL are not
> loaded by Java. I'd double check what you set for java.library.path system
> property.
>
> [1]
> https://stackoverflow.com/questions/6087819/jdbc-sqlserverexception-this-driver-is-not-configured-for-integrated-authentic
>
> Le jeu. 20 févr. 2020 à 02:03, Samarendra Sahoo <
> sahoo.samaren...@gmail.com> a écrit :
>
>> Thanks a lot Peter. Attaching the controller. In addition to the
>> controller setting, adding jdbc 8 driver, we have added mssql-jdbc_auth dll
>> at below locations in the Nifi server.
>>
>>
>> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-3.b16.el6_9.x86_64/bin/mssql-jdbc_auth-8.2.0.x64.dll
>>
>> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-3.b16.el6_9.x86_64/jre/bin/mssql-jdbc_auth-8.2.0.x64.dll
>> - Forwarded message -
>>
>> From: *Pierre Villard* 
>> Date: Thu, 20 Feb, 2020, 12:54 AM
>> Subject: Re: Execute SQL to MSFT SQLServer - getting error
>> To: 
>> Cc: 
>>
>>
>>
>> Hi,
>>
>>
>>
>> Based on the error, it sounds like the driver you are using is not
>> configured for integrated authentication. Can you share more details about
>> how you configured your controller service?
>>
>>
>>
>> Thanks,
>>
>> Pierre
>>
>>
>>
>> Le mer. 19 févr. 2020 à 09:58, Samarendra Sahoo <
>> sahoo.samaren...@gmail.com> a écrit :
>>
>> Dear all,
>> We are trying to connect to SQLServer from Nifi installed on a RHEL Linux
>> VM. We are not able to find a way to connect SQLServer configured with
>> Integrated Security=*SSPI *with Windows login Authentication*  ( Note :
>> Kerberos not enabled )  *. We are trying to use *DBCPConnectionPool *with
>> JDBC database connector 
>> *jdbc:sqlserver://xx.xxx.xx.xxx:port;databaseName=DEMODB;integratedSecurity=SSPI
>> . *However its not allowed us to login to SQLServer. Does nifi provide
>> any connector/Component or to connect to SSPI Security mode ?  Or Is there
>> any other way to connect using custom code connector?
>>
>>
>>
>> Attaching error message for reference
>>
>>

Re: FW: Execute SQL to MSFT SQLServer - getting error

[Pierre Villard]

Based on this thread [1], it looks like it's because the DLL are not loaded
by Java. I'd double check what you set for java.library.path system
property.

[1]
https://stackoverflow.com/questions/6087819/jdbc-sqlserverexception-this-driver-is-not-configured-for-integrated-authentic

Le jeu. 20 févr. 2020 à 02:03, Samarendra Sahoo 
a écrit :

> Thanks a lot Peter. Attaching the controller. In addition to the
> controller setting, adding jdbc 8 driver, we have added mssql-jdbc_auth dll
> at below locations in the Nifi server.
>
>
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-3.b16.el6_9.x86_64/bin/mssql-jdbc_auth-8.2.0.x64.dll
>
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-3.b16.el6_9.x86_64/jre/bin/mssql-jdbc_auth-8.2.0.x64.dll
> - Forwarded message -
>
> From: *Pierre Villard* 
> Date: Thu, 20 Feb, 2020, 12:54 AM
> Subject: Re: Execute SQL to MSFT SQLServer - getting error
> To: 
> Cc: 
>
>
>
> Hi,
>
>
>
> Based on the error, it sounds like the driver you are using is not
> configured for integrated authentication. Can you share more details about
> how you configured your controller service?
>
>
>
> Thanks,
>
> Pierre
>
>
>
> Le mer. 19 févr. 2020 à 09:58, Samarendra Sahoo <
> sahoo.samaren...@gmail.com> a écrit :
>
> Dear all,
> We are trying to connect to SQLServer from Nifi installed on a RHEL Linux
> VM. We are not able to find a way to connect SQLServer configured with
> Integrated Security=*SSPI *with Windows login Authentication*  ( Note :
> Kerberos not enabled )  *. We are trying to use *DBCPConnectionPool *with
> JDBC database connector 
> *jdbc:sqlserver://xx.xxx.xx.xxx:port;databaseName=DEMODB;integratedSecurity=SSPI
> . *However its not allowed us to login to SQLServer. Does nifi provide
> any connector/Component or to connect to SSPI Security mode ?  Or Is there
> any other way to connect using custom code connector?
>
>
>
> Attaching error message for reference
>
>

Fwd: FW: Execute SQL to MSFT SQLServer - getting error

[Samarendra Sahoo]

Thanks a lot Peter. Attaching the controller. In addition to the controller
setting, adding jdbc 8 driver, we have added mssql-jdbc_auth dll at below
locations in the Nifi server.

/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-3.b16.el6_9.x86_64/bin/mssql-jdbc_auth-8.2.0.x64.dll
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-3.b16.el6_9.x86_64/jre/bin/mssql-jdbc_auth-8.2.0.x64.dll
- Forwarded message -

From: *Pierre Villard* 
Date: Thu, 20 Feb, 2020, 12:54 AM
Subject: Re: Execute SQL to MSFT SQLServer - getting error
To: 
Cc: 



Hi,



Based on the error, it sounds like the driver you are using is not
configured for integrated authentication. Can you share more details about
how you configured your controller service?



Thanks,

Pierre



Le mer. 19 févr. 2020 à 09:58, Samarendra Sahoo 
a écrit :

Dear all,
We are trying to connect to SQLServer from Nifi installed on a RHEL Linux
VM. We are not able to find a way to connect SQLServer configured with
Integrated Security=*SSPI *with Windows login Authentication*  ( Note :
Kerberos not enabled )  *. We are trying to use *DBCPConnectionPool *with
JDBC database connector
*jdbc:sqlserver://xx.xxx.xx.xxx:port;databaseName=DEMODB;integratedSecurity=SSPI
. *However its not allowed us to login to SQLServer. Does nifi provide any
connector/Component or to connect to SSPI Security mode ?  Or Is there any
other way to connect using custom code connector?



Attaching error message for reference