Re: NiFi Toolkit CLI Token Creation

[Bryan Bende]

I meant to say that you obviously could generate certs for CLI users, but I
was just mentioning an alternative where you can proxy an identity.

Right now the CLI never obtains a token because it is all cert based.

On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende  wrote:

> Right now the idea is that whoever is running the CLI would have access to
> a NiFi server certificate and then you can proxy any user you want. There
> should be examples of this in the readme or toolkit guide.
>
> Supporting Kerberos auth was something I wanted to do, but it’s definitely
> not a trivial effort.
>
> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto 
> wrote:
>
>> Shawn,
>>
>> I’m not sure I understand your question.
>>
>> I am in the process of refactoring the TLS Toolkit to integrate with
>> public certificate authorities, so in the near future it will be easier to
>> use certificates signed by external authorities rather than self-signed.
>>
>> My understanding is that you are talking about the CLI Toolkit rather
>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>> going to proceed with the understanding that you are referring to the JWT
>> token used to identify an authenticated user when communicating with the
>> NiFi API.
>>
>> You may want to look at JerseyNiFiClient [1], which has methods for
>> getting various clients given an authentication token.
>>
>> You can create the token via the POST /access/kerberos API [2].
>>
>> [1]
>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>> <
>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>> >
>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>
>> Andy LoPresto
>> alopre...@apache.org
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> > On Jun 12, 2019, at 9:39 AM, Shawn Weeks 
>> wrote:
>> >
>> > I work in an environment reluctant to create self signed ssl
>> certificates and I’m looking at the feasibility of having the toolkit cli
>> authenticate via Kerberos. I was expecting it to be as simple as adding
>> another way to get the authentication token but I’m having trouble figuring
>> out exactly when the token is created. I see lots of references to it after
>> it’s been created.
>> >
>> > Thanks
>> > Shawn
>>
>> --
> Sent from Gmail Mobile
>
-- 
Sent from Gmail Mobile

Re: NiFi Toolkit CLI Token Creation

[Bryan Bende]

Right now the idea is that whoever is running the CLI would have access to
a NiFi server certificate and then you can proxy any user you want. There
should be examples of this in the readme or toolkit guide.

Supporting Kerberos auth was something I wanted to do, but it’s definitely
not a trivial effort.

On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto  wrote:

> Shawn,
>
> I’m not sure I understand your question.
>
> I am in the process of refactoring the TLS Toolkit to integrate with
> public certificate authorities, so in the near future it will be easier to
> use certificates signed by external authorities rather than self-signed.
>
> My understanding is that you are talking about the CLI Toolkit rather than
> the TLS Toolkit, but your reference to “token” was ambiguous, so I’m going
> to proceed with the understanding that you are referring to the JWT token
> used to identify an authenticated user when communicating with the NiFi
> API.
>
> You may want to look at JerseyNiFiClient [1], which has methods for
> getting various clients given an authentication token.
>
> You can create the token via the POST /access/kerberos API [2].
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> <
> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> >
> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Jun 12, 2019, at 9:39 AM, Shawn Weeks 
> wrote:
> >
> > I work in an environment reluctant to create self signed ssl
> certificates and I’m looking at the feasibility of having the toolkit cli
> authenticate via Kerberos. I was expecting it to be as simple as adding
> another way to get the authentication token but I’m having trouble figuring
> out exactly when the token is created. I see lots of references to it after
> it’s been created.
> >
> > Thanks
> > Shawn
>
> --
Sent from Gmail Mobile

Re: NiFi Toolkit CLI Token Creation

[Andy LoPresto]

Shawn, 

I’m not sure I understand your question. 

I am in the process of refactoring the TLS Toolkit to integrate with public 
certificate authorities, so in the near future it will be easier to use 
certificates signed by external authorities rather than self-signed. 

My understanding is that you are talking about the CLI Toolkit rather than the 
TLS Toolkit, but your reference to “token” was ambiguous, so I’m going to 
proceed with the understanding that you are referring to the JWT token used to 
identify an authenticated user when communicating with the NiFi API. 

You may want to look at JerseyNiFiClient [1], which has methods for getting 
various clients given an authentication token. 

You can create the token via the POST /access/kerberos API [2]. 

[1] 
https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
 

[2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html 


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 12, 2019, at 9:39 AM, Shawn Weeks  wrote:
> 
> I work in an environment reluctant to create self signed ssl certificates and 
> I’m looking at the feasibility of having the toolkit cli authenticate via 
> Kerberos. I was expecting it to be as simple as adding another way to get the 
> authentication token but I’m having trouble figuring out exactly when the 
> token is created. I see lots of references to it after it’s been created.
> 
> Thanks
> Shawn

NiFi Toolkit CLI Token Creation

[Shawn Weeks]

I work in an environment reluctant to create self signed ssl certificates and 
I’m looking at the feasibility of having the toolkit cli authenticate via 
Kerberos. I was expecting it to be as simple as adding another way to get the 
authentication token but I’m having trouble figuring out exactly when the token 
is created. I see lots of references to it after it’s been created.

Thanks
Shawn

Re: [EXT] Re: GitHub Stuff

[Bryan Bende]

We have a nice guide linked off the website that I think Andy wrote:

https://nifi.apache.org/gpg.html

>From the perspective of Github verified commits, I think it only
really relies on you having added the public GPG key into your account
under settings "SSH and GPG keys", and the fact that the email address
in the key matches an email address you have in Github that is also
the email address of the commits.

Then there is publishing to a key server which would be more for
people to pull down your key and verify signatures you made, and also
the KEYs file that we use for releases which I believe there are two
different files:

https://github.com/apache/nifi/blob/master/KEYS
https://dist.apache.org/repos/dist/release/nifi/KEYS

If you were going to RM a release then your public key needs to be in
the dist KEYS file above.


On Wed, Jun 12, 2019 at 11:37 AM Mike Thomsen  wrote:
>
> I tried once to publish a GPG key I generated on my MBP, but didn't seem to
> be able to get far with it. Are there any good ASF-centric resources for
> setting up a GPG key?
>
> Thanks,
>
> Mike
>
> On Wed, Jun 12, 2019 at 2:20 AM Koji Kawamura 
> wrote:
>
> > Thanks Bryan for the heads up.
> >
> > My GPG key had been expired. I've renewed my KEY by extending expiration.
> > Now I confirmed that my commits is marked as 'verified' on Github.
> >
> > Koji
> >
> > On Wed, Jun 12, 2019 at 5:43 AM Andy LoPresto 
> > wrote:
> > >
> > > Peter,
> > >
> > > If you have specific issues setting it up, I’m happy to help debug. I
> > haven’t done it recently but am willing to investigate with you.
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > alopresto.apa...@gmail.com
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > > On Jun 11, 2019, at 12:55 PM, Bryan Bende  wrote:
> > > >
> > > > I will admit I've never setup GPG signing on Linux. I'm sure there are
> > > > some additional challenges there.
> > > >
> > > > Not sure if it is helpful, but there are a few things related to Linux
> > > > that are mentioned on this Github page:
> > > >
> > > > https://help.github.com/en/articles/telling-git-about-your-signing-key
> > > >
> > > >
> > > > On Tue, Jun 11, 2019 at 3:45 PM Kevin Doran  wrote:
> > > >>
> > > >> Yep, I support these suggestions.
> > > >>
> > > >> Setting up GPG does have a learning curve for folks that haven't done
> > > >> it before, but I think our community would be helpful in assisting
> > > >> folks on the mailing list and Apache NiFi Slack where they run into
> > > >> trouble. It's a good practice to learn and once setup there's not much
> > > >> more to do to get the benefits of it.
> > > >>
> > > >> Setting up GPG is also required when acting as release manager in
> > > >> order to sign convenience binaries (and soon, as Andy brought up,
> > > >> maven release artifacts as well - I think that is also a good idea),
> > > >> so the effort required to get setup for GPG has lots of benefits for
> > > >> folks that are interested in RM'ing as well.
> > > >>
> > > >> Kevin
> > > >>
> > > >> On Tue, Jun 11, 2019 at 3:30 PM Peter Wicks (pwicks) <
> > pwi...@micron.com> wrote:
> > > >>>
> > > >>> I like having signed commits. I develop on both Windows and Linux,
> > but have only had success getting signing working on Windows (which was a
> > bit complicated as it was). You can see when I switched from mostly Windows
> > to mostly Linux by when I stopped signing commits...
> > > >>>
> > > >>> Thanks,
> > > >>>  Peter
> > > >>>
> > > >>> -Original Message-
> > > >>> From: Andy LoPresto 
> > > >>> Sent: Tuesday, June 11, 2019 1:25 PM
> > > >>> To: dev@nifi.apache.org
> > > >>> Subject: [EXT] Re: GitHub Stuff
> > > >>>
> > > >>> I strongly support both of these suggestions. Thanks for starting
> > the conversation Bryan. GPG signing is very important for security and for
> > encouraging the rest of the community to adopt these practices as well.
> > > >>>
> > > >>>
> > > >>> Andy LoPresto
> > > >>> alopre...@apache.org
> > > >>> alopresto.apa...@gmail.com
> > > >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > > >>>
> > >  On Jun 11, 2019, at 11:42 AM, Bryan Bende  wrote:
> > > 
> > >  I had two thoughts related to our GitHub usage that I wanted to
> > throw
> > >  out there for PMC members and committers...
> > > 
> > >  1) I think it would be helpful if everyone setup the link between
> > >  their Apache id and github [1]. Setting up this link puts you into
> > the
> > >  nifi-committers group in Apache (currently 17 of us are in there),
> > and
> > >  I believe this is what controls the list of users that can be
> > selected
> > >  as a reviewer on a pull request. Since PRs are the primary form of
> > >  contribution, it would be nice if all of the PMC/committers were in
> > >  the reviewer list, but of course you can continue to commit against
> > >  Gitbox without doing this.
> > > 
> > > 

Re: [EXT] Re: GitHub Stuff

[Mike Thomsen]

I tried once to publish a GPG key I generated on my MBP, but didn't seem to
be able to get far with it. Are there any good ASF-centric resources for
setting up a GPG key?

Thanks,

Mike

On Wed, Jun 12, 2019 at 2:20 AM Koji Kawamura 
wrote:

> Thanks Bryan for the heads up.
>
> My GPG key had been expired. I've renewed my KEY by extending expiration.
> Now I confirmed that my commits is marked as 'verified' on Github.
>
> Koji
>
> On Wed, Jun 12, 2019 at 5:43 AM Andy LoPresto 
> wrote:
> >
> > Peter,
> >
> > If you have specific issues setting it up, I’m happy to help debug. I
> haven’t done it recently but am willing to investigate with you.
> >
> > Andy LoPresto
> > alopre...@apache.org
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > > On Jun 11, 2019, at 12:55 PM, Bryan Bende  wrote:
> > >
> > > I will admit I've never setup GPG signing on Linux. I'm sure there are
> > > some additional challenges there.
> > >
> > > Not sure if it is helpful, but there are a few things related to Linux
> > > that are mentioned on this Github page:
> > >
> > > https://help.github.com/en/articles/telling-git-about-your-signing-key
> > >
> > >
> > > On Tue, Jun 11, 2019 at 3:45 PM Kevin Doran  wrote:
> > >>
> > >> Yep, I support these suggestions.
> > >>
> > >> Setting up GPG does have a learning curve for folks that haven't done
> > >> it before, but I think our community would be helpful in assisting
> > >> folks on the mailing list and Apache NiFi Slack where they run into
> > >> trouble. It's a good practice to learn and once setup there's not much
> > >> more to do to get the benefits of it.
> > >>
> > >> Setting up GPG is also required when acting as release manager in
> > >> order to sign convenience binaries (and soon, as Andy brought up,
> > >> maven release artifacts as well - I think that is also a good idea),
> > >> so the effort required to get setup for GPG has lots of benefits for
> > >> folks that are interested in RM'ing as well.
> > >>
> > >> Kevin
> > >>
> > >> On Tue, Jun 11, 2019 at 3:30 PM Peter Wicks (pwicks) <
> pwi...@micron.com> wrote:
> > >>>
> > >>> I like having signed commits. I develop on both Windows and Linux,
> but have only had success getting signing working on Windows (which was a
> bit complicated as it was). You can see when I switched from mostly Windows
> to mostly Linux by when I stopped signing commits...
> > >>>
> > >>> Thanks,
> > >>>  Peter
> > >>>
> > >>> -Original Message-
> > >>> From: Andy LoPresto 
> > >>> Sent: Tuesday, June 11, 2019 1:25 PM
> > >>> To: dev@nifi.apache.org
> > >>> Subject: [EXT] Re: GitHub Stuff
> > >>>
> > >>> I strongly support both of these suggestions. Thanks for starting
> the conversation Bryan. GPG signing is very important for security and for
> encouraging the rest of the community to adopt these practices as well.
> > >>>
> > >>>
> > >>> Andy LoPresto
> > >>> alopre...@apache.org
> > >>> alopresto.apa...@gmail.com
> > >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >>>
> >  On Jun 11, 2019, at 11:42 AM, Bryan Bende  wrote:
> > 
> >  I had two thoughts related to our GitHub usage that I wanted to
> throw
> >  out there for PMC members and committers...
> > 
> >  1) I think it would be helpful if everyone setup the link between
> >  their Apache id and github [1]. Setting up this link puts you into
> the
> >  nifi-committers group in Apache (currently 17 of us are in there),
> and
> >  I believe this is what controls the list of users that can be
> selected
> >  as a reviewer on a pull request. Since PRs are the primary form of
> >  contribution, it would be nice if all of the PMC/committers were in
> >  the reviewer list, but of course you can continue to commit against
> >  Gitbox without doing this.
> > 
> >  2) I also think it would be nice if most of the commits in the repo
> >  were signed commits that show up as "Verified" in GitHub [2]. Right
> >  now I think we lose the verification if the user reviewing the
> commit
> >  doesn't have signing setup, because when you amend the commit to add
> >  "This closes ...", it technically produces a new commit hash, thus
> >  making the original signature no longer apply (at least this is
> what I
> >  think is happening, but other may know more).
> > 
> >  These are obviously just my opinions and no one has to do these
> >  things, but just thought I would throw it out there for discussion
> in
> >  case anyone wasn't aware.
> > 
> >  -Bryan
> > 
> >  [1]
> > 
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitb
> >  ox.apache.org%2Fsetup%2Fdata=02%7C01%7Cpwicks%40micron.com
> %7Cc2f2
> > 
> 0a00f6424597c10708d6eea27d65%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C
> > 
> 0%7C636958778999592924sdata=mJ59FD6KSYn1jXHN0yRRagKf6BHdWn7N1ZXmV
> >  

Re: [EXT] Re: GitHub Stuff

[Koji Kawamura]

Thanks Bryan for the heads up.

My GPG key had been expired. I've renewed my KEY by extending expiration.
Now I confirmed that my commits is marked as 'verified' on Github.

Koji

On Wed, Jun 12, 2019 at 5:43 AM Andy LoPresto  wrote:
>
> Peter,
>
> If you have specific issues setting it up, I’m happy to help debug. I haven’t 
> done it recently but am willing to investigate with you.
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Jun 11, 2019, at 12:55 PM, Bryan Bende  wrote:
> >
> > I will admit I've never setup GPG signing on Linux. I'm sure there are
> > some additional challenges there.
> >
> > Not sure if it is helpful, but there are a few things related to Linux
> > that are mentioned on this Github page:
> >
> > https://help.github.com/en/articles/telling-git-about-your-signing-key
> >
> >
> > On Tue, Jun 11, 2019 at 3:45 PM Kevin Doran  wrote:
> >>
> >> Yep, I support these suggestions.
> >>
> >> Setting up GPG does have a learning curve for folks that haven't done
> >> it before, but I think our community would be helpful in assisting
> >> folks on the mailing list and Apache NiFi Slack where they run into
> >> trouble. It's a good practice to learn and once setup there's not much
> >> more to do to get the benefits of it.
> >>
> >> Setting up GPG is also required when acting as release manager in
> >> order to sign convenience binaries (and soon, as Andy brought up,
> >> maven release artifacts as well - I think that is also a good idea),
> >> so the effort required to get setup for GPG has lots of benefits for
> >> folks that are interested in RM'ing as well.
> >>
> >> Kevin
> >>
> >> On Tue, Jun 11, 2019 at 3:30 PM Peter Wicks (pwicks)  
> >> wrote:
> >>>
> >>> I like having signed commits. I develop on both Windows and Linux, but 
> >>> have only had success getting signing working on Windows (which was a bit 
> >>> complicated as it was). You can see when I switched from mostly Windows 
> >>> to mostly Linux by when I stopped signing commits...
> >>>
> >>> Thanks,
> >>>  Peter
> >>>
> >>> -Original Message-
> >>> From: Andy LoPresto 
> >>> Sent: Tuesday, June 11, 2019 1:25 PM
> >>> To: dev@nifi.apache.org
> >>> Subject: [EXT] Re: GitHub Stuff
> >>>
> >>> I strongly support both of these suggestions. Thanks for starting the 
> >>> conversation Bryan. GPG signing is very important for security and for 
> >>> encouraging the rest of the community to adopt these practices as well.
> >>>
> >>>
> >>> Andy LoPresto
> >>> alopre...@apache.org
> >>> alopresto.apa...@gmail.com
> >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>
>  On Jun 11, 2019, at 11:42 AM, Bryan Bende  wrote:
> 
>  I had two thoughts related to our GitHub usage that I wanted to throw
>  out there for PMC members and committers...
> 
>  1) I think it would be helpful if everyone setup the link between
>  their Apache id and github [1]. Setting up this link puts you into the
>  nifi-committers group in Apache (currently 17 of us are in there), and
>  I believe this is what controls the list of users that can be selected
>  as a reviewer on a pull request. Since PRs are the primary form of
>  contribution, it would be nice if all of the PMC/committers were in
>  the reviewer list, but of course you can continue to commit against
>  Gitbox without doing this.
> 
>  2) I also think it would be nice if most of the commits in the repo
>  were signed commits that show up as "Verified" in GitHub [2]. Right
>  now I think we lose the verification if the user reviewing the commit
>  doesn't have signing setup, because when you amend the commit to add
>  "This closes ...", it technically produces a new commit hash, thus
>  making the original signature no longer apply (at least this is what I
>  think is happening, but other may know more).
> 
>  These are obviously just my opinions and no one has to do these
>  things, but just thought I would throw it out there for discussion in
>  case anyone wasn't aware.
> 
>  -Bryan
> 
>  [1]
>  https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitb
>  ox.apache.org%2Fsetup%2Fdata=02%7C01%7Cpwicks%40micron.com%7Cc2f2
>  0a00f6424597c10708d6eea27d65%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C
>  0%7C636958778999592924sdata=mJ59FD6KSYn1jXHN0yRRagKf6BHdWn7N1ZXmV
>  4BtBi8%3Dreserved=0 [2]
>  https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp
>  .github.com%2Fen%2Farticles%2Fsigning-commitsdata=02%7C01%7Cpwick
>  s%40micron.com%7Cc2f20a00f6424597c10708d6eea27d65%7Cf38a5ecd28134862b1
>  1bac1d563c806f%7C0%7C0%7C636958778999592924sdata=%2BiByT0SfcxSsoL
>  XgS4VFLI1DTBn9BW3vD1iPvCCqRSI%3Dreserved=0
> >>>
>