Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Athar]

Hi Mike,

Thank you for quick response. But I have requirement where different users
provide ASCII-armored format Keys (pubring.asc) and I have to encrypt the
data through PGP algorithm by using those key. I can convert the
ASCII-armored  keys into binary through GPG commands. But now next challenge
is "Public Keyring File" property doesn't support expression language.  


Thanks
Athar Iqbal



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/EncryptContent-issues-after-NIFI-1257-and-NIFI-1259-tp8581p15657.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Michael Moser]

Hello,

I believe the EncryptContent "Public Keyring File" property is expecting
the binary key that you generated in step 1.  You do not need to export the
public key into ASCII format.

Kind Regards,
-- Mike


On Wed, May 3, 2017 at 6:40 AM, Athar <athar.iqba...@gmail.com> wrote:

> I am getting this issue in even nifi 1.0.0 .  I am using "PGP_ASCII_ARMOR"
> encryption algorithm.
>
> I performed the following steps.
> 1 )  I  created the binary key using "GnuPG v2.0.14"  and executed the
> "PGP"
> encryption algorithm. Its
> executing properly.
> 2) I exported the public key in ASCII format  and configure
> "PGP_ASCII_ARMOR".  Its displaying  "Invalid header encountered"
>
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/
> n15629/nifi_Error.png>
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/EncryptContent-issues-after-
> NIFI-1257-and-NIFI-1259-tp8581p15629.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Athar]

I am getting this issue in even nifi 1.0.0 .  I am using "PGP_ASCII_ARMOR"
encryption algorithm.

I performed the following steps.
1 )  I  created the binary key using "GnuPG v2.0.14"  and executed the "PGP"
encryption algorithm. Its 
executing properly. 
2) I exported the public key in ASCII format  and configure
"PGP_ASCII_ARMOR".  Its displaying  "Invalid header encountered"

<http://apache-nifi-developer-list.39713.n7.nabble.com/file/n15629/nifi_Error.png>
  
  



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/EncryptContent-issues-after-NIFI-1257-and-NIFI-1259-tp8581p15629.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

Added to the release notes on the wiki.

Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 29, 2016, at 2:45 PM, Sean Busbey  wrote:
> 
> In the mean time can we call this out in the release notes as a known
> issue so that folks using things as Alan was know about it before htey
> upgrade?
> 
> On Tue, Mar 29, 2016 at 12:58 PM, Andy LoPresto
>  wrote:
>> Thanks Alan. I don’t anticipate it being a large effort. I have it marked as
>> minor and will bump it if resources are strained.
>> 
>> Andy LoPresto
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>> On Mar 29, 2016, at 10:32 AM, Alan Jackoway  wrote:
>> 
>> Honestly, it's not clear to me that we should handle this scenario. The
>> only reason I would propose fixing it is to handle people (like me) who did
>> it wrong and then upgraded. Requiring a keyring isn't that unusual, and the
>> docs are pretty specific. I just didn't read them.
>> 
>> Alan
>> 
>> On Tue, Mar 29, 2016 at 1:28 PM, Andy LoPresto 
>> wrote:
>> 
>> Alan,
>> 
>> The processor properties for public keyring file and secret keyring file
>> are fairly explicit in their names, so when I upgraded the BouncyCastle
>> dependencies, I wrote logic that performs strict validation on the file
>> format because the underlying library code changed substantially. I was
>> unaware anyone was using the individual key file there.
>> 
>> I have created a Jira [1] for 0.7.0 to add custom logic to handle this
>> scenario.
>> 
>> [1] https://issues.apache.org/jira/browse/NIFI-1694
>> 
>> Andy LoPresto
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>> On Mar 29, 2016, at 8:03 AM, Alan Jackoway  wrote:
>> 
>> I don't get a stacktrace. Probably because it is a validation failure and
>> the error is caught at
>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptContent.java#L288
>> 
>> I couldn't get your template to work without the gpgkeyring file. However,
>> that clued me into what I believe is the problem.
>> 
>> I have not been using a public keyring file, but rather the public key
>> itself. Somehow that used to work, but the parameter has always been called
>> Public Keyring File so I was using it wrong the whole time.
>> 
>> I attached the encrypt template that is working for me back in 0.3.0 (and
>> should work in 0.4.1 but not 0.5.1)
>> 
>> To fix it for 0.5.1, I had to make a real keyring file AND change the user
>> id to be the right thing.
>> 
>> This feels like a regression to me, but one where I was not following the
>> instructions all along.
>> 
>> Thanks,
>> Alan
>> 
>> On Tue, Mar 29, 2016 at 1:15 AM, Andy LoPresto > 
>> wrote:
>> 
>> 
>> The only other thing I can think of off the top of my head is that the
>> userID specification may have changed with the BouncyCastle upgrade and the
>> provided userID of just an email may be incomplete? In my testing, I had to
>> specify the "name", "description", and "email" fields from the key in the
>> format below in order to match the exact format that the library reads from
>> the keyring.
>> 
>> userID = "Name (Description) "
>> 
>> You can test this and evaluate what the library sees as the key userID by
>> attaching a remote debugger to your running instance and evaluating inside
>> the iterator loop here [1].
>> 
>> I'm not sure what version of GPG you're running, but it is worth
>> investigating if the format of the stored key no longer matches how NiFi
>> was reading it.
>> 
>> [1]
>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200
>> 
>> 
>> 
>> Andy LoPresto
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>> On Mar 28, 2016, at 18:24, Andy LoPresto 
>> 
>> wrote:
>> 
>> 
>> Forgot to mention you’ll want to change the input/output directories in
>> 
>> the GetFile and PutFile processors, as well as the paths to the public and
>> secret keyring, the user ID, and the password for the EncryptContent
>> processors.
>> 
>> 
>> Andy LoPresto
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>> On Mar 28, 2016, at 4:04 PM, Andy LoPresto 
>> 
>> wrote:
>> 
>> 
>> Hi Alan,
>> 
>> I am investigating this issue (spinning up an instance, setting up a
>> 
>> flow that involves PGP encryption and decryption, etc.) to verify.
>> 
>> 
>> As an aside, the setting for “Key Derivation Function” is 

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Sean Busbey]

In the mean time can we call this out in the release notes as a known
issue so that folks using things as Alan was know about it before htey
upgrade?

On Tue, Mar 29, 2016 at 12:58 PM, Andy LoPresto
 wrote:
> Thanks Alan. I don’t anticipate it being a large effort. I have it marked as
> minor and will bump it if resources are strained.
>
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Mar 29, 2016, at 10:32 AM, Alan Jackoway  wrote:
>
> Honestly, it's not clear to me that we should handle this scenario. The
> only reason I would propose fixing it is to handle people (like me) who did
> it wrong and then upgraded. Requiring a keyring isn't that unusual, and the
> docs are pretty specific. I just didn't read them.
>
> Alan
>
> On Tue, Mar 29, 2016 at 1:28 PM, Andy LoPresto 
> wrote:
>
> Alan,
>
> The processor properties for public keyring file and secret keyring file
> are fairly explicit in their names, so when I upgraded the BouncyCastle
> dependencies, I wrote logic that performs strict validation on the file
> format because the underlying library code changed substantially. I was
> unaware anyone was using the individual key file there.
>
> I have created a Jira [1] for 0.7.0 to add custom logic to handle this
> scenario.
>
> [1] https://issues.apache.org/jira/browse/NIFI-1694
>
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Mar 29, 2016, at 8:03 AM, Alan Jackoway  wrote:
>
> I don't get a stacktrace. Probably because it is a validation failure and
> the error is caught at
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptContent.java#L288
>
> I couldn't get your template to work without the gpgkeyring file. However,
> that clued me into what I believe is the problem.
>
> I have not been using a public keyring file, but rather the public key
> itself. Somehow that used to work, but the parameter has always been called
> Public Keyring File so I was using it wrong the whole time.
>
> I attached the encrypt template that is working for me back in 0.3.0 (and
> should work in 0.4.1 but not 0.5.1)
>
> To fix it for 0.5.1, I had to make a real keyring file AND change the user
> id to be the right thing.
>
> This feels like a regression to me, but one where I was not following the
> instructions all along.
>
> Thanks,
> Alan
>
> On Tue, Mar 29, 2016 at 1:15 AM, Andy LoPresto 
> wrote:
>
>
> The only other thing I can think of off the top of my head is that the
> userID specification may have changed with the BouncyCastle upgrade and the
> provided userID of just an email may be incomplete? In my testing, I had to
> specify the "name", "description", and "email" fields from the key in the
> format below in order to match the exact format that the library reads from
> the keyring.
>
> userID = "Name (Description) "
>
> You can test this and evaluate what the library sees as the key userID by
> attaching a remote debugger to your running instance and evaluating inside
> the iterator loop here [1].
>
> I'm not sure what version of GPG you're running, but it is worth
> investigating if the format of the stored key no longer matches how NiFi
> was reading it.
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200
>
>
>
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Mar 28, 2016, at 18:24, Andy LoPresto 
>
> wrote:
>
>
> Forgot to mention you’ll want to change the input/output directories in
>
> the GetFile and PutFile processors, as well as the paths to the public and
> secret keyring, the user ID, and the password for the EncryptContent
> processors.
>
>
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Mar 28, 2016, at 4:04 PM, Andy LoPresto 
>
> wrote:
>
>
> Hi Alan,
>
> I am investigating this issue (spinning up an instance, setting up a
>
> flow that involves PGP encryption and decryption, etc.) to verify.
>
>
> As an aside, the setting for “Key Derivation Function” is irrelevant
>
> if “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is
> required for symmetric encryption (deriving a key from the provided
> password), but not used for PGP encryption/decryption at all.
> Unfortunately, we cannot currently display/hide or change the required-ness
> of processor properties based on the value of other properties. There is an
> existing Jira open [1] to 

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

Thanks Alan. I don’t anticipate it being a large effort. I have it marked as 
minor and will bump it if resources are strained.

Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 29, 2016, at 10:32 AM, Alan Jackoway  wrote:
> 
> Honestly, it's not clear to me that we should handle this scenario. The
> only reason I would propose fixing it is to handle people (like me) who did
> it wrong and then upgraded. Requiring a keyring isn't that unusual, and the
> docs are pretty specific. I just didn't read them.
> 
> Alan
> 
> On Tue, Mar 29, 2016 at 1:28 PM, Andy LoPresto 
> wrote:
> 
>> Alan,
>> 
>> The processor properties for public keyring file and secret keyring file
>> are fairly explicit in their names, so when I upgraded the BouncyCastle
>> dependencies, I wrote logic that performs strict validation on the file
>> format because the underlying library code changed substantially. I was
>> unaware anyone was using the individual key file there.
>> 
>> I have created a Jira [1] for 0.7.0 to add custom logic to handle this
>> scenario.
>> 
>> [1] https://issues.apache.org/jira/browse/NIFI-1694
>> 
>> Andy LoPresto
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>> On Mar 29, 2016, at 8:03 AM, Alan Jackoway  wrote:
>> 
>> I don't get a stacktrace. Probably because it is a validation failure and
>> the error is caught at
>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptContent.java#L288
>> 
>> I couldn't get your template to work without the gpgkeyring file. However,
>> that clued me into what I believe is the problem.
>> 
>> I have not been using a public keyring file, but rather the public key
>> itself. Somehow that used to work, but the parameter has always been called
>> Public Keyring File so I was using it wrong the whole time.
>> 
>> I attached the encrypt template that is working for me back in 0.3.0 (and
>> should work in 0.4.1 but not 0.5.1)
>> 
>> To fix it for 0.5.1, I had to make a real keyring file AND change the user
>> id to be the right thing.
>> 
>> This feels like a regression to me, but one where I was not following the
>> instructions all along.
>> 
>> Thanks,
>> Alan
>> 
>> On Tue, Mar 29, 2016 at 1:15 AM, Andy LoPresto >> wrote:
>> 
>>> The only other thing I can think of off the top of my head is that the
>>> userID specification may have changed with the BouncyCastle upgrade and the
>>> provided userID of just an email may be incomplete? In my testing, I had to
>>> specify the "name", "description", and "email" fields from the key in the
>>> format below in order to match the exact format that the library reads from
>>> the keyring.
>>> 
>>> userID = "Name (Description) "
>>> 
>>> You can test this and evaluate what the library sees as the key userID by
>>> attaching a remote debugger to your running instance and evaluating inside
>>> the iterator loop here [1].
>>> 
>>> I'm not sure what version of GPG you're running, but it is worth
>>> investigating if the format of the stored key no longer matches how NiFi
>>> was reading it.
>>> 
>>> [1]
>>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200
>>> 
>>> 
>>> 
>>> Andy LoPresto
>>> alopresto.apa...@gmail.com
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>> 
 On Mar 28, 2016, at 18:24, Andy LoPresto 
>>> wrote:
 
 Forgot to mention you’ll want to change the input/output directories in
>>> the GetFile and PutFile processors, as well as the paths to the public and
>>> secret keyring, the user ID, and the password for the EncryptContent
>>> processors.
 
 Andy LoPresto
 alopresto.apa...@gmail.com
 PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
 
> On Mar 28, 2016, at 4:04 PM, Andy LoPresto 
>>> wrote:
> 
> Hi Alan,
> 
> I am investigating this issue (spinning up an instance, setting up a
>>> flow that involves PGP encryption and decryption, etc.) to verify.
> 
> As an aside, the setting for “Key Derivation Function” is irrelevant
>>> if “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is
>>> required for symmetric encryption (deriving a key from the provided
>>> password), but not used for PGP encryption/decryption at all.
>>> Unfortunately, we cannot currently display/hide or change the required-ness
>>> of processor properties based on the value of other properties. There is an
>>> existing Jira open [1] to enhance this functionality. Perhaps this can be
>>> better 

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

Alan,

The processor properties for public keyring file and secret keyring file are 
fairly explicit in their names, so when I upgraded the BouncyCastle 
dependencies, I wrote logic that performs strict validation on the file format 
because the underlying library code changed substantially. I was unaware anyone 
was using the individual key file there.

I have created a Jira [1] for 0.7.0 to add custom logic to handle this scenario.

[1] https://issues.apache.org/jira/browse/NIFI-1694 


Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 29, 2016, at 8:03 AM, Alan Jackoway  wrote:
> 
> I don't get a stacktrace. Probably because it is a validation failure and the 
> error is caught at 
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptContent.java#L288
>  
> 
> 
> I couldn't get your template to work without the gpgkeyring file. However, 
> that clued me into what I believe is the problem.
> 
> I have not been using a public keyring file, but rather the public key 
> itself. Somehow that used to work, but the parameter has always been called 
> Public Keyring File so I was using it wrong the whole time.
> 
> I attached the encrypt template that is working for me back in 0.3.0 (and 
> should work in 0.4.1 but not 0.5.1)
> 
> To fix it for 0.5.1, I had to make a real keyring file AND change the user id 
> to be the right thing.
> 
> This feels like a regression to me, but one where I was not following the 
> instructions all along.
> 
> Thanks,
> Alan
> 
> On Tue, Mar 29, 2016 at 1:15 AM, Andy LoPresto  > wrote:
> The only other thing I can think of off the top of my head is that the userID 
> specification may have changed with the BouncyCastle upgrade and the provided 
> userID of just an email may be incomplete? In my testing, I had to specify 
> the "name", "description", and "email" fields from the key in the format 
> below in order to match the exact format that the library reads from the 
> keyring.
> 
> userID = "Name (Description) "
> 
> You can test this and evaluate what the library sees as the key userID by 
> attaching a remote debugger to your running instance and evaluating inside 
> the iterator loop here [1].
> 
> I'm not sure what version of GPG you're running, but it is worth 
> investigating if the format of the stored key no longer matches how NiFi was 
> reading it.
> 
> [1] 
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200
>  
> 
> 
> 
> 
> Andy LoPresto
> alopresto.apa...@gmail.com 
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
> > On Mar 28, 2016, at 18:24, Andy LoPresto  > > wrote:
> >
> > Forgot to mention you’ll want to change the input/output directories in the 
> > GetFile and PutFile processors, as well as the paths to the public and 
> > secret keyring, the user ID, and the password for the EncryptContent 
> > processors.
> >
> > Andy LoPresto
> > alopresto.apa...@gmail.com 
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> >> On Mar 28, 2016, at 4:04 PM, Andy LoPresto  >> > wrote:
> >>
> >> Hi Alan,
> >>
> >> I am investigating this issue (spinning up an instance, setting up a flow 
> >> that involves PGP encryption and decryption, etc.) to verify.
> >>
> >> As an aside, the setting for “Key Derivation Function” is irrelevant if 
> >> “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is 
> >> required for symmetric encryption (deriving a key from the provided 
> >> password), but not used for PGP encryption/decryption at all. 
> >> Unfortunately, we cannot currently display/hide or change the 
> >> required-ness of processor properties based on the value of other 
> >> properties. There is an existing Jira open [1] to enhance this 
> >> functionality. Perhaps this can be better documented in the Admin Guide 
> >> [2].
> >>
> >> Can you also provide the full stacktrace and your system configuration, if 
> >> possible, to help with the troubleshooting? Thank you.
> 

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Alan Jackoway]

I don't get a stacktrace. Probably because it is a validation failure and
the error is caught at
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptContent.java#L288

I couldn't get your template to work without the gpgkeyring file. However,
that clued me into what I believe is the problem.

I have not been using a public keyring file, but rather the public key
itself. Somehow that used to work, but the parameter has always been called
Public Keyring File so I was using it wrong the whole time.

I attached the encrypt template that is working for me back in 0.3.0 (and
should work in 0.4.1 but not 0.5.1)

To fix it for 0.5.1, I had to make a real keyring file AND change the user
id to be the right thing.

This feels like a regression to me, but one where I was not following the
instructions all along.

Thanks,
Alan

On Tue, Mar 29, 2016 at 1:15 AM, Andy LoPresto 
wrote:

> The only other thing I can think of off the top of my head is that the
> userID specification may have changed with the BouncyCastle upgrade and the
> provided userID of just an email may be incomplete? In my testing, I had to
> specify the "name", "description", and "email" fields from the key in the
> format below in order to match the exact format that the library reads from
> the keyring.
>
> userID = "Name (Description) "
>
> You can test this and evaluate what the library sees as the key userID by
> attaching a remote debugger to your running instance and evaluating inside
> the iterator loop here [1].
>
> I'm not sure what version of GPG you're running, but it is worth
> investigating if the format of the stored key no longer matches how NiFi
> was reading it.
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200
>
>
>
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Mar 28, 2016, at 18:24, Andy LoPresto 
> wrote:
> >
> > Forgot to mention you’ll want to change the input/output directories in
> the GetFile and PutFile processors, as well as the paths to the public and
> secret keyring, the user ID, and the password for the EncryptContent
> processors.
> >
> > Andy LoPresto
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> >> On Mar 28, 2016, at 4:04 PM, Andy LoPresto 
> wrote:
> >>
> >> Hi Alan,
> >>
> >> I am investigating this issue (spinning up an instance, setting up a
> flow that involves PGP encryption and decryption, etc.) to verify.
> >>
> >> As an aside, the setting for “Key Derivation Function” is irrelevant if
> “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is
> required for symmetric encryption (deriving a key from the provided
> password), but not used for PGP encryption/decryption at all.
> Unfortunately, we cannot currently display/hide or change the required-ness
> of processor properties based on the value of other properties. There is an
> existing Jira open [1] to enhance this functionality. Perhaps this can be
> better documented in the Admin Guide [2].
> >>
> >> Can you also provide the full stacktrace and your system configuration,
> if possible, to help with the troubleshooting? Thank you.
> >>
> >> [1] https://issues.apache.org/jira/browse/NIFI-1121
> >> [2]
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encryption
> >>
> >>
> >> Andy LoPresto
> >> alopresto.apa...@gmail.com
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>
> >>> On Mar 28, 2016, at 2:18 PM, Alan Jackoway  wrote:
> >>>
> >>> Hello,
> >>>
> >>> I had an EncryptContent processor running with PGP public key
> encryption when we were running NiFi 0.4.x.
> >>>
> >>> We recently went up to a 0.5.x, which includes NIFI-1257 and
> NIFI-1259. Now my EncryptContent processors are failing to validate my key
> with an error message:
> >>> 'Public Keyring File' is invalid because Invalid Public Keyring File
> filename because java.io.IOException: invalid header encountered
> >>>
> >>> I tried all the key derivation functions, but in all cases I got the
> same error.
> >>>
> >>> Is there an easy way to talk NiFi into using my key again?
> >>>
> >>> I have attached a public key that works on 0.3.0 (I didn't have 0.4 on
> my machine for some reason) but fails in 0.5.1. The user id is
> al...@cloudera.com
> >>>
> >>> Is there any easy fix? Should I file a jira?
> >>>
> >>> Since it said invalid header, I tried taking out the comment at the
> top of the key. That didn't work.
> >>>
> >>> Thanks,
> >>> Alan
> >>> 
> >
>
Encrypts using only a public key (not 

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

The only other thing I can think of off the top of my head is that the userID 
specification may have changed with the BouncyCastle upgrade and the provided 
userID of just an email may be incomplete? In my testing, I had to specify the 
"name", "description", and "email" fields from the key in the format below in 
order to match the exact format that the library reads from the keyring. 

userID = "Name (Description) "

You can test this and evaluate what the library sees as the key userID by 
attaching a remote debugger to your running instance and evaluating inside the 
iterator loop here [1]. 

I'm not sure what version of GPG you're running, but it is worth investigating 
if the format of the stored key no longer matches how NiFi was reading it. 

[1] 
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200



Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 28, 2016, at 18:24, Andy LoPresto  wrote:
> 
> Forgot to mention you’ll want to change the input/output directories in the 
> GetFile and PutFile processors, as well as the paths to the public and secret 
> keyring, the user ID, and the password for the EncryptContent processors. 
> 
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Mar 28, 2016, at 4:04 PM, Andy LoPresto  
>> wrote:
>> 
>> Hi Alan,
>> 
>> I am investigating this issue (spinning up an instance, setting up a flow 
>> that involves PGP encryption and decryption, etc.) to verify. 
>> 
>> As an aside, the setting for “Key Derivation Function” is irrelevant if 
>> “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is 
>> required for symmetric encryption (deriving a key from the provided 
>> password), but not used for PGP encryption/decryption at all. Unfortunately, 
>> we cannot currently display/hide or change the required-ness of processor 
>> properties based on the value of other properties. There is an existing Jira 
>> open [1] to enhance this functionality. Perhaps this can be better 
>> documented in the Admin Guide [2]. 
>> 
>> Can you also provide the full stacktrace and your system configuration, if 
>> possible, to help with the troubleshooting? Thank you. 
>> 
>> [1] https://issues.apache.org/jira/browse/NIFI-1121
>> [2] 
>> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encryption
>> 
>> 
>> Andy LoPresto
>> alopresto.apa...@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>>> On Mar 28, 2016, at 2:18 PM, Alan Jackoway  wrote:
>>> 
>>> Hello,
>>> 
>>> I had an EncryptContent processor running with PGP public key encryption 
>>> when we were running NiFi 0.4.x.
>>> 
>>> We recently went up to a 0.5.x, which includes NIFI-1257 and NIFI-1259. Now 
>>> my EncryptContent processors are failing to validate my key with an error 
>>> message:
>>> 'Public Keyring File' is invalid because Invalid Public Keyring File 
>>> filename because java.io.IOException: invalid header encountered
>>> 
>>> I tried all the key derivation functions, but in all cases I got the same 
>>> error.
>>> 
>>> Is there an easy way to talk NiFi into using my key again?
>>> 
>>> I have attached a public key that works on 0.3.0 (I didn't have 0.4 on my 
>>> machine for some reason) but fails in 0.5.1. The user id is 
>>> al...@cloudera.com
>>> 
>>> Is there any easy fix? Should I file a jira?
>>> 
>>> Since it said invalid header, I tried taking out the comment at the top of 
>>> the key. That didn't work.
>>> 
>>> Thanks,
>>> Alan
>>> 
> 

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

Forgot to mention you’ll want to change the input/output directories in the 
GetFile and PutFile processors, as well as the paths to the public and secret 
keyring, the user ID, and the password for the EncryptContent processors.

Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 28, 2016, at 4:04 PM, Andy LoPresto  wrote:
> 
> Hi Alan,
> 
> I am investigating this issue (spinning up an instance, setting up a flow 
> that involves PGP encryption and decryption, etc.) to verify.
> 
> As an aside, the setting for “Key Derivation Function” is irrelevant if 
> “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is 
> required for symmetric encryption (deriving a key from the provided 
> password), but not used for PGP encryption/decryption at all. Unfortunately, 
> we cannot currently display/hide or change the required-ness of processor 
> properties based on the value of other properties. There is an existing Jira 
> open [1] to enhance this functionality. Perhaps this can be better documented 
> in the Admin Guide [2].
> 
> Can you also provide the full stacktrace and your system configuration, if 
> possible, to help with the troubleshooting? Thank you.
> 
> [1] https://issues.apache.org/jira/browse/NIFI-1121 
> 
> [2] 
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encryption
>  
> 
> 
> 
> Andy LoPresto
> alopresto.apa...@gmail.com 
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Mar 28, 2016, at 2:18 PM, Alan Jackoway > > wrote:
>> 
>> Hello,
>> 
>> I had an EncryptContent processor running with PGP public key encryption 
>> when we were running NiFi 0.4.x.
>> 
>> We recently went up to a 0.5.x, which includes NIFI-1257 and NIFI-1259. Now 
>> my EncryptContent processors are failing to validate my key with an error 
>> message:
>> 'Public Keyring File' is invalid because Invalid Public Keyring File 
>> filename because java.io.IOException: invalid header encountered
>> 
>> I tried all the key derivation functions, but in all cases I got the same 
>> error.
>> 
>> Is there an easy way to talk NiFi into using my key again?
>> 
>> I have attached a public key that works on 0.3.0 (I didn't have 0.4 on my 
>> machine for some reason) but fails in 0.5.1. The user id is 
>> al...@cloudera.com 
>> 
>> Is there any easy fix? Should I file a jira?
>> 
>> Since it said invalid header, I tried taking out the comment at the top of 
>> the key. That didn't work.
>> 
>> Thanks,
>> Alan
>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

Hi Alan,

I have created a template [1] which should be able to test the issue you are 
encountering. It works for me (Mac OS X 10.11, NiFi 0.6.0-SNAPSHOT, gpg 
2.0.28), so I am hoping you can run it on your installation and verify. I 
understand you are running NiFi 0.5.1, but to my knowledge, nothing in the 
encryption processing changed between 0.5.1 and 0.6.0.

The only issue I encountered is that “~” expansion does not work if the file 
path you provide to the public or secret keyring starts with the “~” shortcut 
for the user home directory. I do not believe this changed between 0.3.0 and 
0.5.1, but it could have been a dependency change (BouncyCastle was upgraded 
from the legacy jdk16 version to the current and updated jdk15on [2]. I have 
filed a Jira for this issue [3].

Please let me know if this was the issue you were encountering, and if not, any 
additional information to help resolve your issue.


[1] https://gist.github.com/alopresto/87494d245c9298c69352 

[2] https://issues.apache.org/jira/browse/NIFI-1324 

[3] https://issues.apache.org/jira/browse/NIFI-1693 


Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 28, 2016, at 4:04 PM, Andy LoPresto  wrote:
> 
> Hi Alan,
> 
> I am investigating this issue (spinning up an instance, setting up a flow 
> that involves PGP encryption and decryption, etc.) to verify.
> 
> As an aside, the setting for “Key Derivation Function” is irrelevant if 
> “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is 
> required for symmetric encryption (deriving a key from the provided 
> password), but not used for PGP encryption/decryption at all. Unfortunately, 
> we cannot currently display/hide or change the required-ness of processor 
> properties based on the value of other properties. There is an existing Jira 
> open [1] to enhance this functionality. Perhaps this can be better documented 
> in the Admin Guide [2].
> 
> Can you also provide the full stacktrace and your system configuration, if 
> possible, to help with the troubleshooting? Thank you.
> 
> [1] https://issues.apache.org/jira/browse/NIFI-1121 
> 
> [2] 
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encryption
>  
> 
> 
> 
> Andy LoPresto
> alopresto.apa...@gmail.com 
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Mar 28, 2016, at 2:18 PM, Alan Jackoway > > wrote:
>> 
>> Hello,
>> 
>> I had an EncryptContent processor running with PGP public key encryption 
>> when we were running NiFi 0.4.x.
>> 
>> We recently went up to a 0.5.x, which includes NIFI-1257 and NIFI-1259. Now 
>> my EncryptContent processors are failing to validate my key with an error 
>> message:
>> 'Public Keyring File' is invalid because Invalid Public Keyring File 
>> filename because java.io.IOException: invalid header encountered
>> 
>> I tried all the key derivation functions, but in all cases I got the same 
>> error.
>> 
>> Is there an easy way to talk NiFi into using my key again?
>> 
>> I have attached a public key that works on 0.3.0 (I didn't have 0.4 on my 
>> machine for some reason) but fails in 0.5.1. The user id is 
>> al...@cloudera.com 
>> 
>> Is there any easy fix? Should I file a jira?
>> 
>> Since it said invalid header, I tried taking out the comment at the top of 
>> the key. That didn't work.
>> 
>> Thanks,
>> Alan
>> 
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: EncryptContent issues after NIFI-1257 and NIFI-1259

[Andy LoPresto]

Hi Alan,

I am investigating this issue (spinning up an instance, setting up a flow that 
involves PGP encryption and decryption, etc.) to verify.

As an aside, the setting for “Key Derivation Function” is irrelevant if 
“Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is 
required for symmetric encryption (deriving a key from the provided password), 
but not used for PGP encryption/decryption at all. Unfortunately, we cannot 
currently display/hide or change the required-ness of processor properties 
based on the value of other properties. There is an existing Jira open [1] to 
enhance this functionality. Perhaps this can be better documented in the Admin 
Guide [2].

Can you also provide the full stacktrace and your system configuration, if 
possible, to help with the troubleshooting? Thank you.

[1] https://issues.apache.org/jira/browse/NIFI-1121 

[2] 
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encryption
 



Andy LoPresto
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Mar 28, 2016, at 2:18 PM, Alan Jackoway  wrote:
> 
> Hello,
> 
> I had an EncryptContent processor running with PGP public key encryption when 
> we were running NiFi 0.4.x.
> 
> We recently went up to a 0.5.x, which includes NIFI-1257 and NIFI-1259. Now 
> my EncryptContent processors are failing to validate my key with an error 
> message:
> 'Public Keyring File' is invalid because Invalid Public Keyring File filename 
> because java.io.IOException: invalid header encountered
> 
> I tried all the key derivation functions, but in all cases I got the same 
> error.
> 
> Is there an easy way to talk NiFi into using my key again?
> 
> I have attached a public key that works on 0.3.0 (I didn't have 0.4 on my 
> machine for some reason) but fails in 0.5.1. The user id is 
> al...@cloudera.com 
> 
> Is there any easy fix? Should I file a jira?
> 
> Since it said invalid header, I tried taking out the comment at the top of 
> the key. That didn't work.
> 
> Thanks,
> Alan
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail