I don't get a stacktrace. Probably because it is a validation failure and
the error is caught at
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncryptContent.java#L288
I couldn't get your template to work without the gpgkeyring file. However,
that clued me into what I believe is the problem.
I have not been using a public keyring file, but rather the public key
itself. Somehow that used to work, but the parameter has always been called
Public Keyring File so I was using it wrong the whole time.
I attached the encrypt template that is working for me back in 0.3.0 (and
should work in 0.4.1 but not 0.5.1)
To fix it for 0.5.1, I had to make a real keyring file AND change the user
id to be the right thing.
This feels like a regression to me, but one where I was not following the
instructions all along.
Thanks,
Alan
On Tue, Mar 29, 2016 at 1:15 AM, Andy LoPresto <alopresto.apa...@gmail.com>
wrote:
> The only other thing I can think of off the top of my head is that the
> userID specification may have changed with the BouncyCastle upgrade and the
> provided userID of just an email may be incomplete? In my testing, I had to
> specify the "name", "description", and "email" fields from the key in the
> format below in order to match the exact format that the library reads from
> the keyring.
>
> userID = "Name (Description) <Email>"
>
> You can test this and evaluate what the library sees as the key userID by
> attaching a remote debugger to your running instance and evaluating inside
> the iterator loop here [1].
>
> I'm not sure what version of GPG you're running, but it is worth
> investigating if the format of the stored key no longer matches how NiFi
> was reading it.
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/crypto/OpenPGPKeyBasedEncryptor.java#L200
>
>
>
> Andy LoPresto
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> > On Mar 28, 2016, at 18:24, Andy LoPresto <alopresto.apa...@gmail.com>
> wrote:
> >
> > Forgot to mention you’ll want to change the input/output directories in
> the GetFile and PutFile processors, as well as the paths to the public and
> secret keyring, the user ID, and the password for the EncryptContent
> processors.
> >
> > Andy LoPresto
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >
> >> On Mar 28, 2016, at 4:04 PM, Andy LoPresto <alopresto.apa...@gmail.com>
> wrote:
> >>
> >> Hi Alan,
> >>
> >> I am investigating this issue (spinning up an instance, setting up a
> flow that involves PGP encryption and decryption, etc.) to verify.
> >>
> >> As an aside, the setting for “Key Derivation Function” is irrelevant if
> “Encryption Algorithm” is set to “PGP” or “PGP_ASCII_ARMOR”. The KDF is
> required for symmetric encryption (deriving a key from the provided
> password), but not used for PGP encryption/decryption at all.
> Unfortunately, we cannot currently display/hide or change the required-ness
> of processor properties based on the value of other properties. There is an
> existing Jira open [1] to enhance this functionality. Perhaps this can be
> better documented in the Admin Guide [2].
> >>
> >> Can you also provide the full stacktrace and your system configuration,
> if possible, to help with the troubleshooting? Thank you.
> >>
> >> [1] https://issues.apache.org/jira/browse/NIFI-1121
> >> [2]
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encryption
> >>
> >>
> >> Andy LoPresto
> >> alopresto.apa...@gmail.com
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >>
> >>> On Mar 28, 2016, at 2:18 PM, Alan Jackoway <al...@cloudera.com> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I had an EncryptContent processor running with PGP public key
> encryption when we were running NiFi 0.4.x.
> >>>
> >>> We recently went up to a 0.5.x, which includes NIFI-1257 and
> NIFI-1259. Now my EncryptContent processors are failing to validate my key
> with an error message:
> >>> 'Public Keyring File' is invalid because Invalid Public Keyring File
> filename because java.io.IOException: invalid header encountered
> >>>
> >>> I tried all the key derivation functions, but in all cases I got the
> same error.
> >>>
> >>> Is there an easy way to talk NiFi into using my key again?
> >>>
> >>> I have attached a public key that works on 0.3.0 (I didn't have 0.4 on
> my machine for some reason) but fails in 0.5.1. The user id is
> al...@cloudera.com
> >>>
> >>> Is there any easy fix? Should I file a jira?
> >>>
> >>> Since it said invalid header, I tried taking out the comment at the
> top of the key. That didn't work.
> >>>
> >>> Thanks,
> >>> Alan
> >>> <TestPublicKey.asc>
> >
>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template><description>Encrypts using only a public key (not keyring)</description><name>AlanEncryptTemplate</name><snippet><processors><id>9f817fd9-854f-4950-8daf-fdb3fc358756</id><parentGroupId>46e23601-c4b5-4328-8759-2b58ff4defbf</parentGroupId><position><x>2905.9559587098915</x><y>142.95112106092614</y></position><config><bulletinLevel>WARN</bulletinLevel><comments></comments><concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount><defaultConcurrentTasks><entry><key>TIMER_DRIVEN</key><value>1</value></entry><entry><key>EVENT_DRIVEN</key><value>0</value></entry><entry><key>CRON_DRIVEN</key><value>1</value></entry></defaultConcurrentTasks><defaultSchedulingPeriod><entry><key>TIMER_DRIVEN</key><value>0 sec</value></entry><entry><key>CRON_DRIVEN</key><value>* * * * * ?</value></entry></defaultSchedulingPeriod><descriptors><entry><key>Directory</key><value><description>The directory to which files should be written. You may use expression language such as /aa/bb/${path}</description><displayName>Directory</displayName><dynamic>false</dynamic><name>Directory</name><required>true</required><sensitive>false</sensitive><supportsEl>true</supportsEl></value></entry><entry><key>Conflict Resolution Strategy</key><value><allowableValues><displayName>replace</displayName><value>replace</value></allowableValues><allowableValues><displayName>ignore</displayName><value>ignore</value></allowableValues><allowableValues><displayName>fail</displayName><value>fail</value></allowableValues><defaultValue>fail</defaultValue><description>Indicates what should happen when a file with the same name already exists in the output directory</description><displayName>Conflict Resolution Strategy</displayName><dynamic>false</dynamic><name>Conflict Resolution Strategy</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Create Missing Directories</key><value><allowableValues><displayName>true</displayName><value>true</value></allowableValues><allowableValues><displayName>false</displayName><value>false</value></allowableValues><defaultValue>true</defaultValue><description>If true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure.</description><displayName>Create Missing Directories</displayName><dynamic>false</dynamic><name>Create Missing Directories</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Maximum File Count</key><value><description>Specifies the maximum number of files that can exist in the output directory</description><displayName>Maximum File Count</displayName><dynamic>false</dynamic><name>Maximum File Count</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Last Modified Time</key><value><description>Sets the lastModifiedTime on the output file to the value of this attribute. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}.</description><displayName>Last Modified Time</displayName><dynamic>false</dynamic><name>Last Modified Time</name><required>false</required><sensitive>false</sensitive><supportsEl>true</supportsEl></value></entry><entry><key>Permissions</key><value><description>Sets the permissions on the output file to the value of this attribute. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). You may also use expression language such as ${file.permissions}.</description><displayName>Permissions</displayName><dynamic>false</dynamic><name>Permissions</name><required>false</required><sensitive>false</sensitive><supportsEl>true</supportsEl></value></entry><entry><key>Owner</key><value><description>Sets the owner on the output file to the value of this attribute. You may also use expression language such as ${file.owner}.</description><displayName>Owner</displayName><dynamic>false</dynamic><name>Owner</name><required>false</required><sensitive>false</sensitive><supportsEl>true</supportsEl></value></entry><entry><key>Group</key><value><description>Sets the group on the output file to the value of this attribute. You may also use expression language such as ${file.group}.</description><displayName>Group</displayName><dynamic>false</dynamic><name>Group</name><required>false</required><sensitive>false</sensitive><supportsEl>true</supportsEl></value></entry></descriptors><lossTolerant>false</lossTolerant><penaltyDuration>30 sec</penaltyDuration><properties><entry><key>Directory</key><value>/tmp/encryptOut</value></entry><entry><key>Conflict Resolution Strategy</key></entry><entry><key>Create Missing Directories</key></entry><entry><key>Maximum File Count</key></entry><entry><key>Last Modified Time</key></entry><entry><key>Permissions</key></entry><entry><key>Owner</key></entry><entry><key>Group</key></entry></properties><runDurationMillis>0</runDurationMillis><schedulingPeriod>0 sec</schedulingPeriod><schedulingStrategy>TIMER_DRIVEN</schedulingStrategy><yieldDuration>1 sec</yieldDuration></config><name>PutFile</name><relationships><autoTerminate>true</autoTerminate><description>Files that could not be written to the output directory for some reason are transferred to this relationship</description><name>failure</name></relationships><relationships><autoTerminate>true</autoTerminate><description>Files that have been successfully written to the output directory are transferred to this relationship</description><name>success</name></relationships><state>RUNNING</state><style/><supportsEventDriven>false</supportsEventDriven><supportsParallelProcessing>true</supportsParallelProcessing><type>org.apache.nifi.processors.standard.PutFile</type></processors><processors><id>541a7c8d-8c12-4972-b03f-a64d6c69cca1</id><parentGroupId>46e23601-c4b5-4328-8759-2b58ff4defbf</parentGroupId><position><x>2904.578526104455</x><y>-88.45896513460156</y></position><config><bulletinLevel>WARN</bulletinLevel><comments></comments><concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount><defaultConcurrentTasks><entry><key>TIMER_DRIVEN</key><value>1</value></entry><entry><key>EVENT_DRIVEN</key><value>0</value></entry><entry><key>CRON_DRIVEN</key><value>1</value></entry></defaultConcurrentTasks><defaultSchedulingPeriod><entry><key>TIMER_DRIVEN</key><value>0 sec</value></entry><entry><key>CRON_DRIVEN</key><value>* * * * * ?</value></entry></defaultSchedulingPeriod><descriptors><entry><key>Mode</key><value><allowableValues><displayName>Encrypt</displayName><value>Encrypt</value></allowableValues><allowableValues><displayName>Decrypt</displayName><value>Decrypt</value></allowableValues><defaultValue>Encrypt</defaultValue><description>Specifies whether the content should be encrypted or decrypted</description><displayName>Mode</displayName><dynamic>false</dynamic><name>Mode</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Encryption Algorithm</key><value><allowableValues><displayName>MD5_128AES</displayName><value>MD5_128AES</value></allowableValues><allowableValues><displayName>MD5_256AES</displayName><value>MD5_256AES</value></allowableValues><allowableValues><displayName>SHA1_RC2</displayName><value>SHA1_RC2</value></allowableValues><allowableValues><displayName>SHA1_DES</displayName><value>SHA1_DES</value></allowableValues><allowableValues><displayName>MD5_192AES</displayName><value>MD5_192AES</value></allowableValues><allowableValues><displayName>MD5_DES</displayName><value>MD5_DES</value></allowableValues><allowableValues><displayName>MD5_RC2</displayName><value>MD5_RC2</value></allowableValues><allowableValues><displayName>SHA_192AES</displayName><value>SHA_192AES</value></allowableValues><allowableValues><displayName>SHA_40RC4</displayName><value>SHA_40RC4</value></allowableValues><allowableValues><displayName>SHA256_128AES</displayName><value>SHA256_128AES</value></allowableValues><allowableValues><displayName>SHA_128RC2</displayName><value>SHA_128RC2</value></allowableValues><allowableValues><displayName>SHA_128AES</displayName><value>SHA_128AES</value></allowableValues><allowableValues><displayName>SHA256_192AES</displayName><value>SHA256_192AES</value></allowableValues><allowableValues><displayName>SHA_2KEYTRIPLEDES</displayName><value>SHA_2KEYTRIPLEDES</value></allowableValues><allowableValues><displayName>SHA256_256AES</displayName><value>SHA256_256AES</value></allowableValues><allowableValues><displayName>SHA_40RC2</displayName><value>SHA_40RC2</value></allowableValues><allowableValues><displayName>SHA_256AES</displayName><value>SHA_256AES</value></allowableValues><allowableValues><displayName>SHA_3KEYTRIPLEDES</displayName><value>SHA_3KEYTRIPLEDES</value></allowableValues><allowableValues><displayName>SHA_TWOFISH</displayName><value>SHA_TWOFISH</value></allowableValues><allowableValues><displayName>SHA_128RC4</displayName><value>SHA_128RC4</value></allowableValues><allowableValues><displayName>PGP</displayName><value>PGP</value></allowableValues><allowableValues><displayName>PGP_ASCII_ARMOR</displayName><value>PGP_ASCII_ARMOR</value></allowableValues><defaultValue>MD5_256AES</defaultValue><description>The Encryption Algorithm to use</description><displayName>Encryption Algorithm</displayName><dynamic>false</dynamic><name>Encryption Algorithm</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Password</key><value><description>The Password to use for encrypting or decrypting the data</description><displayName>Password</displayName><dynamic>false</dynamic><name>Password</name><required>false</required><sensitive>true</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>public-keyring-file</key><value><description>In a PGP encrypt mode, this keyring contains the public key of the recipient</description><displayName>Public Keyring File</displayName><dynamic>false</dynamic><name>public-keyring-file</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>public-key-user-id</key><value><description>In a PGP encrypt mode, this user id of the recipient</description><displayName>Public Key User Id</displayName><dynamic>false</dynamic><name>public-key-user-id</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>private-keyring-file</key><value><description>In a PGP decrypt mode, this keyring contains the private key of the recipient</description><displayName>Private Keyring File</displayName><dynamic>false</dynamic><name>private-keyring-file</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>private-keyring-passphrase</key><value><description>In a PGP decrypt mode, this is the private keyring passphrase</description><displayName>Private Keyring Passphrase</displayName><dynamic>false</dynamic><name>private-keyring-passphrase</name><required>false</required><sensitive>true</sensitive><supportsEl>false</supportsEl></value></entry></descriptors><lossTolerant>false</lossTolerant><penaltyDuration>30 sec</penaltyDuration><properties><entry><key>Mode</key><value>Encrypt</value></entry><entry><key>Encryption Algorithm</key><value>PGP</value></entry><entry><key>Password</key></entry><entry><key>public-keyring-file</key><value>/Users/alanj/Documents/TestPublicKey.asc</value></entry><entry><key>public-key-user-id</key><value>al...@cloudera.com</value></entry><entry><key>private-keyring-file</key></entry><entry><key>private-keyring-passphrase</key></entry></properties><runDurationMillis>0</runDurationMillis><schedulingPeriod>0 sec</schedulingPeriod><schedulingStrategy>TIMER_DRIVEN</schedulingStrategy><yieldDuration>1 sec</yieldDuration></config><name>EncryptContent</name><relationships><autoTerminate>true</autoTerminate><description>Any FlowFile that cannot be encrypted or decrypted will be routed to failure</description><name>failure</name></relationships><relationships><autoTerminate>false</autoTerminate><description>Any FlowFile that is successfully encrypted or decrypted will be routed to success</description><name>success</name></relationships><state>RUNNING</state><style/><supportsEventDriven>true</supportsEventDriven><supportsParallelProcessing>true</supportsParallelProcessing><type>org.apache.nifi.processors.standard.EncryptContent</type></processors><processors><id>3f89b96c-62f5-42a7-a2c8-4b1566e5f7cf</id><parentGroupId>46e23601-c4b5-4328-8759-2b58ff4defbf</parentGroupId><position><x>2904.5785053345357</x><y>-301.96232959440795</y></position><config><bulletinLevel>WARN</bulletinLevel><comments></comments><concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount><defaultConcurrentTasks><entry><key>TIMER_DRIVEN</key><value>1</value></entry><entry><key>EVENT_DRIVEN</key><value>0</value></entry><entry><key>CRON_DRIVEN</key><value>1</value></entry></defaultConcurrentTasks><defaultSchedulingPeriod><entry><key>TIMER_DRIVEN</key><value>0 sec</value></entry><entry><key>CRON_DRIVEN</key><value>* * * * * ?</value></entry></defaultSchedulingPeriod><descriptors><entry><key>Input Directory</key><value><description>The input directory from which to pull files</description><displayName>Input Directory</displayName><dynamic>false</dynamic><name>Input Directory</name><required>true</required><sensitive>false</sensitive><supportsEl>true</supportsEl></value></entry><entry><key>File Filter</key><value><defaultValue>[^\.].*</defaultValue><description>Only files whose names match the given regular expression will be picked up</description><displayName>File Filter</displayName><dynamic>false</dynamic><name>File Filter</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Path Filter</key><value><description>When Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scanned</description><displayName>Path Filter</displayName><dynamic>false</dynamic><name>Path Filter</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Batch Size</key><value><defaultValue>10</defaultValue><description>The maximum number of files to pull in each iteration</description><displayName>Batch Size</displayName><dynamic>false</dynamic><name>Batch Size</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Keep Source File</key><value><allowableValues><displayName>true</displayName><value>true</value></allowableValues><allowableValues><displayName>false</displayName><value>false</value></allowableValues><defaultValue>false</defaultValue><description>If true, the file is not deleted after it has been copied to the Content Repository; this causes the file to be picked up continually and is useful for testing purposes. If not keeping original NiFi will need write permissions on the directory it is pulling from otherwise it will ignore the file.</description><displayName>Keep Source File</displayName><dynamic>false</dynamic><name>Keep Source File</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Recurse Subdirectories</key><value><allowableValues><displayName>true</displayName><value>true</value></allowableValues><allowableValues><displayName>false</displayName><value>false</value></allowableValues><defaultValue>true</defaultValue><description>Indicates whether or not to pull files from subdirectories</description><displayName>Recurse Subdirectories</displayName><dynamic>false</dynamic><name>Recurse Subdirectories</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Polling Interval</key><value><defaultValue>0 sec</defaultValue><description>Indicates how long to wait before performing a directory listing</description><displayName>Polling Interval</displayName><dynamic>false</dynamic><name>Polling Interval</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Ignore Hidden Files</key><value><allowableValues><displayName>true</displayName><value>true</value></allowableValues><allowableValues><displayName>false</displayName><value>false</value></allowableValues><defaultValue>true</defaultValue><description>Indicates whether or not hidden files should be ignored</description><displayName>Ignore Hidden Files</displayName><dynamic>false</dynamic><name>Ignore Hidden Files</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Minimum File Age</key><value><defaultValue>0 sec</defaultValue><description>The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored</description><displayName>Minimum File Age</displayName><dynamic>false</dynamic><name>Minimum File Age</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Maximum File Age</key><value><description>The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored</description><displayName>Maximum File Age</displayName><dynamic>false</dynamic><name>Maximum File Age</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Minimum File Size</key><value><defaultValue>0 B</defaultValue><description>The minimum size that a file must be in order to be pulled</description><displayName>Minimum File Size</displayName><dynamic>false</dynamic><name>Minimum File Size</name><required>true</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry><entry><key>Maximum File Size</key><value><description>The maximum size that a file can be in order to be pulled</description><displayName>Maximum File Size</displayName><dynamic>false</dynamic><name>Maximum File Size</name><required>false</required><sensitive>false</sensitive><supportsEl>false</supportsEl></value></entry></descriptors><lossTolerant>false</lossTolerant><penaltyDuration>30 sec</penaltyDuration><properties><entry><key>Input Directory</key><value>/tmp/encryptIn</value></entry><entry><key>File Filter</key></entry><entry><key>Path Filter</key></entry><entry><key>Batch Size</key></entry><entry><key>Keep Source File</key></entry><entry><key>Recurse Subdirectories</key></entry><entry><key>Polling Interval</key></entry><entry><key>Ignore Hidden Files</key></entry><entry><key>Minimum File Age</key></entry><entry><key>Maximum File Age</key></entry><entry><key>Minimum File Size</key></entry><entry><key>Maximum File Size</key></entry></properties><runDurationMillis>0</runDurationMillis><schedulingPeriod>0 sec</schedulingPeriod><schedulingStrategy>TIMER_DRIVEN</schedulingStrategy><yieldDuration>1 sec</yieldDuration></config><name>GetFile</name><relationships><autoTerminate>false</autoTerminate><description>All files are routed to success</description><name>success</name></relationships><state>RUNNING</state><style/><supportsEventDriven>false</supportsEventDriven><supportsParallelProcessing>true</supportsParallelProcessing><type>org.apache.nifi.processors.standard.GetFile</type></processors></snippet><timestamp>03/29/2016 10:41:18 EDT</timestamp></template>
