Re: ConsumeKafka to PublishKafka doesn't keep the order of the messages in the destination topic

[edi mari]

Hi Mark,
I tried the combination of FIFO and setting the back pressure to 10k, but
it didn't preserve the order.

Thanks
Edi

On Wed, Dec 13, 2023 at 3:47 PM Mark Payne  wrote:

> Hey Edi,
>
> By default, nifi doesn’t preserve ordering but you can have it do so by
> updating the connection’s configuration and adding the First In First Out
> Prioritizer.
>
> Also of note you will want to keep the backpressure threshold set to
> 10,000 objects rather than increasing it as shown in the image.
>
> Thanks
> Mark
>
>
> Sent from my iPhone
>
> On Dec 13, 2023, at 8:19 AM, edi mari  wrote:
>
> 
>
> Hello ,
> I'm using NIFI v1.20.0 to replicate 250 million messages between Kafka
> topics.
> The problem is that NIFI replicates messages in a non-sequential order,
> resulting in the destination topic storing messages differently than the
> source topic.
>
> for example
> *source topic - partition 0*
> offset:5 key:a value:v1
> offset:6 key:a value:v2
> offset:7 key:a value:v3
>
> *destination topic - partition 0*
> offset:5 key:a value:v2
> offset:6 key:a value:v1
> offset:7 key:a value:v3
>
> The topics are configured with a cleanup policy: compact.
>
> I'm using ConsumeKafka and PublishKafka processors to replicate topics.
>
> 
>
> 
>
> 
>
> 
>
> 
>
> Thanks
> Edi
>
>

RE: [NIFI 1.23.2] Insecure Cipher Provider Algorithm

[Isha Lamboo]

Hi Quentin,

I've encountered similar errors in the past when trying to change the 
encryption algorithm.

Here are two things that may help:


  1.  The password/key needs to be at least 12 characters long before you 
migrate to NIFI_PBKDF2_AES_GCM_256. If it is not, you have to first change the 
password to something long enough with the old algorithm in place. If your key 
is blank, you may have to enter the old default value first: nififtw!
  2.  The command to migrate key algorithm does not support an encrypted 
configuration file. If you have the key encrypted you should replace it with 
the unencrypted version, clear the property  ...sensitivekey.protected=... and 
then migrate. After that you can re-encrypt the configuration using the nifi 
toolkit again.

Regards,

Isha

Van: Quentin HORNEMAN GUTTON 
Verzonden: woensdag 13 december 2023 14:59
Aan: users@nifi.apache.org
Onderwerp: [NIFI 1.23.2] Insecure Cipher Provider Algorithm

You don't often get email from 
qhornemangut...@gmail.com. Learn why this is 
important
Hello,

I'm facing an issue after upgrading NiFi 1.13.2 to 1.23.2.

I have a warn log with Insecure Cipher Provider Algorithm 
[PBEWITHMD5AND256BITAES-CBC-OPENSSL]. I tried to update algorithm with the 
set-sensitive-properties-algorithm command to NIFI_PBKDF2_AES_GCM_256 but I 
have an error message with < Descryption failed with algorithm > caused by < 
pad block corrupted >.

Do you have any informations that could help me ?

Best regards,

Quentin HORNEMAN GUTTON