Cannot dig into it at the moment but it seems that we're expecting integers:
https://github.com/apache/nifi/blob/support/nifi-1.x/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublishKafkaRecord_2_6.java#L608 and https://github.com/apache/nifi/blob/support/nifi-1.x/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-6-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublishKafkaRecord_2_6.java#L624 it should be a fairly easy fix to make sure the returned value is always positive. You may want to file a JIRA for it. Le jeu. 11 juil. 2024 à 18:50, Richter, Robert <[email protected]> a écrit : > Hi, > > > > thank you for this hint. I’ve done some checks and indeed we’re using > schema inference in the sql processor which loads the data into NiFi. > > But the schema inference derives the correct data type. It uses long and > the same pipeline processes values that are larger that the upper value for > integer. > > > > This is the schema that is inferred for the failing record. I tried it > with plain json and avro connected to the schema registry. The schema > registry also provides long as data type. Unfortunately it fails again. > > > > > avro.schemaæ{"type":"record","name":"nifiRecord","namespace":"org.apache.nifi","fields":[{"name":"_hdl_loadstamp","type":["string","null"]},{"name":"Id","type":["string","null"]},{"name":"NumericId","type":"long"},…. > > > > Did I miss something else? > > > > *From: *Pierre Villard <[email protected]> > *Date: *Thursday, 11. July 2024 at 12:09 > *To: *[email protected] <[email protected]> > *Subject: *Re: PublishKafkaRecord_2_6 produces negative partition number > > *Achtung! Die Absender-Adresse ist möglicherweise gefälscht oder wurde > nachträglich manipuliert. Bitte überprüfen Sie die Plausibilität der Email > und lassen bei enthaltenen Anhängen und Links besondere Vorsicht walten.* > Wenden Sie sich im Zweifelsfall an das CIT unter [email protected] oder 06122 > 536 350. > (Stichwort: DKIM Test Fehlgeschlagen) > > * > ----------------------------------------------------------------------------------------------------------------* > > Hi, > > > > Upper value for int type is 2147483647 and your field value is 2147483648 > and I believe that's the reason why you get this error. You may want to > check the schema you're using for your data in the configuration of the > record reader. If you're using schema inference, that may be the reason. > > > > HTH, > > Pierre > > > > Le jeu. 11 juil. 2024 à 16:37, Richter, Robert <[email protected]> a > écrit : > > Hello NiFi Community, > > > > we’ve encountered an error in one of our pipelines. Exactly one record > cannot be processed by the PublishKafkaRecord_2_6 processor. The Processor > throws the following exception. > > > > java.lang.IllegalArgumentException: Invalid partition: -8. Partition > number should always be non-negative or null. > > > > The processor is configured as follows and we’re on NiFi version 1.23.2. > The topic we’re publishing on has 20 partitions: > > > > - partitioner class: record path partitioner > - partition: /NumericId > - Message Key Field: NumericId > > > > The processor fails only for records with value 2147483648 in the message > key field. All other records can be processed without failure. (millions > per day) > > > > We tried different reader and writer. Avro with and without schema > registry and plain json. The result is the same. > > > > Here’s an example record. > > > > [ { > > "_loadstamp" : "20240711075814350", > > "Id" : "04948E4E-94B1-84HD-94CA-AB84020F729", > > "NumericId" : 2147483648, > > "TimeStamp" : "2024-06-12 15:30:58.341 +02:00", > > "SequenceNumber" : 123456789, > > "IsTrendUploaded" : false, > > "CompoundValueId" : null, > > "Value" : 1.0, > > "Score" : 65535, > > "ScoreColor" : 0, > > "CompositeId" : null > > } ] > > > > > > Kind regard > > Robert > > > Das SVA Mail-System ist mit einem Mailverschluesselungs-Gateway > ausgestattet. Wenn Sie moechten, dass an Sie gerichtete E-Mails > verschluesselt werden, senden Sie einfach eine S/MIME-signierte E-Mail oder > Ihren PGP Public Key an [email protected]. > > The SVA mail system is equipped with an email encryption gateway. If you > want email sent to you to be encrypted please send a S/MIME signed email or > your PGP public key to [email protected]. > >
