Re: JoltTransformJSON EL when using file

2023-08-29 Thread Matthew Hawkins
Hi Matt, Had this happen again. Spec: [{ "operation": "default", "spec": { "PartitionID": "${distribute.load.relationship}" }}] I'm abusing DistributeLoad in round robin mode to add a cycling id in a range to the content by sending all relationships to the next processor. Using

Re: JoltTransformJSON EL when using file

2023-08-23 Thread Matthew Hawkins
Hi Matt, Many of the deployments I get involved with are not in a Hadoop ecosystem, so building in resilience becomes part of the challenge, especially when data input is out of your control and often has bad data. In these situations, one thing I like doing is a test flow using DuplicateFlowFile

Re: JoltTransformJSON EL when using file

2023-08-17 Thread Matt Burgess
Matthew, What was your second case trying to use ${now():toNumber()} ? My unit test evaluated the expression into an integer and it worked (versus putting the expression in quotes which would make it a string). Regards, Matt On Tue, Aug 15, 2023 at 4:09 AM Matthew Hawkins wrote: > > Hi Matt, >

Re: JoltTransformJSON EL when using file

2023-08-15 Thread Matthew Hawkins
Hi Matt, The tag will be ${firstname}, the spec is correct. (well, I also get lastname, but it's unimportant) Weird thing is this failed similarly with a file based input on the Record based jolt processor as well on a completely different system. Both Linux (Ubuntu 22.04), both OpenJDK 17,

Re: JoltTransformJSON EL when using file

2023-08-12 Thread Matt Burgess
Just to follow up, I added a unit test to put EL in a JOLT spec and it worked. I noticed you referred to "attrname" in your post but your spec refers to "firstname", is that a typo? Regards, Matt On Thu, Aug 10, 2023 at 3:03 PM Matt Burgess wrote: > > I added file support to JoltTransformJSON

Re: JoltTransformJSON EL when using file

2023-08-10 Thread Matt Burgess
I added file support to JoltTransformJSON under NIFI-4957 [1], a first glance at the code seems like it should work, but I'll try to reproduce it and follow up, thanks for bringing this to our attention! Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-4957 On Thu, Aug 10, 2023 at

JoltTransformJSON EL when using file

2023-08-10 Thread Matthew Hawkins
Hi devs, Using 1.23.0 I have a simple transform adding some flowfile attributes into the content. When putting the spec directly to the processor it works fine. If I have the spec in an external file however it seems to put in the literal string ${attrname} - ie the EL didn't process. Have I