Re: code page conversion

2020-01-15 Thread Mike Thomsen
I'd dump a few of those records to disk with PutFile or view them with the content viewer in the NiFi flow. If you see problems within NiFi, you can add a new parameter for this in conf/bootstrap.conf: -Dfile.encoding=UTF-8 If it looks good, you need to look at how Oracle is configured because it

RE: code page conversion

2020-01-15 Thread סמדג'ה גיא
Thanks for answering The data looks readable all the way, until it enters the oracle DB Other tools like IBM DataStage handles this kind of data correctly, so I guess the problem relays on the NIFI side As I wrote , oracle is configure with ISO8859-8 code page , so I need to do some kind of code p

RE: DistributedMapCacheService - automated creation

2020-01-15 Thread William Gosse
Well we will be deploy Nifi in a docker container and need to configure everything at the time of the container’s creation and startup. To be specific I’m using DistributedMapCacheService with a DetectDuplicate process to track whether a flowfile with a same attribute value had been already cre

Merge and transform JSON

2020-01-15 Thread Tomislav Novosel
Hi Nifi Community, I have situation where I need to merge JSON content from multiple flowfiles into one JSON content (single flowfile). JSON in every flowfile looks like this: { "analyses": "prep_array", "args": "prep_array", "scriptId": "142", "libIds": "141", "job_name": "my_demo_job

Re: DistributedMapCacheService - automated creation

2020-01-15 Thread Bryan Bende
Anything you can do from the UI goes through the REST API. You can open Chrome Dev Tools and see what calls are made to the server when you create the controller service from the UI. On Wed, Jan 15, 2020 at 9:25 AM William Gosse wrote: > > Well we will be deploy Nifi in a docker container and nee

Re: code page conversion

2020-01-15 Thread Mike Thomsen
Looking on Google, I found a number of people having problems with Java converting back and forth between UTF-8 and that Hebrew ISO code page. For example: https://bugs.openjdk.java.net/browse/JDK-8195686 Depending on what JRE you're using, you might need to look very carefully at how you're decl

RE: DistributedMapCacheService - automated creation

2020-01-15 Thread William Gosse
That was a very helpful suggestion. Here's what I came up with which seems to work: #!/bin/bash set -x NIFI_ENCOMPASS_URL=http://xxx.xxx.xx.xxx:16080/nifi-api ROOT_PROCESS=$(curl $NIFI_ENCOMPASS_URL/flow/process-groups/root) ROOT_ID=`echo $ROOT_PROCESS | jq --raw-output '.processGroupFlow.id'`

Holiday scheduling

2020-01-15 Thread Dave Andrews
Hello community, Has anyone come up with a clean internal to NiFi solution to changing workflow based on date/time? For example, a routeonAttribute that will route files differently for the upcoming weekend and Monday holiday? I have written different routing strategies like: ${filename:toUpper(

Re: Holiday scheduling

2020-01-15 Thread Andrew Grande
Maybe a good fit for a drools engine rule set? I remember there was a community processor. Andrew On Wed, Jan 15, 2020, 10:40 AM Dave Andrews wrote: > Hello community, > Has anyone come up with a clean internal to NiFi solution to changing > workflow based on date/time? For example, a routeonA

jolt array name to attribute?

2020-01-15 Thread l vic
i have input json such as the following example: { "Address": [ { "Street": "Washington St", "ZIP": "03420", "unit": "1" } ] } How can i use Jolt transform to add array name as attribute value to array members as following result: { *"type": "Address"*, "Street": "Washington St", "ZIP": "03420

Re: Holiday scheduling

2020-01-15 Thread Mark Payne
A drools engine may be a good solution. But if you wanted to lookup against a database or something like that, you could look at the LookupAttribute processor. It's configured with a LookupService, and there are several different services that you can use to lookup the value. Including SimpleDa

Re: jolt array name to attribute?

2020-01-15 Thread Matt Burgess
Try the following chain spec: [ { "operation": "shift", "spec": { "*": { "$": "type", "*": { "*": "&" } } } } ] This will "flatten" any key with an array value into a "type" field with the name of the array key, and all its entries as

ClassNotFound LogSupport using ExtractEmailHeaders in v1.10.0

2020-01-15 Thread Contacto Control Cobros
Hi all, I am using Apache Nifi 1.10, running with Java 11 and I had an error using the "ExtractEmailHeaders" processor: *WARN [Timer-Driven Process Thread-7] o.a.n.controller.tasks.ConnectableTask Administratively Yielding ExtractEmailHeaders[id=aabe0e09-016f-1000-02c0-2bb7e4f1c5b8] due to uncau