Re: NiFi ram usage

2017-08-30 Thread Adam Lamar
Jeff, This was a new installation so I actually hadn't set up any flows yet. NiFi wouldn't start immediately after installation (before I could configure any flows) because the system had too little ram. The 1.1GB figure is private (RSS) memory usage, which exceeded the 1GB instance limit (and the

Re: NiFi ram usage

2017-08-30 Thread Jeff
HI Adam, Can you provide some more detail about what your NiFi flow is like? Are you using custom processors? I regularly use NiFi with the default bootstrap settings without issue, but if you're bringing lots of data into memory, have lots of flowfiles being processed concurrently, etc, memory

Re: JSON array chunking

2017-08-30 Thread Neil Derraugh
I should have mentioned I tried starting with a JsonPathReader before the AvroReader. I had a property I was calling root with a value of $. I can post details about that too if it would be helpful. On Wed, Aug 30, 2017 at 8:08 PM, Neil Derraugh < neil.derra...@intellifylearning.com> wrote: > I

JSON array chunking

2017-08-30 Thread Neil Derraugh
I have arbitrary JSON arrays that I want to split into chunks. I've been (unsuccessfully) trying to figure this out with InferAvroSchema -> SplitJson(AvroReader, JsonRecordSetWriter). Here's an example payload: [{ "id": "56740f4b-48de-0502-afdc-59a463b3f6dc", "account_id": "b0dad7e2-7bb9-

Re: Adding a Receive Date Time stamp to MQTT message.

2017-08-30 Thread Bruce Lowther
Koji, thank you for your feedback. With a little more review and brain calories, I realized that I had incorrectly accepted default option for 'Skip header line'. Skip Header Line: false --translate--> Consider first line to be header. In my case this is not possible as my MQTT feed does not pr

NiFi ram usage

2017-08-30 Thread Adam Lamar
Hi everybody, I recently started up a new cloud Linux instance with 1GB of ram to do some quick tasks in NiFi. I noticed NiFi kept dying without much information in the logs - it just seemed to stop during startup. Eventually I realized the system was running out of memory and OOM killing the pro

DBCPConnectionPool SqlServer and Kerberos

2017-08-30 Thread Noe Detore
Hello Does anyone have experience or know if DBCPConnectionPool using SqlServer can be configured to authenticate with Kerberos? Thanks Noe

RE: Jolt Quesiton

2017-08-30 Thread Jones, Patrick L.
I’ve got another with Jolt. I have a json where sometimes I have "value": { "$numberLong": "-75928320" } I want to change it to “value” : "-75928320" I want to make this change where every the value. numberLong occurs. So if I have input like: {