Storage buffer separation in a multi-tenant cluster.

2017-01-12 Thread Kristopher Kane
I work with a medium sized Storm cluster that is used by many tenants. As the admins of the Storm cluster we must be mindful of network and CPU IO and adjust, manually, based on usage. Many of these Storm uses would be a better fit with NiFi's inbuilt capabilities and ease of use whilst leaving t

Re: Json Validation for mandatory fields

2017-01-12 Thread Nick Carenza
I have been considering contributing a ValidateJsonSchema processor myself. According to json-schema.org, there are 3 validators in Java. Does anyone have experience with any of them and could support using one of another? http://json-schema.org/implementations On Thu, Jan 12, 2017 at 8:29 AM, Ma

Re: Json Validation for mandatory fields

2017-01-12 Thread Matt Burgess
Selvam, There was a ValidateJson processor submitted under NIFI-1893 [1], but it used a library that has been deemed Category X so it had to be removed. Hopefully it will be refactored and resubmitted. If you are not encumbered by the JSON.org license, you can build the ValidateJson processor you

RE: Having trouble connecting to SQL Server

2017-01-12 Thread Wayne Richard
Carlos, That was my problem. Thanks for the help! I have another error now but I will try to figure it out before I bug everyone. Thanks, Wayne Richard Business Intelligence Engineer II XPOLogistics 13777 Ballantyne Corporate Charlotte, NC 28277 US O: 980-224-5110 | M: 704-798-7941 From: Carlos

Re: Json Validation for mandatory fields

2017-01-12 Thread Selvam Raman
Thanks mark. I have already followed the same approach but we are having more than fifty fields, so thinking about is there any other way? Thanks, selvam R On Thu, Jan 12, 2017 at 3:03 PM, Mark Payne wrote: > Selvam, > > You can use EvaluateJsonPath to extract the particular JSON value into an

RE: Having trouble connecting to SQL Server

2017-01-12 Thread Carlos Manuel Fernandes (DSI)
Hello Wayne, Have you tried this connection on a Sql client tool , like Squirrel, dbvisualizer, etc? I have a connection to SqlServer but the Url connection is : jdbc:sqlserver://:1433;databaseName= using sqljdbc4.jar. Carlos From: Wayne Richard [mailto:wayne.rich...@xpo.com] Sent: quinta-feir

Having trouble connecting to SQL Server

2017-01-12 Thread Wayne Richard
Hello, I am a newbie with Nifi. I am working on a simple task to use as a proof of concept. However, I am not able to connect to SQL Server. It seems that I have not configured this correctly. Here are the DB and Driver settings that I have: Database Connection URL - jdbc:microsoft:sqlserver://

Re: nifi 1.1.1 processor details don't show from summary view

2017-01-12 Thread Matt Gilman
Naz, Thanks for feedback. This issue has been captured in the following JIRA [1]. I apologize for any inconvenience. Matt [1] https://issues.apache.org/jira/browse/NIFI-3306 On Thu, Jan 12, 2017 at 10:22 AM, Irizarry Jr., Nazario wrote: > In NiFi 1.1.1 (and 1.1.0) if you pull up the summary v

nifi 1.1.1 processor details don't show from summary view

2017-01-12 Thread Irizarry Jr., Nazario
In NiFi 1.1.1 (and 1.1.0) if you pull up the summary view then click on the “i” to the left of a processor nothing shows. At the bottom of the page there is a spinning icon that never stops spinning. However, the processor details do work from the Data Provenance page. This works as expected

Re: Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Alessio Palma
Why not. From: Mark Payne Sent: Thursday, January 12, 2017 3:59:03 PM To: users@nifi.apache.org Subject: Re: Cluster is still voting on which Flow is the correct flow for the cluster Alessio, Certainly, we could clarify the log message. Do you mind filing a JIR

Re: Json Validation for mandatory fields

2017-01-12 Thread Mark Payne
Selvam, You can use EvaluateJsonPath to extract the particular JSON value into an attribute and then use RouteOnAttribute to make sure that the field exists (and is valid if necessary). For example, for EvalualteJsonPath you might use something like: name: $.person.name And then in RouteOnAtt

Re: Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Mark Payne
Alessio, Certainly, we could clarify the log message. Do you mind filing a JIRA for that? Thanks -Mark On Jan 12, 2017, at 9:34 AM, Alessio Palma mailto:alessio.pa...@docomodigital.com>> wrote: Thanks Mark, U right, authorizations.xml was out of sync. Now the trick is clear, and when you kno

Re: Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Alessio Palma
Thanks Mark, U right, authorizations.xml was out of sync. Now the trick is clear, and when you know the solution everything looks clear, but is there a way to have a clearer error message into the logs ? From: Mark Payne Sent: Thursday, January 12, 2017 3:24:22

Re: Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Mark Payne
Alessio, It looks like the flow is the not the same on both nodes. Since you only have two nodes, NiFi is not able to come up with a majority of votes. Specifically, it looks like the conf/authorizations.xml or conf/authorizers.xml or conf/users.xml file is out-of-sync. You can try copying these

Json Validation for mandatory fields

2017-01-12 Thread Selvam Raman
Hi, I will get array of json files. There are some fields which are mandatory in json file, if not present i should not process file. How can i achieve it in nifi. -- Selvam Raman "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Alessio Palma
2 host secured custer. 2 zookeeper. From: Andre Sent: Thursday, January 12, 2017 2:39:23 PM To: users@nifi.apache.org Subject: Re: Cluster is still voting on which Flow is the correct flow for the cluster Alessio, May I ask you how many zookeeper form your ens

Re: Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Andre
Alessio, May I ask you how many zookeeper form your ensemble? Kind regards On Fri, Jan 13, 2017 at 12:28 AM, Alessio Palma < alessio.pa...@docomodigital.com> wrote: > Hello all, > > there are 5 minutes of negotiation on the cluster which ends in nothing. > > > These are the logs after the hands

Cluster is still voting on which Flow is the correct flow for the cluster

2017-01-12 Thread Alessio Palma
Hello all, there are 5 minutes of negotiation on the cluster which ends in nothing. These are the logs after the handshake fails: host 1 2017-01-12 13:20:36,448 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from {} to {mid1-t029nifi-2.buongiorno.com:8080