Re: [EXT] Re: Help with frontend-maven-plugin?

Ok, I see the problem now. Thank you for the javascript errors. I opened this ticket to solve the issue: https://issues.apache.org/jira/browse/NIFI-4998 Will you please cherry-pick from this PR: https://github.com/apache/nifi/pull/2571 and confirm that it fixes the issue you are seeing? -Scott

RE: [EXT] Re: Help with frontend-maven-plugin?

Scott, thanks for the response. As you can see in the log/error information I included, it is detecting my proxy settings; and the proxy information it shows (I removed parts of the hostname in my message) are correct. > [INFO] --- frontend-maven-plugin:1.1:npm (npm install) @ nifi-web-ui

Re: FlattenJson

We could start with routeOnJsonPath and do the record path as the need arises? On March 20, 2018 at 16:06:34, Matt Burgess (mattyb...@apache.org) wrote: Rather than restricting it to JSONPath, perhaps we should have a RouteOnRecordPath or RouteRecord using the RecordPath API? Even better would

Re: FlattenJson

The only issue is that typically with record processors you will have many records per flow file, so what do you do in that case? If RouteOnRecordPath is going to send out individual records, then I think you could already achieve this with PartitionRecord + SplitRecord. PartitionRecord would

Re: FlattenJson

Rather than restricting it to JSONPath, perhaps we should have a RouteOnRecordPath or RouteRecord using the RecordPath API? Even better would be the ability to use RecordPath functions in QueryRecord, but that involves digging into Calcite as well. I realize JSONPath might have more capabilities

Re: onPropertyModified doesn't seem to work

Thank you, Mark. That helped. On Mon, Mar 19, 2018 at 10:44 PM, Mark Payne wrote: > Sivaprasanna, > > What package is your processor class in? By default, the logging for the > org.apache.nifi.processors package is set to WARN. I am wondering if you > are > simply not

Re: FlattenJson

Like the idea that Otto suggested. RoutOnJSONPath makes more sense since making the flattened JSON write to attributes is restricted to that processor alone. On Tue, Mar 20, 2018 at 8:37 PM, Otto Fowler wrote: > Why not create a new processor that does routeOnJSONPath

Re: ExecuteHQL failed to reading table from Hive

Hi Matt, Issue got resolved after casting smallint to int, as avro not supported smallint datatype. Thanks. -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Not able to add "Max Rows Per Flow File" property in SelectHiveQL

If you download NiFi 1.5.0, you can copy 'nifi-hive-nar-1.5.0.nar' and 'nifi-hive-services-api-nar-1.5.0.nar' from the lib directory and paste it in your NiFi 1.3.0 lib. No need to remove the older version i.e. 1.3.0 hive nars. NiFi supports multiple versions of bundles to exist. Anyone please

Re: Help with frontend-maven-plugin?

Hey there Peter, I am not sure that the frontend-maven-plugin is out of date or old (the last commit was November 2017). NiFi is running version 1.1 of this plugin and NiFi Registry is running version 1.5. The frontend-maven-plugin downloads versions of Node and npm from https://nodejs.org/dist,

Re: FlattenJson

Why not create a new processor that does routeOnJSONPath and works on the flow file? On March 20, 2018 at 10:39:37, Jorge Machado (jom...@me.com) wrote: So that is what we actually are doing EvaluateJsonPath the problem with that is, that is hard to build something generic if we need to specify

Re: FlattenJson

So that is what we actually are doing EvaluateJsonPath the problem with that is, that is hard to build something generic if we need to specify each property by his name, that’s why this idea. Should I make a PR for this or is this to business specific ? Jorge Machado > On 20 Mar 2018, at

Re: FlattenJson

Ok so I guess it depends whether you end up needing all 30 fields as attributes to achieve the logic in your flow, or if you only need a couple. If you only need a couple you could probably use EvaluateJsonPath after FlattenJson to extract just the couple of fields you need into attributes. If

Re: FlattenJson

From there on we use a lot of routeOnAttritutes and use that values on sql queries to other tables like select * from someTable where id=${myExtractedAttribute} To be honest I tryed JoltTransformJSON but I could not get it working :) Jorge Machado > On 20 Mar 2018, at 15:12, Matt Burgess

Re: FlattenJson

I think Bryan is asking about what happens AFTER this part of the flow. For example, if you are doing routing you can use QueryRecord (and you won't need the SplitJson), if you are doing transformations you can use JoltTransformJSON (often without SplitJson as well), etc. Regards, Matt On Tue,

Re: FlattenJson

Hi Bryan, thanks for the help. Our Flow: ExecuteSql -> convertToJSON -> SplitJson -> ExecuteScript with attachedcode 1. We are now writting a custom processor that does this which is a copy of FlattenJson but instead of putting the result into a flowfile we put it into the attributes.

Re: FlattenJson

Ok it is still not clear what the reason for needing it in attributes is though... Is there another processor you are using after this that only works off attributes? Just trying to understand if there is another way to accomplish what you want to do. On Tue, Mar 20, 2018 at 9:50 AM, Jorge

Re: FlattenJson

We are using nifi for Workflow and we get from a database like job_status and job_name and some nested json columns. (30 columns) We need to put it as attributes from the Flow file and not the content. For the first part (columns without a json is done by groovy script) but then would be nice

Re: FlattenJson

What would be the main use case for wanting all the flattened values in attributes? If the reason was to keep the original content, we could probably just added an original relationship. Also, I think FlattenJson supports flattening a flow file where the root is an array of JSON documents

Re: "Method not supported" error facing GenerateTableFetch and QueryDatabaseTable processor

Rohit, Hive is not supported by the database fetch processors yet, there is a Jira [1] and some PRs [2],[3] under review to add this capability. Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-3093 [2] https://github.com/apache/nifi/pull/1281 [3]

Re: [discuss] nifi 1.6.0 release

Alrighty team. Looks like we're at a good point to cut a 1.6.0 release. I'm starting the RC process now. The only thing 'new' in terms of process is I think we're dropping one of the message digest types. Will look into that as I get to that point. On Tue, Mar 13, 2018 at 2:09 PM, Michael

"Method not supported" error facing GenerateTableFetch and QueryDatabaseTable processor

Hi, I trying to read hive table using QueryDatabaseTable processor and when i executed my flow, i am getting below error. ""' 2018-03-20 00:42:20,717 ERROR [Timer-Driven Process Thread-5] o.a.n.p.standard.GenerateTableFetch GenerateTableFetch[id=3ddee0d9-0162-1000-7696-273b8816bcff] Unable to

Re: Not able to add "Max Rows Per Flow File" property in SelectHiveQL

Thanks.. Is it possible to change only source code in 1.3.0 and add 1.5.0 code. -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Not able to add "Max Rows Per Flow File" property in SelectHiveQL

Thanks for the reply, but i don't have any knowledge to complile NAR object. Is it possible to extract them of 1.5.0 and push it into NiFi 1.3.0 directly? -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: FlattenJson

No I do see how this could be convenient in some cases. My comment was more: you can certainly submit a PR for that feature, but it'll need to be clearly documented using the appropriate annotations, documentation, and property descriptions. 2018-03-20 10:20 GMT+01:00 Jorge Machado

Re: FlattenJson

Hi Pierre, I’m aware of that. So This means the change would not be accepted correct ? Regards Jorge Machado > On 20 Mar 2018, at 09:54, Pierre Villard wrote: > > Hi Jorge, > > I think this should be carefully documented to remind users that the >

Help with frontend-maven-plugin?

The version of "frontend-maven-plugin" used in NiFi is out of date, and the versions of NPM and Node referenced by it are also a bit old. I wouldn't bring this up, except there is a bug in this version of NPM that can cause issues when trying to download NPM packages via proxy server. This is

Re: FlattenJson

Hi Jorge, I think this should be carefully documented to remind users that the attributes are in memory. Doing what you propose would mean having in memory the full content of the flow file as long as the flow file is processed in the workflow (unless you remove attributes using

FlattenJson

Hey guys, I would like to change the FlattenJson Procerssor to be possible to Flatten to the attributes instead of Only to content. Is this a good Idea ? would the PR be accepted ? Cheers Jorge Machado