Re: Handling incoming file names that contain embedded spaces

Yes indeed Joe, it appears from the logs that there are non-ASCII unicode characters preceding and at end of the file name. The log shows them as odd representations of "unprintables" - for example, small inverted question marks in diamonds, etc etc. They are embedded in the file names by the

Re: Handling incoming file names that contain embedded spaces

James, I suspect there is more to the issue than the spaces. GetFile itself should be fine there. Can you share logs showing what is happening with these files? Can you share some sample filenames that it is struggling with? You can also enable debug logging for that processor which could

Handling incoming file names that contain embedded spaces

I am using NiFi 0.6.1. I am trying to use GetFile to read in a large series of files I have preprocessed outside of NiFi from zip files using bash shell commands. GetFile is throwing errors on many of these files because the files contain embedded spaces. Is there a way to tell NiFi to handle each

Re: File causes unexpected stoppage in play

James This article should help you get started https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions especially since you only have to go through the first part only “Maven Processor Archetype” (I am assuming you are familiar with maven). Also, make sure that you pick

Re: File causes unexpected stoppage in play

I've never tried to create and deploy my own custom bundle but would like to try #3 as you recommend Oleg. If you can offer guidance to that process let me try that. Thank you. -Jim On Wed, Dec 14, 2016 at 2:57 PM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > James > > Not sure what

Re: File causes unexpected stoppage in play

James Not sure what kind of liberties you have with your environment, but here is what you can do outside of full upgrade. 1. You can download and drop the bundle (NAR) from NiFi 0.7 release into 0.6 release. That should work (just replace it in the ‘lib’ directory of NiFi home). However,

Re: File causes unexpected stoppage in play

Oleg, I am bound to NiFi 0.6.1 by enterprise application dependencies. So this fix will not be in my baseline if I understand you correctly. Let me ask you this: is there any way I can build this into my code baseline - either through a code mod and rebuild or as a custom plugin feature specific

Re: File causes unexpected stoppage in play

Thank you Joe. While I'm unable to export and share the nifi-app.log file, I can tell you that the processor is throwing the error "...failed to process due to java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /dir/path//_?.jpg " On Wed, Dec

Apache Minifi questions

Hello all, Will latest MINIFI work with Apache NIFI 0.7? In the sense that I build my dataflow and then deploy the dataflow to Apache MINIFI edge node to run? How does the provenance of Apache MINIFI work how can I view it, we will not have Apache NIFI 1 at the "lake" Thank you

Re: File causes unexpected stoppage in play

James Could you also let us know what version of NiFi you are using? The issue with properly handling InvalidPathException was fixed in NiFi 0.7.0 as part of https://issues.apache.org/jira/browse/NIFI-920 It essentially has this catch block: } catch (final ProcessException |

Re: File causes unexpected stoppage in play

James, Can you please share the full log entry for that failure. It is possible the processor is not catching the exception properly and routing the data to failure. It might simply be letting the exception loose thus the framework detects the issue and rollsback the session and yields the

Re: Failing Tests during Build

Hello Those tests rely on DNS which appears to be in a funky state. However, to get the build working otherwise you can run mvn -T C2 clean install -DskipTests Thanks joe On Wed, Dec 14, 2016 at 10:07 AM, Pushkar wrote: > Hi, > > I tried to build NiFi on my linux

Failing Tests during Build

Hi, I tried to build NiFi on my linux machine using 'mvn -T 2.0C clean install'. However there seems to be a few tests failing which I don't seem to be able to fix. I have pasted the stacktrace below. Please let me know how I can fix this and get the build running. Please note, I haven't

Re: Nifi "Hive" with CDH

Unfortunately I don't believe enabling the profile and setting the version will work in this case, because there are API differences between Hive 1.0.0 (the one CDH 5.9 is based on) and Hive 1.2.1 (the one NiFi is based on). These require code changes to get working. Regards, Matt On Wed, Dec

Re: Data Provenance - Disk Full

Hi, I've deleted the files as suggested by James, but I would also like to fine tune so this doesn't happen again. However, I'm not sure what exactly happens with parameter: nifi.provenance.repository.max.storage.size I haven't changed the default (1 GB), so what would happen when the limit is

Re: Nifi "Hive" with CDH

Nicolas, Have you tried building NiFi with the CDH libraries? The functionality is available out of the box and just requires you to enable to correct profile. Cheers On Wed, Dec 14, 2016 at 9:13 PM, Provenzano Nicolas < nicolas.provenz...@gfi.fr> wrote: > Hi all, > > > > We are currently

Nifi "Hive" with CDH

Hi all, We are currently working on NiFi Hive processors (selectHiveQL, putHiveStreaming...). We are testing them with a CDH platform (and with a "Apache" platform). Hive 1.0.0 is deployed with the last releases of CDH (5.9) while Nifi is built with Hive 1.2.1 libraries. We then have serious

Re: Processors on the fly for many sensor devices

Hi Andrew, I noticed you've created a pull request to get this in the Nifi codebase and that there were some review discussions going on. I was wondering what the status is on the GetTCP processor. I've also logged some issues in your Github repo and can create some PRs if you like. --