Toward tighter security...

Is there a way to use an existing reporting task mechanism, to write a custom reporting task or by other means to catch a UI user looking at a flowfile (via right-clicking on the queue, then choosing List queue -> View Details) including which flowfile was looked at (by /filename/ or /uuid/ or

Usage and Configure property defaults do not match (1.1.2)

*Ni**Fi 1.1.2 only* I've never noticed this problem before. I have a custom processor for which I maintain 10 properties. 3 of the 10 are set to "enabled" the rest to "disabled". First up, in an unconfigured instance of my processor, the 3 are "enabled" and the 7 "disabled". This is how

Re: TestRunner question

rc/test/groovy/org/apache/nifi/cdc/mysql/processors/CaptureChangeMySQLTest.groovy On Tue, May 23, 2017 at 5:20 PM, Russell Bateman <r...@windofkeltia.com> wrote: I wondered if it were possible to split the firing of MyProcessor.init()out of TestRunner.run()so that I have finer granularity i

TestRunner question

I wondered if it were possible to split the firing of MyProcessor.init()out of TestRunner.run()so that I have finer granularity in testing? Maybe I'm abusing init()by what I'm doing in it, but I'd like to see that run and give control back to my test case for some assertions and other things,

Re: Service controller list NiFi 0.x to 1.x

beginning in NiFi 1.x. We're producing documentation internally for that. Thanks. On 05/16/2017 03:30 PM, Russell Bateman wrote: We're still testing our migration from 0.7.2 to 1.1.2 which is imminent. We still don't know how much of this is going to have to be reconfigured. Our down-streamers

Re: Service controller list NiFi 0.x to 1.x

il.com <mailto:alopresto.apa...@gmail.com>/ PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On May 16, 2017, at 2:30 PM, Russell Bateman <r...@windofkeltia.com <mailto:r...@windofkeltia.com>> wrote: We're still testing our migration from 0.7.2 to 1.1.2 which is

Re: Service controller list NiFi 0.x to 1.x

nce/display/NIFI/Migration+Guidance On Tue, May 16, 2017 at 3:53 PM, Russell Bateman <r...@windofkeltia.com> wrote: Thanks, Andrew! (From [1]:) "This means that the service will be available to all Processors and Controller Services defined in that Process Group and below."

Service controller list NiFi 0.x to 1.x

It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I look at controller services via the General menu -> Controller Services, what I see is totally different from what I see when I configure controller services for a processor. If I use the General menu to set up my

Re: NiFi JUnit test framework: How to pass through or cause to ignore validation of dynamic property

/org/apache/nifi/components/Validator.java#L39 On Thu, May 11, 2017 at 2:11 PM, Russell Bateman <r...@windofkeltia.com> wrote: In what I'm doing, I don't necessarily have to have a value, only the property's existence (it's going to be an attribute name and I just have to do som

Re: NiFi JUnit test framework: How to pass through or cause to ignore validation of dynamic property

a dynamic property then it must have a value. -Bryan On Thu, May 11, 2017 at 1:51 PM, Russell Bateman <r...@windofkeltia.com> wrote: I'm having trouble feeding a dynamic property a test running a processor I'm writing that expects dynamic properties. I looked at other processor tests I've w

NiFi JUnit test framework: How to pass through or cause to ignore validation of dynamic property

I'm having trouble feeding a dynamic property a test running a processor I'm writing that expects dynamic properties. I looked at other processor tests I've written and I see I've not done a lot of testing involving dynamic properties. Here's some code, followed by output as I run. I Googled,

Re: Pulling out hair over slf4j...

Bryan, Thank you very much for responding. Here's how my project is set up with Maven packaging. This is how we've been building our NAR for a year now. First, the module list. Except for /nifi-shared/, each module contains NiFi custom processors. nifi-pipeline (root, pom packaging)

Pulling out hair over slf4j...

My nightmare today is a whole test suite of JUnit cases that error out with this and it only happens from JUnit tests, never from production code: I have put this in my top-level /pom.xml/ [1.7.25] but that doesn't force-feed the version to NiFi and other dependencies that have

Re: Is it possible to reference python requests module in ExecuteScript?

y well and be fully featured. On Fri, May 5, 2017 at 6:38 PM, Russell Bateman <r...@windofkeltia.com> wrote: We really want to use ExecuteScript because our end users are Pythonistas. They tend to punctuate their flows with the equivalent of PutFile and GetFile with Python scripts doing stuff

Re: Is it possible to reference python requests module in ExecuteScript?

We really want to use /ExecuteScript/ because our end users are Pythonistas. They tend to punctuate their flows with the equivalent of /PutFile/ and /GetFile/ with Python scripts doing stuff on flowfiles that pass out of NiFi before returning into NiFi. However, we find it nearly impossible

Re: Quickly find queues with data...

It shows you how to get the summary view. Click on connections. Sort on the column of interest. Thanks On Fri, May 5, 2017 at 5:35 PM, Russell Bateman <r...@windofkeltia.com> wrote: Is there some NiFi power-user trick to quick-finding queues that have flowfiles sitting in them? I have a

Re: Best practices for backing up an instance of NiFi?

Mark, This echoes if not answers a concern I've had with the whole concept of backing up the repositories. Second, our users have just about completely shut provenance down as not useful enough to justify the disk space and crunch (in a desperate, vague expression of "throw everything

Best practices for backing up an instance of NiFi?

Been Googling hard on this (NiFi docs, Horton docs, these forums, stackoverflow) and I'm not seeing anything about specific wisdom surrounding backing up /content_repository/ and /flowfile_repository/, why I would, why I wouldn't, turning NiFi off when I do, etc. I completely get the rest of

Prepopulate dynamic properties in custom processor?

Is there a way to prepopulate dynamic properties in a custom processor? I'm writing a new processor and my users could benefit from some suggested, in most cases it would be spot-on, configuration. Being able to prepopulate dynamic properties that could be amended or deleted seems the best

Re: ezmlm warning messages threatening removal from list

Annoyingly, I get them from time to time, but just ignore them. I so far haven't been removed from the list. On 04/24/2017 04:44 AM, Andre wrote: Joe, I get them as well. I strongly suspect is caused by your server (GMAIL) bouncing a ezlm message due to its original sender or perhaps message

Re: NiFi property values and line separators

with properties that both support EL and not. Can you verify the loss of line returns by viewing your flow.xml.gz? Matt On Tue, Apr 18, 2017 at 6:24 PM, Russell Bateman <r...@windofkeltia.com> wrote: I'm writing a custom processor. It needs to support dynamic properties the values of which must

Re: How to configure logging via SLF4J ?

Take a look at ${NIFI_ROOT}//conf/logback.xml/. If you've developed a processor and are asking about it, likely this question would fit better in d...@nifi.apache.org, no? Hope this helps. On 04/11/2017 08:24 AM, Vince Cole wrote: Hi I have developed a processor which uses

Re: Ensuring org.slf4j.helpers.MessageFormatter.arrayFormat() in

) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) On 04/06/2017 12:28 PM, Russell Bateman wrote: I'm struggling to figure out how

Re: org.apache.nifi.processor.util.FlowFileSessionWrapper deprecated

In answer to my own question, ... we discovered that the Binclass has been refactored to obviate the FlowFileSessionWrapper. Some light refactoring of our consuming code was required, that's all. Thanks. On 04/03/2017 02:22 PM, Russell Bateman wrote: I've tried building using NiFi JARs from

Ensuring org.slf4j.helpers.MessageFormatter.arrayFormat() in

I'm struggling to figure out how to respond to this problem of ensuring the presence of MessageFormatter(), which I think is in /slf4j-simple-x.y.z.jar/, and/or ensuring the log4j system is properly initialized. The need for slf4j is clearly coming from NiFi; I only need it because it's a

Re: org.apache.nifi.processor.util.FlowFileSessionWrapper deprecated

/SearchServerBatch.java:[34,38] cannot find symbol [ERROR] symbol: class FlowFileSessionWrapper [ERROR] location: package org.apache.nifi.processor.util Thanks. On 04/03/2017 02:05 PM, Russell Bateman wrote: We're moving our code base ahead from NiFi 1.0.0 to 1.1.2 and have some classes

org.apache.nifi.processor.util.FlowFileSessionWrapper deprecated

We're moving our code base ahead from NiFi 1.0.0 to 1.1.2 and have some classes that are using FlowFileSessionWrapper. Appears to wrap up a flowfile and its corresponding session; our use was in some special batching code. I could roll my own, I guess, but there may be a very good reason not

Re: RouteOnContent--how to specify the promised dynamic relationship?

is covered for RouteOnAttribute in the User Guide [1]. [1] http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#properties-tab On Thu, Mar 30, 2017 at 1:32 PM, Russell Bateman <r...@windofkeltia.com <mailto:r...@windofkeltia.com>> wrote: I'm working in NiFi 1.1.1. In thi

RouteOnContent--how to specify the promised dynamic relationship?

I'm working in NiFi 1.1.1. In this processor's usage, I see: Dynamic Relationships: A Dynamic Relationship may be created based on how the user configures the Processor. NameDescription Name from Dynamic Property FlowFiles that match the Dynamic Property's Regular Expression

Re: Apache's Maven repository missing commons-io, most versions

ke Apache /commons-csv/, I'm used to looking at.) Many thanks to sebb for his patience. On 03/30/2017 08:35 AM, Dave Newton wrote: On Thu, Mar 30, 2017 at 10:27 AM, sebb <seb...@gmail.com> wrote: On 30 March 2017 at 15:09, Russell Bateman <r...@windofkeltia.com> wrote: comm

Re: Apache's Maven repository missing commons-io, most versions

ersion was accidentally released with the wrong gid. On 30 March 2017 at 00:31, Russell Bateman <r...@windofkeltia.com> wrote: As I attempt to build, I find https://repo.maven.apache.org/maven2/org/apache/commons/commons-io/2.5/commons-io-2.5.jar to yield a 404. When I go look there, only

Re: Processor missing when attempting Add Processor

(I'm making a list of things to take the magnifying glass to.) On 03/28/2017 07:06 PM, Matt Burgess wrote: Wow, I've been watching this thread all day and I didn't see it either. Bende rocks! On Tue, Mar 28, 2017 at 9:01 PM, Russell Bateman <r...@windofkeltia.com> wrote: Two of us here

Re: Processor missing when attempting Add Processor

, Bryan Bende wrote: It looks like the processors JAR has the service loader file in META_INF instead of META-INF... inflated: META_INF/services/org.apache.nifi.processor.Processor On Tue, Mar 28, 2017 at 8:44 PM, Russell Bateman <r...@windofkeltia.com> wrote: No problem. I'll upload a t

Re: Processor missing when attempting Add Processor

No problem. I'll upload a tarball as soon as I can get it rolled up. On 03/28/2017 06:43 PM, Joseph Niemiec wrote: ugghh.. Any objection to sharing your src project ziped/tared up? On Tue, Mar 28, 2017 at 8:42 PM, Russell Bateman <r...@windofkeltia.com> wrote: With these changes, I'm

Re: Processor missing when attempting Add Processor

With these changes, I'm still not seeing /TikaProcessor/ in the list. On 03/28/2017 06:38 PM, Russell Bateman wrote: (Sheesh. I apologize for trying to do something else concurrently. I meant no disrespect. I'm on board now and the changes are up there again. I'm awaiting NiFi to recycle now

Re: Processor missing when attempting Add Processor

the nars packaged maven depends. Looking at the new parent pom can you change the last zero to a 1 ? 1.0.1 to 1.1.1 On Tue, Mar 28, 2017 at 8:27 PM, Russell Bateman <r...@windofkeltia.com> wrote: Both pom.xml and NAR updated (and still not working) on windofkeltia.com/preview. On 03/28/2017

Re: Processor missing when attempting Add Processor

Tue, Mar 28, 2017 at 7:48 PM, Russell Bateman <r...@windofkeltia.com> wrote: Joseph, You're right. Somehow, /nifi-tika-1.0.1.jar/ is not found among the myriad JARs under /META-INF/bundle//d-dependen//c//ies/ in the NAR. I thought I'd seen it in there, but I had not looked hard enough. I'

Re: Processor missing when attempting Add Processor

Thanks, Joseph. The changed (parent) /pom.xml /plus the new NAR (which still doesn't work for me) are on that path. Russ On 03/28/2017 06:09 PM, Joseph Niemiec wrote: org.apache.nifi nifi-nar-maven-plugin 1.1.0 true

Re: Processor missing when attempting Add Processor

28, 2017 at 7:48 PM, Russell Bateman <r...@windofkeltia.com> wrote: Joseph, You're right. Somehow, /nifi-tika-1.0.1.jar/ is not found among the myriad JARs under /META-INF/bundle//d-dependen//c//ies/ in the NAR. I thought I'd seen it in there, but I had not looked hard enough. I'll have to r

Re: Processor missing when attempting Add Processor

NAR projects. org.apache.nifi nifi-nar-maven-plugin 1.0.1-incubating true On Tue, Mar 28, 2017 at 7:09 PM, Russell Bateman <r...@windofkeltia.com> wrote: Thanks! I'll write some kind of blog post on this once we figure it out. http://www.windofkeltia.com/preview/ On 03/28/2017 04

Re: Processor missing when attempting Add Processor

the content of your /src/main/resources/META-INF/services/org.apache.nifi.processor.Processor file? You also have this entry in some pom's which I dont have in my NAR projects. org.apache.nifi nifi-nar-maven-plugin 1.0.1-incubating true On Tue, Mar 28, 2017 at 7:09 PM, Russell Bateman &l

Re: Processor missing when attempting Add Processor

Thanks! I'll write some kind of blog post on this once we figure it out. http://www.windofkeltia.com/preview/ On 03/28/2017 04:59 PM, Joseph Niemiec wrote: Sure please share, I have alittle time tonight. On Tue, Mar 28, 2017 at 6:53 PM, Russell Bateman <r...@windofkeltia.com> wrote:

Re: Processor missing when attempting Add Processor

and reloading the entire canvas. I have had some funny behaviours before with a stale cache not seeing anything. On Tue, Mar 28, 2017 at 5:57 PM, Russell Bateman <r...@windofkeltia.com> wrote: I've built a NAR containing a custom processor that loads in NiFi, but the processor cannot be

Re: Processor missing when attempting Add Processor

(Yeah, sorry, that is only a typo in my e-mail.) On 03/28/2017 04:36 PM, Andre wrote: Russell, Not sure if this is a typo but have you noticed this? p//rocessor.Processor Cheers On 29 Mar 2017 8:57 AM, "Russell Bateman" <r...@windofkeltia.com> wrote: I've built a NAR con

Processor missing when attempting Add Processor

I've built a NAR containing a custom processor that loads in NiFi, but the processor cannot be found. At the top of /TikaProcessor.java/, I have, among other annotations, this: @Tags( { "tika" } ) I've tried a number of things to solve this, like making it the only NAR besides just the set

Re: Is there a way to prohibit auto-termination...

be a big fan of restricting it personally. Thanks Joe On Fri, Mar 24, 2017 at 4:55 PM, Russell Bateman <r...@windofkeltia.com> wrote: ...of a relationship? In some of our custom processors, we'd like to remove the possibility from our user of ever auto-terminating, in particular, se

Is there a way to prohibit auto-termination...

...of a relationship? In some of our custom processors, we'd like to remove the possibility from our user of ever auto-terminating, in particular, selected processors' failure relationships. (I realize that the user can "drain" the failures away into paths that amount to auto-termination, but

Re: Providing Config and Attributes to NiFi

James, I'm not certain what Joe intends here, as it works for me. I've used it in NiFi 1.1.1 and have notes on it (just today, actually ) for my guys to use, however, my example uses two processors you don't have, a way to create an

Re: [SUGGESTION] Annotation for memory intensive processors (i.e. those that load content into memory)

Joe, I just up-voted this JIRA entry. Our down-streamers are having fits with stuff like this and we have enough custom processors that I'm certain they're confused as to which ones they do what with, etc. This feature (or multiple such annotations) would be comparatively simple to

Re: Creating new sessions...

re is some asynchronous process going on. Does that make sense? Thanks -Mark On Mar 17, 2017, at 12:45 PM, Russell Bateman <r...@windofkeltia.com> wrote: The ability to create a new session is conferred by extending AbstractSessionFactoryProcessor(in lieu of AbstractProcessorwhich further res

Creating new sessions...

The ability to create a new session is conferred by extending AbstractSessionFactoryProcessor(in lieu of AbstractProcessorwhich further restricts Processor, among other things, removing the session factory) giving one the option of calling sessionFactory.createSession(). This begs the question

Re: Closing in on a NiFi 1.2.0 release?

+1 for component versioning in NiFi 1.2! On 03/08/2017 12:40 PM, James Wing wrote: Bryan, I'm 100% in favor of you and Matt Gilman doing all the hard work. Oh, and uh... thanks! :) So the alternatives are: a.) Release 1.2.0 sooner (?), but without component versioning b.) Delay 1.2.0 (?) to

Re: Aw: Re: new Nifi Processors

I too find GPL to be confusing enough that I (and I am not alone) consider it to be poisonous fruit that I simply do not touch. I earn my living working as an employee developing software for companies that sell it, or rights to use it, for money and do not publish their product source code

Re: Deadletter

No confusion, Nick. I hear you. In our case... We attempt to reroute flowfile back through processors that, for whatever reason, might bring them to a state in which they can be successful (much to explain there, but...) and others we route to a similar, "deadletter" store where their

NiFi logging framework (stay in or work outside)

This question may arise a little out of naivety on my part. I need to create a new separate log, not to log the stuff I'm already logging and that appears nicely in /nifi-app.log/, but a sort of auditing log to which I'll write with comparative infrequency and very little verbosity. I'd like

[jira] [Updated] (NIFI-3364) NiFi start-up fails to honor N in definitions java.arg.N as made in conf/bootstrap.conf

[ https://issues.apache.org/jira/browse/NIFI-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Bateman updated NIFI-3364: -- Attachment: RunNiFi.java There are sufficient comments including horizontal rules in this file

[jira] [Commented] (NIFI-3364) NiFi start-up fails to honor N in definitions java.arg.N as made in conf/bootstrap.conf

[ https://issues.apache.org/jira/browse/NIFI-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886457#comment-15886457 ] Russell Bateman commented on NIFI-3364: --- Sorry, I don't know why the attachment didn't take. I'm

Re: Sensitive-value properties

opy the flow, or if you didn't set one in both instances which means you have the same. If you had different keys then one instance wouldn't be able to decrypt the values from the other instance. Thanks, Bryan On Fri, Feb 17, 2017 at 2:35 PM, Russell Bateman <r...@windofkeltia.com <mailt

Sensitive-value properties

1. Where are the values of sensitive-value properties stored? 2. How do they work? (if not obvious from #1) 3. Can /flow.xml.g//z/ be copied wholesale or in part to another instance of NiFi? Thanks, Russ

Re: Deadlocks after upgrade from 0.6.1 to 1.1.1

Mikhail, I have a short article with step-by-step information and comments on how I profile NiFi. You'll want the latest NiFi release, however, because the Java Flight Recorder JVM arguments are very order-dependent. (I'm assuming that NiFi 1.1.2 and 0.7.2 have the fix for

Reporting tasks as processor properties

Reading the docs on Reporting Tasks, it seemed to me that they said that hooking (my word) a reporting task up as a property to a processor is the preferred way [1]. I wanted to confirm this as I have yet to stumble upon a coded reporting task sample that does this (or, should I say, a

Re: Outputting flowfiles to disk

Or, 3? -> MergeContent +-> PutFile AttributesToJson > MergeContent Or, 4? Join the ranks of custom processor writers and write one to do exactly what you want--good idea if this a pretty permanent part of your

Re: Two more updated video tutorials

I don't want to spam this forum unduly, but I want to say that what you're doing, Jenn, is hugely appreciated. I pass your links out. I'm a developer of custom processors mostly and really was writing processors to solve problems months before trying to write actual, production flows of any

Re: Writing back through a python stream callback when the flowfile content is a mix of character and binary

write back to handle all. I'd like to better understand this error I'm getting, too. -Jim On Thu, Feb 2, 2017 at 6:02 PM, Russell Bateman <r...@windofkeltia.com <mailto:r...@windofkeltia.com>> wrote: Could you use /RouteOnContent/ to determine what sort of content yo

Re: Moving ZIP and TAR.GZ "formats" into an explicit profile

Maven profiles could be used to sort this out, one for just the zip, another for just the tarball and a third, maybe the default, to continue working the way it has been working. On 02/02/2017 09:09 AM, Aldrin Piri wrote: I think this could be useful. Only caveat is that I'm sure there are

Build error for UIMA because of Eclipse?

I don't develop this software, only build and use it. Recently, this began to happen. Can anyone enlighten me? From my build: [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache UIMA Java SDK: uimaj-parent

Re: Cadmium rods?

ry needing to be evaluated. This I believe should be far more efficient in recent releases. Can you advise what release you're running on? How large of a content repository in terms of file count are you referring to? Thanks Joe On Mon, Jan 23, 2017 at 9:54 AM, Russell Bateman <r...@windofkeltia.com>

Cadmium rods?

Can we get cadmium rods? Our down-streamers are complaining, and I have some experience through testing myself, of NiFi getting into molasses where the only solution is to bounce it. Here are some comments from my users that I hope are helpful. "I'm getting really burned out over the

Re: [DISCUSS] Idea for visual access control policy application

Telescopic then that there might be flow one has no access to see as well as other flow one can see but not change? Then do we have flow to which one can add, but not subtract, etc.? Can one detach from a protected flow? Can one stop/start protected flows? If a protected flow cannot be

Re: Adding JVM arguments when running NiFi...

n. Thanks -Mark On Jan 18, 2017, at 10:22 AM, Russell Bateman <r...@windofkeltia.com> wrote: For my own purposes, I fixed this in 0.7.1 /nifi-bootstrap///src/main/java/org/apache/nifi/bootstrap/RunNiFi.java/, but at Joe's suggestion, I created JIRA NIFI-3364 hoping some com

Re: Adding JVM arguments when running NiFi...

know this?) the use of non-numeric options for the position of /N/. Russ On 12/14/2016 11:05 AM, Russell Bateman wrote: I've doctored /conf/bootstrap.conf/ to contain these additional lines: java.arg.15=-XX:+UnlockCommercialFeatures java.arg.16=-XX:+FlightRecorder java.arg.17

[jira] [Updated] (NIFI-3364) NiFi start-up fails to honor N in definitions java.arg.N as made in conf/bootstrap.conf

[ https://issues.apache.org/jira/browse/NIFI-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Bateman updated NIFI-3364: -- Fix Version/s: 0.8.0 Affects Version/s: 0.8.0 > NiFi start-up fails to hono

[jira] [Created] (NIFI-3364) NiFi start-up fails to honor N in definitions java.arg.N as made in conf/bootstrap.conf

Russell Bateman created NIFI-3364: - Summary: NiFi start-up fails to honor N in definitions java.arg.N as made in conf/bootstrap.conf Key: NIFI-3364 URL: https://issues.apache.org/jira/browse/NIFI-3364

Re: Deleting millions of files from a queue...

ould be very useful to me. I need a counter that could survive a cluster restart for a few of my workflows. Currently I write out to an SQL server, but this is expensive to maintain. Regards, Kevin *From:*Russell Bateman [mailto:r...@windofkeltia.com] *Sent:* Wednesday, January 11, 2017 1:18 PM

Re: Deleting millions of files from a queue...

with your flow? This would keep the queue more manageable. On Jan 10, 2017, at 4:35 PM, Russell Bateman <r...@windofkeltia.com <mailto:r...@windofkeltia.com>> wrote: To update this thread, ... 1. Setting up a no-op processor to "drain" the queue doesn't seem to prese

Re: Deleting millions of files from a queue...

nt on [4,5], I'll come back and update this thread again. On 01/10/2017 02:50 PM, Russell Bateman wrote: In my case, I'm experimenting with huge flows and huge numbers of files. I wasn't thinking about how much work I'd create for myself by storing up files in a queue at the end (or, in

Re: Deleting millions of files from a queue...

want to retain some output to look at just stop it processing. On 10 Jan 2017, at 21:32, Russell Bateman <r...@windofkeltia.com <mailto:r...@windofkeltia.com>> wrote: If I'm experimenting and have gajillions of flowfiles in a queue that takes a very long time to empty

Re: Deleting millions of files from a queue...

and be faster. Though is clearly a blunt instrument. Do you think we need an express queue killer option? On Jan 10, 2017 1:32 PM, "Russell Bateman" <r...@windofkeltia.com <mailto:r...@windofkeltia.com>> wrote: If I'm experimenting and have gajillions of flowfiles in

Deleting millions of files from a queue...

If I'm experimenting and have gajillions of flowfiles in a queue that takes a very long time to empty from the UI, is there a quicker way? I can certainly bounce NiFi, delete files, both, etc.

Re: GetFile: why must source directory be writeable...

I see that it's okay for the files themselves to be read-only, but it seems heavy-handed to enforce writeability on the parent subdirectory, no? On 01/06/2017 02:00 PM, Russell Bateman wrote: ...when Keep Source Fileis set to trueand the sources are being protected anyway against deletion

GetFile: why must source directory be writeable...

...when Keep Source Fileis set to trueand the sources are being protected anyway against deletion? Is this an oversight that should be JIRA'd?

Re: Legacy Authorized Users File approach not working?

at 3:19 PM, Russell Bateman <r...@windofkeltia.com> wrote: Thanks, Bryan. I will try this again. Meanwhile, I've been studying our 0.x LDAP twists and I am starting to feel sure I can replicate them in 1.x. Both avenues will be useful to us. (But, I really liked nifi-toolkit and your tutorial.)

Re: Legacy Authorized Users File approach not working?

back to a clean state. Thanks, Bryan On Tue, Dec 20, 2016 at 5:10 PM, Russell Bateman <r...@windofkeltia.com> wrote: Bryan, As I'm new to setting this up (why I'm tackling this at all), I initially looked toJames Wing's tutorial <http://www.batchiq.com/nifi-c onfiguring-ssl-

Re: Legacy Authorized Users File approach not working?

able to point set "Legacy Authorized Users File" in authorizers.xml to point to your old file, and leave the Initial Admin blank. On Tue, Dec 20, 2016 at 2:17 PM, Russell Bateman <r...@windofkeltia.com> wrote: Just getting time to return to this... Bryan First, I was trying to f

Re: Legacy Authorized Users File approach not working?

. If you are converting an existing 0.x instance there is a way to auto convert that through a property in authorizers.xml, and it should be explained in the admin guide as well. Hope that helps. -Bryan On Fri, Dec 16, 2016 at 5:21 PM Russell Bateman <r...@windofkeltia.com> wrote: I'm w

Legacy Authorized Users File approach not working?

I'm working on securing NiFi 1.1.0 with SSL. As I'm following a tutorial [1] written for 0.x, I'm attempting to use the legacy option in /conf/authorizers.//xml/: file-provider org.apache.nifi.authorization.FileAuthorizer ./conf/authorizations.xml ./conf/users.xml

Re: Adding JVM arguments when running NiFi...

not ordering the arguments in the same manner supplied. Will need to look into that. Thanks Joe On Wed, Dec 14, 2016 at 9:00 PM, Russell Bateman <r...@windofkeltia.com> wrote: Of course, as many have done, I've run Java applications with JFR enabled using these options against this very JVM (jdk1.8

Re: Adding JVM arguments when running NiFi...

. I'll get back at it tomorrow and report back. On 12/14/2016 11:05 AM, Russell Bateman wrote: I've doctored /conf/bootstrap.conf/ to contain these additional lines: java.arg.15=-XX:+UnlockCommercialFeatures java.arg.16=-XX:+FlightRecorder java.arg.17=-XX:StartFlightRecording

Adding JVM arguments when running NiFi...

I've doctored /conf/bootstrap.conf/ to contain these additional lines: java.arg.15=-XX:+UnlockCommercialFeatures java.arg.16=-XX:+FlightRecorder java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr In the end, NiFi's grumpy about this and won't start (from

Re: Making FlowFiles environment independent

Without completely understanding your scenario, may I suggest you insert UpdateAttributesto do this before PutTCP? It supports Expression Language. Hope this helps. Russ On 12/12/2016 01:04 AM, ddewaele wrote: We have a flowfile that contains a number of environment specific values (ports /

Re: how to aad a specified counter in nifi

= session.get(); if (flowFile != null) { session.adjustCounter("SampleScriptCounter", 1, false); session.transfer(flowFile, REL_SUCCESS); } The performance may not be optimal, but you can experiment with counters immediately. Thanks, James On Fri, Dec 9, 2016 at 7:56 AM, Russell

Re: how to aad a specified counter in nifi

) { session.adjustCounter("SampleScriptCounter", 1, false); session.transfer(flowFile, REL_SUCCESS); } The performance may not be optimal, but you can experiment with counters immediately. Thanks, James On Fri, Dec 9, 2016 at 7:56 AM, Russell Bateman <r...@windofkelt

Re: how to aad a specified counter in nifi

Aldrin, Forgive me for commandeering this thread. My down-streamers are asking for a custom "counting" processor because of what from their point of view is an unreliable volatility in the counts that the UI displays. In particular, they want to know how many documents they're feeding into

Re: ComponentLog missing from 0.7.1 JAR...

Bryan, I'm sorry. I have found the problem and it's entirely my fault. I've been updating /nifi-0.7.1/lib/ with the wrong NAR without realizing that it's the wrong NAR. Thank you for looking and I'm sorry to have wasted your time. Russ On 12/07/2016 09:39 AM, Russell Bateman wrote: Bryan

Re: ComponentLog missing from 0.7.1 JAR...

AM, Russell Bateman <r...@windofkeltia.com> wrote: This should be a simple matter of linking and loading. I mean, if I build/link using /nifi-api-0.7.1.jar/ then load on NiFi 0.7.1, there's no way that any symbol of NiFi origin, in this case, ComponentLog, should go absent. That symbol's use

Re: ComponentLog missing from 0.7.1 JAR...

API JAR and it should of course be satisfied at runtime. What am I missing here? (It's probably staring me in the face, but I don't see it.) On 12/06/2016 02:26 PM, Russell Bateman wrote: I've had this problem with an old version for some time (I'm really concentrating more on NiFi 1.x now

ComponentLog missing from 0.7.1 JAR...

I've had this problem with an old version for some time (I'm really concentrating more on NiFi 1.x now) and I have to fix it. I'm pulling my hair out a bit. We eliminated all use of ProcessorLogeven in this older code that we're trying to keep running on 0.7.1 and maintaining when we must.

Re: Whither NiFi directories and files once NiFi running?

itoring service for things like directory usage,etc.. Since you're only needing this information for passive usage purposes in NiFi there is probably not much of a benefit of hosting it in nifi. What do you think? Thanks Joe On Fri, Dec 2, 2016 at 11:11 AM, Russell Bateman

Whither NiFi directories and files once NiFi running?

/nifi.properties/ locations, such as nifi.content.repository.directory.default, are typically specified as relative paths, but I suppose they could be arbitrarily complex. Whatever the case, I need to discover the actual path to the content subdirectory at runtime not because I want to create,

Re: Another holiday request...

. This will issue a bulletin both on the processor (and visually indicate a note) as well as add it to a global list of bulletins. By default processors ignore anything lower than WARN, but this is adjustable in processor settings. Andrew On Thu, Dec 1, 2016 at 5:47 PM Russell Bateman &l

<    1   2   3   4   5   >