Re: ...not the most recent version of this FlowFile within this session...

2024-05-07 Thread Russell Bateman
Yes, what you described is what was happening, Mark. I didn't display all of the code to the session methods, and I did re-read the in-coming flowfile for different purposes than I had already read and written it. So, I wasn't helpful enough. In the end, however, I had forgotten, immediately

...not the most recent version of this FlowFile within this session...

2024-05-07 Thread Russell Bateman
In /pom.xml/ I specify using NiFi framework libraries from 1.13.2. There are peculiar reasons (that we are trying to fix) that inhibit us from moving forward as all of our customer machines are running 1.1.2. (Don't shoot me, I'm not DevOps, but just the guy who writes custom processors.) I

Re: SELinux and NiFi

2024-03-08 Thread Russell Bateman
to getting a basic setup running. Sent from my iPhone On Mar 8, 2024, at 4:29 PM, Russell Bateman wrote:  We have run on CentOS with SELinux set to enforcing and have run NiFi in that environment for probably 8 or 9 years now. We do install some SELinux policies that allow NiFi to access the

Re: SELinux and NiFi

2024-03-08 Thread Russell Bateman
We have run on CentOS with SELinux set to enforcing and have run NiFi in that environment for probably 8 or 9 years now. We do install some SELinux policies that allow NiFi to access the filesystem underneath itself and not outside that filesystem. What specifically are you asking? On 3/8/24

Re: NiFi 2.0.0-M2

2024-03-06 Thread Russell Bateman
Don't know if this is relevant let along helpful, but we were on CentOS for many years and, when it died, we moved to Alma. Best of luck to you, Russ On 3/5/24 07:56, Marton Szasz wrote: CentOS 7 is a very old distribution, so you may run into issues, but in theory, if you can install Java

Re: Block start/stop processors

2024-02-22 Thread Russell Bateman
Isha, Wait, by "lefthand toolbox" are you referring to the process group's toolbar start button or to the start button in the pallet at the upper left of the page? Please clarify. Thanks On 2/22/24 04:39, Isha Lamboo wrote: always use the right-click context menu on a process group you

Re: Issue with Nifi pipeline

2023-11-28 Thread Russell Bateman
Sonia, It sounds like you may prefer the Users Mailing List (us...@nifi.apache.org)rather than this one which is more for custom processors and other development-related activities. Best regards, Russ On 11/27/23 22:49, Sonia Soleimani wrote: Hello, I am working for Telus and there has

Internationalization and localization of the UI and processors

2023-09-29 Thread Russell Bateman
Looking around, I see there have been statements of intent by folk to localize NiFi [1], but few statements as to how far they got. I saw a question on stackoverflow [2] on how to hack relevant Java-annotated references (@CapabilityDescription) which isn't exactly internationalization, but

Custom-processor configuration suggestions

2023-09-27 Thread Russell Bateman
I'm posting this plea for suggestions as I'm short on imagination here. We have some custom processors that need extraordinary amounts of configuration of the sort a flow writer would have to copy and paste in--huge amounts of Yaml, regular expressions, etc. This is what our flow writers are

Re: new PackageFlowFile processor

2023-09-08 Thread Russell Bateman
Uh, sorry, "Version 3" refers to what exactly? On 9/8/23 12:48, David Handermann wrote: I agree that this would be a useful general feature. I also agree with Joe that format support should be limited to*Version 3* due to the limitations of the earlier versions. This is definitely something

Re: Refreshing HTML displayed for View usage

2023-08-21 Thread Russell Bateman
noticed myself, and it's easily reproducible on the Usage pages for processors, is that Nifi is doing a reverse DNS lookup when loading the page. If your DNS is broken, it can take 10-30 seconds for the page content to appear. Kr, On Fri, 18 Aug 2023, 07:29 Russell Bateman, wrote: Matt, I

Re: Refreshing HTML displayed for View usage

2023-08-17 Thread Russell Bateman
and the version is not changing (x.y.z-SNAPSHOT). This is where the browser cache could result in the behavior your seeing. Matt On Thu, Aug 17, 2023 at 3:21 PM Russell Bateman wrote: Thanks, Matt, (Since I use Chrome 99% of the time, I'm using Chrome here.) Clicking on the reload control next

Re: Refreshing HTML displayed for View usage

2023-08-17 Thread Russell Bateman
that tab and retry from the NiFi UI. Let us know if this doesn't help and we can see if something isn't getting generated and updated correctly. Matt On Thu, Aug 17, 2023 at 2:38 PM Russell Bateman wrote: Seems like a really stupid user/browser question, but I cannot seem to get changes I've made

Refreshing HTML displayed for View usage

2023-08-17 Thread Russell Bateman
Seems like a really stupid user/browser question, but I cannot seem to get changes I've made to properties, relationships, attributes read or written, etc. *for custom processors*. Also, from the Properties tab in Configuring Processor, the cartoon blurbs obtained by hovering over (?) aren't

Custom processor once had properties, has no more now, but they still show up

2023-07-20 Thread Russell Bateman
I have a custom processor I modified losing a few properties because a specification change made them useless. I removed them. The processor works, but in configuration for this processor (in the NiFi UI), the processor appears to have kept them, i.e.: they're not disappearing. I would have

Re: Use of attribute uuid and other "native" attributes

2023-07-18 Thread Russell Bateman
latter setting some other attributes as well (see the Developer Guide [1] for more details). Regards, Matt [1]https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html On Tue, Jul 18, 2023 at 12:25 PM Russell Bateman wrote: I have a custom processor, /SplitHl7v4Resources/, that splits

Use of attribute uuid and other "native" attributes

2023-07-18 Thread Russell Bateman
I have a custom processor, /SplitHl7v4Resources/, that splits out individual FHIR resources (Patients, Observations, Encounters, etc.) from great Bundle flowfiles. So, for a given flowfile, it's split into hundreds of smaller ones. When I do this, I leave the existing NiFi attributes as they

Re: [HAPI-devel] [External] Unable to inspect gender and birthdate in Patient

2023-06-14 Thread Russell Bateman
for more info: https://hapifhir.io/ Incidentally, you are creating a new (empty) Patient object, then trying to fetch the name. It doesn’t have one because you haven’t set one. *From:* Russell Bateman *Sent:* Tuesday, June 13, 2023 1:54 PM *To:* hl7api-devel@lists.sourceforge.net *Subject

[HAPI-devel] Unable to inspect gender and birthdate in Patient

2023-06-13 Thread Russell Bateman
In the code below, I have an incoming Patient which has all the fields that interest me filled in (see debugger image, but if it doesn't come through, it shows that patient is filled out for name, gender, birthdate and address). However, when I attempt to inspect *gender* and *birthdate*,

Re: Possible Docker bug in 1.13.2

2023-06-08 Thread Russell Bateman
SDJT/p1686135561932289?thread_ts=1686135561.932289=CDGMCSDJT On Thu, 8 Jun 2023, 00:10 Russell Bateman, wrote: I'm re-rolling in order to update the Java inside to 11 in order to permit using the new Java HTTP client. This seems to work well; I fixed the bug locally. Maybe too old to be impor

Re: Possible Docker bug in 1.13.2

2023-06-07 Thread Russell Bateman
, until we can shed the old NAR we cannot rebuild, we're stuck at NiFi 1.13.2. On 6/7/23 15:31, Russell Bateman wrote: I downloaded sources to 1.13.2 in order to hand-spin my own container image. When I got down to /nifi-1.13.2/nifi-docker/dockerhub/Dockerfile/, I found: ... ARG

Possible Docker bug in 1.13.2

2023-06-07 Thread Russell Bateman
I downloaded sources to 1.13.2 in order to hand-spin my own container image. When I got down to /nifi-1.13.2/nifi-docker/dockerhub/Dockerfile/, I found: ... ARG NIFI_VERSION=1.13.1 ... and the version is also wrong in /DockerImage.txt/ which /DockerBuild.sh/ consumes. Indeed, the

Re: Usage Documentation for Custom Processors

2023-04-04 Thread Russell Bateman
Matthew, If you feel that the documentation generated from the annotations at the top of your custom processor class (@CapabilityDescription, etc., of which Bryan spoke) is insufficient, it's also possible to supplement it with

Re: Size of CLOB...

2023-03-24 Thread Russell Bateman
3/23/23 14:54, Russell Bateman wrote: In fact, experimentation seems to suggest that the breaking point is a length of 32K+. Whether or not the length of the rest of the INSERT INTO statement counts I do not know for certain, but the delta between MsgContent and the rest is negligeable, so I

Re: Size of CLOB...

2023-03-23 Thread Russell Bateman
In fact, experimentation seems to suggest that the breaking point is a length of 32K+. Whether or not the length of the rest of the INSERT INTO statement counts I do not know for certain, but the delta between MsgContent and the rest is negligeable, so I don't care. On 3/23/23 13:02, Russell

Size of CLOB...

2023-03-23 Thread Russell Bateman
According to what I'm reading, a CLOB should easily hold more than the 128K I'm trying to put into it. My table schema: private static final StringTABLE_SCHEMA ="\n" +" MsgId BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY,\n" +" MsgContent CLOB DEFAULT NULL,\n" +" MsgReceived TIMESTAMP NOT NULL

Re: NiFi failing to start

2022-12-28 Thread Russell Bateman
In case you or someone else wishes only to run, develop, start, stop, start over, etc., and doesn't care to authenticate a (non-production) installation, I have followed this since NiFi 1.14 and last used it for 1.19: https://www.javahotchocolate.com/notes/nifi.html#20210716 If this doesn't

Re: ReplaceText 1.19 not processing Flowfile

2022-12-11 Thread Russell Bateman
Also, this strikes me as a NiFi Users List (us...@nifi.apache.org) question though many of us haunt both forums. On 12/11/22 07:55, Mark Payne wrote: Hello, It looks like the attachment didn’t come through. The mailing list often strips out attachments from emails. Perhaps put them in

Re: Grammar error in error message

2022-11-16 Thread Russell Bateman
Yes it should be the infinitive */receive/* instead of the past participle /received/. On 11/16/22 08:18, Paul Schou wrote: This error message does not look like it is grammatically correct:

Re: On configuring SSLContextService...

2022-08-15 Thread Russell Bateman
If you'll permit, I want to cap this thread I started off a bit by a) thanking the many who contributed to it and b) summing up the solution I am using based on that help. Here are the command lines and germane instructions. For localhost below, substitute the DNS name (or, at least,

Re: On configuring SSLContextService...

2022-08-09 Thread Russell Bateman
3.  The NiFi equivalent to these two instructions is to provide a truststore, which contains a record specifying the certificate being served by your Tomcat server. On Mon, Aug 1, 2022 at 6:27 PM Russell Bateman wrote: Ok

Re: On configuring SSLContextService...

2022-07-29 Thread Russell Bateman
Just a note (for later readers of this thread)... My experience now with this trick seems to say that, as long as "https" is in the URL, a /SSLContextService/ must be supplied. As a URL with "https" and port number 8443 is the only way I have to engage TLS at the far end, I must live with

On configuring SSLContextService...

2022-07-26 Thread Russell Bateman
I have hesitated between providing some huge tl;dr exposé and something shorter. I'll do shorter here. 0. For now, I'm using "changeit" below as password rolling a self-signed certificate for key, key store and trust store. 1. I have a service running in Tomcat that I hit via HTTPs because the

Re: Placement and specification of certificates for StandardRestrictedSSLContextService

2022-07-21 Thread Russell Bateman
David, Sadly, this is my experience. "changeit" works for me. And I tried reconfiguring the three passwords in /StandardRestrictedSSLContextService/ to no avail. ~/dev/nifi/nifi-1.15.0/conf $ *keytool -list -v -keystore mdmi-keystore.jks* Enter keystore password: *changeit*

Placement and specification of certificates for StandardRestrictedSSLContextService

2022-07-20 Thread Russell Bateman
I'm trying to set up TLS for a service using /InvokeHTTP/ against an external-to-NiFi Tomcat-based service and I have configured /StandardRestrictedSSLContextService/ thus: https://www.javahotchocolate.com/notes/nifi-images/mdmi-standard-ssl-context-service.png ...which results in the errors

Re: How to manage security artifacts from a custom processor

2022-07-05 Thread Russell Bateman
representing the third-party service in a truststore and maybe another (a private key) in a keystore. On 7/5/22 16:30, Russell Bateman wrote: From a custom processor, I intend to interface with a third-party service (via simple HTTP client), however, I would need as I understand it to a) maintain

How to manage security artifacts from a custom processor

2022-07-05 Thread Russell Bateman
From a custom processor, I intend to interface with a third-party service (via simple HTTP client), however, I would need as I understand it to a) maintain a private key by which I can identify myself to that third-party service and b) maintain a trusted-store certificate by which I

Re: Reg Nifi java code generation

2022-05-02 Thread Russell Bateman
You don't have to write Java code to benefit from NiFi which is an insanely useful framework all by itself with jillions of super-useful processors ready for use. However, if you plan to code your own, proprietary processor to do something that hasn't been covered, here's a likely place to

Re: how to setup nifi.content.repository.archive.max.retention.period in a Nifi docker image ?

2022-02-22 Thread Russell Bateman
Breno, While we're on this topic, what's best practice for changing something like "the default nifi files on a clean docker image, ..."? Use sed or awk from a RUN command? (This is really a Docker question, but you raised it. Anything you suggest would be helpful.) Russ On 2/22/22 07:07,

Re:

2021-12-20 Thread Russell Bateman
Ismaël, Please send an e-mail to users-unsubscr...@nifi.apache.org. Thanks. On 12/20/21 8:45 AM, ismaelmartin.ri...@post.ch wrote: unsubscribe

Re: How to restrict custom processor execution time

2021-08-12 Thread Russell Bateman
Sanjeet, It occurred to me that you may not be getting replies because you report "building a custom processor" and that's the subject of the Apache NiFi Developers' forum. Try posting there. Best regards, Russ On 8/10/21 7:33 AM, sanjeet rath wrote: Hi , I am building a custom processor

Re: odd performance behavior 1.14

2021-08-02 Thread Russell Bateman
(I'm sorry, I meant to say that I read that the Java Flight Recorder is now available for unlicensed--in the original Oracle sense--for use.) On 8/2/21 12:44 PM, Russell Bateman wrote: Scott, I believe I read somewhere in the last year. I found this: https://developers.redhat.com/blog/2020

Re: odd performance behavior 1.14

2021-08-02 Thread Russell Bateman
Scott, I believe I read somewhere in the last year. I found this: https://developers.redhat.com/blog/2020/08/25/get-started-with-jdk-flight-recorder-in-openjdk-8u And, I used it to look into a problem I had with Apache NiFi a few years ago. For what it's worth, my experience is recorded here:

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-07-23 Thread Russell Bateman
Russell Bateman wrote: Joe, I apologize for the off-topic intrusion, but what replaces templates? The Registry? Templates rocked and we have used them since 0.5.x. Russ On 7/23/21 8:31 AM, Joe Witt wrote: David, I think this is a highly reasonable approach and such a focus will greatly help make

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-07-23 Thread Russell Bateman
Joe, I apologize for the off-topic intrusion, but what replaces templates? The Registry? Templates rocked and we have used them since 0.5.x. Russ On 7/23/21 8:31 AM, Joe Witt wrote: David, I think this is a highly reasonable approach and such a focus will greatly help make a 2.0 release

Re: Problem with the GetFile processor deleting my entire installation

2021-06-04 Thread Russell Bateman
those in. Russ On 6/4/21 11:16 AM, Russell Bateman wrote: Sorry for this behavior of /GetFile/ which is purposeful. If you configure to keep the files instead of removing them, you'll keep getting the same files ingested over and over again as flow files. It's just how it is. The secret

Re: Problem with the GetFile processor deleting my entire installation

2021-06-04 Thread Russell Bateman
Sorry for this behavior of /GetFile/ which is purposeful. If you configure to keep the files instead of removing them, you'll keep getting the same files ingested over and over again as flow files. It's just how it is. The secret was to read the help blurb when configuring this processor.

Re: Penalizing one part of a flow over another

2021-04-22 Thread Russell Bateman
of maybe 10 MB/sec into the sub-flow. Again, that could cause backpressure so you’d want to consider FlowFile Expiration if you’d rather lose the FlowFiles than allow them to affect the main flow. Hope that’s helpful! Thanks -Mark On Apr 22, 2021, at 9:44 AM, Russell Bateman wrote: I have

Penalizing one part of a flow over another

2021-04-22 Thread Russell Bateman
I have a flow performing ETL of HL7v4 (FHIR) document on their way to indexing and storage. Custom processors perform the important transformations. Performance of this flow is at a premium for us. At some point along the way I want to gate off copies of raw or of transformed FHIR records (the

Re: NiFi 1.11.4 Custom Processor Development

2021-04-13 Thread Russell Bateman
I'm sorry. I mistyped (numbers soup). I meant to say NiFi 1.2.0, a version several years ago already. On 4/13/21 9:04 AM, Russell Bateman wrote: There shouldn't be any problem. I have many custom processors in a NAR I haven't rebuilt since 1.12 and we use them successfully. They're just

Re: NiFi 1.11.4 Custom Processor Development

2021-04-13 Thread Russell Bateman
There shouldn't be any problem. I have many custom processors in a NAR I haven't rebuilt since 1.12 and we use them successfully. They're just missing versions because we had not yet upgraded the Maven we used to build the NAR (not exactly relevant, I know, but I thought I'd I would point this

Re: [DISCUSS] Processors Market

2021-03-24 Thread Russell Bateman
Javi, Don't despair. Could just be that folk are busy and haven't had time to reflect upon it. On 3/23/21 11:55 PM, Javi Roman wrote: I see that it has not been well received, I thought it would be a good idea :-( -- Javi Roman Twitter: @javiromanrh GitHub: github.com/javiroman Linkedin:

Re: Preconfiguring dynamic properties

2021-02-24 Thread Russell Bateman
. On Wed, Feb 24, 2021 at 12:02 PM Russell Bateman wrote: I have a dynamic property in a custom processor that my down-streamers struggle a little bit to configure (requires newlines and a peculiar format). I would like to "preconfigure" a dynamic property as an example that they

Preconfiguring dynamic properties

2021-02-24 Thread Russell Bateman
I have a dynamic property in a custom processor that my down-streamers struggle a little bit to configure (requires newlines and a peculiar format). I would like to "preconfigure" a dynamic property as an example that they can either modify or erase to add their own. Most of them would

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Russell Bateman
I second the concerns expressed, but second especially Bryan's pointing out that requiring LDAP/AD to be set up in order even to begin to use our framework would be a bit onerous for developers just interested in getting work done and a barrier to considering the framework should it be erected

Re: java api for changing parameter context

2021-01-27 Thread Russell Bateman
Wait! Can't this be done using the ReST APIs? On 1/27/21 3:24 AM, u...@moosheimer.com wrote: Hello NiFi-Core-Team, Are you planning to create a high-level Java API for setting (and clearing) individual parameters in the parameter context, so we can use this API in processor development?

Re: Static processor design

2021-01-09 Thread Russell Bateman
:09 AM, Mark Bean wrote: Russell, You can use "session.get(N)" where N is an integer. This will get up to N flowfiles per OnTrigger() call. -Mark On Fri, Jan 8, 2021 at 5:07 PM Russell Bateman wrote: Very well, I have decided to force customer flowfiles through this processor (I

Re: Static processor design

2021-01-08 Thread Russell Bateman
Very well, I have decided to force customer flowfiles through this processor (I did check out the /Listen/* processors, but chose this easier solution). This now works. However, It brings up another question: is this the most efficient way to pass flowfiles straight through this processor

Re: Static processor design

2021-01-08 Thread Russell Bateman
I only put the code I want to execute in onTrigger(), I suspected it would not fire there. I know that this isn't what processors do. Configuration is a messy problem to solve when your downstreamers want it made easy. This is supposed to be a solution that allows them to remain in the NiFi UI

Re: Static processor design

2021-01-08 Thread Russell Bateman
statement from onPropertyModified()does reach the log every time I change properties to remove old or introduce new properties). On 1/7/21 6:38 PM, Russell Bateman wrote: (Inadequate title; didn't know what to call it.) I have written a processor that doesn't feature any relationships.

Static processor design

2021-01-07 Thread Russell Bateman
(Inadequate title; didn't know what to call it.) I have written a processor that doesn't feature any relationships. It accepts dynamically properties that, in theory, when created (or removed, or values added or changed), and sets data into a class inside my NAR. I wonder, however, at what

Re: Safely updating custom processors in existing flows...

2020-12-30 Thread Russell Bateman
I will be very happy to read what others contribute. Chris' suggestion, which has huge implications, I think, for my IDE and project structure, will be something to look into. However, I was hoping someone would articulate the difference between *property names* and *property display names*,

Safely updating custom processors in existing flows...

2020-12-30 Thread Russell Bateman
I have a custom processor actively used in customer flows. I need to enhance it, but avoid invalidating it when I update custom's existing NiFi installations. I know that the processor properties, in particular, the conflict between the property .name()and .displayName(), is a good way

Re: ETL to Nifi Migration

2020-12-26 Thread Russell Bateman
Unless you're certain you will need to write custom processors for NiFi, the forum you really want to subscribe to and post in is NiFi Users Best regards! On 12/26/20 7:37 AM, Sumith Karthikeyan wrote: Hi Team, Hope you all doing well !!! This regards a Middle East public sector

Re: Okay to manage ComponentLog via ThreadLocal?

2020-12-08 Thread Russell Bateman
(); logger.warn( “This is a warning!” ); } } Perhaps I’m missing something - or perhaps you made things simpler than they really are for demonstration purposes? Thanks -Mark On Dec 8, 2020, at 4:54 PM, Russell Bateman wrote: Because it's so onerous to pass a reference to the logger down through

Okay to manage ComponentLog via ThreadLocal?

2020-12-08 Thread Russell Bateman
Because it's so onerous to pass a reference to the logger down through parameters lists, I thought I might try using Java's thread-local store. I haven't been using it for anything else either, but I thought I'd start. For now, the logger is the only thing that tempts me. In past lives as a

Re: Tuning for flow with lots of processors

2020-11-27 Thread Russell Bateman
Eric, Mark, This is an example of an exchange in this forum that is of immeasurable help to to on-lookers. Thanks for discussing it so thoroughly! Russ On 11/23/20 5:55 PM, Eric Secules wrote: Hello everyone, I was wondering if there was a metric for the amount of time tImer-driven

Re: Run Nifi in IntelliJ to debug?

2020-10-26 Thread Russell Bateman
*From:* Russell Bateman *Sent:* Monday, October 26, 2020 7:37:13 PM *To:* Darren Govoni ; users@nifi.apache.org *Subject:* Re: Run Nifi in IntelliJ to debug? Darren, This is just Apache NiFi code out of NARs you want to step through or is it yours? You haven't stripped debug information or an

Re: Run Nifi in IntelliJ to debug?

2020-10-26 Thread Russell Bateman
? Generate sources? For future it would be nice if there was a maven goal for debug. Much appreciated! Darren Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android <https://aka.ms/ghei36> *From:* R

Re: Run Nifi in IntelliJ to debug?

2020-10-26 Thread Russell Bateman
Darren, I was out this morning and didn't see your plea until I got in just now. Here's a step by step I wrote up for both IntelliJ IDEA and Eclipse (I'm more an IntelliJ guy). It also covers using an IP tunnel. https://www.javahotchocolate.com/notes/nifi.html#20160323 On 10/26/20 9:52 AM,

Re: Hive NAR not loading because of snappy?

2020-10-13 Thread Russell Bateman
release to see if it works? Has the version of Java changed? Sent from my iPhone On Oct 13, 2020, at 8:13 PM, Russell Bateman wrote:  We didn't want to make deletion or modification of a shipped component to be a required part of its installation since we don't produce the NiFi download

Re: Hive NAR not loading because of snappy?

2020-10-13 Thread Russell Bateman
the native library gets loaded. If you are not using Hive I’d think you could safely delete the nifi-hive-nar and it shouldn’t happen. Hard to tell why the native library couldn’t be installed though. On Oct 13, 2020, at 6:26 PM, Russell Bateman wrote:  No, we don't even use (nor have we ever

Re: Hive NAR not loading because of snappy?

2020-10-13 Thread Russell Bateman
that doesn’t create a unique path under /tmp, do you have multiple PutHiveStreaming processors in the flow? I don’t think that works because we can’t load a single native library into multiple classloaders. On Oct 13, 2020, at 6:15 PM, Russell Bateman wrote:  I see -rwxr-xr-x. 1 nifi

Re: Hive NAR not loading because of snappy?

2020-10-13 Thread Russell Bateman
x defaults to /tmp. Does the NiFi user have write access to that directory? If not you can change the Java temp dir or set it specifically for Snappy (I don’t have the property on hand but a quick Google should find it) Regards, Matt Sent from my iPhone On Oct 13, 2020, at 5:36 PM, Russe

Hive NAR not loading because of snappy?

2020-10-13 Thread Russell Bateman
Should I be seeing this in the log of a vanilla NiFi installation on CentOS? ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.hive.PutHiveStreaming could not be

Re: APACHE NIFI CLUSTER INSTALLATION WALKTHROUGHS FOR 2NODE ON LINUX OS

2020-09-23 Thread Russell Bateman
You have treated Google as your friend and worked through the dozen or so examples by other folk doing this, right? --just a suggestion. On 9/23/20 1:21 PM, Abiodun Adegbile wrote: */Hello Team,/* Still looking forward to your reply. / / /I got this though after i tried setting up the

Re: APACHE NIFI CLUSTER INSTALLATION WALKTHROUGHS FOR 2NODE ON LINUX OS

2020-09-20 Thread Russell Bateman
Google is your friend. https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/nifi-configuration-best-practices/content/basic-cluster-setup.html https://bryanbende.com/development/2018/10/23/apache-nifi-secure-cluster-setup https://mintopsblog.com/2017/11/12/apache-nifi-cluster-configuration/

Re: TestRunner: enqueueing multiple flowfiles

2020-08-31 Thread Russell Bateman
i-mock/src/main/java/org/apache/nifi/util/StandardProcessorTestRunner.java#L443-L448 Instead of creating a new MockProcessSession, you would get the ProcessSessionFactory from the TestRunner and then call createProcessSession(). On Mon, Aug 31, 2020 at 9:01 AM Russell Bateman wrote: In my JUn

TestRunner: enqueueing multiple flowfiles

2020-08-31 Thread Russell Bateman
In my JUnit testing of a custom processor, I need to queue up at least two flowfiles. I see that there is an implementation of TestRunner.enqueue()that takes*a list of flowfiles*, but I'm used to using the implementation of this method that creates me a flowfile from bytes or a stream. I do

Re: From one flowfile to two...

2020-08-27 Thread Russell Bateman
or explain the experience as well. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Aug 27, 2020, at 10:10 AM, Russell Bateman wrote: I needed to get back here... I took this advice to heart and finished

Re: From one flowfile to two...

2020-08-27 Thread Russell Bateman
Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Aug 27, 2020, at 10:10 AM, Russell Bateman wrote: I needed to get back here... I took this advice to heart and finished my processor. Thanks to Matt and Mark for all their suggestions! They cleared up a few things. There was one bug

Re: From one flowfile to two...

2020-08-27 Thread Russell Bateman
, the NiFi mailing lists are a place both for great information and being treated well. Russ On 8/25/20 12:24 PM, Mark Payne wrote: Russ, Several comments here. I’ve included them inline, below. Hope it’s helpful. Thanks -Mark On Aug 25, 2020, at 2:09 PM, Russell Bateman wrote: Thanks for your

Re: From one flowfile to two...

2020-08-25 Thread Russell Bateman
ich is the same of course) and overwriting it on the way out. Regards, Matt On Mon, Aug 24, 2020 at 6:37 PM Russell Bateman wrote: I am writing a custom processor that, upon processing a flowfile, results in two new flowfiles (neither keeping the exact, original content) out two different relatio

From one flowfile to two...

2020-08-24 Thread Russell Bateman
I am writing a custom processor that, upon processing a flowfile, results  in two new flowfiles (neither keeping the exact, original content) out two different relationships. I might like to route the original flowfile to a separate relationship. FlowFile original = session.get(); Do I need

Re: Suggestions for splitting, then reassembling documents

2020-08-21 Thread Russell Bateman
, Jason -- Jason C. Sherman, CSSLP, CISSP Owner Logical Software Solutions, LLC Solid. Secure. Software. http://logicalsoftware.co/ .co? Yes, your data isn't always what you expect. We'll make sense of it. https://www.linkedin.com/in/lss-js/ On Tue, Aug 18, 2020 at 12:38 PM Russell Bateman wrote

Suggestions for splitting, then reassembling documents

2020-08-18 Thread Russell Bateman
I am writing custom processors that juggle medical documents (in a more or less proprietary format). The document are always XML and contain two, major parts: 1. an original document which may be text, HL7v2 or XML and may contain HTML between ... , could be many megabytes in size 2. XML

Re: Change Version not possible due to setting a Parameter Context for a Process Group

2020-08-17 Thread Russell Bateman
Forgive me for asking, but I'm curious. NiFi Registry 0.5.0 is fully two versions behind the current one. Wouldn't using the latest be a precursor to sorting out problems? Bugs may have been fixed. Or, is there something good known to be expected from the older Registry that the newest one

Live development of custom processors and JAVA_HOME--confirm best practice

2020-08-13 Thread Russell Bateman
When installing NiFi in production, Ansible can be used to set up JAVA_HOME. There is zero problem for users of NiFi. However, from a development host, given the now rapid cadence of Java releases, we sometimes run into problems launching a private installation of NiFi in the course of

Re: Need to know if there is multiple template support

2020-08-06 Thread Russell Bateman
Since the advent of the NiFi Registry, templates are sort of deprecated. The Registry is a brilliant design far better able to support what you seem to be asking for. By the way, this is much more a "user" question you might have asked in that forum rather than this "dev" forum. Cheers! On

Re: PUT SQL error attribute

2020-08-03 Thread Russell Bateman
wrote: Yes errors are being written to log but it would be useful if part of error was sent as attribute in failure queue. That will help us to decide on what processor should be called instead of checking logs. Thanks, Asmath On Mon, Aug 3, 2020 at 3:24 PM Russell Bateman <mailt

Re: PUT SQL error attribute

2020-08-03 Thread Russell Bateman
Pardon me, I meant: /${NIFI_ROOT}///logs///nifi-app.log ./ On 8/3/20 2:23 PM, Russell Bateman wrote: Mohammed, I didn't write /PutSQL/ and I haven't dug around to look at its source code, but I wouldn't be surprised to learn that those errors were already written by it to /${NIFI_ROOT

Re: PUT SQL error attribute

2020-08-03 Thread Russell Bateman
Mohammed, I didn't write /PutSQL/ and I haven't dug around to look at its source code, but I wouldn't be surprised to learn that those errors were already written by it to /${NIFI_ROOT}/nifi-app.log/ too. Russ On 8/3/20 8:56 AM, KhajaAsmath Mohammed wrote: Hi, I generally get errors

Re: Failing to update custom processor properties names, displayNames, etc.

2020-07-18 Thread Russell Bateman
code is not being properly loaded/used by NiFi’s server application. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Jul 17, 2020, at 3:48 PM, Russell Bateman wrote: If I have changed a custom

Failing to update custom processor properties names, displayNames, etc.

2020-07-17 Thread Russell Bateman
If I have changed a custom processor's PropertyDescriptor.nameand/or .displayName,including changes I have made to my /additionalDetails.html, /and I have: - removed that processor from my test flow or removed flow.xml.gz altogether - removed my NAR from /${NIFI_ROOT}///custom-lib/ and bounced

Re: Derby as DBCP service, error from Kerberos?

2020-07-14 Thread Russell Bateman
ot;. On Tue, Jul 14, 2020 at 10:24 AM Russell Bateman wrote: Thanks for the responses. I did have this dependency already before mailing to the forum: 1.11.0 ... org.apache.nifi nifi-kerberos-credentials-service ${nifi.version} Oth

Re: Derby as DBCP service, error from Kerberos?

2020-07-14 Thread Russell Bateman
ency in your project... https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/pom.xml#L50-L55 On Mon, Jul 13, 2020 at 5:24 PM Russell Bateman wrote: I'm trying to use Apache Derby as the DBCP controller in JUnit tests. For the fir

Derby as DBCP service, error from Kerberos?

2020-07-13 Thread Russell Bateman
I'm trying to use Apache Derby as the DBCP controller in JUnit tests. For the first test, I start off vetting my ability to inject Derby as the DBCP controller I want to use. But, right off, I get this Kerberos error. I wasn't trying to use Kerberos, but maybe I'm missing configuration to tell

Re: So, Derby works...

2020-07-08 Thread Russell Bateman
s.forName("org.apache.derby.jdbc.EmbeddedDriver")? What are the chain of errors raised? For the record, this is the recommended JDBC way to fault in a driver which was missed at autoloading time. Autoloading occurs on the very first call to DriverManager.getConnection() during th

So, Derby works...

2020-07-08 Thread Russell Bateman
...again, don't know what I changed.* I'm only depending on /derby-10.15.2.0.jar/. What I think is going on is an artifact of being side-by-side with another JDBC driver implementation. I was hoping to keep the original mocked test driver working alongside the Derby implementation at least

Re: Driver registration, Derby drivers and Derby versions

2020-07-08 Thread Russell Bateman
the tools jar. The autoloading driver moved into another package in the engine jar. It is now called org.apache.derby.iapi.jdbc.AutoloadedDriver. Hope this helps, -Rick On 7/7/20 6:00 PM, Russell Bateman wrote: How do the myriad Derby drivers work? I am using Apache Derby in-memory with

  1   2   3   4   5   >