qua., 30 de abr. de 2025 às 13:16, David Gallagher
mailto:dgallag...@cleverdevices.com>> escreveu:
Thanks, Heiko! But, when I use that definition it starts the service and
immediately terminates it.
Apr 30 16:35:46 clever-nifi systemd[1]: Starting nifi.service - Apache NiFi...
Apr 30 16:35:46 c
e=forking
ExecStart=/opt/nifi/bin/nifi.sh start
[Install]
WantedBy=multi-user.target
Regards
Heiko
From: David Gallagher
mailto:dgallag...@cleverdevices.com>>
Sent: Wednesday, April 30, 2025 12:07 AM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Installat
Hi - is there any guidance on how to create a systemd service definition for
NiFi 2.x? I'm finding it very difficult to create a working one, and a lot of
the advice online is apparently very out of date.
Thanks,
Dave
ith the RETRY relationship. Could be
recursively connected to the InvokeHTTP processor itself.
Michael
Am 23.04.25 um 21:15 schrieb David Gallagher:
> I am using Nifi 2.3.0. I want to use an InvokeHTTP processor to poll a REST
> endpoint every 30s and get a flowfile from the response contain
Hi Michael - tried again this morning and InvokeHTTP is working without a GFF,
as you said. If I connect the retry to it, will it automatically back off? Or,
would I have to manually include a wait processor or the like?
From: David Gallagher
Sent: Thursday
I am using Nifi 2.3.0. I want to use an InvokeHTTP processor to poll a REST
endpoint every 30s and get a flowfile from the response containing the payload
(it'll be protobuf). I've found that I have to use a GenerateFlowFile as input
to InvokeHTTP if I want the output on the Response relationshi
We will want to clean that up, too.
Regards,
-- Mike
On Mon, Nov 25, 2024 at 2:43 PM Pongrácz István
mailto:pongracz.ist...@gmail.com>> wrote:
You mean:
bash $NIFIPATH/bin/nifi.sh install
?
Check your path, it should be there and the install command will work.
Cheers,
István
2024
Hi - I'm working with the 2.0 release. Instructions reference 'Installing as a
Service' but install doesn't seem to be an option in the script:
Usage nifi.sh
{start|stop|decommission|run|restart|status|cluster-status|diagnostics|status-history|set-sensitive-properties-algorithm|set-sensitive-pro
vgtLMCe4M_K5Sv2egM0E0XU.MEYCIQDhjEKUCK1Xw3PC1FOGE1K5z2T7Usun_8-agklWYmr9_gIhAMClOvkkrTFFT4RKGF4JhjWs2tkTIajp1bHveDYks91K>
Please let us know if it fixed the issue or you still face it, please share
your nifi logs if there is something still wrong with this recent build!
Thanks, regards,
Arpad
On Mon, Oct 28, 2024 at 6:51 PM David
Hi - I've built a custom Python processor for the 2.0.0.M4 release. I'm finding
that I can't have two of the same processor running at the same time (in
different process groups). They seem to deadlock each other or something. I'm
not sure what to look at to see what's causing the problem. Is th
Hi Luis! You may want to explore building your own Python nifi processor, it
lets you work with flowfiles directly. Joe Witt created a good guide for it:
https://devcenter.datavolo.io/tutorials/build-python-transform/#0
Also as Stephen said JOLT is really handy and fast if you can go that way. I
Email
Protection<https://getskout.com/emailprotection/>
Are you using the docker image or a plain install?
What version of Python do you have? I believe it requires 3.11.something or
newer.
Thanks
On Sat, Sep 28, 2024 at 12:53 PM David Gallagher
mailto:dgallag...@cleverdevices.com>>
Email
Protection<https://getskout.com/emailprotection/>
Are you using the docker image or a plain install?
What version of Python do you have? I believe it requires 3.11.something or
newer.
Thanks
On Sat, Sep 28, 2024 at 12:53 PM David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
ky.com/link?domain=devcenter.datavolo.io&t=h.eJwtzUFuwjAQQNGrIHfb2CQkNmHFVQZ7krhMMsgep0Kody9Wu37S_y9VEqnLQS0ij3wxJuDucRNMOoDAzsQ6spEinCJQNrcSKTSPpyy8NZJgyxOn1Xwc1edB3Wsp37lI4wl3TM07Fz1mE2YggnnBdP2Tf9CeV3Pqp3EY-1uY7NH3Z9eCtcMZsHXt5ELXmdZ1bujt0I3anuoI6-iLUX9Hkeu8QqRaqhTetBWin1-MykfJ.MEUCIHS975J_4AxsMvzH4KZR8uKcszsYTP-wLybGs0kPHEm6AiEA8Bm_MYV2a-3jV9ierZgeI2Y4h4O6HLZeJozl17I2xKo>
Thanks
Joe
On Sat, Sep 28,
Hi - I'm playing around with creating a native Python processor in M4. I've
been able to get them to show up as new Processor options, but they don't have
any Output relationships. /python/extensions is empty in M4, but I moved one of
the extensions from M2 into the directory and same problem. I
ay to do this
automatically, unfortunately it doesn't look like there's a PUT option for
state.
Thanks,
Dave
-Original Message-----
From: David Gallagher
Sent: Thursday, February 13, 2020 3:32 PM
To: users@nifi.apache.org
Subject: RE: NiFi 1.8.0 to 1.11.1 upgrade issue
Yes, re
7172132511735269&sdata=Iy9%2FN5O
> cRxfE7E78NOFWkxj6xX3uGKX7fseKPvFSqI0%3D&reserved=0
>
> On Thu, Feb 13, 2020 at 11:55 AM David Gallagher
> wrote:
> >
> > @Bryan Bende, to add to this it looks like 'All' was an option in 1.8.0 but
> > is no longer an o
it can only be scheduled on the primary node,
and maybe you previously had it schedule on all nodes?
On Thu, Feb 13, 2020 at 9:57 AM David Gallagher
wrote:
>
> Hi Bryan -
>
> The new version of the flow came from a development environment. Thinking
> back on it, I upgraded that de
ou previously had it schedule on all nodes?
On Thu, Feb 13, 2020 at 9:57 AM David Gallagher
wrote:
>
> Hi Bryan -
>
> The new version of the flow came from a development environment. Thinking
> back on it, I upgraded that development environment from 1.8 -> 1.11.1 first
>
Re: NiFi 1.8.0 to 1.11.1 upgrade issue
Hello,
Can you elaborate on what step 3 means? Where did this new version come from
that was imported to registry?
-Bryan
On Thu, Feb 13, 2020 at 9:11 AM David Gallagher
wrote:
>
> Hi – I’m currently converting our flows from 1.8.0 to 1.11.1, w
Hi - I'm currently converting our flows from 1.8.0 to 1.11.1, with a parallel
upgrade from registry 0.2.0 to registry 0.5.0. Currently stuck due to this set
of steps:
1. Upgraded registry from 0.2.0 to 0.5.0
2. Upgraded NiFi from 1.8.0 to 1.11.1
3. Imported a new version of a flow to t
Hi - we have a setup with a single Development registry and multiple Production
NiFi Registries. There's an air gap between the environments. Our intent is to
work on flows in the Development environment, export the flows, and then deploy
them to production environments. We're finding now that t
Bryan / Pierre - problem was definitely on my end. Can confirm that state is
preserved during upgrade. Thanks for your help!
-Original Message-
From: David Gallagher
Sent: Thursday, February 6, 2020 11:44 AM
To: users@nifi.apache.org; Bryan Bende
Subject: RE: Loss of state on
Actually this is probably on me - In the config file I specified the path to
state as ..\..\state, not ..\state. I'll be able to test again in a little
while and see.
-Original Message-
From: David Gallagher
Sent: Thursday, February 6, 2020 10:07 AM
To: users@nifi.apache.org;
ubject: Re: Loss of state on QueryDatabaseTable during upgrade
>
>
>
> Hi David,
>
>
>
> How did you perform the upgrade? Do you have a cluster with external ZK, a
> cluster with embedded ZK, or a single node?
>
>
>
> Thanks,
>
> Pierre
>
>
>
>
upgrade? Do you have a cluster with external ZK, a
cluster with embedded ZK, or a single node?
Thanks,
Pierre
Le jeu. 6 févr. 2020 à 09:15, David Gallagher
mailto:dgallag...@cleverdevices.com>> a écrit :
Hi – I’m upgrading flows from 1.8.0 to 1.11.1. Everything is working nicely,
but it looks
Hi - I'm upgrading flows from 1.8.0 to 1.11.1. Everything is working nicely,
but it looks like I may have lost the State values of my QueryDatabaseTable
processors (I'm using a Maximum ID). I've looked in the state file
(\state\local\snapshot) and the value is there, but it's not showing in the
before PutDatabaseRecord
to remove the unindexed / unnecessary fields, then the generated statement
should only include the necessary fields and be able to take advantage of any
indexes.
Regards,
Matt
On Fri, Sep 20, 2019 at 4:55 PM David Gallagher
wrote:
>
> Hi – I’m looking to
Hi - I'm looking to remove records from a table using the results returned from
a QueryDatabaseTable processor. I want to do this in batches because it's going
to require a lot of deletes, and flowfile per record would be too slow. I've
tried using the PutDatabaseRecord processor, but the DELETE
trying to change the same
key because the Wait sees the value as it flip/flops.
Thanks
Shawn
From: David Gallagher
mailto:dgallag...@cleverdevices.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
mailto:users@nifi.apache.org>>
Date: Wednesday,
Hi - I have a gate set up with a Wait processor listening for a target signal
count of '1'. I have several Notify processors that set the value to '1' when
the system is able to take work. If there is work waiting in the queue while
the 'gate' is closed, I sometimes get this error:
Wait[id=016b
erved=0>
On Jun 7, 2019, at 2:58 PM, David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
Hi Mark - attached.
Thanks,
Dave
From: Mark Payne mailto:marka...@hotmail.com>>
Sent: Thursday, June 6, 2019 12:06 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org
uot;type":"long",
"logicalType":"timestamp-millis"} }
Thanks
-Mark
On Jun 6, 2019, at 11:21 AM, David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
Hi – I’ve got an incoming JSON message with a timestamp that I want to
validate. I have a Validat
illis" only for a long field, not
for a String field.
So I think you'd need to use:
{"name": "activationDate", "type": { "type":"long",
"logicalType":"timestamp-millis"} }
Thanks
-Mark
On Jun 6, 2019, at 11:2
Hi - I've got an incoming JSON message with a timestamp that I want to
validate. I have a ValidateRecord (1.8.0) processor set up with a
JSONPathReader for the message. The relevant field is defined in the schema as:
{"name": "activationDate", "type": { "type":"string",
"logicalType":"timestamp
everything
will get stuck at the Wait processor as the counter will start at 0 (closed)
when it needs to be 1 (open).
Thanks,
Evan
On Tue, May 28, 2019 at 8:44 PM David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
Thanks, Shawn – I had seen your post to the mailing list and
6&sdata=5EhJf0%2FvM%2F4VPKe0z916zcHyVx5UpgzCttu2yUZbCbk%3D&reserved=0>
Thanks
Shawn
Sent from my iPhone
On May 17, 2019, at 1:59 PM, David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
Hi Joe – thanks for answering. I can actually live with the message ack at the
momen
munity has been talking about.
Thanks
On Fri, May 17, 2019 at 12:30 PM David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
Hi – I have a requirement to read one message off RabbitMQ and fully process
that message before I take another message off the queue. I found some great
gui
Hi - I have a requirement to read one message off RabbitMQ and fully process
that message before I take another message off the queue. I found some great
guidance to setting up a 'traffic light' pattern here
(https://gist.github.com/ijokarumawak/9e1a4855934f2bb9661f88ca625bd244), but it
won't w
sider the full user experience here. I'm not certain if
this processor (ExecuteStreamCommand) is like this but hopefully that makes
sense.
In short, the behavior you're describing should be treated like a bug and the
processor logic should be improved.
Thanks
On Fri, Apr 26, 2019 at
Hi - I have an issue where certain processors may hang due to misconfiguration.
For example, an ExecuteStreamCommand with an incorrect path for the executable.
In this scenario, the flowfile waits in the queue for the processor and cannot
proceed. It generates an error message, both visually and
but its really just a preference.
On Wed, Apr 24, 2019 at 12:16 PM David Gallagher
wrote:
>
> Hi - I'm working on a flow with lots of processors and tons of reasons that
> something could fail (failure to match schema, bad return codes, file not
> found, etc). The inten
Hi - I'm working on a flow with lots of processors and tons of reasons that
something could fail (failure to match schema, bad return codes, file not
found, etc). The intent is to send these failures to a ProduceAMQP processor
with an appropriate routing key and error message. It makes the flow
for this to work (your suspicion was correct!). So we recommend doing that and
securing the key using OS filesystem permissions (ie, only the run as user
should have read access to that ssk key file)
On Thu, Mar 28, 2019 at 3:54 PM David Gallagher
wrote:
>
> Thanks for the reply! I've
rds, if on the linux host as
the user that nifi registry runs as, you should be able to push using ssh+git
instead of username / password. then NiFi Registry will be able to do the same.
On Thu, Mar 28, 2019 at 12:09 PM David Gallagher
wrote:
>
> Hi – does anyone know if it is possi
Hi - does anyone know if it is possible to somehow encrypt the providers.xml
file? I'm using the Git persistence provider and want to store the remote user
and password there, but I don't want to leave that value in there unencrypted
if I don't have to. I downloaded the nifi-toolkit, but I don't
t; obviously not going to exist, but it is also unclear how to select the
>>> > appropriate one. What if there were multiple DBCP connection pools
>>> > visible to where the versioned flow is being imported? There would be
>>> > no way to know which one to use.
&
then use that one, but its still hard to know if this is really
> the right service. What if it was for a different database and someone
> didn't realize?
>
> -Bryan
>
> On Thu, Nov 29, 2018 at 12:34 PM David Gallagher
> wrote:
> >
> > Hi - I'm using nifi-1
Hi - I'm using nifi-1.7.1 and nifi-registry-0.2.0. I'd like to have 'global'
DBCPConnectionPool instances at the Nifi Flow level, then import flows from
the registry and have them use the global pools, e.g. in a PutDatabaseRecord
processor. When I try that, though, the processor is invalid and
Hi – not sure if it helps, but you can set a static hostname in your
docker-compose.
Thanks,
Dave
From: Juan Pablo Gardella
Sent: Sunday, September 23, 2018 3:43 PM
To: users@nifi.apache.org
Subject: Nifi with docker and LDAP
Hi all,
I'm using Nifi with docker and it's secure.
I'm facing an
Just following up for the benefit of others - git works just fine with a Linux
VM. Thanks for all your help!
Dave
-Original Message-
From: David Gallagher
Sent: Monday, September 17, 2018 1:38 PM
To: users@nifi.apache.org
Subject: RE: Nifi Registry + Git: target registry ignores git
it actually not there?
[1]
https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#system-requirements
On Wed, Sep 12, 2018 at 6:40 PM, David Gallagher
wrote:
> Hi – ok, now I’ve got a different (but maybe related?) problem. I’ve
> set up the nifi registry on a remote s
didn’t exist
because the bucket id and flow id are not the same.
On Sep 11, 2018, at 4:55 PM, David Gallagher
mailto:dgallag...@cleverdevices.com>> wrote:
Ok, Last question on this one (promise!). I’ve set up the dev / git repository
and it’s working great. I can export flows from de
hout foiloowing the right steps, or
manually modifying the persistence files outside of the NiFi Registry API/UI
I hope this helps you get on the right track. Let me know if you need further
assistance.
Cheers,
Kevin
From: David Gallagher
mailto:dgallag...@cleverdevices.com>>
Reply-To:
appen
before. My guess would be it could be caused due to a configuration error, a
change from one provider to another without foiloowing the right steps, or
manually modifying the persistence files outside of the NiFi Registry API/UI
I hope this helps you get on the right track. Let me know if you nee
ers@nifi.apache.org>>
Subject: Re: Nifi Registry + Git: target registry ignores git flows
It probably has to do with the database the registry used not being migrated to
your new environment. Try setting up a snapshot of that too.
On Sat, Sep 8, 2018 at 1:06 PM David Gallagher
mailto:dgal
Hi - I have a dev registry server (1.7.1) set up with git, and am trying to use
git bundle to move the buckets and flows to a test registry instance (I'm using
git bundle because there's going to be an air gap in production). Everything
looks fine, but when I start the production registry it ign
, Aug 24, 2018 at 10:32 AM, David Gallagher
wrote:
> Hi Matt - I tried changing the order of the union in the Output schema, and
> also tried using a default value. Nothing worked, I still got an 'unmapped
> field' message and could not start the component. But, it looks like
value explicitly.
Prior to NiFi 1.6.0 this was a bug [1], please see the Jira for more details
(and an example).
Regards,
Matt
[1] https://issues.apache.org/jira/browse/NIFI-4893
On Thu, Aug 23, 2018 at 2:29 PM David Gallagher
wrote:
>
> Hi Andy – this is the schema we’re getting from
t 23, 2018 1:28 PM
To: users@nifi.apache.org
Subject: Re: ConvertAvroSchema with unmapped fields
Hi Dave,
Can you provide the two schemas (redact anything necessary). There is a way to
specify an “optional” field [1] by setting the type to an array of null and the
type you support. You can
Hi - I've got a scenario where I'm trying to convert the implicit AVRO schema
associated with a database record to a different AVRO schema. I'm trying to use
the ConvertAvroSchema processor, but it won't validate because there is an
'unmapped' field that exists in the second schema but not the f
61 matches
Mail list logo
