ng that you are receiving a pretty high
> rate if it is at the point of containing over 10K entries
> with a 10 minute timeout. If you are not seeing that many requests, then
> there may be something else going on there.
>
> Thanks
> -Mark
>
>
>
> On Feb 22, 2017, at 10:31 AM
your flow where you
> are not responding to the request?
>
> Thanks
> -Mark
>
>
> > On Feb 22, 2017, at 9:58 AM, James McMahon <jsmcmah...@gmail.com> wrote:
> >
> > I am getting the following errors when my users attempt to use curl or
> python to post to my Handle
I am getting the following errors when my users attempt to use curl or
python to post to my HandleHttpRequest processor (cannot export actual
messages, must select pieces and retype here):
WARNING
Received request from [IP address is here] but could not process it because
too many requests are
gt; [1] http://stackoverflow.com/a/30405947/70465
> [2] http://security.stackexchange.com/a/20847/16485
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
>
> standard manner via their REST API. I don't believe we have any
> generic support for this yet nor do I think the StandardHttpContentMap
> does this. It would be a great thing to create a JIRA for.
>
> Thanks
> Joe
>
> On Thu, Feb 16, 2017 at 6:48 AM, James McMahon <jsm
> Configuration+Management+of+Flows
> [2] https://github.com/aperepel/nifi-api-deploy
> [3] https://github.com/ijokarumawak/nifi-deploy-process-group
>
>
> On Thu, Feb 16, 2017 at 10:05 AM, James McMahon <jsmcmah...@gmail.com>
> wrote:
> > Good morning. Our team has
I am unable to get a response from my NiFi server HandleHttpRequest
processor when I issue a POST from Python code on a remote box. I suspect
my configuration of certs is somehow missing a vital piece of the puzzle,
though that seems contrary to my configuration of HandleHttpRequest (No
Good morning. Our team has been discussing backup, versioning, and restore
strategies to manage the backups we create of our NiFi flow.xml.gz files. I
have a few questions about this.
Currently I manually execute a daily backup of my NiFi workflow from the
Controller Settings UI, General tab,
CE 3C6E F65B 2F7D EF69
>
> On Feb 13, 2017, at 5:19 AM, James McMahon <jsmcmah...@gmail.com> wrote:
>
> Thank you very much Bryan. I will look more closely for archives, and into
> transfers for the time being using flow.xml.gz. I also hope to see if
> there's a way to be a
could just
> > connect your HandleHttpRequest to a different path through the flow
> > that returns an unavailable, and then when everything is back to
> > normal connect it back to the regular path?
> >
> >
> >
> > On Mon, Feb 13, 2017 at 8:15 AM, Ja
Hello. I have a suite of applications that use Http POSTs to push content
to NiFi "on [their] demand". My HandleHttpRequest processor works for the
most part, employing an SSL Context Service and a StandardHttpContextMap.
Yesterday apps were unable to post. We are in development and test, and so
Good morning. How can we export and import our workflow backup snapshots
from one established NiFi server instance to a new instance of NiFi stood
up on a new server?
Also, how does one restore a previous version of a workflow backup snapshot
created from the administrative tool?
Thank you. -Jim
Good morning. The first processor in my workflow is a HandleHttpRequest.
How do we set up to send a HandleHttpResponse if that processor is stopped
and so not in a running state?
Thank you. -Jim Mc.
case, it is performing a very
> simple replacement of text that is returned to the caller.
>
> Please let us know if you have any additional questions.
>
> --aldrin
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/
> Example+Dataflow+Templates
>
> On Tue,
>
> [1] https://community.hortonworks.com/questions/81291/nifi-
> executescript-processor-error-using-string-in.html
>
>
> On Thu, Feb 2, 2017 at 5:38 PM, James McMahon <jsmcmah...@gmail.com>
> wrote:
> > I have a flowfile that has tagged character information I
Good evening. I have a number of customer applications that will be posting
content to a NiFi HandleHTTPRequest processor. These apps need an http
reply so that they know status. I can't find in the configuration any way
to do that. I know that I must be overlooking what must be a commonplace
r.
>
> Regards,
> Matt
>
> [1] https://docs.oracle.com/javase/8/docs/api/java/io/DataInputStream.html
>
>
> On Thu, Feb 2, 2017 at 6:19 PM, James McMahon <jsmcmah...@gmail.com>
> wrote:
> > This is very helpful Russell, but in my case each file
> dealing with, then branch to different *ExecuteScript* processors rigged
> to different Python scripts?
>
> Hope this comment is helpful.
>
>
> On 02/02/2017 03:38 PM, James McMahon wrote:
>
> I have a flowfile that has tagged character information I need to get at
> throughou
I have a flowfile that has tagged character information I need to get at
throughout the first few sections of the file. I need to use regex in
python to select some of those values and to transform others. I am using
an ExecuteScript processor to execute my python code. Here is my approach:
= =
esto
> alopre...@apache.org
> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Jan 16, 2017, at 2:56 PM, James McMahon <jsmcmah...@gmail.com> wrote:
>
> In the example you offer Andy, it se
I am having great difficulty getting a stream callback in python code to
simply add a new metadata attribute property with value to a flowfile. I
execute my code within an ExecuteScript processor. My incoming flowfile has
a number of complex attributes that contain embedded data of high interest
ass an instance of a class extending
> InputStreamCallback.
> Here is an example that I derived from NiFi test case:
> https://gist.github.com/ijokarumawak/1df6d34cd1b2861eb6b7432ee7245ccd
>
> Hope this helps.
>
> Thanks!
> Koji
>
> On Mon, Jan 16, 2017 at 9:02 AM, James McM
Good evening. I often create a custom flowfile payload using a Python
Stream Callback. I define a Python dictionary using
result = {}
and I selectively build my payload as a JSON object, populating it with
flowfile metadata attributes, like so
result['comment'] =
other files, such as multiple journal files,
> snapshot.part files, etc. that are temporary
> artifacts generated in order to provide better performance and ensure
> reliability across system crashes/restarts.
>
> The wali.lock is simply there to ensure that we don't start NiFi twice and
&g
is
> written to ./state/local but this can be configured
> in the conf/state-management.xml. So if you want to be really sure that
> you don't lose the information, you could
> potentially change the location to some place that has a RAID
> configuration for redundancy.
>
> Thanks
fluence/display/NIFI/Upgrading+NiFi
>
> Thanks,
>
> Bryam
>
>
> On Thu, Dec 15, 2016 at 6:49 AM, James McMahon <jsmcmah...@gmail.com>
> wrote:
>
>> NiFi 0.6.1 was the first NiFi baseline I installed in my local data
>> enterprise. I now find that there are
y thoughts.
On Thu, Dec 15, 2016 at 6:07 AM, James McMahon <jsmcmah...@gmail.com> wrote:
> As a representative example using a random Unicode character at the front
> and the back of a notional file name, [U+0932][U+0932][U+0932]+123
> ABC[U+07C1]
>
> On Wed, Dec 14, 2016 at
NiFi 0.6.1 was the first NiFi baseline I installed in my local data
enterprise. I now find that there are refinements in v0.7 that I require,
and so I must install/upgrade my NiFi baseline. I've never done this and
need to better understand the steps required to properly upgrade.
I have a base
As a representative example using a random Unicode character at the front
and the back of a notional file name, [U+0932][U+0932][U+0932]+123
ABC[U+07C1]
On Wed, Dec 14, 2016 at 7:22 PM, James McMahon <jsmcmah...@gmail.com> wrote:
> Yes indeed Joe, it appears from the logs that ther
vide some interesting details as well.
>
> Thanks
> Joe
>
> On Wed, Dec 14, 2016 at 5:03 PM, James McMahon <jsmcmah...@gmail.com>
> wrote:
> > I am using NiFi 0.6.1. I am trying to use GetFile to read in a large
> series
> > of files I have preprocessed outside
I am using NiFi 0.6.1. I am trying to use GetFile to read in a large series
of files I have preprocessed outside of NiFi from zip files using bash
shell commands. GetFile is throwing errors on many of these files because
the files contain embedded spaces. Is there a way to tell NiFi to handle
each
you full control and complete isolation from every other component
> in NiFi.
>
> Let me know what you think so we can guide you thru the process.
>
> Cheers
> Oleg
>
> On Dec 14, 2016, at 2:18 PM, James McMahon <jsmcmah...@gmail.com> wrote:
>
> Oleg, I am bound to
to fix in the processor but please do share the
> full nifi-app.log entry for this.
>
> Thanks
> Joe
>
> On Wed, Dec 14, 2016 at 10:47 AM, James McMahon <jsmcmah...@gmail.com>
> wrote:
>
> Hello. Am wondering if anyone knows how to overcome a challenge with
> unmappable
ting the exception
> loose thus the framework detects the issue and rollsback the session
> and yields the processor.
>
> Likely an easy thing to fix in the processor but please do share the
> full nifi-app.log entry for this.
>
> Thanks
> Joe
>
> On Wed, Dec 14, 2016 at 10:4
ent (even though I believe that you may have configured everything
> correctly, but on the older version of Rabbit).
>
> So, anyway, I’ll file these two bugs while your will try to convince the
> powerful men you’re working for to upgrade RabbitMQ ;)
>
> I’ll follow up with JIRA numb
ava:99)
> at org.apache.nifi.amqp.processors.PublishAMQP.rendezvousWithAmqp(
> PublishAMQP.java:137)
>
> Cheers
> Oleg
>
> On Dec 8, 2016, at 1:04 PM, Oleg Zhurakousky <ozhurakou...@hortonworks.com>
> wrote:
>
> Got it, let me dig in, i got time now
>
> On De
sure I am clear on the second part of your
> explanation. It appears you are saying that when you bound the Exchange to
> a legit routing key all works as expected (in NiFi), but then what?
> Could you please clarify?
>
> Sorry about that.
> Oleg
>
> On Dec 8, 2016, at 11:41
. Nothing shows up in the
exchange or in the queue.
Jim
On Thu, Dec 8, 2016 at 8:26 AM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:
> Fair enough
>
> On Dec 8, 2016, at 8:16 AM, James McMahon <jsmcmah...@gmail.com> wrote:
>
> I'm a captive audience in
s
> Oleg
>
> On Dec 8, 2016, at 8:00 AM, James McMahon <jsmcmah...@gmail.com> wrote:
>
> Yes absolutely Oleg. I will try to figure out how to make my rabbitMQ
> queues explicitly active. I'll need to dig into that because it is not
> plainly obvious to me how / if that ne
> Aside from “closing” failure message, do you see any other errors or stack
> traces in the logs?
> Meanwhile I’ll try to dig more with what I have?
>
> Oleg
>
> On Dec 8, 2016, at 7:13 AM, James McMahon <jsmcmah...@gmail.com> wrote:
>
> Hello I am trying to us
Hello I am trying to use a Publish AMQP step in my NiFi workflow. I
generate a flow file with random text content using Generate Flow File, and
then send to a rabbitMQ Exchange named nifiAbcX using Publish AMQP. My
exchange is bound to queue AbcQ by key **.Abc.**, and my virtual host is
nifiVhost.
t; one to provide the web service response? If not, look at
> "Hello_NiFi_Web_Service" template on the Example Dataflow Templates
> <https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates>
> page.
>
> Regards,
> Joe S
>
> On Wed, Sep
ith HandleHttpRequest and
> HandleHttpResponse. There is an example of this under
> Hello_NiFi_Web_Service [1].
>
> Regards,
> Matt
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/
> Example+Dataflow+Templates
>
> On Sun, Sep 4, 2016 at 7:40 AM, James McMahon
I have a series of applications, each of which I need to redesign so that
they post raw data to NiFi. Is there an example showing how to to construct
and execute RESTful calls that push data to NiFi? My assumption is that I
would establish a ListenHTTP processor in my workflow to receive the
Greetings Chris. I have an idea that *may* work for you to get the file
data into a flow file attribute. Word of caution though: I am relatively
new to NiFi, so this may be harder than it needs to be . If nothing
else, perhaps it will give you some food for thought.
Have you seen this?
ess is the key point.
>
> This is a great place to ask such questions. Welcome to the
> community. Please do have a read through the user guide if you've not
> already as that might also help to understand some of these
> considerations.
>
> Thanks
> Joe
>
> On Wed,
*Hello. Relatively new NiFi developer here. I have a question borne from my
lack of experience. Can anyone offer any guidance? *
*If a flowfile fails in a processor - let's say a Put File fails because of
underlying disk device failure - how do I replay the original flowfile(s)?
I need to
401 - 447 of 447 matches
Mail list logo
