A "proxy" question from the irc channel

2015-11-08 Thread Tony Kurc
" I'd appreciate it if someone could tell me how to easily change a flow with JSON lines like {JSON} to {"x"={JSON}} . The ReplaceText processor seems like the right thing, but when I add almost anything to the ReplacementValue field to change the output, it causes problems and doesn't include th

Re: A "proxy" question from the irc channel

2015-11-08 Thread Tony Kurc
essor, calling a simple BASH script which used "read" to loop through each line of the FlowFile. On Sun, Nov 8, 2015 at 8:09 AM, Tony Kurc wrote: > " I'd appreciate it if someone could tell me how to easily change a flow > with JSON > lines like {JSON}

Re: A "proxy" question from the irc channel

2015-11-08 Thread Tony Kurc
> some time ago, but is the IRC channel's information available anywhere on > > the site? Wasn't able to find anything immediately obvious. > > > > On Sun, Nov 8, 2015 at 12:19 PM, Tony Kurc wrote: > >> > >> and a proxy from irc answer: > >> &

Re: A "proxy" question from the irc channel

2015-11-08 Thread Tony Kurc
ill has one with #user and #dev channels, seems to work pretty well and > has various integrations with other tools (email, GitHub, Jira, etc.) > > Sent from my iPhone > > On Nov 8, 2015, at 12:41 PM, Tony Kurc wrote: > > The irc channel tends to me be or Josh Elser redirecting p

Re: Nifi service fail to start - Removed custom processor

2015-11-10 Thread Tony Kurc
My approach is to modify conf/flow.xml.gz file by hand. There may be bit of a more user friendly approach. gunzip flow.xml.gz vim flow.xml gzip flow.xml On Tue, Nov 10, 2015 at 6:05 PM, Chakrader Dewaragatla < chakrader.dewaraga...@lifelock.com> wrote: > Hi - We developed few custom processors a

Re: Nifi service fail to start - Removed custom processor

2015-11-10 Thread Tony Kurc
And I overlooked one important piece, my approach would be to remove the processors you no longer 'installed' On Nov 10, 2015 6:13 PM, "Tony Kurc" wrote: > My approach is to modify conf/flow.xml.gz file by hand. There may be bit > of a more user friendly approach. &g

Re: Is nifi a good fit for this use case?

2015-11-10 Thread Tony Kurc
Joe, This might be a good basis for a blog post or page on the wiki. On Tue, Nov 10, 2015 at 9:28 PM, Joe Witt wrote: > Darren, > > In short, yes I think NiFi can handle such a case in a generic sense quite > well. > > Read on for the longer response... > > NiFi can process extremely large data,

another proxied question from irc, about machine specs

2015-11-16 Thread Tony Kurc
15:59 < userX> Hey, is there a recommended system specs for a server running nifi -- I am going to our sys admin to get a server virtualized, I need some specs going in to request. I can't seem to find anything hard in the documentation

another proxied question from irc

2015-11-23 Thread Tony Kurc
12:56 < userX> Hey, does anyone have experience setting up the SSL Context in nifi? I realize my knowledge of how certs/ssl is limited, but I was hoping someone could point me in the correct direction. I am getting a "No X509TrustManager available

Re: [announce] open source NATS messaging and Groovy scripting processors for NiFi

2015-11-27 Thread Tony Kurc
It looks like there is a MIT licensed java client underway by the nats.io crew which looks to be early alpha: https://github.com/nats-io/jnats On Fri, Nov 27, 2015 at 10:06 PM, Joe Witt wrote: > Matt, > > Any chance you'd be interested in turning this into a PR or patch for > formal submission?

Re: Mirroring nifi lists on gmane

2015-12-04 Thread Tony Kurc
Similar to nabble, right? On Dec 4, 2015 10:18 PM, "Joe Witt" wrote: > Is anyone aware of any reasons not to do this? > > > On Fri, Dec 4, 2015 at 8:59 AM, Nigel Jones > wrote: > > Would it be ok if I asked gmane to mirror these nifi lists on the gmane > nntp > > server? > > > > I personally fin

Re: Enhance AbstractAWSProcessor to allow S3 Fetch Object to allow cross account access

2015-12-22 Thread Tony Kurc
I can't assign tickets to you, but I say comment on the ticket that you're working it, and some one can assign it to you later On Dec 22, 2015 7:17 PM, "M Singh" wrote: > Hi: > > I am working on a project that requires access to AWS S3 across accounts. > I am using Nifi version 0.4.1. I've added

Re: Patch Submitted - Re: Enhance AbstractAWSProcessor to allow S3 Fetch Object to allow cross account access

2015-12-27 Thread Tony Kurc
Mans, I'll assign this ticket to myself and start digging in on a review! On Sun, Dec 27, 2015 at 6:38 PM, M Singh wrote: > Hi: > > I have submitted a patch for this issue > https://issues.apache.org/jira/browse/NIFI-1325. > > I am a newbie to the world of open source contribution and have follo

Re: Nifi - AW Component - Feedback on changing AWS Abstract Component to use CredentialsProvider rather than Credentials

2015-12-30 Thread Tony Kurc
ss (nifi-1325). While exploring the options with Tony Kurc to > prevent the expiration role based session credentials, I found that the > current nifi aws processors (S3, SNS, SQS) all use credentials directly. > There is an option for all these components to use AWSCredentialsProvider > wh

Re: ListS3 processor?

2016-02-01 Thread Tony Kurc
Joe, There is the possibility of people using S3 compatible "block stores" which don't have the same notification services, I'd say leave them open. Tony On Mon, Feb 1, 2016 at 11:30 AM, Joe Skora wrote: > Agreed, excellent write up. > > When this thread started I had forgotten about prior disc

Re: Binary Build for nifi-0.5.0-RC3

2016-02-15 Thread Tony Kurc
The binaries are staged here: https://dist.apache.org/repos/dist/dev/nifi/nifi-0.5.0/ After the release, they will be moved and available via the web page. On Mon, Feb 15, 2016 at 10:31 PM, indus well wrote: > Hello NiFi: > > I have troubles building nifi-0.5.0-RC3 from source. Is there a build

Re: Authentication error - access denied exporting template

2016-02-22 Thread Tony Kurc
If all goes well, we're likely to put together a release candidate tonight, the voting period is 72 hours. Best case is 3.5 days. On Mon, Feb 22, 2016 at 10:18 AM, Conrad Crampton < conrad.cramp...@secdata.com> wrote: > Matt, > Thanks for the update. I trawled the user list archive for anything >

Re: Nifi KEYS version 0.5.0

2016-02-23 Thread Tony Kurc
Aldrin, It looks like they KEYSfiles here https://dist.apache.org/repos/dist/release/nifi/KEYS differs from ( https://dist.apache.org/repos/dist/dev/nifi/KEYS and https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=blob;f=KEYS;h=22fcb3748a2570e66b84661fc6ad15267ca324ec;hb=24a77755de8d99467a88

Re: CSV/delimited to Parquet conversion via Nifi

2016-03-22 Thread Tony Kurc
Interesting question. A couple discussion points: If we start doing a processor for each of these conversions, it may become unwieldy (P(x,2) processors, where x is number of data formats?) I'd say maybe a more general ConvertFormat processor may be appropriate, but then configuration and code comp

Re: CSV/delimited to Parquet conversion via Nifi

2016-03-22 Thread Tony Kurc
Martin Fowler has blogged about some ideas like this (w.r.t. >> abstracting translation logic), Tika has done some of this but AFAIK its >> focus is on extraction not transformation. In any case, we could certainly >> capture the idea in NiFi. >> >> Regards, >&g

Re: Can NiFi be deployed as a .war file?

2016-05-19 Thread Tony Kurc
Hi! A similar question was asked on the dev mailing list, this was the answer given there, which may answer this for you: http://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3C3B44DF8F-B111-4EAC-8D5F-6BEE8557584F%40gmail.com On May 19, 2016 9:50 PM, "Phu-Thien Tran" wrote: > > We have

Re: Can NiFi be deployed as a .war file?

2016-05-19 Thread Tony Kurc
Sorry, I failed to paste correctly, here is what I intended to link to: http://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3C3B44DF8F-B111-4EAC-8D5F-6BEE8557584F%40gmail.com%3E On May 19, 2016 9:54 PM, "Tony Kurc" wrote: > Hi! A similar question was asked on the dev

Re: Will Nifi eventually become Hortonified?

2016-06-20 Thread Tony Kurc
I think that your "no" is a no to the subject line and not the question in the body. The source code for Apache NiFi has been available since the start of incubation with Apache. Others may have forks of the code (the Apache license v2 allows this), but the canonical repository for Apache NiFi is h

Re: Will Nifi eventually become Hortonified?

2016-06-21 Thread Tony Kurc
n of nifi, > right? > On Jun 20, 2016 3:35 PM, "Tony Kurc" wrote: > >> I think that your "no" is a no to the subject line and not the question >> in the body. The source code for Apache NiFi has been available since the >> start of incubation with Apa

RE: Issue writing file (~50mb) to azure data lake with Nifi

2016-09-06 Thread Tony Kurc
I didn't see the test script that worked in the source code - did I miss it, or is it not in the tree? On Sep 6, 2016 3:17 PM, "Kumiko Yada" wrote: > Joe, > > > > Here is the log (there was no callstack related to this error) and code, > https://github.com/kyada1/dl_sdkworkaround/tree/master/ >

RE: Issue writing file (~50mb) to azure data lake with Nifi

2016-09-06 Thread Tony Kurc
processor is > working for a small size file. > > > > Thanks > > Kumiko > > > > *From:* Tony Kurc [mailto:trk...@gmail.com] > *Sent:* Tuesday, September 6, 2016 2:32 PM > *To:* users@nifi.apache.org > *Cc:* Joe Witt ; #Operations Automation and Tools

RE: Issue writing file (~50mb) to azure data lake with Nifi

2016-09-06 Thread Tony Kurc
inal static *String *LOCATION *= *""*; > *final static *String *TENANT_ID *= *""*; > *final static *String *SUBSCRIPTION_ID *= *""*; > *final static *String *CLIENT_ID *= *""*; > *final static *String *CLIENT_SECRET *= *""*; */* > > >

Re: Issue writing file (~50mb) to azure data lake with Nifi

2016-09-07 Thread Tony Kurc
e same SDK code; however, it’s > failing when the custom processor is tried to uploaded 50 MB file. > > > > Thanks > > Kumiko > > > > *From:* Tony Kurc [mailto:trk...@gmail.com] > *Sent:* Tuesday, September 6, 2016 3:02 PM > *To:* Kumiko Yada > *Cc:* us

Re: Frontend for Nifi support / mailing list

2016-10-27 Thread Tony Kurc
I will look into setting it up today On Oct 27, 2016 5:04 AM, "Davy De Waele" wrote: > Hi, > > Is there a procedure for Nifi users to get nabble posting access to the > Nifi mailing list ? > I can post (obviously) via email, but would prefer to use the nabble web > UI. > > Both myself and a coll

Re: Frontend for Nifi support / mailing list

2016-10-27 Thread Tony Kurc
List has been created on nabble, it will be empty until messages start coming in. I don't yet see a way to backfill with history. I'll get this link on https://nifi.apache.org/mailing_lists.html in a bit. http://apache-nifi-users-list.2361937.n4.nabble.com/ On Thu, Oct 27, 2016 at 6:5

Re: Frontend for Nifi support / mailing list

2016-10-30 Thread Tony Kurc
This one: http://apache-nifi-users-list.2361937.n4.nabble.com/ is the one I set up, and had intended on updating the web page with. Can you post from it? Tony On Sun, Oct 30, 2016 at 7:10 PM, ddewaele wrote: > In order to login to Pony Mail you need to be an ASF committer. Using my > email clie

Re: Deadlocks after upgrade from 0.6.1 to 1.1.1

2017-02-16 Thread Tony Kurc
Mike, also if what Joe asked with the backpressure is "not being applied", if you're good with a profiler, I think joe and I both gravitated to 0x0006c533b770 being locked in at org.apache.nifi.provenance.PersistentProvenanceRepository.persistRecord(PersistentProvenanceRepository.java:757). It

Re: Can nar files built in windows be used in linux nifi?

2017-12-02 Thread Tony Kurc
For all but extreme cases (native code or weird platform specific java stuff), yes, a nar built on windows will work in linux and vice versa. On Sat, Dec 2, 2017 at 7:31 PM, James McMahon wrote: > If I build a custom processor using maven and eclipse in the Windows > environment, can I install t

Re: New Apache NiFi Website Design Launched

2024-01-08 Thread Tony Kurc
Changes look great! Nice job! On Mon, Jan 8, 2024 at 1:24 PM Joe Witt wrote: > Thanks so much to all involved and David for driving this to completion! > > It looks great > > On Mon, Jan 8, 2024 at 11:17 AM David Handermann < > exceptionfact...@apache.org> wrote: > >> Team, >> >> Thanks to a col