Hello,
I would like to pick up development on my processor that uses a ruleengine. It
is working since several month already but I want to convert the processor to
use the new RecordReader/RecordWriter framework. This way I can use any data
format, which is really, really cool and will benefit
Just my two cents...
I have created a processor that generates CSV data.
You can see an example here:
http://datamelt.weebly.com/blog/nifi-processor-generatedata
It generates data based on word lists, regular expressions or purely random.
Date data can be generated in a way that the columns
read in record
at a time. Their are record readers already for CSV, JSON, AVRO, and
Grok with writers in CSV, Avro, JSON, and generic text.
Thanks
Joe
On Fri, Apr 14, 2017 at 11:16 AM, Uwe Geercken wrote:
> Hello everybody,
>
> I have released (Apache License) my NiFi processors a
n a single large set of
rows and putting out many individual rows? Could you have it put out
rows grouped together by some logic or must you end up with individual
splits?
Thanks
Joe
On Fri, Apr 14, 2017 at 11:14 AM, Uwe Geercken wrote:
> Hi everybody,
>
> in my ExecuteRuleEngine
Hello everybody,
I have released (Apache License) my NiFi processors at:
https://github.com/uwegeercken/nifi_processors[https://deref-web-02.de/mail/client/WDRw2DECNaw/dereferrer/?redirectUrl=https%3A%2F%2Fgithub.com%2Fuwegeercken%2Fnifi_processors]
Further below is a summary for the processo
Hi everybody,
in my ExecuteRuleEngine processor I have used a LineIteraotr to loop ofer
incomming rows of data (CSV).
I was wondering if that is the preferred method for (large) files or if there
are better ways to do it. Also, is there a way to influence the number of
buffered rows or bytes
Hello,
I have a little bit of a hard time to design processors correctly. I find it
difficult to decide if a processor should e.g. process a single line from a
flow file or process also flow files with multiples lines of data (e.g. in the
case of CSV files). Another point is the handling of hea
side drop down -> Help)?
>
> Best Regards,
> Uwe
>
> Am 24.02.2017 um 12:35 schrieb Uwe Geercken:
> > Uwe,
> >
> > wenn man in Nifi die Help aufruft, dann erscheinen alle Hilfstexte für die
> > Prozessoren. Nifi genriert diese beim Start im Ordner work/d
Hello,
I remember a while ago that there was a short discussion if it would be good to
have a central place somewhere where people could upload the processors they
created and others could download them from this central point. It would make
life easier compared to surfing the web to find them
but maybe I can
> help you?!
> Send me a direct mail so we can talk in german ;-)
>
> Best Regards,
> Uwe
>
> Am 24.02.2017 um 11:19 schrieb Uwe Geercken:
> > Hello,
> >
> > I am developing some processors for Nifi. They do work in 1.1 but the
> >
Hello,
I am developing some processors for Nifi. They do work in 1.1 but the
documentation does not show up in the Help.
So I set the loglevel to DEBUG and this is what came out:
2017-02-24 11:00:18,538 DEBUG [main] org.apache.nifi.nar.NarUnpacker Expanding
NAR file: /opt/nifi-1.1.1/./lib/c
take
> care to document its intended use and limitations on performance to
> consider, and enable it to limit how many columns/fields get turned
> into attributes maybe by setting a max or by having a
> whitelist/blacklist type model. Even if it won't achieve highest
> sustained pe
Francois,
very nice. Thanks.
I have been working on a simple version a while ago. But it had another scope:
I wnated to have a Nifi processor to merge CSV data with a template from a
template engine (e.g. Apache Velocity). I will review my code and have a look
at your processor.
Where can we
Congratulations Matt.
That's very good for Nifi. I have met Matt in person and he is smart, always
helpful and has great expertise. A great person to work with and also to have a
beer with ;-)
Uwe
> Gesendet: Freitag, 07. Oktober 2016 um 18:52 Uhr
> Von: "Joe Skora"
> An: dev@nifi.apache.or
host should be the ip of the machine it is on, not the machine
> that it expects to receive the data from. That variable is similar to the
> httphost or https host variable, also in the prop file.
>
> Hope that helps!
>
> Sent from my iPhone
>
> > On Sep 10, 2016, at 6
Hello,
I have used Nifi for a while on my laptop. Now I wanted to setup a second
machine communicating with the first one - either using Nifi oder Minifi. For
the moment I have Nifi 1.0.0 running on both and I followed the instructions of
how to setup Nifi. I read documentation, but there must
Hello,
are there any recommendations or best practices available on how to setup a dev
and prod environment with Apache Nifi?
It the simple answer to run Nifi twise on different servers? And if so how
would a deployment from dev to prod look like? Through the use of templates?
Thanks for any f
at I was thinking of using for the CSV processor. Will you
> be committing your version?
>
> - Dmitry
>
> On Tue, Apr 5, 2016 at 1:39 PM, Uwe Geercken wrote:
>
> > Dimitry,
> >
> > I was working on a processor for CSV files and one remark came up that we
> &
Dimitry,
I was working on a processor for CSV files and one remark came up that we might
want to use the opencsv library for parsing the file.
Here is the link: http://opencsv.sourceforge.net/
Greetings,
Uwe
> Gesendet: Dienstag, 05. April 2016 um 13:00 Uhr
> Von: "Dmitry Goldenberg"
> An: d
n import the top level POM into your IDE of choice :)
>
> Regards,
> Matt
>
> > On Mar 21, 2016, at 1:21 PM, Uwe Geercken wrote:
> >
> > I am sorry for that. I have completely overlooked that.
> >
> > So tell me Matt how is it: I create an automatic packa
great progress. That is awesome.
> >
> > Really want to help you get to the bottom of the additional details
> > piece though. We clearly have to do a better job with documenting (or
> > implementing) how to do this. Do you have any more details to share
> > on sy
hat is awesome.
> >
> > Really want to help you get to the bottom of the additional details
> > piece though. We clearly have to do a better job with documenting (or
> > implementing) how to do this. Do you have any more details to share
> > on symptoms you're seei
ilter
> the attributes send to the engine? I don't disagree with this feature, but
> wondering if it's required or just a nicety.
>
> - There are likely some code review and naming items to talk about. For
> example, the reference to the VelocityEngine or Template may not
Hello,
I am writing my first processor.
As described in the documentation, I have added an HTML file to be used when
the user selects "Usage":
docs/com.datamelt.nifi.test.TemplateProcessor/additionalDetails.html
This is located in the root or the Processors nar file.
The processor class is th
Hello,
I am writing my first processor. As described in the documentation, I have
added an HTML file to be used when the user selects "Usage":
docs/com.datamelt.nifi.test.TemplateProcessor/additionalDetails.html
This is located in the root or the Processors nar file.
The processor class is th
d be a nice addition. Making it
> generic to support Velocity, FreeMarker, and others might be really nice.
> Extra bonus points for Markdown or Asciidoc transforms as well (but these
> might be too separate of a use case).
>
> Hope this helps.
>
> Adam
>
> [1] http:/
rs.attributes.UpdateAttribute/additionalDetails.html
>
> On Fri, Mar 18, 2016 at 10:54 AM Uwe Geercken wrote:
>
> > Bryan,
> >
> > all looks ok. I looked into the nifi-home/work/docs folder. There is
> > nothing but a components folder. Inside there is a folde
27;s the "split row into fields and merge with template" that we would
> want to separate into two processors instead of one.
>
> You're very much on the right track, I believe. If the above doesn't help,
> I'll try and jump in on a code example when I can.
y template
transform using flowfile attributes as opposed to CSV? Flowfile attributes
are NIFI's lowest common data representation, many many processors create
attributes which would enable your Velocity processor to be used by more
than just CSV input.
Adam
On Fri, Mar 18, 2016 at 11:0
Hello,
my first mailing here. I am a Java developer, using Apache Velocity, Drill,
Tomcat, Ant, Pentaho ETL, MongoDb, Mysql and more and I am very much a data guy.
I have used Nifi for a while now and started yesterday of coding my first
processor. I basically do it to widen my knowledge and
Hello,
my first mailing here. I am a Java developer, using Apache Velocity, Drill,
Tomcat, Ant, Pentaho ETL, MongoDb, Mysql and more and I am very much a data guy.
I have used Nifi for a while now and started yesterday of coding my first
processor. I basically do it to widen my knowledge and l
bonus points for Markdown or Asciidoc transforms as well (but these
> might be too separate of a use case).
>
> Hope this helps.
>
> Adam
>
> [1] http://nifi.apache.org/developer-guide.html
>
> [2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide
n+Projects+for+Extensions#MavenProjectsforExtensions-ExampleProcessorBundleStructure]
If you think that part is setup correctly, can you check under
nifi_home/work/docs and see if com.datamelt.nifi.test.TemplateProcessor is
there?
-Bryan
On Fri, Mar 18, 2016 at 11:04 AM, Uwe Geercken wrote:
gt; > > into play. For example, if your processor uses a custom query language.
> > >
> > > Generated index.html example:
> > >
> > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.attributes.UpdateAttribute/index.html
> > >
&g
34 matches
Mail list logo