Re: Lowering the barrier to entry

[Mike Thomsen]

> Also on a personal front I struggled with eclipse, but the eclipse tools
are a bit out of date in this regard.  The m2eclipse plugin has many
dependencies that are really challenging to navigate when setting up the
eclipse IDE.

I would strongly recommend trying it with IntelliJ IDEA. I've been a paid
user for a few years now (there is a community version with fewer
features), and can't recommend it enough over Eclipse for reasons like
this. Officially, we probably don't want to risk a religious war over IDE
choice, but I would personally recommend you fire up IDEA. I think you'll
find it drops the barrier to entry substantially here because its Maven
support is incredibly solid and deeply built-in.


On Mon, Feb 4, 2019 at 7:04 PM Ryan Withers  wrote:

> Hello Nifi Community,
>
>   I owe a couple people responses for a welcome to the community.  I made
> a mistake and sent the mailing list a question before being signed up to
> the list itself, LOL.  So instead of replying to the messages directly I
> figured I would thank Andy Lopresto, Kevin Doran, and Matt Burgess for
> their respective messages welcoming me to the project.
>
>   I have spent time re-reading some of the thread "Lowering the Barrier to
> entry" this afternoon.  I concur with many of the sentiments that the
> documentation is great, there is however a lot of it.  Also on a personal
> front I struggled with eclipse, but the eclipse tools are a bit out of date
> in this regard.  The m2eclipse plugin has many dependencies that are really
> challenging to navigate when setting up the eclipse IDE.  I've actually
> been looking for an excuse to adopt intellij (looks like I found it).  I
> didn't have any trouble building the project on the command line or within
> the intellij IDE.  I have also generated a processor and a controller
> service using the maven archetypes without any issue.  I still have a long
> way to go with learning Nifi, but I'm excited by its potential and I am
> learning more every day.
>
>   At the company I work for we've been working on a process we're calling
> Human Centered Agile, and as you might guess it puts the user first.  There
> is a technique that our Human Centered Designers use called card sorting.
> You can watch a short video about card sorting here:
> https://www.optimalworkshop.com/optimalsort.  I have tried to pull out
> what I thought were the main ideas of Nifi into a gsheet to which you can
> request access here:
>
>
> https://docs.google.com/spreadsheets/d/1XEVL5LOgspNJ5BOb5sbf38SxbaY3Dj5oolHJ8ECO2MI/edit?usp=sharing.
> There is a pdf of the same attached.
>
> I have two questions:
>
> 1. Would the community want to go through a card sorting exercise?  If so
> I'm happy to provide access to the optimal sort tool.
>
> 2. Have I captured what everyone believes to be the main ideas?  I'm very
> interested in making sure the list is complete.  (attached as pdf or
> available via the gsheet link above)
>
> Assuming the list is complete with the big ideas then I will make a pass
> at creating definitions.  Using the big ideas along with the definitions
> the goal of the card sort would be to figure out the relationships of the
> data and determine an overall information architecture from those
> relationships to help create a site structure that flows more intuitively.
> Everybody's efforts at sorting the information will create data / insights
> that will reveal relationships between the data that can then be used to
> guide us toward an optimal organization of the data.
>
> Looking forward,
>
> --
> Ryan Withers
> Senior Software Developer / Analyst
>
> http://www.linkedin.com/in/ryanwithers
>

RE: Lowering the barrier to entry

[Ryan Withers]

Hello Nifi Community,

  I owe a couple people responses for a welcome to the community.  I made a
mistake and sent the mailing list a question before being signed up to the
list itself, LOL.  So instead of replying to the messages directly I
figured I would thank Andy Lopresto, Kevin Doran, and Matt Burgess for
their respective messages welcoming me to the project.

  I have spent time re-reading some of the thread "Lowering the Barrier to
entry" this afternoon.  I concur with many of the sentiments that the
documentation is great, there is however a lot of it.  Also on a personal
front I struggled with eclipse, but the eclipse tools are a bit out of date
in this regard.  The m2eclipse plugin has many dependencies that are really
challenging to navigate when setting up the eclipse IDE.  I've actually
been looking for an excuse to adopt intellij (looks like I found it).  I
didn't have any trouble building the project on the command line or within
the intellij IDE.  I have also generated a processor and a controller
service using the maven archetypes without any issue.  I still have a long
way to go with learning Nifi, but I'm excited by its potential and I am
learning more every day.

  At the company I work for we've been working on a process we're calling
Human Centered Agile, and as you might guess it puts the user first.  There
is a technique that our Human Centered Designers use called card sorting.
You can watch a short video about card sorting here:
https://www.optimalworkshop.com/optimalsort.  I have tried to pull out what
I thought were the main ideas of Nifi into a gsheet to which you can
request access here:

https://docs.google.com/spreadsheets/d/1XEVL5LOgspNJ5BOb5sbf38SxbaY3Dj5oolHJ8ECO2MI/edit?usp=sharing.
There is a pdf of the same attached.

I have two questions:

1. Would the community want to go through a card sorting exercise?  If so
I'm happy to provide access to the optimal sort tool.

2. Have I captured what everyone believes to be the main ideas?  I'm very
interested in making sure the list is complete.  (attached as pdf or
available via the gsheet link above)

Assuming the list is complete with the big ideas then I will make a pass at
creating definitions.  Using the big ideas along with the definitions the
goal of the card sort would be to figure out the relationships of the data
and determine an overall information architecture from those relationships
to help create a site structure that flows more intuitively.  Everybody's
efforts at sorting the information will create data / insights that will
reveal relationships between the data that can then be used to guide us
toward an optimal organization of the data.

Looking forward,

-- 
Ryan Withers
Senior Software Developer / Analyst

http://www.linkedin.com/in/ryanwithers

Re: Lowering the barrier of entry

[Andy LoPresto]

There are many, *many* people who use NiFi and never write a line of code. 
Please feel free to peruse the us...@nifi.apache.org 
 list for examples of these (see also: origins of 
NiFi). Separating the User Guide (how to interact with the application via the 
UI and API) and the Developer Guide (how to extend/change/and understand the 
behavior of NiFi by viewing/modifying code) target very different (while 
sometimes overlapping) audiences and I feel strongly should stay separated to 
reduce the likelihood of overwhelming either group with unnecessary information 
and making it harder to find what they are looking for. 

I think moving the Developer Guide link up to the top is a good idea, and there 
has definitely been discussion about this in the past. I can remember no 
objections. 

I don’t know that additional menu items on the main site would help, because 
from the landing page in the documentation section, those titles (User, Admin, 
and hopefully now Developer) are broken out immediately. 

I would vote against including wiki pages as part of the top-level 
documentation. I think linking to them from content in a guide is helpful, but 
as Bryan pointed out, they change frequently and are not deployed with the 
application. I think if we decide there is information in the wiki that is 
stable and valuable, it should be moved into top-level (i.e. static generated) 
documentation and hosted there & deployed with the application. In my opinion, 
the information in the wiki should be frequently changing or under current 
discussion/proposed. There is greater access to modifying that info, but that 
should be fixed by encouraging the community to make PRs to the versioned 
documentation when appropriate. 

Examples:
* feature proposals, process FAQs, release notes, migration guidance, 
articles = wiki
* component documentation, user/dev/admin guides, 
walkthroughs/reference documents = documentation as code in NiFi project proper



Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jan 28, 2019, at 10:46 AM, James Srinivasan  
> wrote:
> 
> Throw away :-)
> 
> In our org, we have some people who can use NiFi to create flows with
> the built-in processors. I guess HWX would all them "Data Flow
> Managers".
> They might know some scripting, so can write Jython scripts.
> 
> We have far fewer (approx 1) person (me) who can write custom
> processors in Java. Sometimes, I also create flows.
> 
> I'd like to have lots more of both kinds of people.
> 
> "User" & "Extending NiFi" actually maps to the internal training I
> run, so I'd be happy :-)
> 
> James
> 
> On Mon, 28 Jan 2019 at 18:01, Andrew Grande  wrote:
>> 
>> Not to throw in a monkey wrench, but does it really make sense to split
>> User and Developer? In all years I've never seen a user who wasn't a
>> developer.
>> 
>> Maybe call it a User and Extending NiFi sections?
>> 
>> On Mon, Jan 28, 2019, 9:18 AM James Srinivasan 
>> wrote:
>> 
>>> Rather than lumping all the documentation together in a single huge
>>> doc, I was thinking separate entries in the top bar of the NiFi site
>>> under "Documentation" for something like:
>>> 
>>> General (Overview & Getting Started)
>>> User (User Guide, Expression Language Guide, Record Path Guide &
>>> detailed Processor Usage)
>>> Admin (Admin Guide)
>>> Developer (All the text currently under Developer)
>>> 
>>> Breaking it out into multiple top-level headings will hopefully help
>>> people find what they need more quickly e.g. with my Developer hat on,
>>> I don't much care about the details of the FooBarProcessor, whereas
>>> with my User hat on, I really want to know about its parameters and
>>> what they mean. Likewise, a non-admin probably doesn't much care about
>>> certificates etc.
>>> 
>>> Does this makes sense? What do others think?
>>> 
>>> On Mon, 28 Jan 2019 at 17:04, Bryan Bende  wrote:
 
 Currently it’s broken into General and Developer, so were you thinking of
 splitting General into User and Admin?
 
 On Mon, Jan 28, 2019 at 11:34 AM James Srinivasan <
 james.sriniva...@gmail.com> wrote:
 
> How about separating out User/Developer/Admin into separate docs?
> 
> On Mon, 28 Jan 2019 at 16:13, Bryan Bende  wrote:
>> 
>> What does everyone think about bumping the "Developer" section of the
>> docs ahead of "Processors" so that it's easier to find?
>> 
>> Here is what it would look like -
>> https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
>> 
>> I also added links to the "Contributor Guide" and the "Maven
>>> Projects"
>> page since I think it would be helpful to make the Developer section
>> be the one-stop place people look for development help, although I
>>> can
>> see an argument for not mixing wiki content with the docs content.
>> 
>> One issue would 

Re: Lowering the barrier of entry

[James Srinivasan]

Throw away :-)

In our org, we have some people who can use NiFi to create flows with
the built-in processors. I guess HWX would all them "Data Flow
Managers".
They might know some scripting, so can write Jython scripts.

We have far fewer (approx 1) person (me) who can write custom
processors in Java. Sometimes, I also create flows.

I'd like to have lots more of both kinds of people.

"User" & "Extending NiFi" actually maps to the internal training I
run, so I'd be happy :-)

James

On Mon, 28 Jan 2019 at 18:01, Andrew Grande  wrote:
>
> Not to throw in a monkey wrench, but does it really make sense to split
> User and Developer? In all years I've never seen a user who wasn't a
> developer.
>
> Maybe call it a User and Extending NiFi sections?
>
> On Mon, Jan 28, 2019, 9:18 AM James Srinivasan 
> wrote:
>
> > Rather than lumping all the documentation together in a single huge
> > doc, I was thinking separate entries in the top bar of the NiFi site
> > under "Documentation" for something like:
> >
> > General (Overview & Getting Started)
> > User (User Guide, Expression Language Guide, Record Path Guide &
> > detailed Processor Usage)
> > Admin (Admin Guide)
> > Developer (All the text currently under Developer)
> >
> > Breaking it out into multiple top-level headings will hopefully help
> > people find what they need more quickly e.g. with my Developer hat on,
> > I don't much care about the details of the FooBarProcessor, whereas
> > with my User hat on, I really want to know about its parameters and
> > what they mean. Likewise, a non-admin probably doesn't much care about
> > certificates etc.
> >
> > Does this makes sense? What do others think?
> >
> > On Mon, 28 Jan 2019 at 17:04, Bryan Bende  wrote:
> > >
> > > Currently it’s broken into General and Developer, so were you thinking of
> > > splitting General into User and Admin?
> > >
> > > On Mon, Jan 28, 2019 at 11:34 AM James Srinivasan <
> > > james.sriniva...@gmail.com> wrote:
> > >
> > > > How about separating out User/Developer/Admin into separate docs?
> > > >
> > > > On Mon, 28 Jan 2019 at 16:13, Bryan Bende  wrote:
> > > > >
> > > > > What does everyone think about bumping the "Developer" section of the
> > > > > docs ahead of "Processors" so that it's easier to find?
> > > > >
> > > > > Here is what it would look like -
> > > > > https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
> > > > >
> > > > > I also added links to the "Contributor Guide" and the "Maven
> > Projects"
> > > > > page since I think it would be helpful to make the Developer section
> > > > > be the one-stop place people look for development help, although I
> > can
> > > > > see an argument for not mixing wiki content with the docs content.
> > > > >
> > > > > One issue would be if we want the docs to be fully usable in an
> > > > > off-line environment, then linking to the wiki won't work, so we
> > could
> > > > > remove those links, or convert those pages to be part of the docs now
> > > > > that they are more stable.
> > > > >
> > > > > For reference, we already have some links in the docs to the wiki:
> > > > >
> > > > >
> > > >
> > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution
> > > > >
> > > > > On Sat, Jan 26, 2019 at 10:49 AM Joe Witt 
> > wrote:
> > > > > >
> > > > > > ...we can.  but the discussion is about how to both lower the bar
> > and
> > > > offer
> > > > > > more routes to the bar.
> > > > > >
> > > > > > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler <
> > ottobackwa...@gmail.com
> > > > wrote:
> > > > > >
> > > > > > > Why wouldn’t we make the archetypes do this?
> > > > > > >
> > > > > > >
> > > > > > > On January 25, 2019 at 18:04:25, Andy LoPresto (
> > alopre...@apache.org
> > > > )
> > > > > > > wrote:
> > > > > > >
> > > > > > > James,
> > > > > > >
> > > > > > > I’m wondering if a page outlining a toy processor (something like
> > > > > > > `CountText` or `ReverseContent`) and doing a line-by-line
> > annotation
> > > > from a
> > > > > > > developer’s perspective would be helpful. It could be a few
> > sections:
> > > > > > >
> > > > > > > 1. How to get to this point
> > > > > > > * running the maven archetype
> > > > > > > * choosing the directory to install to
> > > > > > > * putting the class name in the manifest file
> > > > > > > * etc.
> > > > > > > 2. The code
> > > > > > > * here’s the class
> > > > > > > * here’s what extending AbstractProcessor gets you, etc. A lot of
> > > > this is
> > > > > > > currently in the Developer Guide, but in textbook mode
> > > > > > > * here’s how to modify some code (i.e. write one line of Java
> > that
> > > > switches
> > > > > > > it from straight content pass-through to reversing the text)
> > > > > > > * here’s how to make a unit test (introduce the TestRunner
> > > > framework, etc.)
> > > > > > > 3. Running, building, installing
> > > > > > > * Run your unit test from the IDE/maven
> > > > > > > * Build the NAR module
> > > > > > > * 

Re: Lowering the barrier of entry

[Andrew Grande]

Not to throw in a monkey wrench, but does it really make sense to split
User and Developer? In all years I've never seen a user who wasn't a
developer.

Maybe call it a User and Extending NiFi sections?

On Mon, Jan 28, 2019, 9:18 AM James Srinivasan 
wrote:

> Rather than lumping all the documentation together in a single huge
> doc, I was thinking separate entries in the top bar of the NiFi site
> under "Documentation" for something like:
>
> General (Overview & Getting Started)
> User (User Guide, Expression Language Guide, Record Path Guide &
> detailed Processor Usage)
> Admin (Admin Guide)
> Developer (All the text currently under Developer)
>
> Breaking it out into multiple top-level headings will hopefully help
> people find what they need more quickly e.g. with my Developer hat on,
> I don't much care about the details of the FooBarProcessor, whereas
> with my User hat on, I really want to know about its parameters and
> what they mean. Likewise, a non-admin probably doesn't much care about
> certificates etc.
>
> Does this makes sense? What do others think?
>
> On Mon, 28 Jan 2019 at 17:04, Bryan Bende  wrote:
> >
> > Currently it’s broken into General and Developer, so were you thinking of
> > splitting General into User and Admin?
> >
> > On Mon, Jan 28, 2019 at 11:34 AM James Srinivasan <
> > james.sriniva...@gmail.com> wrote:
> >
> > > How about separating out User/Developer/Admin into separate docs?
> > >
> > > On Mon, 28 Jan 2019 at 16:13, Bryan Bende  wrote:
> > > >
> > > > What does everyone think about bumping the "Developer" section of the
> > > > docs ahead of "Processors" so that it's easier to find?
> > > >
> > > > Here is what it would look like -
> > > > https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
> > > >
> > > > I also added links to the "Contributor Guide" and the "Maven
> Projects"
> > > > page since I think it would be helpful to make the Developer section
> > > > be the one-stop place people look for development help, although I
> can
> > > > see an argument for not mixing wiki content with the docs content.
> > > >
> > > > One issue would be if we want the docs to be fully usable in an
> > > > off-line environment, then linking to the wiki won't work, so we
> could
> > > > remove those links, or convert those pages to be part of the docs now
> > > > that they are more stable.
> > > >
> > > > For reference, we already have some links in the docs to the wiki:
> > > >
> > > >
> > >
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution
> > > >
> > > > On Sat, Jan 26, 2019 at 10:49 AM Joe Witt 
> wrote:
> > > > >
> > > > > ...we can.  but the discussion is about how to both lower the bar
> and
> > > offer
> > > > > more routes to the bar.
> > > > >
> > > > > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler <
> ottobackwa...@gmail.com
> > > wrote:
> > > > >
> > > > > > Why wouldn’t we make the archetypes do this?
> > > > > >
> > > > > >
> > > > > > On January 25, 2019 at 18:04:25, Andy LoPresto (
> alopre...@apache.org
> > > )
> > > > > > wrote:
> > > > > >
> > > > > > James,
> > > > > >
> > > > > > I’m wondering if a page outlining a toy processor (something like
> > > > > > `CountText` or `ReverseContent`) and doing a line-by-line
> annotation
> > > from a
> > > > > > developer’s perspective would be helpful. It could be a few
> sections:
> > > > > >
> > > > > > 1. How to get to this point
> > > > > > * running the maven archetype
> > > > > > * choosing the directory to install to
> > > > > > * putting the class name in the manifest file
> > > > > > * etc.
> > > > > > 2. The code
> > > > > > * here’s the class
> > > > > > * here’s what extending AbstractProcessor gets you, etc. A lot of
> > > this is
> > > > > > currently in the Developer Guide, but in textbook mode
> > > > > > * here’s how to modify some code (i.e. write one line of Java
> that
> > > switches
> > > > > > it from straight content pass-through to reversing the text)
> > > > > > * here’s how to make a unit test (introduce the TestRunner
> > > framework, etc.)
> > > > > > 3. Running, building, installing
> > > > > > * Run your unit test from the IDE/maven
> > > > > > * Build the NAR module
> > > > > > * Install the NAR in NiFi lib/ or custom/
> > > > > > * Restart NiFi
> > > > > > * See the NAR loaded in the log
> > > > > > * Deploy the component on the canvas
> > > > > >
> > > > > > I imagine this being written more conversationally/blog-like than
> > > most of
> > > > > > our current reference documentation to be used as a split-screen
> > > > > > walkthrough. Each section could certainly link to the existing
> > > detailed
> > > > > > documentation for various topics, like the processor lifecycle,
> etc.
> > > > > >
> > > > > > Does this sounds like something that would have helped you?
> > > > > >
> > > > > > Andy LoPresto
> > > > > > alopre...@apache.org
> > > > > > alopresto.apa...@gmail.com
> > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E 

Re: Lowering the barrier of entry

[James Srinivasan]

Rather than lumping all the documentation together in a single huge
doc, I was thinking separate entries in the top bar of the NiFi site
under "Documentation" for something like:

General (Overview & Getting Started)
User (User Guide, Expression Language Guide, Record Path Guide &
detailed Processor Usage)
Admin (Admin Guide)
Developer (All the text currently under Developer)

Breaking it out into multiple top-level headings will hopefully help
people find what they need more quickly e.g. with my Developer hat on,
I don't much care about the details of the FooBarProcessor, whereas
with my User hat on, I really want to know about its parameters and
what they mean. Likewise, a non-admin probably doesn't much care about
certificates etc.

Does this makes sense? What do others think?

On Mon, 28 Jan 2019 at 17:04, Bryan Bende  wrote:
>
> Currently it’s broken into General and Developer, so were you thinking of
> splitting General into User and Admin?
>
> On Mon, Jan 28, 2019 at 11:34 AM James Srinivasan <
> james.sriniva...@gmail.com> wrote:
>
> > How about separating out User/Developer/Admin into separate docs?
> >
> > On Mon, 28 Jan 2019 at 16:13, Bryan Bende  wrote:
> > >
> > > What does everyone think about bumping the "Developer" section of the
> > > docs ahead of "Processors" so that it's easier to find?
> > >
> > > Here is what it would look like -
> > > https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
> > >
> > > I also added links to the "Contributor Guide" and the "Maven Projects"
> > > page since I think it would be helpful to make the Developer section
> > > be the one-stop place people look for development help, although I can
> > > see an argument for not mixing wiki content with the docs content.
> > >
> > > One issue would be if we want the docs to be fully usable in an
> > > off-line environment, then linking to the wiki won't work, so we could
> > > remove those links, or convert those pages to be part of the docs now
> > > that they are more stable.
> > >
> > > For reference, we already have some links in the docs to the wiki:
> > >
> > >
> > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution
> > >
> > > On Sat, Jan 26, 2019 at 10:49 AM Joe Witt  wrote:
> > > >
> > > > ...we can.  but the discussion is about how to both lower the bar and
> > offer
> > > > more routes to the bar.
> > > >
> > > > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler  > wrote:
> > > >
> > > > > Why wouldn’t we make the archetypes do this?
> > > > >
> > > > >
> > > > > On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org
> > )
> > > > > wrote:
> > > > >
> > > > > James,
> > > > >
> > > > > I’m wondering if a page outlining a toy processor (something like
> > > > > `CountText` or `ReverseContent`) and doing a line-by-line annotation
> > from a
> > > > > developer’s perspective would be helpful. It could be a few sections:
> > > > >
> > > > > 1. How to get to this point
> > > > > * running the maven archetype
> > > > > * choosing the directory to install to
> > > > > * putting the class name in the manifest file
> > > > > * etc.
> > > > > 2. The code
> > > > > * here’s the class
> > > > > * here’s what extending AbstractProcessor gets you, etc. A lot of
> > this is
> > > > > currently in the Developer Guide, but in textbook mode
> > > > > * here’s how to modify some code (i.e. write one line of Java that
> > switches
> > > > > it from straight content pass-through to reversing the text)
> > > > > * here’s how to make a unit test (introduce the TestRunner
> > framework, etc.)
> > > > > 3. Running, building, installing
> > > > > * Run your unit test from the IDE/maven
> > > > > * Build the NAR module
> > > > > * Install the NAR in NiFi lib/ or custom/
> > > > > * Restart NiFi
> > > > > * See the NAR loaded in the log
> > > > > * Deploy the component on the canvas
> > > > >
> > > > > I imagine this being written more conversationally/blog-like than
> > most of
> > > > > our current reference documentation to be used as a split-screen
> > > > > walkthrough. Each section could certainly link to the existing
> > detailed
> > > > > documentation for various topics, like the processor lifecycle, etc.
> > > > >
> > > > > Does this sounds like something that would have helped you?
> > > > >
> > > > > Andy LoPresto
> > > > > alopre...@apache.org
> > > > > alopresto.apa...@gmail.com
> > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > > > >
> > > > > > On Jan 25, 2019, at 1:59 PM, James Srinivasan <
> > > > > james.sriniva...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > 9) Oh, and the wiki is a little hard to navigate and the contents
> > rather
> > > > > patchy
> > > > > >
> > > > > > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> > > > > >  wrote:
> > > > > >>
> > > > > >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> > > > > >> realise I could and possibly should submit PRs :)
> > > > > >>
> > > > > >> 1) 

Re: Lowering the barrier of entry

[Bryan Bende]

Currently it’s broken into General and Developer, so were you thinking of
splitting General into User and Admin?

On Mon, Jan 28, 2019 at 11:34 AM James Srinivasan <
james.sriniva...@gmail.com> wrote:

> How about separating out User/Developer/Admin into separate docs?
>
> On Mon, 28 Jan 2019 at 16:13, Bryan Bende  wrote:
> >
> > What does everyone think about bumping the "Developer" section of the
> > docs ahead of "Processors" so that it's easier to find?
> >
> > Here is what it would look like -
> > https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
> >
> > I also added links to the "Contributor Guide" and the "Maven Projects"
> > page since I think it would be helpful to make the Developer section
> > be the one-stop place people look for development help, although I can
> > see an argument for not mixing wiki content with the docs content.
> >
> > One issue would be if we want the docs to be fully usable in an
> > off-line environment, then linking to the wiki won't work, so we could
> > remove those links, or convert those pages to be part of the docs now
> > that they are more stable.
> >
> > For reference, we already have some links in the docs to the wiki:
> >
> >
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution
> >
> > On Sat, Jan 26, 2019 at 10:49 AM Joe Witt  wrote:
> > >
> > > ...we can.  but the discussion is about how to both lower the bar and
> offer
> > > more routes to the bar.
> > >
> > > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler  wrote:
> > >
> > > > Why wouldn’t we make the archetypes do this?
> > > >
> > > >
> > > > On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org
> )
> > > > wrote:
> > > >
> > > > James,
> > > >
> > > > I’m wondering if a page outlining a toy processor (something like
> > > > `CountText` or `ReverseContent`) and doing a line-by-line annotation
> from a
> > > > developer’s perspective would be helpful. It could be a few sections:
> > > >
> > > > 1. How to get to this point
> > > > * running the maven archetype
> > > > * choosing the directory to install to
> > > > * putting the class name in the manifest file
> > > > * etc.
> > > > 2. The code
> > > > * here’s the class
> > > > * here’s what extending AbstractProcessor gets you, etc. A lot of
> this is
> > > > currently in the Developer Guide, but in textbook mode
> > > > * here’s how to modify some code (i.e. write one line of Java that
> switches
> > > > it from straight content pass-through to reversing the text)
> > > > * here’s how to make a unit test (introduce the TestRunner
> framework, etc.)
> > > > 3. Running, building, installing
> > > > * Run your unit test from the IDE/maven
> > > > * Build the NAR module
> > > > * Install the NAR in NiFi lib/ or custom/
> > > > * Restart NiFi
> > > > * See the NAR loaded in the log
> > > > * Deploy the component on the canvas
> > > >
> > > > I imagine this being written more conversationally/blog-like than
> most of
> > > > our current reference documentation to be used as a split-screen
> > > > walkthrough. Each section could certainly link to the existing
> detailed
> > > > documentation for various topics, like the processor lifecycle, etc.
> > > >
> > > > Does this sounds like something that would have helped you?
> > > >
> > > > Andy LoPresto
> > > > alopre...@apache.org
> > > > alopresto.apa...@gmail.com
> > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > > >
> > > > > On Jan 25, 2019, at 1:59 PM, James Srinivasan <
> > > > james.sriniva...@gmail.com>
> > > > wrote:
> > > > >
> > > > > 9) Oh, and the wiki is a little hard to navigate and the contents
> rather
> > > > patchy
> > > > >
> > > > > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> > > > >  wrote:
> > > > >>
> > > > >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> > > > >> realise I could and possibly should submit PRs :)
> > > > >>
> > > > >> 1) I'm used to Java and Maven, so used the archetype. It worked
> fine,
> > > > >> it would have been nice it if set up unit tests for me.
> > > > >> 2) The User and Developer documentation is great and
> comprehensive.
> > > > >> Finding the developer docs is a little painful (handful of items
> at
> > > > >> the end of a scrolling list of 200+ processors)
> > > > >> 3) The Developer docs could possibly do with a little more
> clarity on
> > > > >> processor lifetime i.e. what is called when ^h^h^h - skimming back
> > > > >> over the docs, it looks pretty clear now
> > > > >> 4) Some example code for common operations e.g. getting/setting
> > > > >> attributes or reading/writing/modifying flowfile content would be
> > > > >> great.
> > > > >> 5) When using existing processors for inspiration, best practices
> > > > >> weren't always clear e.g. some generated properties inside
> > > > >> getSupportedPropertyDescriptors(), others generated a private
> static
> > > > >> list on init and returned that. Such differences are inevitable
> in a
> > > 

Re: Lowering the barrier of entry

[James Srinivasan]

How about separating out User/Developer/Admin into separate docs?

On Mon, 28 Jan 2019 at 16:13, Bryan Bende  wrote:
>
> What does everyone think about bumping the "Developer" section of the
> docs ahead of "Processors" so that it's easier to find?
>
> Here is what it would look like -
> https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
>
> I also added links to the "Contributor Guide" and the "Maven Projects"
> page since I think it would be helpful to make the Developer section
> be the one-stop place people look for development help, although I can
> see an argument for not mixing wiki content with the docs content.
>
> One issue would be if we want the docs to be fully usable in an
> off-line environment, then linking to the wiki won't work, so we could
> remove those links, or convert those pages to be part of the docs now
> that they are more stable.
>
> For reference, we already have some links in the docs to the wiki:
>
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution
>
> On Sat, Jan 26, 2019 at 10:49 AM Joe Witt  wrote:
> >
> > ...we can.  but the discussion is about how to both lower the bar and offer
> > more routes to the bar.
> >
> > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler  >
> > > Why wouldn’t we make the archetypes do this?
> > >
> > >
> > > On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org)
> > > wrote:
> > >
> > > James,
> > >
> > > I’m wondering if a page outlining a toy processor (something like
> > > `CountText` or `ReverseContent`) and doing a line-by-line annotation from 
> > > a
> > > developer’s perspective would be helpful. It could be a few sections:
> > >
> > > 1. How to get to this point
> > > * running the maven archetype
> > > * choosing the directory to install to
> > > * putting the class name in the manifest file
> > > * etc.
> > > 2. The code
> > > * here’s the class
> > > * here’s what extending AbstractProcessor gets you, etc. A lot of this is
> > > currently in the Developer Guide, but in textbook mode
> > > * here’s how to modify some code (i.e. write one line of Java that 
> > > switches
> > > it from straight content pass-through to reversing the text)
> > > * here’s how to make a unit test (introduce the TestRunner framework, 
> > > etc.)
> > > 3. Running, building, installing
> > > * Run your unit test from the IDE/maven
> > > * Build the NAR module
> > > * Install the NAR in NiFi lib/ or custom/
> > > * Restart NiFi
> > > * See the NAR loaded in the log
> > > * Deploy the component on the canvas
> > >
> > > I imagine this being written more conversationally/blog-like than most of
> > > our current reference documentation to be used as a split-screen
> > > walkthrough. Each section could certainly link to the existing detailed
> > > documentation for various topics, like the processor lifecycle, etc.
> > >
> > > Does this sounds like something that would have helped you?
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > alopresto.apa...@gmail.com
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > >
> > > > On Jan 25, 2019, at 1:59 PM, James Srinivasan <
> > > james.sriniva...@gmail.com>
> > > wrote:
> > > >
> > > > 9) Oh, and the wiki is a little hard to navigate and the contents rather
> > > patchy
> > > >
> > > > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> > > >  wrote:
> > > >>
> > > >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> > > >> realise I could and possibly should submit PRs :)
> > > >>
> > > >> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
> > > >> it would have been nice it if set up unit tests for me.
> > > >> 2) The User and Developer documentation is great and comprehensive.
> > > >> Finding the developer docs is a little painful (handful of items at
> > > >> the end of a scrolling list of 200+ processors)
> > > >> 3) The Developer docs could possibly do with a little more clarity on
> > > >> processor lifetime i.e. what is called when ^h^h^h - skimming back
> > > >> over the docs, it looks pretty clear now
> > > >> 4) Some example code for common operations e.g. getting/setting
> > > >> attributes or reading/writing/modifying flowfile content would be
> > > >> great.
> > > >> 5) When using existing processors for inspiration, best practices
> > > >> weren't always clear e.g. some generated properties inside
> > > >> getSupportedPropertyDescriptors(), others generated a private static
> > > >> list on init and returned that. Such differences are inevitable in a
> > > >> large project, but it would be nice to have something blessed to start
> > > >> from.
> > > >> 6) (Minor niggle - layout of the docs doesn't work great on a phone
> > > screen)
> > > >> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
> > > >> the great blog posts by Matt Burgess and others were invaluable
> > > >> 8) In case this all sounds too negative, NiFi is fab!
> > > >>
> > > >> On Fri, 25 Jan 

Re: Lowering the barrier of entry

[Andrew Grande]

+1 it's so obvious it should've always been there :)

As for offline access. Well, every time I showed an offline help section to
every user thry were startled :) tells about awareness. The first hunch is
always to google insread of going to a locally hosted doc, so it won't be
an issue, IMO.

Andrew

On Mon, Jan 28, 2019, 8:13 AM Bryan Bende  wrote:

> What does everyone think about bumping the "Developer" section of the
> docs ahead of "Processors" so that it's easier to find?
>
> Here is what it would look like -
> https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f
>
> I also added links to the "Contributor Guide" and the "Maven Projects"
> page since I think it would be helpful to make the Developer section
> be the one-stop place people look for development help, although I can
> see an argument for not mixing wiki content with the docs content.
>
> One issue would be if we want the docs to be fully usable in an
> off-line environment, then linking to the wiki won't work, so we could
> remove those links, or convert those pages to be part of the docs now
> that they are more stable.
>
> For reference, we already have some links in the docs to the wiki:
>
>
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution
>
> On Sat, Jan 26, 2019 at 10:49 AM Joe Witt  wrote:
> >
> > ...we can.  but the discussion is about how to both lower the bar and
> offer
> > more routes to the bar.
> >
> > On Sat, Jan 26, 2019, 10:45 AM Otto Fowler  wrote:
> >
> > > Why wouldn’t we make the archetypes do this?
> > >
> > >
> > > On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org)
> > > wrote:
> > >
> > > James,
> > >
> > > I’m wondering if a page outlining a toy processor (something like
> > > `CountText` or `ReverseContent`) and doing a line-by-line annotation
> from a
> > > developer’s perspective would be helpful. It could be a few sections:
> > >
> > > 1. How to get to this point
> > > * running the maven archetype
> > > * choosing the directory to install to
> > > * putting the class name in the manifest file
> > > * etc.
> > > 2. The code
> > > * here’s the class
> > > * here’s what extending AbstractProcessor gets you, etc. A lot of this
> is
> > > currently in the Developer Guide, but in textbook mode
> > > * here’s how to modify some code (i.e. write one line of Java that
> switches
> > > it from straight content pass-through to reversing the text)
> > > * here’s how to make a unit test (introduce the TestRunner framework,
> etc.)
> > > 3. Running, building, installing
> > > * Run your unit test from the IDE/maven
> > > * Build the NAR module
> > > * Install the NAR in NiFi lib/ or custom/
> > > * Restart NiFi
> > > * See the NAR loaded in the log
> > > * Deploy the component on the canvas
> > >
> > > I imagine this being written more conversationally/blog-like than most
> of
> > > our current reference documentation to be used as a split-screen
> > > walkthrough. Each section could certainly link to the existing detailed
> > > documentation for various topics, like the processor lifecycle, etc.
> > >
> > > Does this sounds like something that would have helped you?
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > alopresto.apa...@gmail.com
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > >
> > > > On Jan 25, 2019, at 1:59 PM, James Srinivasan <
> > > james.sriniva...@gmail.com>
> > > wrote:
> > > >
> > > > 9) Oh, and the wiki is a little hard to navigate and the contents
> rather
> > > patchy
> > > >
> > > > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> > > >  wrote:
> > > >>
> > > >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> > > >> realise I could and possibly should submit PRs :)
> > > >>
> > > >> 1) I'm used to Java and Maven, so used the archetype. It worked
> fine,
> > > >> it would have been nice it if set up unit tests for me.
> > > >> 2) The User and Developer documentation is great and comprehensive.
> > > >> Finding the developer docs is a little painful (handful of items at
> > > >> the end of a scrolling list of 200+ processors)
> > > >> 3) The Developer docs could possibly do with a little more clarity
> on
> > > >> processor lifetime i.e. what is called when ^h^h^h - skimming back
> > > >> over the docs, it looks pretty clear now
> > > >> 4) Some example code for common operations e.g. getting/setting
> > > >> attributes or reading/writing/modifying flowfile content would be
> > > >> great.
> > > >> 5) When using existing processors for inspiration, best practices
> > > >> weren't always clear e.g. some generated properties inside
> > > >> getSupportedPropertyDescriptors(), others generated a private static
> > > >> list on init and returned that. Such differences are inevitable in a
> > > >> large project, but it would be nice to have something blessed to
> start
> > > >> from.
> > > >> 6) (Minor niggle - layout of the docs doesn't work great on a phone
> > > screen)
> 

Re: Lowering the barrier of entry

[Bryan Bende]

What does everyone think about bumping the "Developer" section of the
docs ahead of "Processors" so that it's easier to find?

Here is what it would look like -
https://gist.github.com/bbende/279c983f5c80d4fad1431ae81862060f

I also added links to the "Contributor Guide" and the "Maven Projects"
page since I think it would be helpful to make the Developer section
be the one-stop place people look for development help, although I can
see an argument for not mixing wiki content with the docs content.

One issue would be if we want the docs to be fully usable in an
off-line environment, then linking to the wiki won't work, so we could
remove those links, or convert those pages to be part of the docs now
that they are more stable.

For reference, we already have some links in the docs to the wiki:

https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#supplying-a-contribution

On Sat, Jan 26, 2019 at 10:49 AM Joe Witt  wrote:
>
> ...we can.  but the discussion is about how to both lower the bar and offer
> more routes to the bar.
>
> On Sat, Jan 26, 2019, 10:45 AM Otto Fowler 
> > Why wouldn’t we make the archetypes do this?
> >
> >
> > On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org)
> > wrote:
> >
> > James,
> >
> > I’m wondering if a page outlining a toy processor (something like
> > `CountText` or `ReverseContent`) and doing a line-by-line annotation from a
> > developer’s perspective would be helpful. It could be a few sections:
> >
> > 1. How to get to this point
> > * running the maven archetype
> > * choosing the directory to install to
> > * putting the class name in the manifest file
> > * etc.
> > 2. The code
> > * here’s the class
> > * here’s what extending AbstractProcessor gets you, etc. A lot of this is
> > currently in the Developer Guide, but in textbook mode
> > * here’s how to modify some code (i.e. write one line of Java that switches
> > it from straight content pass-through to reversing the text)
> > * here’s how to make a unit test (introduce the TestRunner framework, etc.)
> > 3. Running, building, installing
> > * Run your unit test from the IDE/maven
> > * Build the NAR module
> > * Install the NAR in NiFi lib/ or custom/
> > * Restart NiFi
> > * See the NAR loaded in the log
> > * Deploy the component on the canvas
> >
> > I imagine this being written more conversationally/blog-like than most of
> > our current reference documentation to be used as a split-screen
> > walkthrough. Each section could certainly link to the existing detailed
> > documentation for various topics, like the processor lifecycle, etc.
> >
> > Does this sounds like something that would have helped you?
> >
> > Andy LoPresto
> > alopre...@apache.org
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >
> > > On Jan 25, 2019, at 1:59 PM, James Srinivasan <
> > james.sriniva...@gmail.com>
> > wrote:
> > >
> > > 9) Oh, and the wiki is a little hard to navigate and the contents rather
> > patchy
> > >
> > > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> > >  wrote:
> > >>
> > >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> > >> realise I could and possibly should submit PRs :)
> > >>
> > >> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
> > >> it would have been nice it if set up unit tests for me.
> > >> 2) The User and Developer documentation is great and comprehensive.
> > >> Finding the developer docs is a little painful (handful of items at
> > >> the end of a scrolling list of 200+ processors)
> > >> 3) The Developer docs could possibly do with a little more clarity on
> > >> processor lifetime i.e. what is called when ^h^h^h - skimming back
> > >> over the docs, it looks pretty clear now
> > >> 4) Some example code for common operations e.g. getting/setting
> > >> attributes or reading/writing/modifying flowfile content would be
> > >> great.
> > >> 5) When using existing processors for inspiration, best practices
> > >> weren't always clear e.g. some generated properties inside
> > >> getSupportedPropertyDescriptors(), others generated a private static
> > >> list on init and returned that. Such differences are inevitable in a
> > >> large project, but it would be nice to have something blessed to start
> > >> from.
> > >> 6) (Minor niggle - layout of the docs doesn't work great on a phone
> > screen)
> > >> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
> > >> the great blog posts by Matt Burgess and others were invaluable
> > >> 8) In case this all sounds too negative, NiFi is fab!
> > >>
> > >> On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
> > >>>
> > >>> I am not against the archetype. But we need to spell out every step of
> > the
> > >>> way. I'd like to see a user thinking about their custom logic ASAP
> > rather
> > >>> than fighting the tools to get started. Those steps should be
> > brain-dead,
> > >>> just reflexes, if you know what I mean. 

Re: Lowering the barrier of entry

[Joe Witt]

...we can.  but the discussion is about how to both lower the bar and offer
more routes to the bar.

On Sat, Jan 26, 2019, 10:45 AM Otto Fowler  Why wouldn’t we make the archetypes do this?
>
>
> On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org)
> wrote:
>
> James,
>
> I’m wondering if a page outlining a toy processor (something like
> `CountText` or `ReverseContent`) and doing a line-by-line annotation from a
> developer’s perspective would be helpful. It could be a few sections:
>
> 1. How to get to this point
> * running the maven archetype
> * choosing the directory to install to
> * putting the class name in the manifest file
> * etc.
> 2. The code
> * here’s the class
> * here’s what extending AbstractProcessor gets you, etc. A lot of this is
> currently in the Developer Guide, but in textbook mode
> * here’s how to modify some code (i.e. write one line of Java that switches
> it from straight content pass-through to reversing the text)
> * here’s how to make a unit test (introduce the TestRunner framework, etc.)
> 3. Running, building, installing
> * Run your unit test from the IDE/maven
> * Build the NAR module
> * Install the NAR in NiFi lib/ or custom/
> * Restart NiFi
> * See the NAR loaded in the log
> * Deploy the component on the canvas
>
> I imagine this being written more conversationally/blog-like than most of
> our current reference documentation to be used as a split-screen
> walkthrough. Each section could certainly link to the existing detailed
> documentation for various topics, like the processor lifecycle, etc.
>
> Does this sounds like something that would have helped you?
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> > On Jan 25, 2019, at 1:59 PM, James Srinivasan <
> james.sriniva...@gmail.com>
> wrote:
> >
> > 9) Oh, and the wiki is a little hard to navigate and the contents rather
> patchy
> >
> > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> >  wrote:
> >>
> >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> >> realise I could and possibly should submit PRs :)
> >>
> >> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
> >> it would have been nice it if set up unit tests for me.
> >> 2) The User and Developer documentation is great and comprehensive.
> >> Finding the developer docs is a little painful (handful of items at
> >> the end of a scrolling list of 200+ processors)
> >> 3) The Developer docs could possibly do with a little more clarity on
> >> processor lifetime i.e. what is called when ^h^h^h - skimming back
> >> over the docs, it looks pretty clear now
> >> 4) Some example code for common operations e.g. getting/setting
> >> attributes or reading/writing/modifying flowfile content would be
> >> great.
> >> 5) When using existing processors for inspiration, best practices
> >> weren't always clear e.g. some generated properties inside
> >> getSupportedPropertyDescriptors(), others generated a private static
> >> list on init and returned that. Such differences are inevitable in a
> >> large project, but it would be nice to have something blessed to start
> >> from.
> >> 6) (Minor niggle - layout of the docs doesn't work great on a phone
> screen)
> >> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
> >> the great blog posts by Matt Burgess and others were invaluable
> >> 8) In case this all sounds too negative, NiFi is fab!
> >>
> >> On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
> >>>
> >>> I am not against the archetype. But we need to spell out every step of
> the
> >>> way. I'd like to see a user thinking about their custom logic ASAP
> rather
> >>> than fighting the tools to get started. Those steps should be
> brain-dead,
> >>> just reflexes, if you know what I mean. Hell, let them create a custom
> >>> processor project or prototype in a script by accident even! :)
> >>>
> >>> On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:
> >>>
>  That makes sense about the best practice for deploying to an
>  additional lib directory.
> 
>  So for the project structure you are saying it would be easier to have
>  a repo somewhere with essentially the same thing that is in the
>  archetype, but they just clone it and rename it themselves (what the
>  archetype does for you)?
> 
>  Something that I think would be awesome is if we could provide a
>  web-based project initializer that would essentially run the archetype
>  behind the scenes and then let you download the archive of the code,
>  just like the spring-boot starter [1]. Not sure if their initializr is
>  something that can be re-used and customized [2].
> 
>  The problem is we would need to host that somewhere.
> 
>  [1] https://start.spring.io/
>  [2] https://github.com/spring-io/initializr
> 
>  On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande 
> wrote:

Re: Lowering the barrier of entry

[Otto Fowler]

Why wouldn’t we make the archetypes do this?


On January 25, 2019 at 18:04:25, Andy LoPresto (alopre...@apache.org) wrote:

James,

I’m wondering if a page outlining a toy processor (something like
`CountText` or `ReverseContent`) and doing a line-by-line annotation from a
developer’s perspective would be helpful. It could be a few sections:

1. How to get to this point
* running the maven archetype
* choosing the directory to install to
* putting the class name in the manifest file
* etc.
2. The code
* here’s the class
* here’s what extending AbstractProcessor gets you, etc. A lot of this is
currently in the Developer Guide, but in textbook mode
* here’s how to modify some code (i.e. write one line of Java that switches
it from straight content pass-through to reversing the text)
* here’s how to make a unit test (introduce the TestRunner framework, etc.)
3. Running, building, installing
* Run your unit test from the IDE/maven
* Build the NAR module
* Install the NAR in NiFi lib/ or custom/
* Restart NiFi
* See the NAR loaded in the log
* Deploy the component on the canvas

I imagine this being written more conversationally/blog-like than most of
our current reference documentation to be used as a split-screen
walkthrough. Each section could certainly link to the existing detailed
documentation for various topics, like the processor lifecycle, etc.

Does this sounds like something that would have helped you?

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69

> On Jan 25, 2019, at 1:59 PM, James Srinivasan 
wrote:
>
> 9) Oh, and the wiki is a little hard to navigate and the contents rather
patchy
>
> On Fri, 25 Jan 2019 at 21:57, James Srinivasan
>  wrote:
>>
>> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
>> realise I could and possibly should submit PRs :)
>>
>> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
>> it would have been nice it if set up unit tests for me.
>> 2) The User and Developer documentation is great and comprehensive.
>> Finding the developer docs is a little painful (handful of items at
>> the end of a scrolling list of 200+ processors)
>> 3) The Developer docs could possibly do with a little more clarity on
>> processor lifetime i.e. what is called when ^h^h^h - skimming back
>> over the docs, it looks pretty clear now
>> 4) Some example code for common operations e.g. getting/setting
>> attributes or reading/writing/modifying flowfile content would be
>> great.
>> 5) When using existing processors for inspiration, best practices
>> weren't always clear e.g. some generated properties inside
>> getSupportedPropertyDescriptors(), others generated a private static
>> list on init and returned that. Such differences are inevitable in a
>> large project, but it would be nice to have something blessed to start
>> from.
>> 6) (Minor niggle - layout of the docs doesn't work great on a phone
screen)
>> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
>> the great blog posts by Matt Burgess and others were invaluable
>> 8) In case this all sounds too negative, NiFi is fab!
>>
>> On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
>>>
>>> I am not against the archetype. But we need to spell out every step of
the
>>> way. I'd like to see a user thinking about their custom logic ASAP
rather
>>> than fighting the tools to get started. Those steps should be
brain-dead,
>>> just reflexes, if you know what I mean. Hell, let them create a custom
>>> processor project or prototype in a script by accident even! :)
>>>
>>> On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:
>>>
 That makes sense about the best practice for deploying to an
 additional lib directory.

 So for the project structure you are saying it would be easier to have
 a repo somewhere with essentially the same thing that is in the
 archetype, but they just clone it and rename it themselves (what the
 archetype does for you)?

 Something that I think would be awesome is if we could provide a
 web-based project initializer that would essentially run the archetype
 behind the scenes and then let you download the archive of the code,
 just like the spring-boot starter [1]. Not sure if their initializr is
 something that can be re-used and customized [2].

 The problem is we would need to host that somewhere.

 [1] https://start.spring.io/
 [2] https://github.com/spring-io/initializr

 On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande 
wrote:
>
> We assume they create new projects from archetypes every day. They
don't.
>
> We also assume they know how to deploy new NARs. Most don't.
Especially
 if
> we want them to follow best practices and create an additional NAR
 bundles
> directory entry im the config (vs dumping into nifi lib).
>
> I can attest that I feel a bit lost myself every 

Re: Lowering the barrier of entry

[James Srinivasan]

That would have been great - the only things I would add are
reading/writing attributes & reading the processor's properties. Maybe
something about testing in NiFi using GenerateFlowFile too?

I ended up referring to the bundled processors a lot, but it was
sometimes hard to see the wood for the trees.

On Fri, 25 Jan 2019 at 23:04, Andy LoPresto  wrote:
>
> James,
>
> I’m wondering if a page outlining a toy processor (something like `CountText` 
> or `ReverseContent`) and doing a line-by-line annotation from a developer’s 
> perspective would be helpful. It could be a few sections:
>
> 1. How to get to this point
> * running the maven archetype
> * choosing the directory to install to
> * putting the class name in the manifest file
> * etc.
> 2. The code
> * here’s the class
> * here’s what extending AbstractProcessor gets you, etc. A lot of 
> this is currently in the Developer Guide, but in textbook mode
> * here’s how to modify some code (i.e. write one line of Java that 
> switches it from straight content pass-through to reversing the text)
> * here’s how to make a unit test (introduce the TestRunner framework, 
> etc.)
> 3. Running, building, installing
> * Run your unit test from the IDE/maven
> * Build the NAR module
> * Install the NAR in NiFi lib/ or custom/
> * Restart NiFi
> * See the NAR loaded in the log
> * Deploy the component on the canvas
>
> I imagine this being written more conversationally/blog-like than most of our 
> current reference documentation to be used as a split-screen walkthrough. 
> Each section could certainly link to the existing detailed documentation for 
> various topics, like the processor lifecycle, etc.
>
> Does this sounds like something that would have helped you?
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Jan 25, 2019, at 1:59 PM, James Srinivasan  
> > wrote:
> >
> > 9) Oh, and the wiki is a little hard to navigate and the contents rather 
> > patchy
> >
> > On Fri, 25 Jan 2019 at 21:57, James Srinivasan
> >  wrote:
> >>
> >> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> >> realise I could and possibly should submit PRs :)
> >>
> >> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
> >> it would have been nice it if set up unit tests for me.
> >> 2) The User and Developer documentation is great and comprehensive.
> >> Finding the developer docs is a little painful (handful of items at
> >> the end of a scrolling list of 200+ processors)
> >> 3) The Developer docs could possibly do with a little more clarity on
> >> processor lifetime i.e. what is called when ^h^h^h - skimming back
> >> over the docs, it looks pretty clear now
> >> 4) Some example code for common operations e.g. getting/setting
> >> attributes or reading/writing/modifying flowfile content would be
> >> great.
> >> 5) When using existing processors for inspiration, best practices
> >> weren't always clear e.g. some generated properties inside
> >> getSupportedPropertyDescriptors(), others generated a private static
> >> list on init and returned that. Such differences are inevitable in a
> >> large project, but it would be nice to have something blessed to start
> >> from.
> >> 6) (Minor niggle - layout of the docs doesn't work great on a phone screen)
> >> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
> >> the great blog posts by Matt Burgess and others were invaluable
> >> 8) In case this all sounds too negative, NiFi is fab!
> >>
> >> On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
> >>>
> >>> I am not against the archetype. But we need to spell out every step of the
> >>> way. I'd like to see a user thinking about their custom logic ASAP rather
> >>> than fighting the tools to get started. Those steps should be brain-dead,
> >>> just reflexes, if you know what I mean. Hell, let them create a custom
> >>> processor project or prototype in a script by accident even! :)
> >>>
> >>> On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:
> >>>
>  That makes sense about the best practice for deploying to an
>  additional lib directory.
> 
>  So for the project structure you are saying it would be easier to have
>  a repo somewhere with essentially the same thing that is in the
>  archetype, but they just clone it and rename it themselves (what the
>  archetype does for you)?
> 
>  Something that I think would be awesome is if we could provide a
>  web-based project initializer that would essentially run the archetype
>  behind the scenes and then let you download the archive of the code,
>  just like the spring-boot starter [1]. Not sure if their initializr is
>  something that can be re-used and customized [2].
> 
>  The problem is we would need 

Re: Lowering the barrier of entry

[Andy LoPresto]

James,

I’m wondering if a page outlining a toy processor (something like `CountText` 
or `ReverseContent`) and doing a line-by-line annotation from a developer’s 
perspective would be helpful. It could be a few sections:

1. How to get to this point
* running the maven archetype
* choosing the directory to install to
* putting the class name in the manifest file
* etc.
2. The code
* here’s the class
* here’s what extending AbstractProcessor gets you, etc. A lot of this 
is currently in the Developer Guide, but in textbook mode
* here’s how to modify some code (i.e. write one line of Java that 
switches it from straight content pass-through to reversing the text)
* here’s how to make a unit test (introduce the TestRunner framework, 
etc.)
3. Running, building, installing
* Run your unit test from the IDE/maven
* Build the NAR module
* Install the NAR in NiFi lib/ or custom/
* Restart NiFi
* See the NAR loaded in the log
* Deploy the component on the canvas

I imagine this being written more conversationally/blog-like than most of our 
current reference documentation to be used as a split-screen walkthrough. Each 
section could certainly link to the existing detailed documentation for various 
topics, like the processor lifecycle, etc. 

Does this sounds like something that would have helped you?

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jan 25, 2019, at 1:59 PM, James Srinivasan  
> wrote:
> 
> 9) Oh, and the wiki is a little hard to navigate and the contents rather 
> patchy
> 
> On Fri, 25 Jan 2019 at 21:57, James Srinivasan
>  wrote:
>> 
>> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
>> realise I could and possibly should submit PRs :)
>> 
>> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
>> it would have been nice it if set up unit tests for me.
>> 2) The User and Developer documentation is great and comprehensive.
>> Finding the developer docs is a little painful (handful of items at
>> the end of a scrolling list of 200+ processors)
>> 3) The Developer docs could possibly do with a little more clarity on
>> processor lifetime i.e. what is called when ^h^h^h - skimming back
>> over the docs, it looks pretty clear now
>> 4) Some example code for common operations e.g. getting/setting
>> attributes or reading/writing/modifying flowfile content would be
>> great.
>> 5) When using existing processors for inspiration, best practices
>> weren't always clear e.g. some generated properties inside
>> getSupportedPropertyDescriptors(), others generated a private static
>> list on init and returned that. Such differences are inevitable in a
>> large project, but it would be nice to have something blessed to start
>> from.
>> 6) (Minor niggle - layout of the docs doesn't work great on a phone screen)
>> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
>> the great blog posts by Matt Burgess and others were invaluable
>> 8) In case this all sounds too negative, NiFi is fab!
>> 
>> On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
>>> 
>>> I am not against the archetype. But we need to spell out every step of the
>>> way. I'd like to see a user thinking about their custom logic ASAP rather
>>> than fighting the tools to get started. Those steps should be brain-dead,
>>> just reflexes, if you know what I mean. Hell, let them create a custom
>>> processor project or prototype in a script by accident even! :)
>>> 
>>> On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:
>>> 
 That makes sense about the best practice for deploying to an
 additional lib directory.
 
 So for the project structure you are saying it would be easier to have
 a repo somewhere with essentially the same thing that is in the
 archetype, but they just clone it and rename it themselves (what the
 archetype does for you)?
 
 Something that I think would be awesome is if we could provide a
 web-based project initializer that would essentially run the archetype
 behind the scenes and then let you download the archive of the code,
 just like the spring-boot starter [1]. Not sure if their initializr is
 something that can be re-used and customized [2].
 
 The problem is we would need to host that somewhere.
 
 [1] https://start.spring.io/
 [2] https://github.com/spring-io/initializr
 
 On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande  wrote:
> 
> We assume they create new projects from archetypes every day. They don't.
> 
> We also assume they know how to deploy new NARs. Most don't. Especially
 if
> we want them to follow best practices and create an additional NAR
 bundles
> directory entry im the config (vs dumping into nifi lib).
> 
> I can attest 

Re: Lowering the barrier of entry

[James Srinivasan]

9) Oh, and the wiki is a little hard to navigate and the contents rather patchy

On Fri, 25 Jan 2019 at 21:57, James Srinivasan
 wrote:
>
> As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
> realise I could and possibly should submit PRs :)
>
> 1) I'm used to Java and Maven, so used the archetype. It worked fine,
> it would have been nice it if set up unit tests for me.
> 2) The User and Developer documentation is great and comprehensive.
> Finding the developer docs is a little painful (handful of items at
> the end of a scrolling list of 200+ processors)
> 3) The Developer docs could possibly do with a little more clarity on
> processor lifetime i.e. what is called when ^h^h^h - skimming back
> over the docs, it looks pretty clear now
> 4) Some example code for common operations e.g. getting/setting
> attributes or reading/writing/modifying flowfile content would be
> great.
> 5) When using existing processors for inspiration, best practices
> weren't always clear e.g. some generated properties inside
> getSupportedPropertyDescriptors(), others generated a private static
> list on init and returned that. Such differences are inevitable in a
> large project, but it would be nice to have something blessed to start
> from.
> 6) (Minor niggle - layout of the docs doesn't work great on a phone screen)
> 7) I couldn't find (m?)any docs about the Groovy scripting API, but
> the great blog posts by Matt Burgess and others were invaluable
> 8) In case this all sounds too negative, NiFi is fab!
>
> On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
> >
> > I am not against the archetype. But we need to spell out every step of the
> > way. I'd like to see a user thinking about their custom logic ASAP rather
> > than fighting the tools to get started. Those steps should be brain-dead,
> > just reflexes, if you know what I mean. Hell, let them create a custom
> > processor project or prototype in a script by accident even! :)
> >
> > On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:
> >
> > > That makes sense about the best practice for deploying to an
> > > additional lib directory.
> > >
> > > So for the project structure you are saying it would be easier to have
> > > a repo somewhere with essentially the same thing that is in the
> > > archetype, but they just clone it and rename it themselves (what the
> > > archetype does for you)?
> > >
> > > Something that I think would be awesome is if we could provide a
> > > web-based project initializer that would essentially run the archetype
> > > behind the scenes and then let you download the archive of the code,
> > > just like the spring-boot starter [1]. Not sure if their initializr is
> > > something that can be re-used and customized [2].
> > >
> > > The problem is we would need to host that somewhere.
> > >
> > > [1] https://start.spring.io/
> > > [2] https://github.com/spring-io/initializr
> > >
> > > On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande  wrote:
> > > >
> > > > We assume they create new projects from archetypes every day. They 
> > > > don't.
> > > >
> > > > We also assume they know how to deploy new NARs. Most don't. Especially
> > > if
> > > > we want them to follow best practices and create an additional NAR
> > > bundles
> > > > directory entry im the config (vs dumping into nifi lib).
> > > >
> > > > I can attest that I feel a bit lost myself every time I need to come 
> > > > back
> > > > to this and refresh my brain synapses. If we could make these not 
> > > > require
> > > > any of that and make simple thinga dead simple
> > > >
> > > > Andrew
> > > >
> > > > On Fri, Jan 25, 2019, 9:47 AM Bryan Bende  wrote:
> > > >
> > > > > Andrew,
> > > > >
> > > > > I'm not disagreeing with your points, but I'm curious how you see
> > > > > those two ideas being different from the processor archetype and the
> > > > > wiki page with the archetype commands?
> > > > >
> > > > > Is it just that people don't know about it?
> > > > >
> > > > > -Bryan
> > > > >
> > > > > [1]
> > > > >
> > > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
> > > > >
> > > > > On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler 
> > > > > wrote:
> > > > > >
> > > > > > I think this ties into my other discuss thread on refreshing the
> > > > > archetypes
> > > > > >
> > > > > >
> > > > > > On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com)
> > > > > wrote:
> > > > > >
> > > > > > I consistently see my users struggling when they move up the nifi
> > > food
> > > > > > chain and start looking at custom processors. The good content about
> > > > > > prototyping processsors via scripting processors and finalizing with
> > > a
> > > > > full
> > > > > > NAR bundle is everywhere but where it should be.
> > > > > >
> > > > > > A few simple changes could help (not *more* docs). They are great,
> > > much
> > > > > > better than in many other projecta, but people are already drowning
> > > in
> > > > > > those.
> > > > > >
> > > 

Re: Lowering the barrier of entry

[James Srinivasan]

As someone relatively new to NiFi dev, here's my £0.02. (Yes, I
realise I could and possibly should submit PRs :)

1) I'm used to Java and Maven, so used the archetype. It worked fine,
it would have been nice it if set up unit tests for me.
2) The User and Developer documentation is great and comprehensive.
Finding the developer docs is a little painful (handful of items at
the end of a scrolling list of 200+ processors)
3) The Developer docs could possibly do with a little more clarity on
processor lifetime i.e. what is called when ^h^h^h - skimming back
over the docs, it looks pretty clear now
4) Some example code for common operations e.g. getting/setting
attributes or reading/writing/modifying flowfile content would be
great.
5) When using existing processors for inspiration, best practices
weren't always clear e.g. some generated properties inside
getSupportedPropertyDescriptors(), others generated a private static
list on init and returned that. Such differences are inevitable in a
large project, but it would be nice to have something blessed to start
from.
6) (Minor niggle - layout of the docs doesn't work great on a phone screen)
7) I couldn't find (m?)any docs about the Groovy scripting API, but
the great blog posts by Matt Burgess and others were invaluable
8) In case this all sounds too negative, NiFi is fab!

On Fri, 25 Jan 2019 at 18:47, Andrew Grande  wrote:
>
> I am not against the archetype. But we need to spell out every step of the
> way. I'd like to see a user thinking about their custom logic ASAP rather
> than fighting the tools to get started. Those steps should be brain-dead,
> just reflexes, if you know what I mean. Hell, let them create a custom
> processor project or prototype in a script by accident even! :)
>
> On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:
>
> > That makes sense about the best practice for deploying to an
> > additional lib directory.
> >
> > So for the project structure you are saying it would be easier to have
> > a repo somewhere with essentially the same thing that is in the
> > archetype, but they just clone it and rename it themselves (what the
> > archetype does for you)?
> >
> > Something that I think would be awesome is if we could provide a
> > web-based project initializer that would essentially run the archetype
> > behind the scenes and then let you download the archive of the code,
> > just like the spring-boot starter [1]. Not sure if their initializr is
> > something that can be re-used and customized [2].
> >
> > The problem is we would need to host that somewhere.
> >
> > [1] https://start.spring.io/
> > [2] https://github.com/spring-io/initializr
> >
> > On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande  wrote:
> > >
> > > We assume they create new projects from archetypes every day. They don't.
> > >
> > > We also assume they know how to deploy new NARs. Most don't. Especially
> > if
> > > we want them to follow best practices and create an additional NAR
> > bundles
> > > directory entry im the config (vs dumping into nifi lib).
> > >
> > > I can attest that I feel a bit lost myself every time I need to come back
> > > to this and refresh my brain synapses. If we could make these not require
> > > any of that and make simple thinga dead simple
> > >
> > > Andrew
> > >
> > > On Fri, Jan 25, 2019, 9:47 AM Bryan Bende  wrote:
> > >
> > > > Andrew,
> > > >
> > > > I'm not disagreeing with your points, but I'm curious how you see
> > > > those two ideas being different from the processor archetype and the
> > > > wiki page with the archetype commands?
> > > >
> > > > Is it just that people don't know about it?
> > > >
> > > > -Bryan
> > > >
> > > > [1]
> > > >
> > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
> > > >
> > > > On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler 
> > > > wrote:
> > > > >
> > > > > I think this ties into my other discuss thread on refreshing the
> > > > archetypes
> > > > >
> > > > >
> > > > > On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com)
> > > > wrote:
> > > > >
> > > > > I consistently see my users struggling when they move up the nifi
> > food
> > > > > chain and start looking at custom processors. The good content about
> > > > > prototyping processsors via scripting processors and finalizing with
> > a
> > > > full
> > > > > NAR bundle is everywhere but where it should be.
> > > > >
> > > > > A few simple changes could help (not *more* docs). They are great,
> > much
> > > > > better than in many other projecta, but people are already drowning
> > in
> > > > > those.
> > > > >
> > > > > How about:
> > > > >
> > > > > + ISP has a pre-populated processor sceleton. A simple no-op to fill
> > in
> > > > is
> > > > > miles better than a blank text area (which invokes a blank stare).
> > > > >
> > > > > + As much as we may loook down on this, but... A simple guide to a
> > full
> > > > NAR
> > > > > build as a series of copy/paste commands.
> > > > >
> > > > > 

Re: Lowering the barrier of entry

[Andrew Grande]

I am not against the archetype. But we need to spell out every step of the
way. I'd like to see a user thinking about their custom logic ASAP rather
than fighting the tools to get started. Those steps should be brain-dead,
just reflexes, if you know what I mean. Hell, let them create a custom
processor project or prototype in a script by accident even! :)

On Fri, Jan 25, 2019, 10:43 AM Bryan Bende  wrote:

> That makes sense about the best practice for deploying to an
> additional lib directory.
>
> So for the project structure you are saying it would be easier to have
> a repo somewhere with essentially the same thing that is in the
> archetype, but they just clone it and rename it themselves (what the
> archetype does for you)?
>
> Something that I think would be awesome is if we could provide a
> web-based project initializer that would essentially run the archetype
> behind the scenes and then let you download the archive of the code,
> just like the spring-boot starter [1]. Not sure if their initializr is
> something that can be re-used and customized [2].
>
> The problem is we would need to host that somewhere.
>
> [1] https://start.spring.io/
> [2] https://github.com/spring-io/initializr
>
> On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande  wrote:
> >
> > We assume they create new projects from archetypes every day. They don't.
> >
> > We also assume they know how to deploy new NARs. Most don't. Especially
> if
> > we want them to follow best practices and create an additional NAR
> bundles
> > directory entry im the config (vs dumping into nifi lib).
> >
> > I can attest that I feel a bit lost myself every time I need to come back
> > to this and refresh my brain synapses. If we could make these not require
> > any of that and make simple thinga dead simple
> >
> > Andrew
> >
> > On Fri, Jan 25, 2019, 9:47 AM Bryan Bende  wrote:
> >
> > > Andrew,
> > >
> > > I'm not disagreeing with your points, but I'm curious how you see
> > > those two ideas being different from the processor archetype and the
> > > wiki page with the archetype commands?
> > >
> > > Is it just that people don't know about it?
> > >
> > > -Bryan
> > >
> > > [1]
> > >
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
> > >
> > > On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler 
> > > wrote:
> > > >
> > > > I think this ties into my other discuss thread on refreshing the
> > > archetypes
> > > >
> > > >
> > > > On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com)
> > > wrote:
> > > >
> > > > I consistently see my users struggling when they move up the nifi
> food
> > > > chain and start looking at custom processors. The good content about
> > > > prototyping processsors via scripting processors and finalizing with
> a
> > > full
> > > > NAR bundle is everywhere but where it should be.
> > > >
> > > > A few simple changes could help (not *more* docs). They are great,
> much
> > > > better than in many other projecta, but people are already drowning
> in
> > > > those.
> > > >
> > > > How about:
> > > >
> > > > + ISP has a pre-populated processor sceleton. A simple no-op to fill
> in
> > > is
> > > > miles better than a blank text area (which invokes a blank stare).
> > > >
> > > > + As much as we may loook down on this, but... A simple guide to a
> full
> > > NAR
> > > > build as a series of copy/paste commands.
> > > >
> > > > There's more, but this should fit the context for now.
> > > >
> > > > Andrew
> > > >
> > > > On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen 
> > > wrote:
> > > >
> > > > > One of the changes we should make is to create a separate guide for
> > > > product
> > > > > vendors on how to build and maintain a bundle. We're at that point
> > > where
> > > > > vendors will have to do it on their own as extension providers, so
> it
> > > > would
> > > > > be very helpful for them to have a simple and straight forward
> document
> > > > > showing them what should be there, best practices for
> maintainability
> > > and
> > > > > where to announce it.
> > > > >
> > > > > On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende 
> wrote:
> > > > >
> > > > > > I think we have a lot more documentation than most projects, but
> I
> > > > > > think an issue is that content is scattered in many different
> > > > > > locations, and some of the docs are huge reference guides where
> it
> > > can
> > > > > > be hard to find all the pieces of what you are trying to do.
> > > > > >
> > > > > > The first thing a new contributor wants to do is get the code
> and run
> > > > > > a build, and we do have a quick-start guide linked to on the
> site,
> > > but
> > > > > > I think there is a lot of extra information in there that is not
> > > > > > really relevant to someone just wanting get the code and build.
> We
> > > > > > could have separate guides per OS like "Build NiFi on Linux",
> "Build
> > > > > > NiFi on Windows", etc, where each guide was 4-5 steps like:
> > > > > >
> > > > > > - Clone repo
> > 

Re: Lowering the barrier of entry

[Bryan Bende]

That makes sense about the best practice for deploying to an
additional lib directory.

So for the project structure you are saying it would be easier to have
a repo somewhere with essentially the same thing that is in the
archetype, but they just clone it and rename it themselves (what the
archetype does for you)?

Something that I think would be awesome is if we could provide a
web-based project initializer that would essentially run the archetype
behind the scenes and then let you download the archive of the code,
just like the spring-boot starter [1]. Not sure if their initializr is
something that can be re-used and customized [2].

The problem is we would need to host that somewhere.

[1] https://start.spring.io/
[2] https://github.com/spring-io/initializr

On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande  wrote:
>
> We assume they create new projects from archetypes every day. They don't.
>
> We also assume they know how to deploy new NARs. Most don't. Especially if
> we want them to follow best practices and create an additional NAR bundles
> directory entry im the config (vs dumping into nifi lib).
>
> I can attest that I feel a bit lost myself every time I need to come back
> to this and refresh my brain synapses. If we could make these not require
> any of that and make simple thinga dead simple
>
> Andrew
>
> On Fri, Jan 25, 2019, 9:47 AM Bryan Bende  wrote:
>
> > Andrew,
> >
> > I'm not disagreeing with your points, but I'm curious how you see
> > those two ideas being different from the processor archetype and the
> > wiki page with the archetype commands?
> >
> > Is it just that people don't know about it?
> >
> > -Bryan
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
> >
> > On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler 
> > wrote:
> > >
> > > I think this ties into my other discuss thread on refreshing the
> > archetypes
> > >
> > >
> > > On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com)
> > wrote:
> > >
> > > I consistently see my users struggling when they move up the nifi food
> > > chain and start looking at custom processors. The good content about
> > > prototyping processsors via scripting processors and finalizing with a
> > full
> > > NAR bundle is everywhere but where it should be.
> > >
> > > A few simple changes could help (not *more* docs). They are great, much
> > > better than in many other projecta, but people are already drowning in
> > > those.
> > >
> > > How about:
> > >
> > > + ISP has a pre-populated processor sceleton. A simple no-op to fill in
> > is
> > > miles better than a blank text area (which invokes a blank stare).
> > >
> > > + As much as we may loook down on this, but... A simple guide to a full
> > NAR
> > > build as a series of copy/paste commands.
> > >
> > > There's more, but this should fit the context for now.
> > >
> > > Andrew
> > >
> > > On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen 
> > wrote:
> > >
> > > > One of the changes we should make is to create a separate guide for
> > > product
> > > > vendors on how to build and maintain a bundle. We're at that point
> > where
> > > > vendors will have to do it on their own as extension providers, so it
> > > would
> > > > be very helpful for them to have a simple and straight forward document
> > > > showing them what should be there, best practices for maintainability
> > and
> > > > where to announce it.
> > > >
> > > > On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:
> > > >
> > > > > I think we have a lot more documentation than most projects, but I
> > > > > think an issue is that content is scattered in many different
> > > > > locations, and some of the docs are huge reference guides where it
> > can
> > > > > be hard to find all the pieces of what you are trying to do.
> > > > >
> > > > > The first thing a new contributor wants to do is get the code and run
> > > > > a build, and we do have a quick-start guide linked to on the site,
> > but
> > > > > I think there is a lot of extra information in there that is not
> > > > > really relevant to someone just wanting get the code and build. We
> > > > > could have separate guides per OS like "Build NiFi on Linux", "Build
> > > > > NiFi on Windows", etc, where each guide was 4-5 steps like:
> > > > >
> > > > > - Clone repo
> > > > > - checkout master
> > > > > - run maven
> > > > > - cd to assembly
> > > > > - ./bin/nifi.sh
> > > > >
> > > > > The next thing they want to do is contribute a change, and we have a
> > > > > great contributor guide, but again I think there could be a very
> > short
> > > > > tutorial for the most common steps:
> > > > >
> > > > > - fork repo
> > > > > - clone fork
> > > > > - create branch
> > > > > - make changes
> > > > > - push branch
> > > > > - submit pr
> > > > >
> > > > > and then say something like "for a more detailed description of the
> > > > > contribution process, please reference the Contributor Guide".
> > > > >
> > > > > If we then make 

Re: Lowering the barrier of entry

[Andrew Grande]

We assume they create new projects from archetypes every day. They don't.

We also assume they know how to deploy new NARs. Most don't. Especially if
we want them to follow best practices and create an additional NAR bundles
directory entry im the config (vs dumping into nifi lib).

I can attest that I feel a bit lost myself every time I need to come back
to this and refresh my brain synapses. If we could make these not require
any of that and make simple thinga dead simple

Andrew

On Fri, Jan 25, 2019, 9:47 AM Bryan Bende  wrote:

> Andrew,
>
> I'm not disagreeing with your points, but I'm curious how you see
> those two ideas being different from the processor archetype and the
> wiki page with the archetype commands?
>
> Is it just that people don't know about it?
>
> -Bryan
>
> [1]
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
>
> On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler 
> wrote:
> >
> > I think this ties into my other discuss thread on refreshing the
> archetypes
> >
> >
> > On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com)
> wrote:
> >
> > I consistently see my users struggling when they move up the nifi food
> > chain and start looking at custom processors. The good content about
> > prototyping processsors via scripting processors and finalizing with a
> full
> > NAR bundle is everywhere but where it should be.
> >
> > A few simple changes could help (not *more* docs). They are great, much
> > better than in many other projecta, but people are already drowning in
> > those.
> >
> > How about:
> >
> > + ISP has a pre-populated processor sceleton. A simple no-op to fill in
> is
> > miles better than a blank text area (which invokes a blank stare).
> >
> > + As much as we may loook down on this, but... A simple guide to a full
> NAR
> > build as a series of copy/paste commands.
> >
> > There's more, but this should fit the context for now.
> >
> > Andrew
> >
> > On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen 
> wrote:
> >
> > > One of the changes we should make is to create a separate guide for
> > product
> > > vendors on how to build and maintain a bundle. We're at that point
> where
> > > vendors will have to do it on their own as extension providers, so it
> > would
> > > be very helpful for them to have a simple and straight forward document
> > > showing them what should be there, best practices for maintainability
> and
> > > where to announce it.
> > >
> > > On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:
> > >
> > > > I think we have a lot more documentation than most projects, but I
> > > > think an issue is that content is scattered in many different
> > > > locations, and some of the docs are huge reference guides where it
> can
> > > > be hard to find all the pieces of what you are trying to do.
> > > >
> > > > The first thing a new contributor wants to do is get the code and run
> > > > a build, and we do have a quick-start guide linked to on the site,
> but
> > > > I think there is a lot of extra information in there that is not
> > > > really relevant to someone just wanting get the code and build. We
> > > > could have separate guides per OS like "Build NiFi on Linux", "Build
> > > > NiFi on Windows", etc, where each guide was 4-5 steps like:
> > > >
> > > > - Clone repo
> > > > - checkout master
> > > > - run maven
> > > > - cd to assembly
> > > > - ./bin/nifi.sh
> > > >
> > > > The next thing they want to do is contribute a change, and we have a
> > > > great contributor guide, but again I think there could be a very
> short
> > > > tutorial for the most common steps:
> > > >
> > > > - fork repo
> > > > - clone fork
> > > > - create branch
> > > > - make changes
> > > > - push branch
> > > > - submit pr
> > > >
> > > > and then say something like "for a more detailed description of the
> > > > contribution process, please reference the Contributor Guide".
> > > >
> > > > If we then make these getting started guides more prominent right in
> > > > the middle of the NiFi homepage, then maybe they will be easier to
> > > > find for new community members.
> > > >
> > > > We can keep extending this idea to other common tasks beyond just
> > > > building and contributing.
> > > >
> > > >
> > > > On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto 
> > > > wrote:
> > > > >
> > > > > Hi folks,
> > > > >
> > > > > Based on some recent (and long-term) experiences, I wanted to
> discuss
> > > > with the community what we could do to lower the barrier of entry to
> > > using
> > > > & contributing to NiFi. I hope to get some good feedback from both
> > > > long-time and newer members, and determine some immediate concrete
> > steps
> > > we
> > > > can take.
> > > > >
> > > > > Problems identified:
> > > > > * NiFi has a number of custom profiles, so a simple “mvn clean
> > install”
> > > > in project root doesn’t get a new developer up and running
> immediately
> > > > > * The API is very well defined, but for new contributors, it can
> 

Re: Lowering the barrier of entry

[Bryan Bende]

Andrew,

I'm not disagreeing with your points, but I'm curious how you see
those two ideas being different from the processor archetype and the
wiki page with the archetype commands?

Is it just that people don't know about it?

-Bryan

[1] 
https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions

On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler  wrote:
>
> I think this ties into my other discuss thread on refreshing the archetypes
>
>
> On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com) wrote:
>
> I consistently see my users struggling when they move up the nifi food
> chain and start looking at custom processors. The good content about
> prototyping processsors via scripting processors and finalizing with a full
> NAR bundle is everywhere but where it should be.
>
> A few simple changes could help (not *more* docs). They are great, much
> better than in many other projecta, but people are already drowning in
> those.
>
> How about:
>
> + ISP has a pre-populated processor sceleton. A simple no-op to fill in is
> miles better than a blank text area (which invokes a blank stare).
>
> + As much as we may loook down on this, but... A simple guide to a full NAR
> build as a series of copy/paste commands.
>
> There's more, but this should fit the context for now.
>
> Andrew
>
> On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen  wrote:
>
> > One of the changes we should make is to create a separate guide for
> product
> > vendors on how to build and maintain a bundle. We're at that point where
> > vendors will have to do it on their own as extension providers, so it
> would
> > be very helpful for them to have a simple and straight forward document
> > showing them what should be there, best practices for maintainability and
> > where to announce it.
> >
> > On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:
> >
> > > I think we have a lot more documentation than most projects, but I
> > > think an issue is that content is scattered in many different
> > > locations, and some of the docs are huge reference guides where it can
> > > be hard to find all the pieces of what you are trying to do.
> > >
> > > The first thing a new contributor wants to do is get the code and run
> > > a build, and we do have a quick-start guide linked to on the site, but
> > > I think there is a lot of extra information in there that is not
> > > really relevant to someone just wanting get the code and build. We
> > > could have separate guides per OS like "Build NiFi on Linux", "Build
> > > NiFi on Windows", etc, where each guide was 4-5 steps like:
> > >
> > > - Clone repo
> > > - checkout master
> > > - run maven
> > > - cd to assembly
> > > - ./bin/nifi.sh
> > >
> > > The next thing they want to do is contribute a change, and we have a
> > > great contributor guide, but again I think there could be a very short
> > > tutorial for the most common steps:
> > >
> > > - fork repo
> > > - clone fork
> > > - create branch
> > > - make changes
> > > - push branch
> > > - submit pr
> > >
> > > and then say something like "for a more detailed description of the
> > > contribution process, please reference the Contributor Guide".
> > >
> > > If we then make these getting started guides more prominent right in
> > > the middle of the NiFi homepage, then maybe they will be easier to
> > > find for new community members.
> > >
> > > We can keep extending this idea to other common tasks beyond just
> > > building and contributing.
> > >
> > >
> > > On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto 
> > > wrote:
> > > >
> > > > Hi folks,
> > > >
> > > > Based on some recent (and long-term) experiences, I wanted to discuss
> > > with the community what we could do to lower the barrier of entry to
> > using
> > > & contributing to NiFi. I hope to get some good feedback from both
> > > long-time and newer members, and determine some immediate concrete
> steps
> > we
> > > can take.
> > > >
> > > > Problems identified:
> > > > * NiFi has a number of custom profiles, so a simple “mvn clean
> install”
> > > in project root doesn’t get a new developer up and running immediately
> > > > * The API is very well defined, but for new contributors, it can be a
> > > challenge to know where to put functionality, and building a custom
> > > processor + NAR and deploying isn’t a one-step process
> > > > * Project size (and build size/time) is large. This can restrict the
> > > minimum hardware necessary, elongate the development cycle, etc.
> > > > * Some new users do not receive mailing list replies
> > > >
> > > > Possible solutions:
> > > > * On a clean git clone, “mvn clean install” should build a working
> > > instance. Maybe we provide a quickstart.sh script to handle the default
> > > maven build, change to the target directory, and start NiFi?
> > > > * Individual contributors have written excellent blogs, and
> > > documentation exists, but making it more prominent or more easily
> > accessed
> > > could help?
> > > > * 

Re: Lowering the barrier of entry

[Otto Fowler]

I think this ties into my other discuss thread on refreshing the archetypes


On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com) wrote:

I consistently see my users struggling when they move up the nifi food
chain and start looking at custom processors. The good content about
prototyping processsors via scripting processors and finalizing with a full
NAR bundle is everywhere but where it should be.

A few simple changes could help (not *more* docs). They are great, much
better than in many other projecta, but people are already drowning in
those.

How about:

+ ISP has a pre-populated processor sceleton. A simple no-op to fill in is
miles better than a blank text area (which invokes a blank stare).

+ As much as we may loook down on this, but... A simple guide to a full NAR
build as a series of copy/paste commands.

There's more, but this should fit the context for now.

Andrew

On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen  wrote:

> One of the changes we should make is to create a separate guide for
product
> vendors on how to build and maintain a bundle. We're at that point where
> vendors will have to do it on their own as extension providers, so it
would
> be very helpful for them to have a simple and straight forward document
> showing them what should be there, best practices for maintainability and
> where to announce it.
>
> On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:
>
> > I think we have a lot more documentation than most projects, but I
> > think an issue is that content is scattered in many different
> > locations, and some of the docs are huge reference guides where it can
> > be hard to find all the pieces of what you are trying to do.
> >
> > The first thing a new contributor wants to do is get the code and run
> > a build, and we do have a quick-start guide linked to on the site, but
> > I think there is a lot of extra information in there that is not
> > really relevant to someone just wanting get the code and build. We
> > could have separate guides per OS like "Build NiFi on Linux", "Build
> > NiFi on Windows", etc, where each guide was 4-5 steps like:
> >
> > - Clone repo
> > - checkout master
> > - run maven
> > - cd to assembly
> > - ./bin/nifi.sh
> >
> > The next thing they want to do is contribute a change, and we have a
> > great contributor guide, but again I think there could be a very short
> > tutorial for the most common steps:
> >
> > - fork repo
> > - clone fork
> > - create branch
> > - make changes
> > - push branch
> > - submit pr
> >
> > and then say something like "for a more detailed description of the
> > contribution process, please reference the Contributor Guide".
> >
> > If we then make these getting started guides more prominent right in
> > the middle of the NiFi homepage, then maybe they will be easier to
> > find for new community members.
> >
> > We can keep extending this idea to other common tasks beyond just
> > building and contributing.
> >
> >
> > On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto 
> > wrote:
> > >
> > > Hi folks,
> > >
> > > Based on some recent (and long-term) experiences, I wanted to discuss
> > with the community what we could do to lower the barrier of entry to
> using
> > & contributing to NiFi. I hope to get some good feedback from both
> > long-time and newer members, and determine some immediate concrete
steps
> we
> > can take.
> > >
> > > Problems identified:
> > > * NiFi has a number of custom profiles, so a simple “mvn clean
install”
> > in project root doesn’t get a new developer up and running immediately
> > > * The API is very well defined, but for new contributors, it can be a
> > challenge to know where to put functionality, and building a custom
> > processor + NAR and deploying isn’t a one-step process
> > > * Project size (and build size/time) is large. This can restrict the
> > minimum hardware necessary, elongate the development cycle, etc.
> > > * Some new users do not receive mailing list replies
> > >
> > > Possible solutions:
> > > * On a clean git clone, “mvn clean install” should build a working
> > instance. Maybe we provide a quickstart.sh script to handle the default
> > maven build, change to the target directory, and start NiFi?
> > > * Individual contributors have written excellent blogs, and
> > documentation exists, but making it more prominent or more easily
> accessed
> > could help?
> > > * Extension registry will solve all the world’s problems (related to
> > bundling and build time)
> > > * Not sure about this one — I don’t know if it’s because they’re not
> > subscribed, their mail client is blocking them, etc.
> > >
> > > I’ve said my bit, now I am eager to hear from other community members
> on
> > their experiences, steps that helped them, and suggestions for the
future
> > to continue to make the NiFi community welcoming to new users. Thanks.
> > >
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > alopresto.apa...@gmail.com
> > > PGP Fingerprint: 70EC B3E5 98A6 

Re: Lowering the barrier of entry

[Mike Thomsen]

I should be able to find some time to set up some skeletons based on code
that I've written. Would storing them in the docs folder work, so we can do
PRs?

On Fri, Jan 25, 2019 at 11:50 AM Andrew Grande  wrote:

> I consistently see my users struggling when they move up the nifi food
> chain and start looking at custom processors. The good content about
> prototyping processsors via scripting processors and finalizing with a full
> NAR bundle is everywhere but where it should be.
>
> A few simple changes could help (not *more* docs). They are great, much
> better than in many other projecta, but people are already drowning in
> those.
>
> How about:
>
> + ISP has a pre-populated processor sceleton. A simple no-op to fill in is
> miles better than a blank text area (which invokes a blank stare).
>
> + As much as we may loook down on this, but... A simple guide to a full NAR
> build as a series of copy/paste commands.
>
> There's more, but this should fit the context for now.
>
> Andrew
>
> On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen  wrote:
>
> > One of the changes we should make is to create a separate guide for
> product
> > vendors on how to build and maintain a bundle. We're at that point where
> > vendors will have to do it on their own as extension providers, so it
> would
> > be very helpful for them to have a simple and straight forward document
> > showing them what should be there, best practices for maintainability and
> > where to announce it.
> >
> > On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:
> >
> > > I think we have a lot more documentation than most projects, but I
> > > think an issue is that content is scattered in many different
> > > locations, and some of the docs are huge reference guides where it can
> > > be hard to find all the pieces of what you are trying to do.
> > >
> > > The first thing a new contributor wants to do is get the code and run
> > > a build, and we do have a quick-start guide linked to on the site, but
> > > I think there is a lot of extra information in there that is not
> > > really relevant to someone just wanting get the code and build. We
> > > could have separate guides per OS like "Build NiFi on Linux", "Build
> > > NiFi on Windows", etc, where each guide was 4-5 steps like:
> > >
> > > - Clone repo
> > > - checkout master
> > > - run maven
> > > - cd to assembly
> > > - ./bin/nifi.sh
> > >
> > > The next thing they want to do is contribute a change, and we have a
> > > great contributor guide, but again I think there could be a very short
> > > tutorial for the most common steps:
> > >
> > > - fork repo
> > > - clone fork
> > > - create branch
> > > - make changes
> > > - push branch
> > > - submit pr
> > >
> > > and then say something like "for a more detailed description of the
> > > contribution process, please reference the Contributor Guide".
> > >
> > > If we then make these getting started guides more prominent right in
> > > the middle of the NiFi homepage, then maybe they will be easier to
> > > find for new community members.
> > >
> > > We can keep extending this idea to other common tasks beyond just
> > > building and contributing.
> > >
> > >
> > > On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto 
> > > wrote:
> > > >
> > > > Hi folks,
> > > >
> > > > Based on some recent (and long-term) experiences, I wanted to discuss
> > > with the community what we could do to lower the barrier of entry to
> > using
> > > & contributing to NiFi. I hope to get some good feedback from both
> > > long-time and newer members, and determine some immediate concrete
> steps
> > we
> > > can take.
> > > >
> > > > Problems identified:
> > > > * NiFi has a number of custom profiles, so a simple “mvn clean
> install”
> > > in project root doesn’t get a new developer up and running immediately
> > > > * The API is very well defined, but for new contributors, it can be a
> > > challenge to know where to put functionality, and building a custom
> > > processor + NAR and deploying isn’t a one-step process
> > > > * Project size (and build size/time) is large. This can restrict the
> > > minimum hardware necessary, elongate the development cycle, etc.
> > > > * Some new users do not receive mailing list replies
> > > >
> > > > Possible solutions:
> > > > * On a clean git clone, “mvn clean install” should build a working
> > > instance. Maybe we provide a quickstart.sh script to handle the default
> > > maven build, change to the target directory, and start NiFi?
> > > > * Individual contributors have written excellent blogs, and
> > > documentation exists, but making it more prominent or more easily
> > accessed
> > > could help?
> > > > * Extension registry will solve all the world’s problems (related to
> > > bundling and build time)
> > > > * Not sure about this one — I don’t know if it’s because they’re not
> > > subscribed, their mail client is blocking them, etc.
> > > >
> > > > I’ve said my bit, now I am eager to hear from other community members
> > on
> > > 

Re: Lowering the barrier of entry

[Andrew Grande]

I consistently see my users struggling when they move up the nifi food
chain and start looking at custom processors. The good content about
prototyping processsors via scripting processors and finalizing with a full
NAR bundle is everywhere but where it should be.

A few simple changes could help (not *more* docs). They are great, much
better than in many other projecta, but people are already drowning in
those.

How about:

+ ISP has a pre-populated processor sceleton. A simple no-op to fill in is
miles better than a blank text area (which invokes a blank stare).

+ As much as we may loook down on this, but... A simple guide to a full NAR
build as a series of copy/paste commands.

There's more, but this should fit the context for now.

Andrew

On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen  wrote:

> One of the changes we should make is to create a separate guide for product
> vendors on how to build and maintain a bundle. We're at that point where
> vendors will have to do it on their own as extension providers, so it would
> be very helpful for them to have a simple and straight forward document
> showing them what should be there, best practices for maintainability and
> where to announce it.
>
> On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:
>
> > I think we have a lot more documentation than most projects, but I
> > think an issue is that content is scattered in many different
> > locations, and some of the docs are huge reference guides where it can
> > be hard to find all the pieces of what you are trying to do.
> >
> > The first thing a new contributor wants to do is get the code and run
> > a build, and we do have a quick-start guide linked to on the site, but
> > I think there is a lot of extra information in there that is not
> > really relevant to someone just wanting get the code and build. We
> > could have separate guides per OS like "Build NiFi on Linux", "Build
> > NiFi on Windows", etc, where each guide was 4-5 steps like:
> >
> > - Clone repo
> > - checkout master
> > - run maven
> > - cd to assembly
> > - ./bin/nifi.sh
> >
> > The next thing they want to do is contribute a change, and we have a
> > great contributor guide, but again I think there could be a very short
> > tutorial for the most common steps:
> >
> > - fork repo
> > - clone fork
> > - create branch
> > - make changes
> > - push branch
> > - submit pr
> >
> > and then say something like "for a more detailed description of the
> > contribution process, please reference the Contributor Guide".
> >
> > If we then make these getting started guides more prominent right in
> > the middle of the NiFi homepage, then maybe they will be easier to
> > find for new community members.
> >
> > We can keep extending this idea to other common tasks beyond just
> > building and contributing.
> >
> >
> > On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto 
> > wrote:
> > >
> > > Hi folks,
> > >
> > > Based on some recent (and long-term) experiences, I wanted to discuss
> > with the community what we could do to lower the barrier of entry to
> using
> > & contributing to NiFi. I hope to get some good feedback from both
> > long-time and newer members, and determine some immediate concrete steps
> we
> > can take.
> > >
> > > Problems identified:
> > > * NiFi has a number of custom profiles, so a simple “mvn clean install”
> > in project root doesn’t get a new developer up and running immediately
> > > * The API is very well defined, but for new contributors, it can be a
> > challenge to know where to put functionality, and building a custom
> > processor + NAR and deploying isn’t a one-step process
> > > * Project size (and build size/time) is large. This can restrict the
> > minimum hardware necessary, elongate the development cycle, etc.
> > > * Some new users do not receive mailing list replies
> > >
> > > Possible solutions:
> > > * On a clean git clone, “mvn clean install” should build a working
> > instance. Maybe we provide a quickstart.sh script to handle the default
> > maven build, change to the target directory, and start NiFi?
> > > * Individual contributors have written excellent blogs, and
> > documentation exists, but making it more prominent or more easily
> accessed
> > could help?
> > > * Extension registry will solve all the world’s problems (related to
> > bundling and build time)
> > > * Not sure about this one — I don’t know if it’s because they’re not
> > subscribed, their mail client is blocking them, etc.
> > >
> > > I’ve said my bit, now I am eager to hear from other community members
> on
> > their experiences, steps that helped them, and suggestions for the future
> > to continue to make the NiFi community welcoming to new users. Thanks.
> > >
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > alopresto.apa...@gmail.com
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> >
>

Re: Lowering the barrier of entry

[Mike Thomsen]

One of the changes we should make is to create a separate guide for product
vendors on how to build and maintain a bundle. We're at that point where
vendors will have to do it on their own as extension providers, so it would
be very helpful for them to have a simple and straight forward document
showing them what should be there, best practices for maintainability and
where to announce it.

On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende  wrote:

> I think we have a lot more documentation than most projects, but I
> think an issue is that content is scattered in many different
> locations, and some of the docs are huge reference guides where it can
> be hard to find all the pieces of what you are trying to do.
>
> The first thing a new contributor wants to do is get the code and run
> a build, and we do have a quick-start guide linked to on the site, but
> I think there is a lot of extra information in there that is not
> really relevant to someone just wanting get the code and build. We
> could have separate guides per OS like "Build NiFi on Linux", "Build
> NiFi on Windows", etc, where each guide was 4-5 steps like:
>
> - Clone repo
> - checkout master
> - run maven
> - cd to assembly
> - ./bin/nifi.sh
>
> The next thing they want to do is contribute a change, and we have a
> great contributor guide, but again I think there could be a very short
> tutorial for the most common steps:
>
> - fork repo
> - clone fork
> - create branch
> - make changes
> - push branch
> - submit pr
>
> and then say something like "for a more detailed description of the
> contribution process, please reference the Contributor Guide".
>
> If we then make these getting started guides more prominent right in
> the middle of the NiFi homepage, then maybe they will be easier to
> find for new community members.
>
> We can keep extending this idea to other common tasks beyond just
> building and contributing.
>
>
> On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto 
> wrote:
> >
> > Hi folks,
> >
> > Based on some recent (and long-term) experiences, I wanted to discuss
> with the community what we could do to lower the barrier of entry to using
> & contributing to NiFi. I hope to get some good feedback from both
> long-time and newer members, and determine some immediate concrete steps we
> can take.
> >
> > Problems identified:
> > * NiFi has a number of custom profiles, so a simple “mvn clean install”
> in project root doesn’t get a new developer up and running immediately
> > * The API is very well defined, but for new contributors, it can be a
> challenge to know where to put functionality, and building a custom
> processor + NAR and deploying isn’t a one-step process
> > * Project size (and build size/time) is large. This can restrict the
> minimum hardware necessary, elongate the development cycle, etc.
> > * Some new users do not receive mailing list replies
> >
> > Possible solutions:
> > * On a clean git clone, “mvn clean install” should build a working
> instance. Maybe we provide a quickstart.sh script to handle the default
> maven build, change to the target directory, and start NiFi?
> > * Individual contributors have written excellent blogs, and
> documentation exists, but making it more prominent or more easily accessed
> could help?
> > * Extension registry will solve all the world’s problems (related to
> bundling and build time)
> > * Not sure about this one — I don’t know if it’s because they’re not
> subscribed, their mail client is blocking them, etc.
> >
> > I’ve said my bit, now I am eager to hear from other community members on
> their experiences, steps that helped them, and suggestions for the future
> to continue to make the NiFi community welcoming to new users. Thanks.
> >
> >
> > Andy LoPresto
> > alopre...@apache.org
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
>

Re: Lowering the barrier of entry

[Bryan Bende]

I think we have a lot more documentation than most projects, but I
think an issue is that content is scattered in many different
locations, and some of the docs are huge reference guides where it can
be hard to find all the pieces of what you are trying to do.

The first thing a new contributor wants to do is get the code and run
a build, and we do have a quick-start guide linked to on the site, but
I think there is a lot of extra information in there that is not
really relevant to someone just wanting get the code and build. We
could have separate guides per OS like "Build NiFi on Linux", "Build
NiFi on Windows", etc, where each guide was 4-5 steps like:

- Clone repo
- checkout master
- run maven
- cd to assembly
- ./bin/nifi.sh

The next thing they want to do is contribute a change, and we have a
great contributor guide, but again I think there could be a very short
tutorial for the most common steps:

- fork repo
- clone fork
- create branch
- make changes
- push branch
- submit pr

and then say something like "for a more detailed description of the
contribution process, please reference the Contributor Guide".

If we then make these getting started guides more prominent right in
the middle of the NiFi homepage, then maybe they will be easier to
find for new community members.

We can keep extending this idea to other common tasks beyond just
building and contributing.


On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto  wrote:
>
> Hi folks,
>
> Based on some recent (and long-term) experiences, I wanted to discuss with 
> the community what we could do to lower the barrier of entry to using & 
> contributing to NiFi. I hope to get some good feedback from both long-time 
> and newer members, and determine some immediate concrete steps we can take.
>
> Problems identified:
> * NiFi has a number of custom profiles, so a simple “mvn clean install” in 
> project root doesn’t get a new developer up and running immediately
> * The API is very well defined, but for new contributors, it can be a 
> challenge to know where to put functionality, and building a custom processor 
> + NAR and deploying isn’t a one-step process
> * Project size (and build size/time) is large. This can restrict the minimum 
> hardware necessary, elongate the development cycle, etc.
> * Some new users do not receive mailing list replies
>
> Possible solutions:
> * On a clean git clone, “mvn clean install” should build a working instance. 
> Maybe we provide a quickstart.sh script to handle the default maven build, 
> change to the target directory, and start NiFi?
> * Individual contributors have written excellent blogs, and documentation 
> exists, but making it more prominent or more easily accessed could help?
> * Extension registry will solve all the world’s problems (related to bundling 
> and build time)
> * Not sure about this one — I don’t know if it’s because they’re not 
> subscribed, their mail client is blocking them, etc.
>
> I’ve said my bit, now I am eager to hear from other community members on 
> their experiences, steps that helped them, and suggestions for the future to 
> continue to make the NiFi community welcoming to new users. Thanks.
>
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>

Lowering the barrier of entry

[Andy LoPresto]

Hi folks,

Based on some recent (and long-term) experiences, I wanted to discuss with the 
community what we could do to lower the barrier of entry to using & 
contributing to NiFi. I hope to get some good feedback from both long-time and 
newer members, and determine some immediate concrete steps we can take. 

Problems identified:
* NiFi has a number of custom profiles, so a simple “mvn clean install” in 
project root doesn’t get a new developer up and running immediately
* The API is very well defined, but for new contributors, it can be a challenge 
to know where to put functionality, and building a custom processor + NAR and 
deploying isn’t a one-step process
* Project size (and build size/time) is large. This can restrict the minimum 
hardware necessary, elongate the development cycle, etc. 
* Some new users do not receive mailing list replies

Possible solutions:
* On a clean git clone, “mvn clean install” should build a working instance. 
Maybe we provide a quickstart.sh script to handle the default maven build, 
change to the target directory, and start NiFi?
* Individual contributors have written excellent blogs, and documentation 
exists, but making it more prominent or more easily accessed could help? 
* Extension registry will solve all the world’s problems (related to bundling 
and build time)
* Not sure about this one — I don’t know if it’s because they’re not 
subscribed, their mail client is blocking them, etc. 

I’ve said my bit, now I am eager to hear from other community members on their 
experiences, steps that helped them, and suggestions for the future to continue 
to make the NiFi community welcoming to new users. Thanks. 


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69