9) Oh, and the wiki is a little hard to navigate and the contents rather patchy
On Fri, 25 Jan 2019 at 21:57, James Srinivasan <james.sriniva...@gmail.com> wrote: > > As someone relatively new to NiFi dev, here's my £0.02. (Yes, I > realise I could and possibly should submit PRs :) > > 1) I'm used to Java and Maven, so used the archetype. It worked fine, > it would have been nice it if set up unit tests for me. > 2) The User and Developer documentation is great and comprehensive. > Finding the developer docs is a little painful (handful of items at > the end of a scrolling list of 200+ processors) > 3) The Developer docs could possibly do with a little more clarity on > processor lifetime i.e. what is called when ^h^h^h - skimming back > over the docs, it looks pretty clear now > 4) Some example code for common operations e.g. getting/setting > attributes or reading/writing/modifying flowfile content would be > great. > 5) When using existing processors for inspiration, best practices > weren't always clear e.g. some generated properties inside > getSupportedPropertyDescriptors(), others generated a private static > list on init and returned that. Such differences are inevitable in a > large project, but it would be nice to have something blessed to start > from. > 6) (Minor niggle - layout of the docs doesn't work great on a phone screen) > 7) I couldn't find (m?)any docs about the Groovy scripting API, but > the great blog posts by Matt Burgess and others were invaluable > 8) In case this all sounds too negative, NiFi is fab! > > On Fri, 25 Jan 2019 at 18:47, Andrew Grande <apere...@gmail.com> wrote: > > > > I am not against the archetype. But we need to spell out every step of the > > way. I'd like to see a user thinking about their custom logic ASAP rather > > than fighting the tools to get started. Those steps should be brain-dead, > > just reflexes, if you know what I mean. Hell, let them create a custom > > processor project or prototype in a script by accident even! :) > > > > On Fri, Jan 25, 2019, 10:43 AM Bryan Bende <bbe...@gmail.com> wrote: > > > > > That makes sense about the best practice for deploying to an > > > additional lib directory. > > > > > > So for the project structure you are saying it would be easier to have > > > a repo somewhere with essentially the same thing that is in the > > > archetype, but they just clone it and rename it themselves (what the > > > archetype does for you)? > > > > > > Something that I think would be awesome is if we could provide a > > > web-based project initializer that would essentially run the archetype > > > behind the scenes and then let you download the archive of the code, > > > just like the spring-boot starter [1]. Not sure if their initializr is > > > something that can be re-used and customized [2]. > > > > > > The problem is we would need to host that somewhere. > > > > > > [1] https://start.spring.io/ > > > [2] https://github.com/spring-io/initializr > > > > > > On Fri, Jan 25, 2019 at 12:56 PM Andrew Grande <apere...@gmail.com> wrote: > > > > > > > > We assume they create new projects from archetypes every day. They > > > > don't. > > > > > > > > We also assume they know how to deploy new NARs. Most don't. Especially > > > if > > > > we want them to follow best practices and create an additional NAR > > > bundles > > > > directory entry im the config (vs dumping into nifi lib). > > > > > > > > I can attest that I feel a bit lost myself every time I need to come > > > > back > > > > to this and refresh my brain synapses. If we could make these not > > > > require > > > > any of that and make simple thinga dead simple.... > > > > > > > > Andrew > > > > > > > > On Fri, Jan 25, 2019, 9:47 AM Bryan Bende <bbe...@gmail.com> wrote: > > > > > > > > > Andrew, > > > > > > > > > > I'm not disagreeing with your points, but I'm curious how you see > > > > > those two ideas being different from the processor archetype and the > > > > > wiki page with the archetype commands? > > > > > > > > > > Is it just that people don't know about it? > > > > > > > > > > -Bryan > > > > > > > > > > [1] > > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions > > > > > > > > > > On Fri, Jan 25, 2019 at 12:23 PM Otto Fowler <ottobackwa...@gmail.com> > > > > > wrote: > > > > > > > > > > > > I think this ties into my other discuss thread on refreshing the > > > > > archetypes > > > > > > > > > > > > > > > > > > On January 25, 2019 at 11:50:10, Andrew Grande (apere...@gmail.com) > > > > > wrote: > > > > > > > > > > > > I consistently see my users struggling when they move up the nifi > > > food > > > > > > chain and start looking at custom processors. The good content about > > > > > > prototyping processsors via scripting processors and finalizing with > > > a > > > > > full > > > > > > NAR bundle is everywhere but where it should be. > > > > > > > > > > > > A few simple changes could help (not *more* docs). They are great, > > > much > > > > > > better than in many other projecta, but people are already drowning > > > in > > > > > > those. > > > > > > > > > > > > How about: > > > > > > > > > > > > + ISP has a pre-populated processor sceleton. A simple no-op to fill > > > in > > > > > is > > > > > > miles better than a blank text area (which invokes a blank stare). > > > > > > > > > > > > + As much as we may loook down on this, but... A simple guide to a > > > full > > > > > NAR > > > > > > build as a series of copy/paste commands. > > > > > > > > > > > > There's more, but this should fit the context for now. > > > > > > > > > > > > Andrew > > > > > > > > > > > > On Fri, Jan 25, 2019, 8:13 AM Mike Thomsen <mikerthom...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > One of the changes we should make is to create a separate guide > > > > > > > for > > > > > > product > > > > > > > vendors on how to build and maintain a bundle. We're at that point > > > > > where > > > > > > > vendors will have to do it on their own as extension providers, so > > > it > > > > > > would > > > > > > > be very helpful for them to have a simple and straight forward > > > document > > > > > > > showing them what should be there, best practices for > > > maintainability > > > > > and > > > > > > > where to announce it. > > > > > > > > > > > > > > On Fri, Jan 25, 2019 at 9:59 AM Bryan Bende <bbe...@gmail.com> > > > wrote: > > > > > > > > > > > > > > > I think we have a lot more documentation than most projects, but > > > I > > > > > > > > think an issue is that content is scattered in many different > > > > > > > > locations, and some of the docs are huge reference guides where > > > it > > > > > can > > > > > > > > be hard to find all the pieces of what you are trying to do. > > > > > > > > > > > > > > > > The first thing a new contributor wants to do is get the code > > > and run > > > > > > > > a build, and we do have a quick-start guide linked to on the > > > site, > > > > > but > > > > > > > > I think there is a lot of extra information in there that is not > > > > > > > > really relevant to someone just wanting get the code and build. > > > We > > > > > > > > could have separate guides per OS like "Build NiFi on Linux", > > > "Build > > > > > > > > NiFi on Windows", etc, where each guide was 4-5 steps like: > > > > > > > > > > > > > > > > - Clone repo > > > > > > > > - checkout master > > > > > > > > - run maven > > > > > > > > - cd to assembly > > > > > > > > - ./bin/nifi.sh > > > > > > > > > > > > > > > > The next thing they want to do is contribute a change, and we > > > have a > > > > > > > > great contributor guide, but again I think there could be a very > > > > > short > > > > > > > > tutorial for the most common steps: > > > > > > > > > > > > > > > > - fork repo > > > > > > > > - clone fork > > > > > > > > - create branch > > > > > > > > - make changes > > > > > > > > - push branch > > > > > > > > - submit pr > > > > > > > > > > > > > > > > and then say something like "for a more detailed description of > > > the > > > > > > > > contribution process, please reference the Contributor Guide". > > > > > > > > > > > > > > > > If we then make these getting started guides more prominent > > > right in > > > > > > > > the middle of the NiFi homepage, then maybe they will be easier > > > to > > > > > > > > find for new community members. > > > > > > > > > > > > > > > > We can keep extending this idea to other common tasks beyond > > > > > > > > just > > > > > > > > building and contributing. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jan 24, 2019 at 8:03 PM Andy LoPresto < > > > alopre...@apache.org> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > Based on some recent (and long-term) experiences, I wanted to > > > > > discuss > > > > > > > > with the community what we could do to lower the barrier of > > > entry to > > > > > > > using > > > > > > > > & contributing to NiFi. I hope to get some good feedback from > > > both > > > > > > > > long-time and newer members, and determine some immediate > > > concrete > > > > > > steps > > > > > > > we > > > > > > > > can take. > > > > > > > > > > > > > > > > > > Problems identified: > > > > > > > > > * NiFi has a number of custom profiles, so a simple “mvn clean > > > > > > install” > > > > > > > > in project root doesn’t get a new developer up and running > > > > > immediately > > > > > > > > > * The API is very well defined, but for new contributors, it > > > can > > > > > be a > > > > > > > > challenge to know where to put functionality, and building a > > > custom > > > > > > > > processor + NAR and deploying isn’t a one-step process > > > > > > > > > * Project size (and build size/time) is large. This can > > > restrict > > > > > the > > > > > > > > minimum hardware necessary, elongate the development cycle, etc. > > > > > > > > > * Some new users do not receive mailing list replies > > > > > > > > > > > > > > > > > > Possible solutions: > > > > > > > > > * On a clean git clone, “mvn clean install” should build a > > > working > > > > > > > > instance. Maybe we provide a quickstart.sh script to handle the > > > > > default > > > > > > > > maven build, change to the target directory, and start NiFi? > > > > > > > > > * Individual contributors have written excellent blogs, and > > > > > > > > documentation exists, but making it more prominent or more > > > > > > > > easily > > > > > > > accessed > > > > > > > > could help? > > > > > > > > > * Extension registry will solve all the world’s problems > > > (related > > > > > to > > > > > > > > bundling and build time) > > > > > > > > > * Not sure about this one — I don’t know if it’s because > > > they’re > > > > > not > > > > > > > > subscribed, their mail client is blocking them, etc. > > > > > > > > > > > > > > > > > > I’ve said my bit, now I am eager to hear from other community > > > > > members > > > > > > > on > > > > > > > > their experiences, steps that helped them, and suggestions for > > > the > > > > > > future > > > > > > > > to continue to make the NiFi community welcoming to new users. > > > > > Thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > Andy LoPresto > > > > > > > > > alopre...@apache.org > > > > > > > > > alopresto.apa...@gmail.com > > > > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D > > > EF69 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
