Re: [DISCUSS] Extension Registry

2018-11-13 Thread Andy LoPresto
From a security perspective, I’d like to see attention paid to the following topics: * verifiable checksums over extensions/bundles to ensure the code being deployed from a remote system is trusted * fuzzy hashing (complementary) to provide “change factor” on new version of the same extension/b

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Andrew Grande
I would like to see a clear separation of blob and metadata storage. Most often you'd see some object storage being already distributed and replicated, let's think about an easy way to backup or migrate the metadata between registry instances. Andrew On Tue, Nov 13, 2018, 11:32 AM Michael Moser

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Michael Moser
I have thought about this in the past, too. Here's a scenario where I could never really lay down an approach I was happy with. Consider that a NiFi user searches the NiFi registry and finds the PutMongo processor. Registry knows the PutMongo processor is in the nifi-mongodb-nar, and through its

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Bryan Bende
Mark, I think there are a couple of ways that could be solved, but ultimately it would be up to how the users choose to setup/manage the registry, or registries. The NiFi Registry security model is based around permissions to buckets (read/write), and all versioned items belong to a bucket. So yo

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Joe Witt
Group selection based on tag names for bundles could probably do that. Meaning it could be a sorting/filtering mechanism in the NiFi/Registry interface perhaps. Will be good to consider that UX as that progresses. As far as the different environments NiFi instances would certainly be able to load

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Mark Bean
Joe, I envision the Registry being able to provide a subset of NARs required for a specific NiFi instance. The user may have a relatively small set of NARs required for a NiFi used for basic routing/distribution, and a different more extensive set of NARs required for a more robust NiFi instance w

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Mike Thomsen
What are the plans for enabling the registry/NiFi to pull NARs and deploy them? Off the top of my head, piggybacking off of the Maven libraries/repos might be a good way to go there if there are no other plans. Private repositories should be baked in from the start here. Ideal would be something w

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Joe Witt
Mark Can you describe your use case from the user perspective both for the entity that would upload the items and demarcate them as a group as well as the user that would consume those bundles? I ask because the point here is that nars are themselves a 'group' in that they are a logical/contained

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Mark Bean
I would like to see a "group" capability in the Registry for NAR bundles. Then, users can create their own customized grouping of relevant NARs. Managing bundles one at a time will become tedious. Thanks, Mark On Tue, Nov 13, 2018 at 12:48 PM Joe Witt wrote: > Sivaprasanna - yes absolutely. Th

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Joe Witt
Sivaprasanna - yes absolutely. That is the core point I think of Bryan's first bullet under the 'NiFi' section. On Tue, Nov 13, 2018 at 12:47 PM Sivaprasanna wrote: > > One quick question. With the extension registry, my understanding is that > we would try to reduce the overall NiFi size by offl

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Sivaprasanna
One quick question. With the extension registry, my understanding is that we would try to reduce the overall NiFi size by offloading certain existing NAR bundles to the extension registry. So are we expecting the extension registry to also come up with the ability/mechanism that lets an user to dow

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Joe Witt
i'm not gonna lie - i googled elusive vs illusive. Well played, Rick. Well played. On Tue, Nov 13, 2018 at 12:44 PM Richard St. John wrote: > > Bryan, > > Good luck. Godspeed, my friend. As a hard-core NiFi user, I can’t wait for > the illusive extension registry to be a reality. > > Rick. > >

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Richard St. John
Bryan, Good luck.  Godspeed, my friend.  As a hard-core NiFi user, I can’t wait for the illusive extension registry to be a reality. Rick. -- Richard St. John, PhD Asymmetrik AGILITY | EXPERIENCE | RESULTS 308 Sentinel Drive, Suite 400 Annapolis Junction, MD 20701 On Nov 13, 2018, 12:37 PM -050

Re: [DISCUSS] Extension Registry

2018-11-13 Thread Joe Witt
Bryan Very exciting to see this under way!!! We desperately have to get our core nifi build size way down and make it far easier for people to contribute new processors. We have a long line of extensions that could be really useful/valuable and this will help unlock that while improving the user

[DISCUSS] Extension Registry

2018-11-13 Thread Bryan Bende
All, We've needed the elusive extension registry for quite some time now, and with NiFi Registry I think we are in a good place to make some progress in this area. I've started looking into adding "extension bundles" to NiFi Registry as the next type of versioned item, along side the existing ver