These probably can be cleaned up some now, as we now have a much more unified
build, with a high level dependency management.
Careful, though: nifi-properties cannot be merged with anything else! It is on
the root of the lib/ directory, so it cannot have any transitive dependencies
at all.
-Mark
> Date: Fri, 19 Dec 2014 09:17:34 -0600
> Subject: Re: nifi/commons consolidation suggestion
> From: [email protected]
> To: [email protected]
>
> Those sound like really good candidates for consolidation. It might
> also be worth looking at the dependency graph to find a lot of
> co-occurence. If every module that depends on A also depends on B,
> there's less of an argument to keep them separate.
>
> -Joey
>
> On Thu, Dec 18, 2014 at 7:37 PM, Michael Moser <[email protected]> wrote:
> > You didn't fall too far, Joe. core-flowfile-attributes,
> > naive-search-ring-buffer, nifi-properties, nifi-stream-utils, and
> > processor-utilities have no dependencies. And nifi-logging-utils only
> > depends on the provided slf4j-api. Just sayin' ;)
> >
> > -- Mike
> >
> > On Thu, Dec 18, 2014 at 8:06 PM, Joe Witt <[email protected]> wrote:
> >> i'm sure mark payne was laughing at my response on this thread pretty
> >> hard. I've given him grief before for the many split utilities jars we
> >> have and he'd each time quickly remind me that it was to avoid pulling
> >> needless deps into spaces we dont want them. So fell into the trap again
> >> today...
> >>
> >> Thanks
> >> Joe
> >>
> >> On Thu, Dec 18, 2014 at 5:47 PM, Michael Moser <[email protected]> wrote:
> >>>
> >>> Joey makes a good point. Without nifi-socket-utils, the transitive
> >>> dependencies will be commons-codec, commons-compress and
> >>> commons-lang3. slf4j-api is also in there but that's marked provided
> >>> by all of NiFi.
> >>>
> >>> When you add nifi-socket-utils to the equation, that adds commons-io
> >>> and commons-net. Interestingly, nifi-socket-utils also depends on
> >>> three of the other nifi-*-utils. So if you need nifi-socket-utils,
> >>> you also get nifi-properties, nifi-logging-utils, and nifi-utils
> >>> anyway.
> >>>
> >>> It's probably worth leaving nifi-socket-utils separate for now. It
> >>> had the biggest footprint of all of the utils to begin with.
> >>>
> >>> flowfile-packager is the only one that pulls in commons-compress.
> >>> nifi-file-utils is the only one that pulls in commons-codec (though
> >>> that dependency could be removed with a clever refactor of
> >>> computeMd5Digest(File file) using just the JDK).
> >>>
> >>> -- Mike
> >>>
> >>>
> >>> On Thu, Dec 18, 2014 at 2:38 PM, Joey Echeverria <[email protected]>
> >>> wrote:
> >>> > Do we know how many transitive dependencies this ends up mixing
> >>> > together?
> >>> >
> >>> > I bring it up because that's often a reason for splitting a small
> >>> > number of classes into their own module. For example, if I care about
> >>> > socket-based data flow maybe I don't need the dependancies utilities
> >>> > related to file-based data flow. I'll try to take a look at the actual
> >>> > modules, but I thought I would throw that out there for others to
> >>> > think on.
> >>> >
> >>> > One thing I've seen work well is creating a dependency aggregator
> >>> > module for users that don't care about the extra dependencies.
> >>> >
> >>> > -Joey
> >>> >
> >>> > On Thu, Dec 18, 2014 at 12:13 PM, Joe Witt <[email protected]> wrote:
> >>> >> Mike,
> >>> >>
> >>> >> I think this is a great point and a great analysis.
> >>> >>
> >>> >> +1 and unless anyone specifically objects I'll go ahead and do this
> >>> >> tonight. If i run into any curveballs I'll throw it on this thread.
> >>> >>
> >>> >> Thanks
> >>> >> Joe
> >>> >>
> >>> >> On Thu, Dec 18, 2014 at 11:04 AM, Michael Moser <[email protected]>
> >>> wrote:
> >>> >>>
> >>> >>> I'm not sure if this is the most appropriate forum or I should have
> >>> >>> just written a Jira ticket, but here goes.
> >>> >>>
> >>> >>> I believe we should consolidate the number of artifacts we have in the
> >>> >>> nifi/commons module. We create three jars that contain just 1 class
> >>> >>> each and there are three more jars with 3 or fewer classes in them.
> >>> >>> This makes it annoying (especially for beginners) to find the location
> >>> >>> of classes that you need and slightly bloats our footprint for number
> >>> >>> of artifacts that nifi create. I believe we can improve this.
> >>> >>>
> >>> >>> I analyzed all of the nar-bundles to find where each common library
> >>> >>> was used. Several are used by many framework, services, and
> >>> >>> processors bundles already, so consolidating these common jars is a
> >>> >>> no-brainer. Other jars that are used more sparingly contain just 1 or
> >>> >>> 2 classes, so it really will have minimal impact to consolidate them
> >>> >>> even if the classes aren't needed by a nar.
> >>> >>>
> >>> >>> So, I propose we consolidate these artifacts into the nifi-utils
> >>> >>> artifact. The number in (parentheses) is the number of classes in
> >>> >>> them.
> >>> >>>
> >>> >>> core-flowfile-attributes (2)
> >>> >>> flowfile-packager (9)
> >>> >>> naive-search-ring-buffer (1)
> >>> >>> nifi-file-utils (1)
> >>> >>> nifi-logging-utils (1)
> >>> >>> nifi-properties (2)
> >>> >>> nifi-security-utils (5)
> >>> >>> nifi-socket-utils (24)
> >>> >>> nifi-stream-utils (17)
> >>> >>> processor-utilities (3) (this would also resolve why the name doesn't
> >>> >>> start with "nifi")
> >>> >>>
> >>> >>> nifi-utils would go from 24 classes to 89 classes.
> >>> >>>
> >>> >>> nifi-web-utils (3), remote-communications-utils (13), and search-utils
> >>> >>> (5) I did not include because their use is limited to just one or two
> >>> >>> places.
> >>> >>>
> >>> >>> Thanks,
> >>> >>> -- Mike
> >>> >>>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Joey Echeverria
> >>>
>
>
>
> --
> Joey Echeverria
