https://issues.apache.org/jira/browse/NIFI-189
On Fri, Dec 19, 2014 at 11:05 AM, Mark Payne <[email protected]> wrote: > These probably can be cleaned up some now, as we now have a much more > unified build, with a high level dependency management. > Careful, though: nifi-properties cannot be merged with anything else! It > is on the root of the lib/ directory, so it cannot have any transitive > dependencies at all. > -Mark > > Date: Fri, 19 Dec 2014 09:17:34 -0600 > > Subject: Re: nifi/commons consolidation suggestion > > From: [email protected] > > To: [email protected] > > > > Those sound like really good candidates for consolidation. It might > > also be worth looking at the dependency graph to find a lot of > > co-occurence. If every module that depends on A also depends on B, > > there's less of an argument to keep them separate. > > > > -Joey > > > > On Thu, Dec 18, 2014 at 7:37 PM, Michael Moser <[email protected]> > wrote: > > > You didn't fall too far, Joe. core-flowfile-attributes, > > > naive-search-ring-buffer, nifi-properties, nifi-stream-utils, and > > > processor-utilities have no dependencies. And nifi-logging-utils only > > > depends on the provided slf4j-api. Just sayin' ;) > > > > > > -- Mike > > > > > > On Thu, Dec 18, 2014 at 8:06 PM, Joe Witt <[email protected]> wrote: > > >> i'm sure mark payne was laughing at my response on this thread pretty > > >> hard. I've given him grief before for the many split utilities jars > we > > >> have and he'd each time quickly remind me that it was to avoid pulling > > >> needless deps into spaces we dont want them. So fell into the trap > again > > >> today... > > >> > > >> Thanks > > >> Joe > > >> > > >> On Thu, Dec 18, 2014 at 5:47 PM, Michael Moser <[email protected]> > wrote: > > >>> > > >>> Joey makes a good point. Without nifi-socket-utils, the transitive > > >>> dependencies will be commons-codec, commons-compress and > > >>> commons-lang3. slf4j-api is also in there but that's marked provided > > >>> by all of NiFi. > > >>> > > >>> When you add nifi-socket-utils to the equation, that adds commons-io > > >>> and commons-net. Interestingly, nifi-socket-utils also depends on > > >>> three of the other nifi-*-utils. So if you need nifi-socket-utils, > > >>> you also get nifi-properties, nifi-logging-utils, and nifi-utils > > >>> anyway. > > >>> > > >>> It's probably worth leaving nifi-socket-utils separate for now. It > > >>> had the biggest footprint of all of the utils to begin with. > > >>> > > >>> flowfile-packager is the only one that pulls in commons-compress. > > >>> nifi-file-utils is the only one that pulls in commons-codec (though > > >>> that dependency could be removed with a clever refactor of > > >>> computeMd5Digest(File file) using just the JDK). > > >>> > > >>> -- Mike > > >>> > > >>> > > >>> On Thu, Dec 18, 2014 at 2:38 PM, Joey Echeverria <[email protected]> > > >>> wrote: > > >>> > Do we know how many transitive dependencies this ends up mixing > together? > > >>> > > > >>> > I bring it up because that's often a reason for splitting a small > > >>> > number of classes into their own module. For example, if I care > about > > >>> > socket-based data flow maybe I don't need the dependancies > utilities > > >>> > related to file-based data flow. I'll try to take a look at the > actual > > >>> > modules, but I thought I would throw that out there for others to > > >>> > think on. > > >>> > > > >>> > One thing I've seen work well is creating a dependency aggregator > > >>> > module for users that don't care about the extra dependencies. > > >>> > > > >>> > -Joey > > >>> > > > >>> > On Thu, Dec 18, 2014 at 12:13 PM, Joe Witt <[email protected]> > wrote: > > >>> >> Mike, > > >>> >> > > >>> >> I think this is a great point and a great analysis. > > >>> >> > > >>> >> +1 and unless anyone specifically objects I'll go ahead and do > this > > >>> >> tonight. If i run into any curveballs I'll throw it on this > thread. > > >>> >> > > >>> >> Thanks > > >>> >> Joe > > >>> >> > > >>> >> On Thu, Dec 18, 2014 at 11:04 AM, Michael Moser < > [email protected]> > > >>> wrote: > > >>> >>> > > >>> >>> I'm not sure if this is the most appropriate forum or I should > have > > >>> >>> just written a Jira ticket, but here goes. > > >>> >>> > > >>> >>> I believe we should consolidate the number of artifacts we have > in the > > >>> >>> nifi/commons module. We create three jars that contain just 1 > class > > >>> >>> each and there are three more jars with 3 or fewer classes in > them. > > >>> >>> This makes it annoying (especially for beginners) to find the > location > > >>> >>> of classes that you need and slightly bloats our footprint for > number > > >>> >>> of artifacts that nifi create. I believe we can improve this. > > >>> >>> > > >>> >>> I analyzed all of the nar-bundles to find where each common > library > > >>> >>> was used. Several are used by many framework, services, and > > >>> >>> processors bundles already, so consolidating these common jars > is a > > >>> >>> no-brainer. Other jars that are used more sparingly contain > just 1 or > > >>> >>> 2 classes, so it really will have minimal impact to consolidate > them > > >>> >>> even if the classes aren't needed by a nar. > > >>> >>> > > >>> >>> So, I propose we consolidate these artifacts into the nifi-utils > > >>> >>> artifact. The number in (parentheses) is the number of classes > in > > >>> >>> them. > > >>> >>> > > >>> >>> core-flowfile-attributes (2) > > >>> >>> flowfile-packager (9) > > >>> >>> naive-search-ring-buffer (1) > > >>> >>> nifi-file-utils (1) > > >>> >>> nifi-logging-utils (1) > > >>> >>> nifi-properties (2) > > >>> >>> nifi-security-utils (5) > > >>> >>> nifi-socket-utils (24) > > >>> >>> nifi-stream-utils (17) > > >>> >>> processor-utilities (3) (this would also resolve why the name > doesn't > > >>> >>> start with "nifi") > > >>> >>> > > >>> >>> nifi-utils would go from 24 classes to 89 classes. > > >>> >>> > > >>> >>> nifi-web-utils (3), remote-communications-utils (13), and > search-utils > > >>> >>> (5) I did not include because their use is limited to just one > or two > > >>> >>> places. > > >>> >>> > > >>> >>> Thanks, > > >>> >>> -- Mike > > >>> >>> > > >>> > > > >>> > > > >>> > > > >>> > -- > > >>> > Joey Echeverria > > >>> > > > > > > > > -- > > Joey Echeverria > >
