i'm sure mark payne was laughing at my response on this thread pretty hard. I've given him grief before for the many split utilities jars we have and he'd each time quickly remind me that it was to avoid pulling needless deps into spaces we dont want them. So fell into the trap again today...
Thanks Joe On Thu, Dec 18, 2014 at 5:47 PM, Michael Moser <[email protected]> wrote: > > Joey makes a good point. Without nifi-socket-utils, the transitive > dependencies will be commons-codec, commons-compress and > commons-lang3. slf4j-api is also in there but that's marked provided > by all of NiFi. > > When you add nifi-socket-utils to the equation, that adds commons-io > and commons-net. Interestingly, nifi-socket-utils also depends on > three of the other nifi-*-utils. So if you need nifi-socket-utils, > you also get nifi-properties, nifi-logging-utils, and nifi-utils > anyway. > > It's probably worth leaving nifi-socket-utils separate for now. It > had the biggest footprint of all of the utils to begin with. > > flowfile-packager is the only one that pulls in commons-compress. > nifi-file-utils is the only one that pulls in commons-codec (though > that dependency could be removed with a clever refactor of > computeMd5Digest(File file) using just the JDK). > > -- Mike > > > On Thu, Dec 18, 2014 at 2:38 PM, Joey Echeverria <[email protected]> > wrote: > > Do we know how many transitive dependencies this ends up mixing together? > > > > I bring it up because that's often a reason for splitting a small > > number of classes into their own module. For example, if I care about > > socket-based data flow maybe I don't need the dependancies utilities > > related to file-based data flow. I'll try to take a look at the actual > > modules, but I thought I would throw that out there for others to > > think on. > > > > One thing I've seen work well is creating a dependency aggregator > > module for users that don't care about the extra dependencies. > > > > -Joey > > > > On Thu, Dec 18, 2014 at 12:13 PM, Joe Witt <[email protected]> wrote: > >> Mike, > >> > >> I think this is a great point and a great analysis. > >> > >> +1 and unless anyone specifically objects I'll go ahead and do this > >> tonight. If i run into any curveballs I'll throw it on this thread. > >> > >> Thanks > >> Joe > >> > >> On Thu, Dec 18, 2014 at 11:04 AM, Michael Moser <[email protected]> > wrote: > >>> > >>> I'm not sure if this is the most appropriate forum or I should have > >>> just written a Jira ticket, but here goes. > >>> > >>> I believe we should consolidate the number of artifacts we have in the > >>> nifi/commons module. We create three jars that contain just 1 class > >>> each and there are three more jars with 3 or fewer classes in them. > >>> This makes it annoying (especially for beginners) to find the location > >>> of classes that you need and slightly bloats our footprint for number > >>> of artifacts that nifi create. I believe we can improve this. > >>> > >>> I analyzed all of the nar-bundles to find where each common library > >>> was used. Several are used by many framework, services, and > >>> processors bundles already, so consolidating these common jars is a > >>> no-brainer. Other jars that are used more sparingly contain just 1 or > >>> 2 classes, so it really will have minimal impact to consolidate them > >>> even if the classes aren't needed by a nar. > >>> > >>> So, I propose we consolidate these artifacts into the nifi-utils > >>> artifact. The number in (parentheses) is the number of classes in > >>> them. > >>> > >>> core-flowfile-attributes (2) > >>> flowfile-packager (9) > >>> naive-search-ring-buffer (1) > >>> nifi-file-utils (1) > >>> nifi-logging-utils (1) > >>> nifi-properties (2) > >>> nifi-security-utils (5) > >>> nifi-socket-utils (24) > >>> nifi-stream-utils (17) > >>> processor-utilities (3) (this would also resolve why the name doesn't > >>> start with "nifi") > >>> > >>> nifi-utils would go from 24 classes to 89 classes. > >>> > >>> nifi-web-utils (3), remote-communications-utils (13), and search-utils > >>> (5) I did not include because their use is limited to just one or two > >>> places. > >>> > >>> Thanks, > >>> -- Mike > >>> > > > > > > > > -- > > Joey Echeverria >
