Hi Isha, I think what Shubham meant to say is any operator which has a dependency on external entity (not library) should go in contrib. Others should go in library.
For eg. DB related operators needs an instance/server of respect database to be running. Without that running server, the operator cannot satisfy the functionality expected. Hence it goes into contrib. I think none of the parsers will have dependency on external entity, hence they should go in library and not contrib. - Chinmay. On 18 Dec 2015 00:42, "Isha Arkatkar" <[email protected]> wrote: > Hi Shubham, > > I think it is somewhat subjective what goes in Contrib Vs Library. > Here is what I understood about general guideline: > An operator would go in malhar-library if it does not have any > other library dependency than what is already available. If operator needs > to include a dependency, it could be added to Contrib but not library. > > If we add an external library dependency in Malhar-library, the size > of the lib jar keeps on growing. So, if we refer malhar-lib in a project, > the size of the total package would increase, even if we may not directly > use all external libs. > > For this reason, in pull request #137 > <https://github.com/apache/incubator-apex-malhar/pull/137> , I move only > Json and XML parsers to malhar-library. Csv one had library reference to > supercsv, so it is still in contrib. > > Please correct if I missed something. :) > > Thanks! > Isha > > On Wed, Dec 16, 2015 at 11:27 PM, Shubham Pathak <[email protected]> > wrote: > > > Hi, > > > > What do we mean by "additional dependency" in case of deciding contrib vs > > lib ? > > As i understand, "additional dependency" is when an operator is > > interacting with external technologies . For e.g kafka, MQ , HBase etc. > > By this definition even CSV parsers need to be moved to malhar-lib. > > > > Thanks, > > Shubham > > > > > > On Thu, Dec 17, 2015 at 5:39 AM, Isha Arkatkar <[email protected]> > > wrote: > > > > > Hi Chandni, > > > > > > I have moved that as well to lib, as the parsers depended on that. > > > > > > Thanks, > > > Isha > > > > > > On Wed, Dec 16, 2015 at 1:01 PM, Chandni Singh < > [email protected]> > > > wrote: > > > > > > > There is a converter package under com.datatorrent.contrib which has > a > > > > Converter API. This belongs in library as well. > > > > > > > > Thanks, > > > > Chandni > > > > > > > > On Tue, Dec 15, 2015 at 1:29 PM, Chandni Singh < > > [email protected]> > > > > wrote: > > > > > > > > > Isha, > > > > > > > > > > Thanks for moving this. When you move these files, please place > then > > > > under > > > > > a package which reflects its functionality. I don't see the need > for > > > > > package called schema. > > > > > > > > > > Thanks, > > > > > Chandni > > > > > > > > > > On Tue, Dec 15, 2015 at 12:31 PM, Isha Arkatkar < > > [email protected]> > > > > > wrote: > > > > > > > > > >> Hi, > > > > >> > > > > >> For csv parser there is an additional dependency. So, I'll move > > only > > > > >> json > > > > >> and xml to new location. > > > > >> > > > > >> Thanks, > > > > >> Isha > > > > >> > > > > >> On Tue, Dec 15, 2015 at 11:42 AM, Thomas Weise < > > > [email protected]> > > > > >> wrote: > > > > >> > > > > >> > As long as the operators don't introduce additional dependencies > > > they > > > > >> > should be in lib. > > > > >> > > > > > >> > > > > > >> > On Tue, Dec 15, 2015 at 9:34 AM, Shubham Pathak < > > > > >> [email protected]> > > > > >> > wrote: > > > > >> > > > > > >> > > Hi Chandni, > > > > >> > > > > > > >> > > I had written those operators. > > > > >> > > Here is the jira for that > > > > >> https://malhar.atlassian.net/browse/MLHR-1838 > > > > >> > > You would find the entire discussion there. > > > > >> > > > > > > >> > > Why are all these operator under Malhar/contrib and not > > Malhar/lib > > > > >> > > When i was writing the code i saw AbstractCsvParser in > contriib > > > and > > > > >> hence > > > > >> > > added there. > > > > >> > > > > > > >> > > Recently i got to know which operators must go in contrib and > > what > > > > >> must > > > > >> > go > > > > >> > > in lib. > > > > >> > > By that definition, these operators must belong to lib. > > > > >> > > > > > > >> > > Thanks, > > > > >> > > Shubham > > > > >> > > > > > > >> > > > > > > >> > > On Tue, Dec 15, 2015 at 1:32 PM, Chandni Singh < > > > > >> [email protected]> > > > > >> > > wrote: > > > > >> > > > > > > >> > > > Hi, > > > > >> > > > > > > > >> > > > I just came across couple of formatter and parser operators > > > which > > > > >> are > > > > >> > > under > > > > >> > > > Malhar/contrib/schema. > > > > >> > > > > > > > >> > > > I have couple of questions: > > > > >> > > > 1. What does schema denote here? > > > > >> > > > 2. Why formatter/parser which are functions are placed under > > > > schema > > > > >> > > > package? > > > > >> > > > 2. Why are all these operator under Malhar/contrib and not > > > > >> Malhar/lib > > > > >> > > > > > > > >> > > > Thanks, > > > > >> > > > Chandni > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
