Chinmay, Can you explain the use case a bit more. Why would the user not supply the classes instead of providing information to generate classes.
Thanks On Mon, Jan 11, 2016 at 7:30 AM, Chinmay Kolhatkar <[email protected]> wrote: > Hi All, > > I've added APEXCORE Jira for this: > https://issues.apache.org/jira/browse/APEXCORE-304. > > Here is how I'm thinking about achieving it: > 1) LogicalPlan class will have a new method as follows: > > *void addResource(String localPath, boolean deleteOnceCopied);* > > This method will just keep the resource information i.e. jar file name in a > data structure in logical plan. > > For application containing modules, if this method is used by module, then > during flattening of the DAG, this resource will be copied to the parent > DAG. > > 2) In StramClient, this added resource will also be copied to lib folder of > the application in HDFS. > After copying to HDFS, if the delete flag is set for this resource, the > local resource will be deleted. > > > Please share your thoughts on this approach. > > Thanks, > Chinmay. > > > > > On Fri, Jan 8, 2016 at 1:11 PM, Tushar Gosavi <[email protected]> > wrote: > > > +1 > > In past we achieved this by writing generated class to a jar file in /tmp > > directory and adding that jar in LIBJARS_CONF_KEY_NAME > > conf.set(StramAppLauncher.LIBJARS_CONF_KEY_NAME, > conf.get(StramAppLauncher. > > LIBJARS_CONF_KEY_NAME) + "," + generatedPojoJar); > > This is a kind of hack and generated jar file does not get cleaned up as > we > > don't get any callback after application is submitted in > > StreamingApplication. It may be > > good idea to provide a api to register temp jar which will get copied to > > application directory, and deleted (optionally) from local machine after > > application is submitted. > > > > Regards, > > -Tushar. > > > > > > On Fri, Jan 8, 2016 at 12:07 PM, Chinmay Kolhatkar < > > [email protected]> > > wrote: > > > > > Hi Everyone, > > > > > > I have a requirement where I need to add certain classes to classpath > > > during application started (during initialization). > > > > > > I get to know the classes to be added to classpath only via property of > > > application i.e. populateDAG method is the first point where I get to > > know > > > what need to be added to classpath. > > > The class I want to add primary defines what a tuple would look like. > > > Basically properties will have the field information, populateDAG needs > > to > > > generate the class and add it to classpath for all the operators. > > > > > > Creating a class is not a problem, but adding to classpath is. > > > Is there a hook present in Apex platform where I can add given .class > > file > > > to the classpath in populateDAG? > > > > > > If not, I was wondering if this will be a positive addition to the > > > platform. > > > I have some code already created for this. If everyone agrees this to > be > > a > > > good addition to platform, I'll be happy to work on this and contribute > > to > > > the platform. > > > > > > Please share your thoughts. > > > > > > Thanks, > > > Chinmay. > > > > > >
