Got it. I think approach 2 is still possible with json based schema. So basically, the operator can create json schema as mention in first mail based on column name and type. Then pass that json schema to this utility to get class to be used directly.
Thanks, Chinmay. On 22 Jan 2016 18:38, "Priyanka Gugale" <[email protected]> wrote: > Say I need to emit pojo from JDBCInputOperator. I will use on fly POJO > generator for this purpose instead of providing predefined class. > Now to generate POJO on fly there are two options: > 1. Get json as config from user as you specified. > 2. Based on column name and type generate pojo fields and hence pojo class > without much of user configuration. (I am only considering one level here, > not the nested pojo) > > Option 1 is must have to provide user a way to control how we generate > pojo. But option 2 is useful when user simply want to put column values in > some object. > > -Priyanka > > On Fri, Jan 22, 2016 at 6:30 PM, Chinmay Kolhatkar < > [email protected]> > wrote: > > > Hi Priyanka, > > > > Can you give an example? > > > > Thanks, > > Chinmay. > > > > ~ Chinmay > > On 22 Jan 2016 18:14, "Priyanka Gugale" <[email protected]> > wrote: > > > > > We do need configuration in some form like json, but I was thinking we > > can > > > have default wherever possible. This is an option and not replacement > to > > > json config. > > > > > > -Priyanka > > > > > > On Fri, Jan 22, 2016 at 4:34 PM, Chinmay Kolhatkar < > > > [email protected]> > > > wrote: > > > > > > > @priyanka > > > > > > > > Advantage with JSON schema is that one can represent nested POJO > > > definition > > > > in that. If you're referring to FieldInfo, I'm not sure if that is > > > possible > > > > there. > > > > > > > > Even if we chose not to have nested POJO now, I think JSON schema > input > > > to > > > > this utility gives more chance to expand easily later. > > > > > > > > > > > > > > > > On Fri, Jan 22, 2016 at 3:30 PM, Chinmay Kolhatkar < > > > > [email protected]> > > > > wrote: > > > > > > > > > Sure. We can make this utility to provide byte[] for compiled > class. > > > > > Though the utility will give a OutputStream object.. One can get > > byte[] > > > > > out of that. > > > > > > > > > > On Fri, Jan 22, 2016 at 2:56 PM, Timothy Farkas < > [email protected] > > > > > > > > wrote: > > > > > > > > > >> Suggestion: +1 for automatic generation of schema. The user should > > be > > > > able > > > > >> to override the a manually specified schema though. > > > > >> > > > > >> Question: If the mechanism used to generate the class can produce > a > > > > byte[] > > > > >> array, then you can send the byte[] array for the class to > > downstream > > > > >> operators and load the class from the byte[] array in each > operator. > > > An > > > > >> example of how to do this is here: > > > > >> > > > > >> > > > > >> > > > > > > > > > > http://stackoverflow.com/questions/1781091/java-how-to-load-class-stored-as-byte-into-the-jvm > > > > >> > > > > >> Thanks, > > > > >> Tim > > > > >> > > > > >> On Fri, Jan 22, 2016 at 12:59 AM, Priyanka Gugale < > > > > >> [email protected]> > > > > >> wrote: > > > > >> > > > > >> > Hi, > > > > >> > > > > > >> > Suggestion: > > > > >> > This came up with one of the discussion with Pramod, will it be > a > > > good > > > > >> > idea, for database input operators to generate pojo based on the > > > > >> selected > > > > >> > column field names and types? No need to accept json input from > > > user. > > > > >> > > > > > >> > Question: > > > > >> > How can we share this POJO among multiple operators when > > application > > > > is > > > > >> > already launched and class is generated on the fly? > > > > >> > > > > > >> > -Priyanka > > > > >> > > > > > >> > On Thu, Jan 21, 2016 at 11:16 PM, Chinmay Kolhatkar < > > > > >> > [email protected] > > > > >> > > wrote: > > > > >> > > > > > >> > > Hi All, > > > > >> > > > > > > >> > > We're planning to add a utility in malhar-library for > > generating a > > > > >> POJO > > > > >> > > class on the fly from given JSON schema. > > > > >> > > > > > > >> > > Use case is where the application is provided with schema and > > that > > > > >> needs > > > > >> > to > > > > >> > > be used in one or more operators either as a tuple over the > > stream > > > > OR > > > > >> for > > > > >> > > processing. > > > > >> > > > > > > >> > > General Design: > > > > >> > > 1. Utility will be provided with fqcn of the class and schema > > > > >> definition > > > > >> > > provided as json. > > > > >> > > 2. The schema definition will look like following: > > > > >> > > { > > > > >> > > "fqcn":"<qualified class name>", > > > > >> > > "fields": [ > > > > >> > > { > > > > >> > > "name":"field1", > > > > >> > > "type":"long" > > > > >> > > }, > > > > >> > > { > > > > >> > > "name":"field2", > > > > >> > > "type": "string" > > > > >> > > } > > > > >> > > ] > > > > >> > > } > > > > >> > > 3. Supported types identified in "type" JSON field are: > > > > >> > > boolean, char, byte, short, int, float, long, double > > > > >> > > 4. The output of this utility will be a generated .class file > in > > > the > > > > >> form > > > > >> > > of FSDataOutputStream. > > > > >> > > 5. Xbean asm5 library will be used for this. > > > > >> > > 6. Following methods will be added to the generated class: > > > > >> > > a. Getter/Setter methods for given fields. > > > > >> > > b. simple toString - Generate string equivalent for all > the > > > > fields > > > > >> > > c. hashCode method - calculate the overall hashCode using > > > > >> individual > > > > >> > > field hashcodes, similar to how String.hashCode generates > > hashCode > > > > of > > > > >> > > string using chars in it. > > > > >> > > d. equals method - Similar to how String.equals method has > > > done > > > > it > > > > >> > > using individual char. > > > > >> > > > > > > >> > > > > > > >> > > Questions I have about the functionality: > > > > >> > > 1. Should the utility also support nested pojo definition via > > > schema > > > > >> > json? > > > > >> > > In such case the field definition can look like following: > > > > >> > > { "name":"nestedField", "type":"<fqcn of nested class>", > > > > >> > > "fields":[....] } > > > > >> > > The nested class will be provided > > > > >> > > > > > > >> > > 2. Should the utility also support giving out the > > JarOutputStream > > > > >> which > > > > >> > > will have a generated jar? > > > > >> > > > > > > >> > > Please let me know your thoughts. > > > > >> > > > > > > >> > > Thanks, > > > > >> > > Chinmay. > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
