Yes. So in the JDBC output case, a json object could be readily constructed by the operator to get a POJO class rather than playing around with JSON syntax.
On Fri, Jan 22, 2016 at 6:54 PM, Chinmay Kolhatkar <[email protected]> wrote: > @bhupesh, would it help if JSONObject is taken as input to this utility > rather than json string? > > Thanks, > Chinmay > On 22 Jan 2016 18:45, "Chinmay Kolhatkar" <[email protected]> wrote: > > > Got it. I think approach 2 is still possible with json based schema. > > So basically, the operator can create json schema as mention in first > mail > > based on column name and type. Then pass that json schema to this utility > > to get class to be used directly. > > > > Thanks, > > Chinmay. > > On 22 Jan 2016 18:38, "Priyanka Gugale" <[email protected]> > wrote: > > > >> Say I need to emit pojo from JDBCInputOperator. I will use on fly POJO > >> generator for this purpose instead of providing predefined class. > >> Now to generate POJO on fly there are two options: > >> 1. Get json as config from user as you specified. > >> 2. Based on column name and type generate pojo fields and hence pojo > class > >> without much of user configuration. (I am only considering one level > here, > >> not the nested pojo) > >> > >> Option 1 is must have to provide user a way to control how we generate > >> pojo. But option 2 is useful when user simply want to put column values > in > >> some object. > >> > >> -Priyanka > >> > >> On Fri, Jan 22, 2016 at 6:30 PM, Chinmay Kolhatkar < > >> [email protected]> > >> wrote: > >> > >> > Hi Priyanka, > >> > > >> > Can you give an example? > >> > > >> > Thanks, > >> > Chinmay. > >> > > >> > ~ Chinmay > >> > On 22 Jan 2016 18:14, "Priyanka Gugale" <[email protected]> > >> wrote: > >> > > >> > > We do need configuration in some form like json, but I was thinking > we > >> > can > >> > > have default wherever possible. This is an option and not > replacement > >> to > >> > > json config. > >> > > > >> > > -Priyanka > >> > > > >> > > On Fri, Jan 22, 2016 at 4:34 PM, Chinmay Kolhatkar < > >> > > [email protected]> > >> > > wrote: > >> > > > >> > > > @priyanka > >> > > > > >> > > > Advantage with JSON schema is that one can represent nested POJO > >> > > definition > >> > > > in that. If you're referring to FieldInfo, I'm not sure if that is > >> > > possible > >> > > > there. > >> > > > > >> > > > Even if we chose not to have nested POJO now, I think JSON schema > >> input > >> > > to > >> > > > this utility gives more chance to expand easily later. > >> > > > > >> > > > > >> > > > > >> > > > On Fri, Jan 22, 2016 at 3:30 PM, Chinmay Kolhatkar < > >> > > > [email protected]> > >> > > > wrote: > >> > > > > >> > > > > Sure. We can make this utility to provide byte[] for compiled > >> class. > >> > > > > Though the utility will give a OutputStream object.. One can get > >> > byte[] > >> > > > > out of that. > >> > > > > > >> > > > > On Fri, Jan 22, 2016 at 2:56 PM, Timothy Farkas < > >> [email protected] > >> > > > >> > > > > wrote: > >> > > > > > >> > > > >> Suggestion: +1 for automatic generation of schema. The user > >> should > >> > be > >> > > > able > >> > > > >> to override the a manually specified schema though. > >> > > > >> > >> > > > >> Question: If the mechanism used to generate the class can > >> produce a > >> > > > byte[] > >> > > > >> array, then you can send the byte[] array for the class to > >> > downstream > >> > > > >> operators and load the class from the byte[] array in each > >> operator. > >> > > An > >> > > > >> example of how to do this is here: > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > > >> > > > >> > > >> > http://stackoverflow.com/questions/1781091/java-how-to-load-class-stored-as-byte-into-the-jvm > >> > > > >> > >> > > > >> Thanks, > >> > > > >> Tim > >> > > > >> > >> > > > >> On Fri, Jan 22, 2016 at 12:59 AM, Priyanka Gugale < > >> > > > >> [email protected]> > >> > > > >> wrote: > >> > > > >> > >> > > > >> > Hi, > >> > > > >> > > >> > > > >> > Suggestion: > >> > > > >> > This came up with one of the discussion with Pramod, will it > >> be a > >> > > good > >> > > > >> > idea, for database input operators to generate pojo based on > >> the > >> > > > >> selected > >> > > > >> > column field names and types? No need to accept json input > from > >> > > user. > >> > > > >> > > >> > > > >> > Question: > >> > > > >> > How can we share this POJO among multiple operators when > >> > application > >> > > > is > >> > > > >> > already launched and class is generated on the fly? > >> > > > >> > > >> > > > >> > -Priyanka > >> > > > >> > > >> > > > >> > On Thu, Jan 21, 2016 at 11:16 PM, Chinmay Kolhatkar < > >> > > > >> > [email protected] > >> > > > >> > > wrote: > >> > > > >> > > >> > > > >> > > Hi All, > >> > > > >> > > > >> > > > >> > > We're planning to add a utility in malhar-library for > >> > generating a > >> > > > >> POJO > >> > > > >> > > class on the fly from given JSON schema. > >> > > > >> > > > >> > > > >> > > Use case is where the application is provided with schema > and > >> > that > >> > > > >> needs > >> > > > >> > to > >> > > > >> > > be used in one or more operators either as a tuple over the > >> > stream > >> > > > OR > >> > > > >> for > >> > > > >> > > processing. > >> > > > >> > > > >> > > > >> > > General Design: > >> > > > >> > > 1. Utility will be provided with fqcn of the class and > schema > >> > > > >> definition > >> > > > >> > > provided as json. > >> > > > >> > > 2. The schema definition will look like following: > >> > > > >> > > { > >> > > > >> > > "fqcn":"<qualified class name>", > >> > > > >> > > "fields": [ > >> > > > >> > > { > >> > > > >> > > "name":"field1", > >> > > > >> > > "type":"long" > >> > > > >> > > }, > >> > > > >> > > { > >> > > > >> > > "name":"field2", > >> > > > >> > > "type": "string" > >> > > > >> > > } > >> > > > >> > > ] > >> > > > >> > > } > >> > > > >> > > 3. Supported types identified in "type" JSON field are: > >> > > > >> > > boolean, char, byte, short, int, float, long, double > >> > > > >> > > 4. The output of this utility will be a generated .class > >> file in > >> > > the > >> > > > >> form > >> > > > >> > > of FSDataOutputStream. > >> > > > >> > > 5. Xbean asm5 library will be used for this. > >> > > > >> > > 6. Following methods will be added to the generated class: > >> > > > >> > > a. Getter/Setter methods for given fields. > >> > > > >> > > b. simple toString - Generate string equivalent for all > >> the > >> > > > fields > >> > > > >> > > c. hashCode method - calculate the overall hashCode > using > >> > > > >> individual > >> > > > >> > > field hashcodes, similar to how String.hashCode generates > >> > hashCode > >> > > > of > >> > > > >> > > string using chars in it. > >> > > > >> > > d. equals method - Similar to how String.equals method > >> has > >> > > done > >> > > > it > >> > > > >> > > using individual char. > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > Questions I have about the functionality: > >> > > > >> > > 1. Should the utility also support nested pojo definition > via > >> > > schema > >> > > > >> > json? > >> > > > >> > > In such case the field definition can look like > >> following: > >> > > > >> > > { "name":"nestedField", "type":"<fqcn of nested > class>", > >> > > > >> > > "fields":[....] } > >> > > > >> > > The nested class will be provided > >> > > > >> > > > >> > > > >> > > 2. Should the utility also support giving out the > >> > JarOutputStream > >> > > > >> which > >> > > > >> > > will have a generated jar? > >> > > > >> > > > >> > > > >> > > Please let me know your thoughts. > >> > > > >> > > > >> > > > >> > > Thanks, > >> > > > >> > > Chinmay. > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
