Thank you Yingyi. I will try to figure out a solution from that direction. Best, Xikui
On Fri, Apr 29, 2016 at 3:48 PM, Yingyi Bu <[email protected]> wrote: > Yeah, I think so:-) > > Best, > Yingyi > > On Fri, Apr 29, 2016 at 3:46 PM, Mike Carey <[email protected]> wrote: > > > This indeed might be cleaner? > > > > > > On 4/29/16 3:28 PM, Yingyi Bu wrote: > > > >> I'm guessing that you can do similar things to CastRecordDescriptor > >>>> if you want to handle general cases in that region. > >>>> > >>> Or, you can inject a cast-record function in the loading pipeline > >> so that you can defer the runtime-type-check/cast to that function > instead > >> of doing that in the parser. > >> > >> > >> On Fri, Apr 29, 2016 at 3:25 PM, Yingyi Bu <[email protected]> wrote: > >> > >> My answer is inlined. > >>> > >>> My feeling is SerializerDeserializer offers another level of > abstraction > >>>>> but with output I can write value directly without construct AType > >>>>> > >>>> object. > >>> > >>>> I am wondering if there are any preferences over these two? > >>>>> > >>>> I agree with you. However, a SerializerDeserializer has to be > stateless, > >>> hence it cannot be used at runtime for complex type objects such as > >>> records and lists, > >>> because it will create a lot Java objects. > >>> > >>> in other words, parser has to guarantee that the > >>>>> processed records has to match the dataset definition(non-optional > >>>>> attribute cannot have null value). I tried to assign null value to > >>>>> > >>>> non-null > >>> > >>>> attributes. It will be inserted successfully but read records will > have > >>>>> problem. > >>>>> > >>>> That sounds right to me. Please file a JIRA issue and assign to you ( > >>> if you're working on that). > >>> I'm guessing that you can do similar things to CastRecordDescriptor > >>> if you want to handle general cases in that region. > >>> > >>> 3. Set to null or skip > >>>>> For optional(nullable) attributes, if I want to insert a record with > >>>>> > >>>> null > >>> > >>>> value on that attribute. Should I assign null value or should I just > >>>>> > >>>> skip > >>> > >>>> it? (Probably this is related to the missing attribute that Yingyi > >>>>> mentioned today?) > >>>>> > >>>> Assign null value. > >>> Missing means the field doesn't exist in a record at all. > >>> > >>> Best, > >>> Yingyi > >>> > >>> > >>> On Fri, Apr 29, 2016 at 2:06 PM, Xikui Wang <[email protected]> wrote: > >>> > >>> Hi devs, > >>>> > >>>> I came across several questions while I was constructing records in > >>>> AsterixDB. Hope someone can help me clear the confusion. :) > >>>> > >>>> 1. Write directly to data output or use SerializerDeserializer > >>>> I am working with AbstractDataParser now. I see people using different > >>>> ways > >>>> to append attributes to data output. Either use: > >>>> output.Write(typetag.serialize()); > >>>> output.WriteInt(0); > >>>> to write into data output directly, or > >>>> use AInt8SerializerDeserializer.serialize(int8Serde) to serialize a > >>>> AINT8 > >>>> instance to output. *SerializerDeserializer uses writeByte to write > >>>> output. > >>>> > >>>> My feeling is SerializerDeserializer offers another level of > abstraction > >>>> but with output I can write value directly without construct AType > >>>> object. > >>>> I am wondering if there are any preferences over these two? > >>>> > >>>> 2. RecordType validation after parser but before add to frame? > >>>> My observation is after parser finish writing the output and pass it > to > >>>> next level, there is no such validation that checks whether > non-optional > >>>> field is null or not. In other words, parser has to guarantee that the > >>>> processed records has to match the dataset definition(non-optional > >>>> attribute cannot have null value). I tried to assign null value to > >>>> non-null > >>>> attributes. It will be inserted successfully but read records will > have > >>>> problem. > >>>> > >>>> 3. Set to null or skip > >>>> For optional(nullable) attributes, if I want to insert a record with > >>>> null > >>>> value on that attribute. Should I assign null value or should I just > >>>> skip > >>>> it? (Probably this is related to the missing attribute that Yingyi > >>>> mentioned today?) > >>>> > >>>> Thanks for your help. > >>>> > >>>> Best, > >>>> Xikui > >>>> > >>>> > >>> > > >
