Yeah, I think so:-) Best, Yingyi
On Fri, Apr 29, 2016 at 3:46 PM, Mike Carey <[email protected]> wrote: > This indeed might be cleaner? > > > On 4/29/16 3:28 PM, Yingyi Bu wrote: > >> I'm guessing that you can do similar things to CastRecordDescriptor >>>> if you want to handle general cases in that region. >>>> >>> Or, you can inject a cast-record function in the loading pipeline >> so that you can defer the runtime-type-check/cast to that function instead >> of doing that in the parser. >> >> >> On Fri, Apr 29, 2016 at 3:25 PM, Yingyi Bu <[email protected]> wrote: >> >> My answer is inlined. >>> >>> My feeling is SerializerDeserializer offers another level of abstraction >>>>> but with output I can write value directly without construct AType >>>>> >>>> object. >>> >>>> I am wondering if there are any preferences over these two? >>>>> >>>> I agree with you. However, a SerializerDeserializer has to be stateless, >>> hence it cannot be used at runtime for complex type objects such as >>> records and lists, >>> because it will create a lot Java objects. >>> >>> in other words, parser has to guarantee that the >>>>> processed records has to match the dataset definition(non-optional >>>>> attribute cannot have null value). I tried to assign null value to >>>>> >>>> non-null >>> >>>> attributes. It will be inserted successfully but read records will have >>>>> problem. >>>>> >>>> That sounds right to me. Please file a JIRA issue and assign to you ( >>> if you're working on that). >>> I'm guessing that you can do similar things to CastRecordDescriptor >>> if you want to handle general cases in that region. >>> >>> 3. Set to null or skip >>>>> For optional(nullable) attributes, if I want to insert a record with >>>>> >>>> null >>> >>>> value on that attribute. Should I assign null value or should I just >>>>> >>>> skip >>> >>>> it? (Probably this is related to the missing attribute that Yingyi >>>>> mentioned today?) >>>>> >>>> Assign null value. >>> Missing means the field doesn't exist in a record at all. >>> >>> Best, >>> Yingyi >>> >>> >>> On Fri, Apr 29, 2016 at 2:06 PM, Xikui Wang <[email protected]> wrote: >>> >>> Hi devs, >>>> >>>> I came across several questions while I was constructing records in >>>> AsterixDB. Hope someone can help me clear the confusion. :) >>>> >>>> 1. Write directly to data output or use SerializerDeserializer >>>> I am working with AbstractDataParser now. I see people using different >>>> ways >>>> to append attributes to data output. Either use: >>>> output.Write(typetag.serialize()); >>>> output.WriteInt(0); >>>> to write into data output directly, or >>>> use AInt8SerializerDeserializer.serialize(int8Serde) to serialize a >>>> AINT8 >>>> instance to output. *SerializerDeserializer uses writeByte to write >>>> output. >>>> >>>> My feeling is SerializerDeserializer offers another level of abstraction >>>> but with output I can write value directly without construct AType >>>> object. >>>> I am wondering if there are any preferences over these two? >>>> >>>> 2. RecordType validation after parser but before add to frame? >>>> My observation is after parser finish writing the output and pass it to >>>> next level, there is no such validation that checks whether non-optional >>>> field is null or not. In other words, parser has to guarantee that the >>>> processed records has to match the dataset definition(non-optional >>>> attribute cannot have null value). I tried to assign null value to >>>> non-null >>>> attributes. It will be inserted successfully but read records will have >>>> problem. >>>> >>>> 3. Set to null or skip >>>> For optional(nullable) attributes, if I want to insert a record with >>>> null >>>> value on that attribute. Should I assign null value or should I just >>>> skip >>>> it? (Probably this is related to the missing attribute that Yingyi >>>> mentioned today?) >>>> >>>> Thanks for your help. >>>> >>>> Best, >>>> Xikui >>>> >>>> >>> >
