I should also note that the IDL introduced another concept that we
previously were dealing with... protocol. Is the protocol just a grouping
of related records?
FYI, we are using Avro strictly for serialization/de-serialization and no
RPC features.
On Thu, Feb 20, 2014 at 4:19 PM, Software Dev <static.void....@gmail.com>wrote:
> We have a similar use case and I would like to just "flatten" out the
> schema by including the fields from the parent into the child without
> nesting. The reason I don't want to nest is because it doesnt play well
> with some of our tools (Impala, Pig, etc)
>
> Now back to your initial response. I started playing around with the IDL
> but I still can't seem to figure out how to inherit all the fields from a
> parent into a child record.
>
> I would like the final "Chile" record to look like:
>
> {
> "type": "record",
> "name": "Child",
> "fields" : [
> {
> "name": "bar",
> "type": "string"
> },
> {
> "name": "foo",
> "type": "string"
> }
> ]
> }
>
>
> On Thu, Feb 20, 2014 at 12:55 PM, Lewis John Mcgibbney <
> lewis.mcgibb...@gmail.com> wrote:
>
>> Going back to your initial question... why don't you wish to include
>> "foo" within nested "parent"?
>> I am not quite getting it here.
>> In my case, the nested records were of substantial size including dozens
>> of fields, which then had nested records. It did not scale to write out
>> AVSC definitions for the data model.
>>
>>
>> On Thu, Feb 20, 2014 at 8:28 PM, Software Dev
>> <static.void....@gmail.com>wrote:
>>
>>> Thanks for the input. I'm guessing then the above problem can only
>>> solved with IDL and not AVSC?
>>>
>>>
>>> On Thu, Feb 20, 2014 at 11:45 AM, Lewis John Mcgibbney <
>>> lewis.mcgibb...@gmail.com> wrote:
>>>
>>>> Hey,
>>>> Did you check out the IDL documentation?
>>>> http://avro.apache.org/docs/current/idl.html
>>>> I had similar data modeling issues a while back and this helped out A
>>>> LOT.
>>>> hth
>>>>
>>>>
>>>> On Thu, Feb 20, 2014 at 6:58 PM, Software Dev <
>>>> static.void....@gmail.com> wrote:
>>>>
>>>>> Is there anyway to include the fields of another schema into our
>>>>> schema WITHOUT it creating a nested record?
>>>>>
>>>>>
>>>>> {
>>>>> "type": "record",
>>>>> "name": "Parent",
>>>>> "fields" : [
>>>>> {
>>>>> "name": "foo",
>>>>> "type": "string"
>>>>> }
>>>>> ]
>>>>> }
>>>>>
>>>>> {
>>>>> "type": "record",
>>>>> "name": "Child",
>>>>> "fields" : [
>>>>> {
>>>>> "name": "bar",
>>>>> "type": "string"
>>>>> },
>>>>> // I dont want it nested like this
>>>>> // {
>>>>> // "name": "parent",
>>>>> // "type": "Parent"
>>>>> // }
>>>>> ]
>>>>> }
>>>>>
>>>>> So in this example is there a way to have child include both the "bar"
>>>>> field as well as "foo" without it nested under parent?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>>>
>>
>>
>> --
>> *Lewis*
>>
>
>
