Re: Parameterized deserializers?
If you make your Serialization implement Configurable it will be given
a Configuration object that it can pass to the Deserializer on
construction.
Also, this thread may be related:
http://www.nabble.com/Serialization-with-additional-schema-info-td19260579.html
Tom
On Sat, Sep 13, 2008 at 12:38 AM, Pete Wyckoff <[EMAIL PROTECTED]> wrote:
>
> I should mention this is out of the context of SequenceFiles where we get
> the class names in the file itself. Here there is some information inserted
> into the JobConf that tells me the class of the records in the input file.
>
>
> -- pete
>
>
> On 9/12/08 3:26 PM, "Pete Wyckoff" <[EMAIL PROTECTED]> wrote:
>
>>
>> If I have a generic Serializer/Deserializers that take some runtime
>> information to instantiate, how would this work in the current
>> serializer/deserializer APIs? And depending on this runtime information, may
>> return different Objects although they may all derive from the same class.
>>
>> For example, for Thrift, I may have something called a ThriftSerializer that
>> is general:
>>
>> {code}
>> Public class ThriftDeserializer implements
>> Deserializer {
>> T deserialize(T);
>> }
>> {code}
>>
>> How would I instantiate this, since the current getDeserializer takes only
>> the Class but not configuration object.
>> How would I implement createKey in RecordReader
>>
>>
>> In other words, I think we need a {code}Class getClass(); {code} method
>> in Deserializer() and a {code}Deserializer getDeserializer(Class,
>> Configuration conf); {code} method in Serializer.java.
>>
>> Or is there another way to do this?
>>
>> IF not, I can open a JIRA for implementing parameterized serializers.
>>
>> Thanks, pete
>>
>>
>>
>
>
Re: Parameterized deserializers?
I should mention this is out of the context of SequenceFiles where we get
the class names in the file itself. Here there is some information inserted
into the JobConf that tells me the class of the records in the input file.
-- pete
On 9/12/08 3:26 PM, "Pete Wyckoff" <[EMAIL PROTECTED]> wrote:
>
> If I have a generic Serializer/Deserializers that take some runtime
> information to instantiate, how would this work in the current
> serializer/deserializer APIs? And depending on this runtime information, may
> return different Objects although they may all derive from the same class.
>
> For example, for Thrift, I may have something called a ThriftSerializer that
> is general:
>
> {code}
> Public class ThriftDeserializer implements
> Deserializer {
> T deserialize(T);
> }
> {code}
>
> How would I instantiate this, since the current getDeserializer takes only
> the Class but not configuration object.
> How would I implement createKey in RecordReader
>
>
> In other words, I think we need a {code}Class getClass(); {code} method
> in Deserializer() and a {code}Deserializer getDeserializer(Class,
> Configuration conf); {code} method in Serializer.java.
>
> Or is there another way to do this?
>
> IF not, I can open a JIRA for implementing parameterized serializers.
>
> Thanks, pete
>
>
>
Parameterized deserializers?
If I have a generic Serializer/Deserializers that take some runtime
information to instantiate, how would this work in the current
serializer/deserializer APIs? And depending on this runtime information, may
return different Objects although they may all derive from the same class.
For example, for Thrift, I may have something called a ThriftSerializer that
is general:
{code}
Public class ThriftDeserializer implements
Deserializer {
T deserialize(T);
}
{code}
How would I instantiate this, since the current getDeserializer takes only
the Class but not configuration object.
How would I implement createKey in RecordReader
In other words, I think we need a {code}Class getClass(); {code} method
in Deserializer() and a {code}Deserializer getDeserializer(Class,
Configuration conf); {code} method in Serializer.java.
Or is there another way to do this?
IF not, I can open a JIRA for implementing parameterized serializers.
Thanks, pete
