Re: storm bolts receiving tuples with null values
Nathan,
That is exactly what was happening.
Thanks,
On Fri, Sep 19, 2014 at 2:11 PM, Vincent Russell
wrote:
> Thank you Nathan. I just looked at the serializer and there is one case
> where it can return null. I am going to add more debugging and retry.
>
> Thanks,
>
> On Fri, Sep 19, 2014 at 1:35 PM, Nathan Leung wrote:
>
>> What does the code for your kryo serializer look like? Are you sure that
>> it is not returning null? Kryo will only be used to serialize if your
>> tuple is crossing worker boundaries; when you have 1 worker everything is
>> more or less passed by reference (through some queues and whatnot, but it
>> does not get serialized). Since you are using custom serializer, I would
>> take a look at those first.
>>
>> On Fri, Sep 19, 2014 at 1:23 PM, Vincent Russell <
>> vincent.russ...@gmail.com> wrote:
>>
>>>
>>> Just to add additional information to this tread.
>>>
>>>
>>>- I am using custom kryo serializers that I have registered.
>>>- I turned on debug logging and I see debug messages (in my
>>>serializer) for the serialization of the event put in the tuple but I see
>>>no corresponding deserialization debug log so it looks like
>>> deserialization
>>>isn't even being attempted.
>>>- The .toString() on the tuple looks like this: (source: parse:30,
>>>stream: data, id: {-8767240739148856227=2139281104057768924}, [null])
>>>- I see no errors in the logs from storm
>>>
>>>
>>> Any assistance would be greatly appreciated.
>>>
>>> Thanks,
>>>
>>> On Wed, Sep 17, 2014 at 9:48 PM, Vincent Russell <
>>> vincent.russ...@gmail.com> wrote:
>>>
All,
When I change my topology workers from 1 to 2 my topology starts
sending tuples with null values. Is there a reason why this might happen?
The tuple isn't null, but when I call getValueByField I get a null value.
Is there a size limit for a tuple that I might be crossing?
I'm using storm version 0.9.0.1.
Thank you,
Vincent
>>>
>>>
>>
>
Re: storm bolts receiving tuples with null values
Thank you Nathan. I just looked at the serializer and there is one case
where it can return null. I am going to add more debugging and retry.
Thanks,
On Fri, Sep 19, 2014 at 1:35 PM, Nathan Leung wrote:
> What does the code for your kryo serializer look like? Are you sure that
> it is not returning null? Kryo will only be used to serialize if your
> tuple is crossing worker boundaries; when you have 1 worker everything is
> more or less passed by reference (through some queues and whatnot, but it
> does not get serialized). Since you are using custom serializer, I would
> take a look at those first.
>
> On Fri, Sep 19, 2014 at 1:23 PM, Vincent Russell <
> vincent.russ...@gmail.com> wrote:
>
>>
>> Just to add additional information to this tread.
>>
>>
>>- I am using custom kryo serializers that I have registered.
>>- I turned on debug logging and I see debug messages (in my
>>serializer) for the serialization of the event put in the tuple but I see
>>no corresponding deserialization debug log so it looks like
>> deserialization
>>isn't even being attempted.
>>- The .toString() on the tuple looks like this: (source: parse:30,
>>stream: data, id: {-8767240739148856227=2139281104057768924}, [null])
>>- I see no errors in the logs from storm
>>
>>
>> Any assistance would be greatly appreciated.
>>
>> Thanks,
>>
>> On Wed, Sep 17, 2014 at 9:48 PM, Vincent Russell <
>> vincent.russ...@gmail.com> wrote:
>>
>>>
>>> All,
>>>
>>> When I change my topology workers from 1 to 2 my topology starts sending
>>> tuples with null values. Is there a reason why this might happen? The
>>> tuple isn't null, but when I call getValueByField I get a null value. Is
>>> there a size limit for a tuple that I might be crossing?
>>>
>>>
>>> I'm using storm version 0.9.0.1.
>>>
>>>
>>> Thank you,
>>> Vincent
>>>
>>
>>
>
Re: storm bolts receiving tuples with null values
What does the code for your kryo serializer look like? Are you sure that
it is not returning null? Kryo will only be used to serialize if your
tuple is crossing worker boundaries; when you have 1 worker everything is
more or less passed by reference (through some queues and whatnot, but it
does not get serialized). Since you are using custom serializer, I would
take a look at those first.
On Fri, Sep 19, 2014 at 1:23 PM, Vincent Russell
wrote:
>
> Just to add additional information to this tread.
>
>
>- I am using custom kryo serializers that I have registered.
>- I turned on debug logging and I see debug messages (in my
>serializer) for the serialization of the event put in the tuple but I see
>no corresponding deserialization debug log so it looks like deserialization
>isn't even being attempted.
>- The .toString() on the tuple looks like this: (source: parse:30,
>stream: data, id: {-8767240739148856227=2139281104057768924}, [null])
>- I see no errors in the logs from storm
>
>
> Any assistance would be greatly appreciated.
>
> Thanks,
>
> On Wed, Sep 17, 2014 at 9:48 PM, Vincent Russell <
> vincent.russ...@gmail.com> wrote:
>
>>
>> All,
>>
>> When I change my topology workers from 1 to 2 my topology starts sending
>> tuples with null values. Is there a reason why this might happen? The
>> tuple isn't null, but when I call getValueByField I get a null value. Is
>> there a size limit for a tuple that I might be crossing?
>>
>>
>> I'm using storm version 0.9.0.1.
>>
>>
>> Thank you,
>> Vincent
>>
>
>
Re: storm bolts receiving tuples with null values
Just to add additional information to this tread.
- I am using custom kryo serializers that I have registered.
- I turned on debug logging and I see debug messages (in my serializer)
for the serialization of the event put in the tuple but I see no
corresponding deserialization debug log so it looks like deserialization
isn't even being attempted.
- The .toString() on the tuple looks like this: (source: parse:30,
stream: data, id: {-8767240739148856227=2139281104057768924}, [null])
- I see no errors in the logs from storm
Any assistance would be greatly appreciated.
Thanks,
On Wed, Sep 17, 2014 at 9:48 PM, Vincent Russell
wrote:
>
> All,
>
> When I change my topology workers from 1 to 2 my topology starts sending
> tuples with null values. Is there a reason why this might happen? The
> tuple isn't null, but when I call getValueByField I get a null value. Is
> there a size limit for a tuple that I might be crossing?
>
>
> I'm using storm version 0.9.0.1.
>
>
> Thank you,
> Vincent
>
