[jira] [Updated] (STORM-3581) Change log level to info to show the config classes being used for validation

2020-02-14 Thread Ethan Li (Jira) 


 [ 
https://issues.apache.org/jira/browse/STORM-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Li updated STORM-3581:

Affects Version/s: 2.0.0
   2.1.0

> Change log level to info to show the config classes being used for validation
> -
>
> Key: STORM-3581
> URL: https://issues.apache.org/jira/browse/STORM-3581
> Project: Apache Storm
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Trivial
>  Labels: pull-request-available
> Fix For: 2.2.0, 2.1.1
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/validation/ConfigValidation.java#L82]
> This is trivial but since it's caused some confusion, it's better to have it 
> in the log as INFO instead of DEBUG
> {code:java}
> LOG.debug("Will use {} for validation", ret);
> {code}
>  
> Because the classes being used for validation depends on whether the 
> following file is in the classpath or not
>  
> [https://github.com/apache/storm/blob/master/storm-server/src/main/resources/META-INF/services/org.apache.storm.validation.Validated]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (STORM-3581) Change log level to info to show the config classes being used for validation

2020-02-14 Thread Ethan Li (Jira) 


 [ 
https://issues.apache.org/jira/browse/STORM-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Li resolved STORM-3581.
-
Fix Version/s: 2.1.1
   2.2.0
   Resolution: Fixed

Merged to master

> Change log level to info to show the config classes being used for validation
> -
>
> Key: STORM-3581
> URL: https://issues.apache.org/jira/browse/STORM-3581
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Trivial
>  Labels: pull-request-available
> Fix For: 2.2.0, 2.1.1
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/validation/ConfigValidation.java#L82]
> This is trivial but since it's caused some confusion, it's better to have it 
> in the log as INFO instead of DEBUG
> {code:java}
> LOG.debug("Will use {} for validation", ret);
> {code}
>  
> Because the classes being used for validation depends on whether the 
> following file is in the classpath or not
>  
> [https://github.com/apache/storm/blob/master/storm-server/src/main/resources/META-INF/services/org.apache.storm.validation.Validated]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (STORM-3582) Kryo errors when using custom serialization

2020-02-14 Thread Julien Nioche (Jira) 
Julien Nioche created STORM-3582:


 Summary: Kryo errors when using custom serialization
 Key: STORM-3582
 URL: https://issues.apache.org/jira/browse/STORM-3582
 Project: Apache Storm
  Issue Type: Bug
  Components: storm-core
Affects Versions: 2.1.0
Reporter: Julien Nioche


(this has been reported on the user list in January and previously by a 
different user)
 

 
My code works fine with Storm 1.x but the workers crash constantly with Storm 
2.x 
 
Some exceptions look like 
 
{{_com.esotericsoftware.kryo.KryoException: Buffer underflow._}}
{{_at com.esotericsoftware.kryo.io.Input.require(Input.java:199) 
~[kryo-3.0.3.jar:?]_}}
{{_at com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575) 
~[kryo-3.0.3.jar:?]_}}
{{_at com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553) 
~[kryo-3.0.3.jar:?]_}}
{{_at com.esotericsoftware.kryo.io.Input.readString(Input.java:483) 
~[kryo-3.0.3.jar:?]_}}

{{whereas others are}}
 
{{_2020-01-21 11:13:39.368 o.a.s.m.n.StormServerHandler 
Netty-server-localhost-6701-worker-1 [ERROR] server errors in handling the 
request_}}
{{_com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 
*95*
at 
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
 ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781) 
~[kryo-3.0.3.jar:?]
at 
com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
 ~[kryo-3.0.3.jar:?]
at 
com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39) 
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793) 
~[kryo-3.0.3.jar:?]
at 
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
 ~[kryo-3.0.3.jar:?]
at 
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
 ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689) ~[kryo-3.0.3.jar:?]
at 
org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31)
 ~[storm-client-2.1.0.jar:2.1.0]_}}


{{the class ID values are random integers. }}

{{}}
The tuples in my topologies contain mostly standard classes like String; the 
only exception is the following class 
 
[https://github.com/DigitalPebble/storm-crawler/blob/2.x/core/src/main/java/com/digitalpebble/stormcrawler/Metadata.java#L41]
 
for which we specify a custom serialization for Kryo.
 
My configurations contain
 
| topology.kryo.register:|- com.digitalpebble.stormcrawler.Metadata|
to register the custom class.
 
The user who had reported the problem first also used custom serialization.
 
 
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)