Hello,
I found a breaking change using Apache Storm 2.6.0 snapshot build
dated 20231103 and without changing my topologies' code (keeping the
same big jars).
This seems to be tied to a datatype change in Map argument of
prepare() method of Bolt.
For example, this Bolt is KO at runtime:
public class MyKOBolt extends BaseRichBolt {
@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
cacheMaxSize = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE);
cacheTtlMin = (Long) stormConf.get(ConfKeys.SVC_DEF_CACHE_TTL_MIN);
as it throws this error:
2023-11-03 16:42:20.230 o.a.s.e.b.BoltExecutor
Thread-14-__system-executor[-1, -1] [INFO] Preparing bolt
__system:[-1]
2023-11-03 16:42:20.193 o.a.s.u.Utils
Thread-15-evaluateTriggers-executor[11, 11] [ERROR] Async loop died!
java.lang.ClassCastException: java.lang.Integer incompatible with java.lang.Long
at com.acme.storm.alerting.MyKOBolt .prepare(EvaluationBolt.java:71)
~[stormjar.jar:?]
at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
~[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at org.apache.storm.utils.Utils$1.run(Utils.java:393)
[storm-client-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:857) [?:?]
whereas this bolt is fine with Storm 2.6.0:
public class MyOKBolt extends BaseBasicBolt {
@Override
public void prepare(final Map stormConf, final TopologyContext context) {
super.prepare(stormConf, context);
this.redisPort = Integer.parseInt((String)
stormConf.get(ConfKeys.REDIS_PORT));
That said, I can modify the code of "MyKOBolt" to use same
Integer.parseInt or Integer.parseLong trick, but it's the first time
in my long history of upgrades that I have seen such runtime
incompatibility.
Maybe "MyKOBolt" was badly written since the beginning and I have just
hit the punishment for it: is there a documentation which clarifies
datatypes of the Map argument of prepare() method?
Also worth noting: "MyKOBolt" derives from BaseRichBolt , while
"MyOKBolt" derives from BaseBasicBolt => could this have any impact on
this finding ?
Last, I'm running this on Redhat Linux 8 and IBM Semeru JDK 17.0.8.1.
Kind regards,
Alexandre
Le ven. 3 nov. 2023 à 15:16, Julien Nioche
<lists.digitalpeb...@gmail.com> a écrit :
>
> Thanks Richard.
>
> Tried the latest snapshot with StormCrawler both in local and deployed mode
> and did not find any issues.
> Will try it on a topology generating WARC files next week to check that the
> dependency changes on Hadoop have not broken anything.
>
> Have a good week end
>
> Julien
>
> On Thu, 2 Nov 2023 at 19:25, Richard Zowalla <r...@apache.org> wrote:
>
> > Short update. Sorted out an issue with the tar.gz/zip with Julien today
> > and re-uploaded them to the nightlies area.
> >
> > This new bundle works as expected in my deployment but happy to receive
> > additional feedback before getting up a first release candidate ;-)
> >
> > Am Montag, dem 30.10.2023 um 08:21 +0100 schrieb Richard Zowalla:
> > > Hi Alexandre,
> > >
> > > we are not in a hurry here :) - take as much as time you need.
> > >
> > > Gruß
> > > Richard
> > >
> > >
> > > Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
> > > Vermeerbergen:
> > > > Hello Richard,
> > > >
> > > > Okay, I'm more than happy to do that with my pre-production cluster
> > > > (~10 topologies) using the binary artifacts.
> > > > Would it be OK if I can use up to end of this week so that I'll be
> > > > able to have enough time to check all potential issues that this
> > > > upgrade bring?
> > > >
> > > > Thanks,
> > > > Alexandre
> > > >
> > > > Le lun. 30 oct. 2023 à 07:35, Richard Zowalla <r...@apache.org> a
> > > > écrit :
> > > > >
> > > > > Hi all,
> > > > >
> > > > > before starting a release of 2.6.0-SNAPSHOT, I would like to
> > > > > receive
> > > > > some feedback on the current SNAPSHOT build.
> > > > >
> > > > > I just uploaded a 2.6.0-SNAPSHOT of Storm build from
> > > > >
> > https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
> > > > >
> > > > > It is available via the ASF Snapshot repository:
> > > > > https://repository.apache.org/content/repositories/snapshots/
> > > > >
> > > > > You can consume by adding
> > > > >
> > > > > <repository>
> > > > > <id>apache.snapshots</id>
> > > > > <name>Apache Snapshot Repository</name>
> > > > > <url>https://repository.apache.org/snapshots</url>
> > > > > <releases>
> > > > > <enabled>false</enabled>
> > > > > </releases>
> > > > > </repository>
> > > > >
> > > > > to your project pom. As we do not deploy SNAPSHOTS automatically,
> > > > > it
> > > > > should be easy to just consume the latest SNAPSHOT.
> > > > >
> > > > > The packaged binaries are available at nightlies.apache.org:
> > > > >
> > > > >
> > https://nightlies.apache.org/storm/2.6.0-SNAPSHOT/8f883086032669a8f04b09a3b312d60af5b44533/
> > > > >
> > > > > If you have some minutes left: Please test and report any issues
> > > > > with
> > > > > this binaries, so we can fix before attempting to release.
> > > > >
> > > > > The most significant changes are in the are of hadoop/hbase/hdfs
> > > > > as
> > > > > we
> > > > > upgraded from 2.x to 3.x - our own test coverage within the build
> > > > > looks
> > > > > good but would be nice to get some real world use-case feedback
> > > > > on
> > > > > this. In addition, we had quite a lof of 3rd party dependency
> > > > > upgrades.
> > > > >
> > > > > In addition, it contains the pruning of external modules as
> > > > > listed
> > > > > in
> > > > >
> > > > > https://issues.apache.org/jira/browse/STORM-3988
> > > > >
> > > > > A summaryin Jira is here:
> > > > >
> > > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314820&version=12353484
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > >
> >
> >
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble <http://twitter.com/digitalpebble>
