I am going to prepare an RC1 for VOTE today.
Am Mittwoch, dem 15.11.2023 um 19:33 +0100 schrieb Alexandre
Vermeerbergen:
> Great !
> On a side note, my preproduction has been running with 2.6.0 build
> 20231114 at scale since the last 10 hours, using my usual set:
> RedHat Linux 8
> IBM Java
Great !
On a side note, my preproduction has been running with 2.6.0 build
20231114 at scale since the last 10 hours, using my usual set:
RedHat Linux 8
IBM Java Semeru 17.0.8.1 (aka OpenJ9) - 6 big machines for Supervisors
14 topologies (with one having 5 different spouts and 9 bolts, all
w
Thx. The related PR is merged.
Am Mittwoch, dem 15.11.2023 um 15:13 +0100 schrieb Richard Zowalla:
> Yes, feel free to provide a fix / pr + jira issue.
>
> It is just a matter of adding
>
>
> jakarta.xml.bind
> jakarta.xml.bind-api
>
>
>
> to the related pom in the /externa
Okay!
I just created https://issues.apache.org/jira/browse/STORM-4003 for
the tracking.
Le mer. 15 nov. 2023 à 15:16, Richard Zowalla a écrit :
>
> Yes, feel free to provide a fix / pr + jira issue.
>
> It is just a matter of adding
>
>
> jakarta.xml.bind
> jakarta.xml.bind-api
>
Yes, feel free to provide a fix / pr + jira issue.
It is just a matter of adding
jakarta.xml.bind
jakarta.xml.bind-api
to the related pom in the /external module [1]
Gruß
Richard
[1]
https://github.com/apache/storm/blob/master/external/storm-kafka-monitor/pom.xml
Am Mitt
I had found yet another thing that must be fixed before starting 2.6.0
vote: while reviewing my shells to install Storm on my Nimbus hosts
and Supervision hosts, I found this old by-pass which I used to put in
place when switching from Java 11 to Java 17 at runtime:
cp storm-stable/lib-worker/jaka
Good news on my side:
* First test with 2.6.0 snapshot 2023/11/14 with a single Supervisor host is OK
=> I will soon (tomorrow) be able to upgrade my pre production
cluster to have a wider scope & performance results "at scale"
* My crazy attempt to convert all my topologies code from Java EE AP
Thanks Richard,
I have tested the latest snapshot with a simple crawl topology in local and
deployed mode + a more complex one which generates WARC files (and
therefore the HDFS resouces), again both in local and remote mode.
No issues found, everything is looking fine
Julien
On Tue, 14 Nov 2023
Thanks Richard !
I am starting a release of my code based on this new Storm 2.6.0
snapshot 20231114, and I will upgrade our pre-production (running with
previous 2.6.0 snapshot of 2nd of November + few hand applies patches)
On a side note, I have made a crazy test yesterday, which was to
migrate
Hi all,
as written in the other thread, I have prepared another SNAPSHOT
artifact incorporting the feedbach received in the first round.
I would like to receive some feedback on the current SNAPSHOT build
again.
I just uploaded a 2.6.0-SNAPSHOT of Storm build from
https://github.com/apache/stor
Here is the fix:
https://github.com/apache/storm/commit/d640cea65b1cbc71c85c220073e68442c3269a93
I will provide a new snapshot soon (= new thread as this one is really
long), so Julien can do his tests on the hdfs/hadoop stuff in the
context of crawling work ;-)
Gruß
Richard
Am Montag, dem 13.1
Yet another regression on in-work 2.6.0 : the supervisor.html view is
broken, there's a Javascript syntax error (probably my fault again).
It' s bit late for me, I'll check that tomorrow...
Alexandre
Le lun. 13 nov. 2023 à 15:36, Julien Nioche
a écrit :
>
> I agree with Richard. The next release
I agree with Richard. The next release contains enough big changes and it
would be good to release it ASAP before embarking on another big
modification.
Julien
On Mon, 13 Nov 2023 at 12:08, Richard Zowalla wrote:
> Hi,
>
> yes - in principal I agree, that switching to slf4j 2.x would be great
>
Hi,
yes - in principal I agree, that switching to slf4j 2.x would be great
in the near future. However, it requires some more work due to our old
dependency stack ("external modules", ...) and most likely a lot of
exclusions.
Therefore, personally, I wouldn't see it in a 2.6.0 release but as a
t
Thank Richard for the great work to cleanup CVEs on in-work 2.6.0 !
One last "regret" I have with current in-work 2.6.0 is that we still
rely on very old slf4j 1.x.
While it was not on slf4j itself, the log4shell CVE should remind us
that we should keep loggers as up to date as possible.
I see in
Ok, most of the stuff reported by grype should be fixed now.
I might be able to push a new snapshot within the next week, so we can
do some testing and than go for a release candiate / vote :)
Gruß
Richard
Am Donnerstag, dem 09.11.2023 um 08:53 +0100 schrieb Richard Zowalla:
> Maybe just run gr
At this moment that's the CVE which I found using Storm 2.6.0 snapshot 20231102.
And since there's a commons-text-1.11.0.jar available, I guess we
should directly jump to this version.
Le jeu. 9 nov. 2023 à 08:49, Alexandre Vermeerbergen
a écrit :
>
> Yet another finding on my side to "not call y
Maybe just run grype [1] on the lib directories and post the output on
the list / thread, so we can create tickets for it
There might be some false positives but will give good insights.
Gruß
Richard
[1] https://github.com/anchore/grype
Am Donnerstag, dem 09.11.2023 um 08:49 +0100 schrieb Alexa
Yet another finding on my side to "not call yet for vote on a Storm
2.6.0", we have some CVEs and we shoudn't release a version having
known CVEs.
For example, we have
find . -name "commons-text*jar" -print
./lib-webapp/commons-text-1.9.jar
./external/storm-autocreds/commons-text-1.10.0.jar
The
Oh thank you very much for this cherry-pinking, I was too busy doing
other unrelated stuff..
Le mar. 7 nov. 2023 à 17:35, Richard Zowalla a écrit :
>
> I cherry-picked your commit and put that on master
>
>
> Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> > You pushed on yo
I cherry-picked your commit and put that on master
Am Dienstag, dem 07.11.2023 um 17:25 +0100 schrieb Richard Zowalla:
> You pushed on your own branch. You would need to push to apache/storm
> main branch or open a PR ;-)
>
> Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
> Vermeerb
We dropped "org.json.simple.*" and replaced it with JSON from
"net.minidev".
So it would be best to replace that dependency (should be a namespace
shift only).
Am Samstag, dem 04.11.2023 um 20:57 +0100 schrieb Alexandre
Vermeerbergen:
> Yes another regression I found with my topologies (built w
You pushed on your own branch. You would need to push to apache/storm
main branch or open a PR ;-)
Am Samstag, dem 04.11.2023 um 10:36 +0100 schrieb Alexandre
Vermeerbergen:
> Just did a 'git push' with that trivial fix... is that enough?
>
> Le sam. 4 nov. 2023 à 09:34, Richard Zowalla a
> écri
Hello Julien,
Thanks for your suggestion to rebuild my code, but as I mentioned, my
first test with Storm 2.6.0 SNAPSHOT was to deploy existing topologies
without rebuilding these topologies.
It turned out to be a worthy experience, given the "old bad practices"
in my topologies' code that this ex
Alexandre,
I had a similar issue when trying the initial snapshot made by Richard but
it seems to have disappeared with the new one.
Maybe try building your code with *mvn clean package -U *or delete the
content of* ~/.m2/repository/org/apache/storm/ *beforehand?
Julien
On Sat, 4 Nov 2023 at 19
Interestingly enough, to clean our topologies' code from its use of
Storm shaded classes, I removed storm-shaded-deps.jar from our
dependencies... only to find that in such case we have a compilation
error:
The type org.apache.storm.thrift.TException cannot be resolved. It is
indirectly referenced
Yes another regression I found with my topologies (built with 2.5.0
dependencies) running to Storm 2.6.0 SNAPSHOT 2023/11/02: I have seen
several exceptions like this one:
2023-11-04 18:43:31.696 o.a.s.e.e.ReportError
Thread-16-alertsHistoryUMToKafka-executor[27, 27] [ERROR] Error
java.lang.Runtim
Just did a 'git push' with that trivial fix... is that enough?
Le sam. 4 nov. 2023 à 09:34, Richard Zowalla a écrit :
>
> You can do as you like ;-)
>
> Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
> :
> >I mean :
> >
> >Can I keep using my
> >'STORM-3958-Capacity-to-set-St
>
You can do as you like ;-)
Am 4. November 2023 08:54:48 MEZ schrieb Alexandre Vermeerbergen
:
>I mean :
>
>Can I keep using my
>'STORM-3958-Capacity-to-set-St
>orm-UI's-title-in-conf/storm.yaml'
>branch to push this fix, or should I create a new one?
>
>Le sam. 4 nov. 2023 à 08:53, Alexandre Verm
I mean :
Can I keep using my
'STORM-3958-Capacity-to-set-St
orm-UI's-title-in-conf/storm.yaml'
branch to push this fix, or should I create a new one?
Le sam. 4 nov. 2023 à 08:53, Alexandre Vermeerbergen
a écrit :
>
> Can you keep using my
> 'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/st
Can you keep using my
'STORM-3958-Capacity-to-set-Storm-UI's-title-in-conf/storm.yaml'
branch to push this fix, or should I create a new one?
Le sam. 4 nov. 2023 à 08:14, Richard Zowalla a écrit :
>
> Just push a fix for it (referencing the original issue) ;-) - we hadn't a
> release in between,
Just push a fix for it (referencing the original issue) ;-) - we hadn't a
release in between, so not a big deal.
Am 4. November 2023 08:01:54 MEZ schrieb Alexandre Vermeerbergen
:
>Temporary workaround for the Storm UI title's regression: run this
>command on the title-template.html file (in
>st
Temporary workaround for the Storm UI title's regression: run this
command on the title-template.html file (in
storm-stable/public/templates directory) after having extracted the
binary archive:
sed -i 's/{{ui\.title}}/{{title}}/' title-template.html
Again sorry for having introduced this regress
Updates regarding my test with Storm 2.6.0 Snapshot 2023/11/02:
1. I have modified my KO bolts using this pattern in their prepare() method:
long cacheMaxSize =
Long.parseLong(stormConf.get(ConfKeys.SVC_DEF_CACHE_SIZE).toString());
instead of:
long cacheMaxSize = (Long) sto
Thanks Richard for the analysis.
My thought on this are very simple: let me fix my own old code, after
all this was impacting only one of my topologies, the other one were
using more careful Integer.parseInt or Integer.parseLong to avoid
making assumption on the way the values in the config Map we
Hi,
thanks for the feedback.
The main difference to 2.5.0 is, that we switched the JSON
implementation in Utils, which relied on a
super old JSON library. What is happening here is, that the new JSON
library (net.minidev) behaves differently.
We can see it by writing some simple code without St
On a side note, the Long value that "MyKOBolt" is set by a
DefaultTopologyConfig to a value with is a java.lang.Long instance...
hence it's surprising to see it internally converted into a
java.lang.Integer in the Map passed to prepare() method...
Le ven. 3 nov. 2023 à 18:27, Alexandre Vermeerber
Hello,
I found a breaking change using Apache Storm 2.6.0 snapshot build
dated 20231103 and without changing my topologies' code (keeping the
same big jars).
This seems to be tied to a datatype change in Map argument of
prepare() method of Bolt.
For example, this Bolt is KO at runtime:
public c
Thanks Richard.
Tried the latest snapshot with StormCrawler both in local and deployed mode
and did not find any issues.
Will try it on a topology generating WARC files next week to check that the
dependency changes on Hadoop have not broken anything.
Have a good week end
Julien
On Thu, 2 Nov 2
Apologies: I have been way too busy many unrelated stuff (including,
but not limited to bad weather, which is ironic when thinking about
Storm ;); so I haven't yet started testing pre-vote Storm 2.6.0.
Trying to bootstrap this activity tomorrow, if the sky is nice enough
not try falling on my head
Short update. Sorted out an issue with the tar.gz/zip with Julien today
and re-uploaded them to the nightlies area.
This new bundle works as expected in my deployment but happy to receive
additional feedback before getting up a first release candidate ;-)
Am Montag, dem 30.10.2023 um 08:21 +0100
Hi Alexandre,
we are not in a hurry here :) - take as much as time you need.
Gruß
Richard
Am Montag, dem 30.10.2023 um 07:50 +0100 schrieb Alexandre
Vermeerbergen:
> Hello Richard,
>
> Okay, I'm more than happy to do that with my pre-production cluster
> (~10 topologies) using the binary artif
Hello Richard,
Okay, I'm more than happy to do that with my pre-production cluster
(~10 topologies) using the binary artifacts.
Would it be OK if I can use up to end of this week so that I'll be
able to have enough time to check all potential issues that this
upgrade bring?
Thanks,
Alexandre
Le
Hi all,
before starting a release of 2.6.0-SNAPSHOT, I would like to receive
some feedback on the current SNAPSHOT build.
I just uploaded a 2.6.0-SNAPSHOT of Storm build from
https://github.com/apache/storm/commit/8f883086032669a8f04b09a3b312d60af5b44533
It is available via the ASF Snapshot repo
44 matches
Mail list logo
