Hello,
thank you for your work on Storm. There is a few questions me and my colleagues
stumbled upon while working with the framework for tha past few months, and
that peaked when the new version was released:
1. Because of "Supervisor v2" ("Simplify Threading Model of the Supervisor",
STORM-2018), the new version 1.0.3 needs the ability to create symlinks ...
while in Local Mode, on Windows. Both `LocalCluster.submitTopology()` and
`Testing.withSimulatedTimeLocalCluster() ... Testing.completeTopology()` do
this, and it's a major pain for local development where our IDEs constantly run
tests in local mode.
Yes, we know about
http://storm.apache.org/releases/1.0.3/windows-users-guide.html (which 404s for
1.0.1 and 1.0.2, by the way), and running our IDE as Administrator helps, but
it might at least be a good idea to add this little change into the release
notes - "Running in local mode now requires the symlink creation permission,
too.". Or don't introduce new major features in .build versions :(. The
question: Is there any setting we might turn on to revert to the old 1.0.2
behaviour?
2. Since 1.0.3, every time we run any test in _local mode_, there is an extra
directory being created in the *root of our project*:
./logs/workers-artifacts/topologytest-random-uuid, and it contains a single
file, "worker.yaml". Not cool. What do we have to set up to move this? I went
through the release notes and could not find anything even remotely close.
3. How does `Testing.completeTopology()` know the topology is completed? We're
not acking any tuples, so I'd expect the method to return once all tuples have
internally timed out (or the `CompleteTopologyParam` timeout has passed).
However, the method returns much sooner (sooner than we'd say our topology is
"completed"), implying a more clever strategy. Is this deterministic? Are there
any knobs to turn?
4. Does local mode not honor `conf.registerSerialization()`? This seems
strange, but if we're sending an instance of `OurData` class in local mode,
the serialization fails with `NotSerializableException`, like this:
java.io.NotSerializableException: com.our.company.data.OurData
at org.apache.storm.utils.Utils.javaSerialize(Utils.java:236)
at
org.apache.storm.thrift$serialize_component_object.invoke(thrift.clj:172)
at
org.apache.storm.testing$complete_topology.doInvoke(testing.clj:514)
at clojure.lang.RestFn.invoke(RestFn.java:1124)
at
org.apache.storm.testing4j$_completeTopology.invoke(testing4j.clj:63)
at org.apache.storm.Testing.completeTopology(Unknown Source)
...even though we have used `conf.registerSerialization(OurData.class,
OurDataSerializer.class);` Slapping `Serilizable` on the class fixes the issue,
but obviously that's not the solution - we don't want to change our class
because of local testing, and we definitely want local testing to use the same
serialization mechanism as production. I'm very sure there's a lot of people
using this functionality, so we probably just overlooked something? We even
tried:
conf.put(Config.TOPOLOGY_TESTING_ALWAYS_TRY_SERIALIZE, true); //
Thanks for fixing this in 1.0.3!
conf.put(Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION, false);
By the way, as far as I know, Kryo can serialize nonserializable classes,
too, via using the `FieldSerializer`. Is there any hidden option to enable this
by default instead of Java serialization? Do you have any plans on using this
instead of Java serialization?
Thank you in advance for any responses, we've been scratching our heads lately.
Petr Janeček
---------- Původní zpráva ----------
Od: P. Taylor Goetz
Komu: user@storm.apache.org, d...@storm.apache.org
Datum: 14. 2. 2017 23:04:21
Předmět: Re: [ANNOUNCE] Apache Storm 1.0.3 Released
GitHub releases are created every time you create a tag, for example during the
release candidate process. They should not be considered official releases.
Official releases are also always signed with the GPG signature of an Apache
committer.
It’s always best to download releases from an official mirror.
-Taylor
On Feb 14, 2017, at 4:50 PM, Andrew Xor wrote:
from my understanding you can always fall back to github releases as well....
Best,
A.
On Tue, Feb 14, 2017 at 9:44 PM, P. Taylor Goetz wrote:
My apologies for sending this out early, not all web servers are in sync yet,
so you may get a 404 depending on which server you hit.
Direct link for download:
http://www.apache.org/dyn/closer.lua/storm/apache-storm-1.0.3/
Changelog: https://github.com/apache/storm/blob/v1.0.3/CHANGELOG.md
-Taylor
> On Feb 14, 2017, at 3:43 PM, P. Taylor Goetz wrote:
>
> The Apache Storm community is pleased to announce the release of Apache Storm
> version 1.0.3.
>
> Storm is a distributed, fault-tolerant, and high-performance realtime
> computation system that provides strong guarantees on the processing of data.
> You can read more about Storm on the project website:
>
> http://storm.apache.org
>
> Downloads of source and binary distributions are listed in our download
> section:
>
> http://storm.apache.org/downloads.html
>
> You can read more about this release in the following blog post:
>
> http://storm.apache.org/2017/02/14/storm103-released.html
>
> Distribution artifacts are available in Maven Central at the following
> coordinates:
>
> groupId: org.apache.storm
> artifactId: storm-core
> version: 1.0.3
>
> The full list of changes is available here[1]. Please let us know [2] if you
> encounter any problems.
>
> Regards,
>
> The Apache Storm Team
>
> [1]: https://github.com/apache/storm/blob/v1.0.3/CHANGELOG.md
> [2]: https://issues.apache.org/jira/browse/STORM
