[jira] [Updated] (STORM-2988) "Error on initialization of server mk-worker" when using org.apache.storm.metrics2.reporters.JmxStormReporter on worker
[
https://issues.apache.org/jira/browse/STORM-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated STORM-2988:
Affects Version/s: 2.0.0
> "Error on initialization of server mk-worker" when using
> org.apache.storm.metrics2.reporters.JmxStormReporter on worker
> ---
>
> Key: STORM-2988
> URL: https://issues.apache.org/jira/browse/STORM-2988
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-client, storm-core
>Affects Versions: 2.0.0, 1.2.1
> Environment: CentOS 7.4
> java.version=1.8.0_161
> java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/jre
>Reporter: Federico Chiacchiaretta
>Assignee: Artem Ervits
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> As per documentation, I configured metrics v2 in my storm.yaml using the
> following configuration:
>
> {code:yaml}
> storm.metrics.reporters:
> - class: "org.apache.storm.metrics2.reporters.JmxStormReporter"
> daemons:
> - "supervisor"
> - "nimbus"
> - "worker"
> report.period: 10
> report.period.units: "SECONDS"
> {code}
> When I start nimbus and supervisors everything works properly, I can see
> metrics reported to JMX, and logs (for nimbus in this example) report:
> {code}
> 2018-03-07 15:35:22.201 o.a.s.d.m.MetricsUtils main [INFO] Using statistics
> reporter
> plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
> 2018-03-07 15:35:22.203 o.a.s.d.m.r.JmxPreparableReporter main [INFO]
> Preparing...
> 2018-03-07 15:35:22.221 o.a.s.d.common main [INFO] Started statistics report
> plugin...
> {code}
> When I submit a topology, workers cannot initialize and report this error
> {code:java}
> 2018-03-07 15:39:19.136 o.a.s.d.worker main [INFO] Launching worker for
> stp_topology-1-1520433551 on [... cut ...]
> 2018-03-07 15:39:19.169 o.a.s.m.StormMetricRegistry main [INFO] Starting
> metrics reporters...
> 2018-03-07 15:39:19.172 o.a.s.m.StormMetricRegistry main [INFO] Attempting to
> instantiate reporter class:
> org.apache.storm.metrics2.reporters.JmxStormReporter
> 2018-03-07 15:39:19.175 o.a.s.m.r.JmxStormReporter main [INFO] Preparing...
> 2018-03-07 15:39:19.182 o.a.s.d.worker main [ERROR] Error on initialization
> of server mk-worker
> java.lang.IllegalArgumentException: Don't know how to convert {"class"
> "org.apache.storm.metrics2.reporters.JmxStormReporter", "daemons"
> ["supervisor" "nimbus" "worker"], "report.period" 10, "report.period.units"
> "SECONDS"} + to String
> at org.apache.storm.utils.Utils.getString(Utils.java:848)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.reporters.JmxStormReporter.getMetricsJMXDomain(JmxStormReporter.java:70)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.reporters.JmxStormReporter.prepare(JmxStormReporter.java:51)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.StormMetricRegistry.startReporter(StormMetricRegistry.java:119)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.StormMetricRegistry.start(StormMetricRegistry.java:102)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.daemon.worker$fn__5545$exec_fn__1369__auto5546.invoke(worker.clj:611)
> ~[storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker$_main.invoke(worker.clj:787)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker.main(Unknown Source)
> [storm-core-1.2.1.jar:1.2.1]
> 2018-03-07 15:39:19.195 o.a.s.util main [ERROR] Halting process: ("Error on
> initialization")
> java.lang.RuntimeException: ("Error on initialization")
> at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.
[jira] [Updated] (STORM-2988) "Error on initialization of server mk-worker" when using org.apache.storm.metrics2.reporters.JmxStormReporter on worker
[
https://issues.apache.org/jira/browse/STORM-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated STORM-2988:
Component/s: storm-client
> "Error on initialization of server mk-worker" when using
> org.apache.storm.metrics2.reporters.JmxStormReporter on worker
> ---
>
> Key: STORM-2988
> URL: https://issues.apache.org/jira/browse/STORM-2988
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-client, storm-core
>Affects Versions: 2.0.0, 1.2.1
> Environment: CentOS 7.4
> java.version=1.8.0_161
> java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/jre
>Reporter: Federico Chiacchiaretta
>Assignee: Artem Ervits
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> As per documentation, I configured metrics v2 in my storm.yaml using the
> following configuration:
>
> {code:yaml}
> storm.metrics.reporters:
> - class: "org.apache.storm.metrics2.reporters.JmxStormReporter"
> daemons:
> - "supervisor"
> - "nimbus"
> - "worker"
> report.period: 10
> report.period.units: "SECONDS"
> {code}
> When I start nimbus and supervisors everything works properly, I can see
> metrics reported to JMX, and logs (for nimbus in this example) report:
> {code}
> 2018-03-07 15:35:22.201 o.a.s.d.m.MetricsUtils main [INFO] Using statistics
> reporter
> plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
> 2018-03-07 15:35:22.203 o.a.s.d.m.r.JmxPreparableReporter main [INFO]
> Preparing...
> 2018-03-07 15:35:22.221 o.a.s.d.common main [INFO] Started statistics report
> plugin...
> {code}
> When I submit a topology, workers cannot initialize and report this error
> {code:java}
> 2018-03-07 15:39:19.136 o.a.s.d.worker main [INFO] Launching worker for
> stp_topology-1-1520433551 on [... cut ...]
> 2018-03-07 15:39:19.169 o.a.s.m.StormMetricRegistry main [INFO] Starting
> metrics reporters...
> 2018-03-07 15:39:19.172 o.a.s.m.StormMetricRegistry main [INFO] Attempting to
> instantiate reporter class:
> org.apache.storm.metrics2.reporters.JmxStormReporter
> 2018-03-07 15:39:19.175 o.a.s.m.r.JmxStormReporter main [INFO] Preparing...
> 2018-03-07 15:39:19.182 o.a.s.d.worker main [ERROR] Error on initialization
> of server mk-worker
> java.lang.IllegalArgumentException: Don't know how to convert {"class"
> "org.apache.storm.metrics2.reporters.JmxStormReporter", "daemons"
> ["supervisor" "nimbus" "worker"], "report.period" 10, "report.period.units"
> "SECONDS"} + to String
> at org.apache.storm.utils.Utils.getString(Utils.java:848)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.reporters.JmxStormReporter.getMetricsJMXDomain(JmxStormReporter.java:70)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.reporters.JmxStormReporter.prepare(JmxStormReporter.java:51)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.StormMetricRegistry.startReporter(StormMetricRegistry.java:119)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.StormMetricRegistry.start(StormMetricRegistry.java:102)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.daemon.worker$fn__5545$exec_fn__1369__auto5546.invoke(worker.clj:611)
> ~[storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker$_main.invoke(worker.clj:787)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker.main(Unknown Source)
> [storm-core-1.2.1.jar:1.2.1]
> 2018-03-07 15:39:19.195 o.a.s.util main [ERROR] Halting process: ("Error on
> initialization")
> java.lang.RuntimeException: ("Error on initialization")
> at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon
[jira] [Updated] (STORM-2988) "Error on initialization of server mk-worker" when using org.apache.storm.metrics2.reporters.JmxStormReporter on worker
[
https://issues.apache.org/jira/browse/STORM-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-2988:
--
Labels: pull-request-available (was: )
> "Error on initialization of server mk-worker" when using
> org.apache.storm.metrics2.reporters.JmxStormReporter on worker
> ---
>
> Key: STORM-2988
> URL: https://issues.apache.org/jira/browse/STORM-2988
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 1.2.1
> Environment: CentOS 7.4
> java.version=1.8.0_161
> java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/jre
>Reporter: Federico Chiacchiaretta
>Assignee: Artem Ervits
>Priority: Major
> Labels: pull-request-available
>
> As per documentation, I configured metrics v2 in my storm.yaml using the
> following configuration:
>
> {code:yaml}
> storm.metrics.reporters:
> - class: "org.apache.storm.metrics2.reporters.JmxStormReporter"
> daemons:
> - "supervisor"
> - "nimbus"
> - "worker"
> report.period: 10
> report.period.units: "SECONDS"
> {code}
> When I start nimbus and supervisors everything works properly, I can see
> metrics reported to JMX, and logs (for nimbus in this example) report:
> {code}
> 2018-03-07 15:35:22.201 o.a.s.d.m.MetricsUtils main [INFO] Using statistics
> reporter
> plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
> 2018-03-07 15:35:22.203 o.a.s.d.m.r.JmxPreparableReporter main [INFO]
> Preparing...
> 2018-03-07 15:35:22.221 o.a.s.d.common main [INFO] Started statistics report
> plugin...
> {code}
> When I submit a topology, workers cannot initialize and report this error
> {code:java}
> 2018-03-07 15:39:19.136 o.a.s.d.worker main [INFO] Launching worker for
> stp_topology-1-1520433551 on [... cut ...]
> 2018-03-07 15:39:19.169 o.a.s.m.StormMetricRegistry main [INFO] Starting
> metrics reporters...
> 2018-03-07 15:39:19.172 o.a.s.m.StormMetricRegistry main [INFO] Attempting to
> instantiate reporter class:
> org.apache.storm.metrics2.reporters.JmxStormReporter
> 2018-03-07 15:39:19.175 o.a.s.m.r.JmxStormReporter main [INFO] Preparing...
> 2018-03-07 15:39:19.182 o.a.s.d.worker main [ERROR] Error on initialization
> of server mk-worker
> java.lang.IllegalArgumentException: Don't know how to convert {"class"
> "org.apache.storm.metrics2.reporters.JmxStormReporter", "daemons"
> ["supervisor" "nimbus" "worker"], "report.period" 10, "report.period.units"
> "SECONDS"} + to String
> at org.apache.storm.utils.Utils.getString(Utils.java:848)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.reporters.JmxStormReporter.getMetricsJMXDomain(JmxStormReporter.java:70)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.reporters.JmxStormReporter.prepare(JmxStormReporter.java:51)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.StormMetricRegistry.startReporter(StormMetricRegistry.java:119)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.metrics2.StormMetricRegistry.start(StormMetricRegistry.java:102)
> ~[storm-core-1.2.1.jar:1.2.1]
> at
> org.apache.storm.daemon.worker$fn__5545$exec_fn__1369__auto5546.invoke(worker.clj:611)
> ~[storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker$_main.invoke(worker.clj:787)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker.main(Unknown Source)
> [storm-core-1.2.1.jar:1.2.1]
> 2018-03-07 15:39:19.195 o.a.s.util main [ERROR] Halting process: ("Error on
> initialization")
> java.lang.RuntimeException: ("Error on initialization")
> at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
> at
> org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
> [storm-core-1.2.1.jar:1.2.1]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker$_main.invoke(worker.clj:787)
> [storm-core-1.2.1.j
[jira] [Updated] (STORM-2988) "Error on initialization of server mk-worker" when using org.apache.storm.metrics2.reporters.JmxStormReporter on worker
[
https://issues.apache.org/jira/browse/STORM-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Weathers updated STORM-2988:
-
Description:
As per documentation, I configured metrics v2 in my storm.yaml using the
following configuration:
{code:yaml}
storm.metrics.reporters:
- class: "org.apache.storm.metrics2.reporters.JmxStormReporter"
daemons:
- "supervisor"
- "nimbus"
- "worker"
report.period: 10
report.period.units: "SECONDS"
{code}
When I start nimbus and supervisors everything works properly, I can see
metrics reported to JMX, and logs (for nimbus in this example) report:
{code}
2018-03-07 15:35:22.201 o.a.s.d.m.MetricsUtils main [INFO] Using statistics
reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
2018-03-07 15:35:22.203 o.a.s.d.m.r.JmxPreparableReporter main [INFO]
Preparing...
2018-03-07 15:35:22.221 o.a.s.d.common main [INFO] Started statistics report
plugin...
{code}
When I submit a topology, workers cannot initialize and report this error
{code:java}
2018-03-07 15:39:19.136 o.a.s.d.worker main [INFO] Launching worker for
stp_topology-1-1520433551 on [... cut ...]
2018-03-07 15:39:19.169 o.a.s.m.StormMetricRegistry main [INFO] Starting
metrics reporters...
2018-03-07 15:39:19.172 o.a.s.m.StormMetricRegistry main [INFO] Attempting to
instantiate reporter class: org.apache.storm.metrics2.reporters.JmxStormReporter
2018-03-07 15:39:19.175 o.a.s.m.r.JmxStormReporter main [INFO] Preparing...
2018-03-07 15:39:19.182 o.a.s.d.worker main [ERROR] Error on initialization of
server mk-worker
java.lang.IllegalArgumentException: Don't know how to convert {"class"
"org.apache.storm.metrics2.reporters.JmxStormReporter", "daemons" ["supervisor"
"nimbus" "worker"], "report.period" 10, "report.period.units" "SECONDS"} + to
String
at org.apache.storm.utils.Utils.getString(Utils.java:848)
~[storm-core-1.2.1.jar:1.2.1]
at
org.apache.storm.metrics2.reporters.JmxStormReporter.getMetricsJMXDomain(JmxStormReporter.java:70)
~[storm-core-1.2.1.jar:1.2.1]
at
org.apache.storm.metrics2.reporters.JmxStormReporter.prepare(JmxStormReporter.java:51)
~[storm-core-1.2.1.jar:1.2.1]
at
org.apache.storm.metrics2.StormMetricRegistry.startReporter(StormMetricRegistry.java:119)
~[storm-core-1.2.1.jar:1.2.1]
at
org.apache.storm.metrics2.StormMetricRegistry.start(StormMetricRegistry.java:102)
~[storm-core-1.2.1.jar:1.2.1]
at
org.apache.storm.daemon.worker$fn__5545$exec_fn__1369__auto5546.invoke(worker.clj:611)
~[storm-core-1.2.1.jar:1.2.1]
at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
at
org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
[storm-core-1.2.1.jar:1.2.1]
at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
at org.apache.storm.daemon.worker$_main.invoke(worker.clj:787)
[storm-core-1.2.1.jar:1.2.1]
at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
at org.apache.storm.daemon.worker.main(Unknown Source)
[storm-core-1.2.1.jar:1.2.1]
2018-03-07 15:39:19.195 o.a.s.util main [ERROR] Halting process: ("Error on
initialization")
java.lang.RuntimeException: ("Error on initialization")
at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
[storm-core-1.2.1.jar:1.2.1]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
at
org.apache.storm.daemon.worker$fn__5545$mk_worker__5636.doInvoke(worker.clj:598)
[storm-core-1.2.1.jar:1.2.1]
at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
at org.apache.storm.daemon.worker$_main.invoke(worker.clj:787)
[storm-core-1.2.1.jar:1.2.1]
at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
at org.apache.storm.daemon.worker.main(Unknown Source)
[storm-core-1.2.1.jar:1.2.1]
{code}
Looking at
org.apache.storm.metrics2.reporters.JmxStormReporter.getMetricsJMXDomain() I
found that it passes "reporterConf" map to Utils.getString() instead of a
string:
{code:java}
public static String getMetricsJMXDomain(Map reporterConf) {
return Utils.getString(reporterConf, JMX_DOMAIN);
}
{code}
The "prepare" method in
org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter used by nimbus
and supervisor correctly passes a string to Utils.getString():
{code:java}
public void prepare(MetricRegistry metricsRegistry, Map stormConf) {
LOG.info("Preparing...");
JmxReporter.Builder builder = JmxReporter.forRegistry(me
