[jira] [Assigned] (STORM-2808) Main UI gets some errors.
[
https://issues.apache.org/jira/browse/STORM-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Joseph Evans reassigned STORM-2808:
--
Assignee: Govind Menon (was: Robert Joseph Evans)
> Main UI gets some errors.
> -
>
> Key: STORM-2808
> URL: https://issues.apache.org/jira/browse/STORM-2808
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Robert Joseph Evans
>Assignee: Govind Menon
>
> I noticed that the UI is throwing exceptions...
> {code}
> java.lang.NullPointerException
> at clojure.lang.Numbers.minus(Numbers.java:3708)
> at
> org.apache.storm.ui.core$cluster_summary$iter__3372__3376$fn__3377.invoke(core.clj:430)
> at clojure.lang.LazySeq.sval(LazySeq.java:40)
> at clojure.lang.LazySeq.seq(LazySeq.java:49)
> at clojure.lang.RT.seq(RT.java:507)
> at clojure.core$seq__4128.invoke(core.clj:137)
> at clojure.core.protocols$seq_reduce.invoke(protocols.clj:26)
> at clojure.core.protocols$fn__6506.invoke(protocols.clj:100)
> at
> clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
> at clojure.core$reduce.invoke(core.clj:6515)
> at org.apache.storm.ui.core$cluster_summary.invoke(core.clj:426)
> at org.apache.storm.ui.core$cluster_summary.invoke(core.clj:412)
> at org.apache.storm.ui.core$fn__3865.invoke(core.clj:1258)
> at compojure.core$make_route$fn__382.invoke(core.clj:100)
> at compojure.core$if_route$fn__370.invoke(core.clj:46)
> at compojure.core$if_method$fn__363.invoke(core.clj:31)
> at compojure.core$routing$fn__388.invoke(core.clj:113)
> at clojure.core$some.invoke(core.clj:2570)
> at compojure.core$routing.doInvoke(core.clj:113)
> at clojure.lang.RestFn.applyTo(RestFn.java:139)
> at clojure.core$apply.invoke(core.clj:632)
> at compojure.core$routes$fn__392.invoke(core.clj:118)
> at ring.middleware.json$wrap_json_params$fn__1800.invoke(json.clj:74)
> at
> ring.middleware.multipart_params$wrap_multipart_params$fn__1038.invoke(multipart_params.clj:172)
> at ring.middleware.reload$wrap_reload$fn__844.invoke(reload.clj:39)
> at
> org.apache.storm.ui.helpers$requests_middleware$fn__3110.invoke(helpers.clj:46)
> at org.apache.storm.ui.core$catch_errors$fn__4073.invoke(core.clj:1587)
> at
> ring.middleware.keyword_params$wrap_keyword_params$fn__2851.invoke(keyword_params.clj:36)
> at
> ring.middleware.nested_params$wrap_nested_params$fn__2891.invoke(nested_params.clj:89)
> at ring.middleware.params$wrap_params$fn__2823.invoke(params.clj:67)
> at
> ring.middleware.multipart_params$wrap_multipart_params$fn__1038.invoke(multipart_params.clj:172)
> at ring.middleware.flash$wrap_flash$fn__3096.invoke(flash.clj:39)
> at ring.middleware.session$wrap_session$fn__3081.invoke(session.clj:108)
> at
> ring.util.servlet$make_blocking_service_method$fn__2718.invoke(servlet.clj:113)
> at ring.util.servlet$servlet$fn__2729.invoke(servlet.clj:147)
> at
> ring.util.servlet.proxy$javax.servlet.http.HttpServlet$ff19274a.service(Unknown
> Source)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
> at
> org.apache.storm.logging.filters.AccessLoggingFilter.handle(AccessLoggingFilter.java:51)
> at
> org.apache.storm.logging.filters.AccessLoggingFilter.doFilter(AccessLoggingFilter.java:43)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at
> org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:308)
> at
> org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:262)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
[jira] [Created] (STORM-2808) Main UI gets some errors.
Robert Joseph Evans created STORM-2808:
--
Summary: Main UI gets some errors.
Key: STORM-2808
URL: https://issues.apache.org/jira/browse/STORM-2808
Project: Apache Storm
Issue Type: Bug
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans
I noticed that the UI is throwing exceptions...
{code}
java.lang.NullPointerException
at clojure.lang.Numbers.minus(Numbers.java:3708)
at
org.apache.storm.ui.core$cluster_summary$iter__3372__3376$fn__3377.invoke(core.clj:430)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:507)
at clojure.core$seq__4128.invoke(core.clj:137)
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:26)
at clojure.core.protocols$fn__6506.invoke(protocols.clj:100)
at
clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
at clojure.core$reduce.invoke(core.clj:6515)
at org.apache.storm.ui.core$cluster_summary.invoke(core.clj:426)
at org.apache.storm.ui.core$cluster_summary.invoke(core.clj:412)
at org.apache.storm.ui.core$fn__3865.invoke(core.clj:1258)
at compojure.core$make_route$fn__382.invoke(core.clj:100)
at compojure.core$if_route$fn__370.invoke(core.clj:46)
at compojure.core$if_method$fn__363.invoke(core.clj:31)
at compojure.core$routing$fn__388.invoke(core.clj:113)
at clojure.core$some.invoke(core.clj:2570)
at compojure.core$routing.doInvoke(core.clj:113)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:632)
at compojure.core$routes$fn__392.invoke(core.clj:118)
at ring.middleware.json$wrap_json_params$fn__1800.invoke(json.clj:74)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__1038.invoke(multipart_params.clj:172)
at ring.middleware.reload$wrap_reload$fn__844.invoke(reload.clj:39)
at
org.apache.storm.ui.helpers$requests_middleware$fn__3110.invoke(helpers.clj:46)
at org.apache.storm.ui.core$catch_errors$fn__4073.invoke(core.clj:1587)
at
ring.middleware.keyword_params$wrap_keyword_params$fn__2851.invoke(keyword_params.clj:36)
at
ring.middleware.nested_params$wrap_nested_params$fn__2891.invoke(nested_params.clj:89)
at ring.middleware.params$wrap_params$fn__2823.invoke(params.clj:67)
at
ring.middleware.multipart_params$wrap_multipart_params$fn__1038.invoke(multipart_params.clj:172)
at ring.middleware.flash$wrap_flash$fn__3096.invoke(flash.clj:39)
at ring.middleware.session$wrap_session$fn__3081.invoke(session.clj:108)
at
ring.util.servlet$make_blocking_service_method$fn__2718.invoke(servlet.clj:113)
at ring.util.servlet$servlet$fn__2729.invoke(servlet.clj:147)
at
ring.util.servlet.proxy$javax.servlet.http.HttpServlet$ff19274a.service(Unknown
Source)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
at
org.apache.storm.logging.filters.AccessLoggingFilter.handle(AccessLoggingFilter.java:51)
at
org.apache.storm.logging.filters.AccessLoggingFilter.doFilter(AccessLoggingFilter.java:43)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at
org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:308)
at
org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:262)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:561)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:334)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
[jira] [Updated] (STORM-2796) Flux: Provide means for invoking static factory methods
[
https://issues.apache.org/jira/browse/STORM-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-2796:
--
Labels: pull-request-available (was: )
> Flux: Provide means for invoking static factory methods
> ---
>
> Key: STORM-2796
> URL: https://issues.apache.org/jira/browse/STORM-2796
> Project: Apache Storm
> Issue Type: Improvement
> Components: Flux
>Affects Versions: 2.0.0, 1.1.1, 1.2.0, 1.0.6
>Reporter: P. Taylor Goetz
>Assignee: P. Taylor Goetz
> Labels: pull-request-available
>
> Provide a means to invoke static factory methods for flux components. E.g:
> Java signature:
> {code}
> public static MyComponent newInstance(String... params)
> {code}
> Yaml:
> {code}
> className: "org.apache.storm.flux.test.MyComponent"
> factory: "newInstance"
> factoryArgs: ["a", "b", "c"]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (STORM-2546) Kafka spout can stall / get stuck due to edge case with failing tuples
[ https://issues.apache.org/jira/browse/STORM-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stig Rohde Døssing updated STORM-2546: -- Affects Version/s: (was: 1.x) 1.1.0 > Kafka spout can stall / get stuck due to edge case with failing tuples > -- > > Key: STORM-2546 > URL: https://issues.apache.org/jira/browse/STORM-2546 > Project: Apache Storm > Issue Type: Bug > Components: storm-kafka-client >Affects Versions: 2.0.0, 1.1.0 >Reporter: Prasanna Ranganathan >Assignee: Stig Rohde Døssing > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > The mechanism for replaying a failed tuple involves seeking the kafka > consumer to the failing offset and then re-emitting it into the topology. A > tuple, when emitted the first time, will have an entry created in > OffsetManager. This entry will be removed only after the tuple is > successfully acknowledged and its offset successfully committed. Till then, > commits for offsets beyond the failing offset for that TopicPartition will be > blocked. > It is possible that when the spout seeks the consumer to the failing offset, > the corresponding kafka message is not returned in the poll response. This > can happen due to that offset being deleted or compacted away. In this > scenario that partition will be blocked from committing and progressing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2549) The fix for STORM-2343 is incomplete, and the spout can still get stuck on failed tuples
[ https://issues.apache.org/jira/browse/STORM-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stig Rohde Døssing updated STORM-2549: -- Fix Version/s: 2.0.0 > The fix for STORM-2343 is incomplete, and the spout can still get stuck on > failed tuples > > > Key: STORM-2549 > URL: https://issues.apache.org/jira/browse/STORM-2549 > Project: Apache Storm > Issue Type: Bug > Components: storm-kafka-client >Affects Versions: 2.0.0, 1.1.0 >Reporter: Stig Rohde Døssing >Assignee: Stig Rohde Døssing > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Example: > Say maxUncommittedOffsets is 10, maxPollRecords is 5, and the committedOffset > is 0. > The spout will initially emit up to offset 10, because it is allowed to poll > until numNonRetriableTuples is >= maxUncommittedOffsets > The spout will be allowed to emit another 5 tuples if offset 10 fails, so if > that happens, offsets 10-14 will get emitted. If offset 1 fails and 2-14 get > acked, the spout gets stuck because it will count the "extra tuples" 11-14 in > numNonRetriableTuples. > An similar case is the one where maxPollRecords doesn't divide > maxUncommittedOffsets evenly. If it were 3 in the example above, the spout > might just immediately emit offsets 1-12. If 2-12 get acked, offset 1 cannot > be reemitted. > The proposed solution is the following: > * Enforce maxUncommittedOffsets on a per partition basis (i.e. actual limit > will be multiplied by the number of partitions) by always allowing poll for > retriable tuples that are within maxUncommittedOffsets tuples of the > committed offset. Pause any non-retriable partitions if the partition has > passed the maxUncommittedOffsets limit, and some other partition is polling > for retries while also at the maxUncommittedOffsets limit. > Example of this functionality: > MaxUncommittedOffsets is 100 > MaxPollRecords is 10 > Committed offset for partition 0 and 1 is 0. > Partition 0 has emitted 0 > Partition 1 has emitted 0...95, 97, 99, 101, 103 (some offsets compacted away) > Partition 1, message 99 is retriable > We check that message 99 is within 100 emitted tuples of offset 0 (it is the > 97th tuple after offset 0, so it is) > We do not pause partition 0 because that partition isn't at the > maxUncommittedOffsets limit. > Seek to offset 99 on partition 1 and poll > We get back offset 99, 101, 103 and potentially 7 new tuples. Say the lowest > of these is at offset 104. > The spout emits offset 99, filters out 101 and 103 because they were already > emitted, and emits the 7 new tuples. > If offset 104 (or later) become retriable, they are not retried until the > committed offset moves. This is because offset 104 is the 101st tuple emitted > after offset 0, so it isn't allowed to retry until the committed offset moves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2807) Integration test should shut down topologies immediately after the test
[ https://issues.apache.org/jira/browse/STORM-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated STORM-2807: -- Labels: pull-request-available (was: ) > Integration test should shut down topologies immediately after the test > --- > > Key: STORM-2807 > URL: https://issues.apache.org/jira/browse/STORM-2807 > Project: Apache Storm > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Stig Rohde Døssing >Assignee: Stig Rohde Døssing >Priority: Minor > Labels: pull-request-available > > The integration test kills topologies with the default 30 second timeout. > This is unnecessary and delays the following tests, because the killed > topology is still occupying worker slots. > When the integration test kills topologies, it tries sending the kill message > to Nimbus once, and may fail quietly. This breaks following tests, because > the default Storm install has only 4 worker slots, and the test topologies > each take up 3. When a topology is not shut down, it prevents the following > topologies from being assigned. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (STORM-2807) Integration test should shut down topologies immediately after the test
Stig Rohde Døssing created STORM-2807: - Summary: Integration test should shut down topologies immediately after the test Key: STORM-2807 URL: https://issues.apache.org/jira/browse/STORM-2807 Project: Apache Storm Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Stig Rohde Døssing Assignee: Stig Rohde Døssing Priority: Minor The integration test kills topologies with the default 30 second timeout. This is unnecessary and delays the following tests, because the killed topology is still occupying worker slots. When the integration test kills topologies, it tries sending the kill message to Nimbus once, and may fail quietly. This breaks following tests, because the default Storm install has only 4 worker slots, and the test topologies each take up 3. When a topology is not shut down, it prevents the following topologies from being assigned. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
