[jira] [Created] (STORM-3675) Eliminate ps command and use /proc/ status in ServerUtils
Bipin Prasad created STORM-3675: --- Summary: Eliminate ps command and use /proc/ status in ServerUtils Key: STORM-3675 URL: https://issues.apache.org/jira/browse/STORM-3675 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad In ServerUtils.java, the *ps* command is very expensive on loaded Linux system. when trying to determine if all worker processes are dead. Changed ownership or missing /proc/ directory for should be sufficient to detect death of process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3675) Eliminate ps command and use /proc/ status in ServerUtils
[ https://issues.apache.org/jira/browse/STORM-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3675: --- Assignee: Bipin Prasad > Eliminate ps command and use /proc/ status in ServerUtils > -- > > Key: STORM-3675 > URL: https://issues.apache.org/jira/browse/STORM-3675 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > In ServerUtils.java, the *ps* command is very expensive on loaded Linux > system. when trying to determine if all worker processes are dead. Changed > ownership or missing /proc/ directory for should be sufficient to detect > death of process. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3683) Check for consistency in JVM options for worker launch
Bipin Prasad created STORM-3683: --- Summary: Check for consistency in JVM options for worker launch Key: STORM-3683 URL: https://issues.apache.org/jira/browse/STORM-3683 Project: Apache Storm Issue Type: Improvement Components: storm-client Reporter: Bipin Prasad If GC options are specified in topology.worker.childopts, they could conflict with the cluster settings and cause problems launching the JVM. This leads to storm alerts due to failures launching containers. We could catch these on submission and prevent the topology from launching with an error message. *Kishor Patil's further comments:* Submit time validation of JVM Options includes.. Running `java -showversion` kind of command while topology is being submitted using JVM_OPTIONS derived using # topology.worker.childopts # topology.worker.gc.childopts # topology.worker.logwriter.childopts # worker.childopts # worker.gc.childopts # worker.profiler.childopts And removing replacement strings such as "%ID%", "%WORKER-ID%", "%TOPOLOGY-ID%", "%WORKER-PORT", "%OFF-HEAP-MEM%", "%LIMIT-MEM%" with dummy values. To perform these replacements with dummy values, I would use externalize and use the logic from [BasicContainer#substituteChildOptsInternal|https://git.vzbuilders.com/storm/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L440] method. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3683) Check for consistency in JVM options for worker launch
[ https://issues.apache.org/jira/browse/STORM-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3683: --- Assignee: Bipin Prasad > Check for consistency in JVM options for worker launch > -- > > Key: STORM-3683 > URL: https://issues.apache.org/jira/browse/STORM-3683 > Project: Apache Storm > Issue Type: Improvement > Components: storm-client >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > If GC options are specified in topology.worker.childopts, they could conflict > with the cluster settings and cause problems launching the JVM. This leads to > storm alerts due to failures launching containers. > We could catch these on submission and prevent the topology from launching > with an error message. > *Kishor Patil's further comments:* > Submit time validation of JVM Options includes.. > Running `java -showversion` kind of command while topology is being submitted > using JVM_OPTIONS derived using > # topology.worker.childopts > # topology.worker.gc.childopts > # topology.worker.logwriter.childopts > # worker.childopts > # worker.gc.childopts > # worker.profiler.childopts > And removing replacement strings such as "%ID%", "%WORKER-ID%", > "%TOPOLOGY-ID%", "%WORKER-PORT", "%OFF-HEAP-MEM%", "%LIMIT-MEM%" with dummy > values. To perform these replacements with dummy values, I would use > externalize and use the logic from > [BasicContainer#substituteChildOptsInternal|https://git.vzbuilders.com/storm/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L440] > method. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3685) Detect Loops in Topology at Submit
Bipin Prasad created STORM-3685: --- Summary: Detect Loops in Topology at Submit Key: STORM-3685 URL: https://issues.apache.org/jira/browse/STORM-3685 Project: Apache Storm Issue Type: Improvement Components: storm-client Reporter: Bipin Prasad Assignee: Bipin Prasad Topology graph is expected to be a Directed Acyclic Graph (DAG). Detect cycles in DAG when Topology is submitted and prevent cycles in Topology. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3684) receive-queue V2 metrics shouldn't have "_system" as componentId
[ https://issues.apache.org/jira/browse/STORM-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3684: --- Assignee: Bipin Prasad (was: Rui Li) > receive-queue V2 metrics shouldn't have "_system" as componentId > > > Key: STORM-3684 > URL: https://issues.apache.org/jira/browse/STORM-3684 > Project: Apache Storm > Issue Type: Bug >Reporter: Rui Li >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > When storm registers receive-queue for executors, > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/worker/WorkerState.java#L722-L724] > it set componentId as "__system" for any task. This is wrong for V2 metrics. > V1 metrics are fine since the component Id used for V1 metric is not from the > same place. > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/executor/Executor.java#L342-L343] > It is from the member variable of the executor object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3684) receive-queue V2 metrics shouldn't have "_system" as componentId
[ https://issues.apache.org/jira/browse/STORM-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3684: --- Assignee: Rui Li (was: Bipin Prasad) > receive-queue V2 metrics shouldn't have "_system" as componentId > > > Key: STORM-3684 > URL: https://issues.apache.org/jira/browse/STORM-3684 > Project: Apache Storm > Issue Type: Bug >Reporter: Rui Li >Assignee: Rui Li >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > When storm registers receive-queue for executors, > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/worker/WorkerState.java#L722-L724] > it set componentId as "__system" for any task. This is wrong for V2 metrics. > V1 metrics are fine since the component Id used for V1 metric is not from the > same place. > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/executor/Executor.java#L342-L343] > It is from the member variable of the executor object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3689) Storm build fails when obtaining third party license
Bipin Prasad created STORM-3689: --- Summary: Storm build fails when obtaining third party license Key: STORM-3689 URL: https://issues.apache.org/jira/browse/STORM-3689 Project: Apache Storm Issue Type: Improvement Components: build Reporter: Bipin Prasad Assignee: Bipin Prasad Travis build fails. Ths log shows the following: [INFO] Downloading from clojars: https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties [ERROR] Failed to execute goal org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party (generate-and-check-licenses) on project storm: could not execute goal AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to locate third party descriptor: Could not transfer artifact org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer failed for https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties: Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed: Connection timed out (Connection timed out) -> [Help 1] In fact there is no artifact for [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties] It web site for ASM indicates that it is covered by BSD 3-Clause License. It appears that THIRD-PARTY.properties file needs an entry for ASM version 5.0.3. Nogt sure why this build failure started 15 days ago. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad updated STORM-3689:
Description:
Travis build fails. Ths log shows the following:
{code:java}
// code placeholder
[INFO] Downloading from clojars:
https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project storm: could not execute goal
AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to
locate third party descriptor: Could not transfer artifact
org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
(https://oss.sonatype.org/content/repositories/releases/): Transfer failed for
https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
In fact there is no artifact for
[https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
Its web site for ASM indicates that it is covered by BSD 3-Clause License.
It appears that THIRD-PARTY.properties file needs an entry for ASM version
5.0.3.
Not sure why this build failure started 15 days ago.
was:
Travis build fails. Ths log shows the following:
[INFO] Downloading from clojars:
https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project storm: could not execute goal
AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to
locate third party descriptor: Could not transfer artifact
org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
(https://oss.sonatype.org/content/repositories/releases/): Transfer failed for
https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
In fact there is no artifact for
[https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
It web site for ASM indicates that it is covered by BSD 3-Clause License.
It appears that THIRD-PARTY.properties file needs an entry for ASM version
5.0.3.
Nogt sure why this build failure started 15 days ago.
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180709#comment-17180709
]
Bipin Prasad commented on STORM-3689:
-
Next failure occurs with
{code:java}
// code placeholder
[INFO] Downloading from clojars:
https://clojars.org/repo/jakarta/annotation/jakarta.annotation-api/1.3.4/jakarta.annotation-api-1.3.4-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project storm: could not execute goal
AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to
locate third party descriptor: Could not transfer artifact
jakarta.annotation:jakarta.annotation-api:properties:third-party:1.3.4 from/to
sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer
failed for
https://oss.sonatype.org/content/repositories/releases/jakarta/annotation/jakarta.annotation-api/1.3.4/jakarta.annotation-api-1.3.4-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180709#comment-17180709
]
Bipin Prasad edited comment on STORM-3689 at 8/19/20, 5:25 PM:
---
Next failure occurs with
{code:java}
// code placeholder
[INFO] Downloading from clojars:
https://clojars.org/repo/jakarta/annotation/jakarta.annotation-api/1.3.4/jakarta.annotation-api-1.3.4-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project storm: could not execute goal
AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to
locate third party descriptor: Could not transfer artifact
jakarta.annotation:jakarta.annotation-api:properties:third-party:1.3.4 from/to
sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer
failed for
https://oss.sonatype.org/content/repositories/releases/jakarta/annotation/jakarta.annotation-api/1.3.4/jakarta.annotation-api-1.3.4-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
As per
[https://projects.eclipse.org/projects/ee4j.ca,|https://projects.eclipse.org/projects/ee4j.ca]
the license is
[Eclipse Public License 2.0|https://projects.eclipse.org/license/epl-2.0]
[一 (Secondary) GNU General Public License, version 2 with the GNU Classpath
Exception|https://projects.eclipse.org/license/secondary-gpl-2.0-cp]
was (Author: bipinprasad):
Next failure occurs with
{code:java}
// code placeholder
[INFO] Downloading from clojars:
https://clojars.org/repo/jakarta/annotation/jakarta.annotation-api/1.3.4/jakarta.annotation-api-1.3.4-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project storm: could not execute goal
AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to
locate third party descriptor: Could not transfer artifact
jakarta.annotation:jakarta.annotation-api:properties:third-party:1.3.4 from/to
sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer
failed for
https://oss.sonatype.org/content/repositories/releases/jakarta/annotation/jakarta.annotation-api/1.3.4/jakarta.annotation-api-1.3.4-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180763#comment-17180763
]
Bipin Prasad commented on STORM-3689:
-
Third failure occurs with in build final *Storm Binary Package. Where is this
dependency for stephenc being injected?*
{code:java}
// code placeholder
[INFO] Downloading from sonatype-apache:
https://repository.apache.org/content/repositories/releases/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project apache-storm-bin: could not execute
goal AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException:
Unable to locate third party descriptor: Could not transfer artifact
com.github.stephenc.jcip:jcip-annotations:properties:third-party:1.0-1 from/to
sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer
failed for
https://oss.sonatype.org/content/repositories/releases/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180763#comment-17180763
]
Bipin Prasad edited comment on STORM-3689 at 8/19/20, 7:05 PM:
---
Third failure occurs with in build final *Storm Binary Package. Where is this
dependency for stephenc being injected?*
As per [https://github.com/stephenc/jcip-annotations/blob/master/LICENSE.txt]
JCIP license is Apache License 2.0
{code:java}
// code placeholder
[INFO] Downloading from sonatype-apache:
https://repository.apache.org/content/repositories/releases/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project apache-storm-bin: could not execute
goal AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException:
Unable to locate third party descriptor: Could not transfer artifact
com.github.stephenc.jcip:jcip-annotations:properties:third-party:1.0-1 from/to
sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer
failed for
https://oss.sonatype.org/content/repositories/releases/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
was (Author: bipinprasad):
Third failure occurs with in build final *Storm Binary Package. Where is this
dependency for stephenc being injected?*
{code:java}
// code placeholder
[INFO] Downloading from sonatype-apache:
https://repository.apache.org/content/repositories/releases/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project apache-storm-bin: could not execute
goal AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException:
Unable to locate third party descriptor: Could not transfer artifact
com.github.stephenc.jcip:jcip-annotations:properties:third-party:1.0-1 from/to
sonatype (https://oss.sonatype.org/content/repositories/releases/): Transfer
failed for
https://oss.sonatype.org/content/repositories/releases/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183214#comment-17183214
]
Bipin Prasad commented on STORM-3689:
-
Fourth failure in JDK8:
[https://repo1.maven.org/maven2/org/glassfish/hk2/hk2-api/2.5.0/hk2-api-2.5.0-third-party.properties]
does not exist, but its parent directory does. License is EPL 2.0 as per
[https://mvnrepository.com/artifact/org.glassfish.hk2/hk2-api/2.5.0]
{code:java}
[INFO] Downloading from sonatype-apache:
https://repository.apache.org/content/repositories/releases/org/glassfish/hk2/hk2-api/2.5.0/hk2-api-2.5.0-third-party.properties
[ERROR] Failed to execute goal
org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
(generate-and-check-licenses) on project storm: could not execute goal
AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable to
locate third party descriptor: Could not transfer artifact
org.glassfish.hk2:hk2-api:properties:third-party:2.5.0 from/to sonatype
(https://oss.sonatype.org/content/repositories/releases/): Transfer failed for
https://oss.sonatype.org/content/repositories/releases/org/glassfish/hk2/hk2-api/2.5.0/hk2-api-2.5.0-third-party.properties:
Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
Connection timed out (Connection timed out) -> [Help 1]
{code}
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3691) Refactor Resource Aware Strategies (Base, Generic, Default)
Bipin Prasad created STORM-3691: --- Summary: Refactor Resource Aware Strategies (Base, Generic, Default) Key: STORM-3691 URL: https://issues.apache.org/jira/browse/STORM-3691 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad Code is duplicated in incompatible ways. Refactor BaseResourceAwareStrategy, DefaultResourceAwareStrategy and GenericResourceAwareStrategy to remove duplicated code and incompatibilities. Pushing out changes done internally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3692) Handle UID return from Files.getOwner()
[ https://issues.apache.org/jira/browse/STORM-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3692: --- Assignee: Bipin Prasad > Handle UID return from Files.getOwner() > --- > > Key: STORM-3692 > URL: https://issues.apache.org/jira/browse/STORM-3692 > Project: Apache Storm > Issue Type: Improvement >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Sometime Files.getOwner() returns the userid instead of user name. When this > happens, compare the user ids instead of the names. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3692) Handle UID return from Files.getOwner()
Bipin Prasad created STORM-3692: --- Summary: Handle UID return from Files.getOwner() Key: STORM-3692 URL: https://issues.apache.org/jira/browse/STORM-3692 Project: Apache Storm Issue Type: Improvement Reporter: Bipin Prasad Sometime Files.getOwner() returns the userid instead of user name. When this happens, compare the user ids instead of the names. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3702) Change thrift exception classes to contain getMessage() method
Bipin Prasad created STORM-3702: --- Summary: Change thrift exception classes to contain getMessage() method Key: STORM-3702 URL: https://issues.apache.org/jira/browse/STORM-3702 Project: Apache Storm Issue Type: Improvement Components: storm-client Affects Versions: 2.1.0 Reporter: Bipin Prasad Assignee: Bipin Prasad Thrift classes are currently being generated without camel casing for method names. And the exception classes have a member variable called "msg". This generates, get_msg() and set_msg() method names. Since this class extends java Exception class, when an TException is thrown, the getMessage() method returns null and cannot be relied upon. To get around this problem with getMessage() method, storm code has Wrapper classes, eg WrappedNotAliveException with a getMessage() method that return wraps get_msg() method in base class. Proposed Change: * Change the TException classes to rename member variable and generate camel cased class so that getMessage() is generated * Limit the change to specific TException classes only to avoid large change * Deprecate Wrapped classes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (STORM-3689) Storm build fails when obtaining third party license
[
https://issues.apache.org/jira/browse/STORM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad closed STORM-3689.
---
Resolution: Won't Fix
Build failures stopped.
> Storm build fails when obtaining third party license
>
>
> Key: STORM-3689
> URL: https://issues.apache.org/jira/browse/STORM-3689
> Project: Apache Storm
> Issue Type: Improvement
> Components: build
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Travis build fails. Ths log shows the following:
> {code:java}
> // code placeholder
> [INFO] Downloading from clojars:
> https://clojars.org/repo/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties
> [ERROR] Failed to execute goal
> org.codehaus.mojo:license-maven-plugin:2.0.0:aggregate-add-third-party
> (generate-and-check-licenses) on project storm: could not execute goal
> AggregatorAddThirdPartyMojo for reason : ArtifactResolutionException: Unable
> to locate third party descriptor: Could not transfer artifact
> org.ow2.asm:asm:properties:third-party:5.0.3 from/to sonatype
> (https://oss.sonatype.org/content/repositories/releases/): Transfer failed
> for
> https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties:
> Connect to repo1.maven.org:443 [repo1.maven.org/199.232.64.209] failed:
> Connection timed out (Connection timed out) -> [Help 1]
>
> {code}
> In fact there is no artifact for
> [https://oss.sonatype.org/content/repositories/releases/org/ow2/asm/asm/5.0.3/asm-5.0.3-third-party.properties]
> Its web site for ASM indicates that it is covered by BSD 3-Clause License.
> It appears that THIRD-PARTY.properties file needs an entry for ASM version
> 5.0.3.
> Not sure why this build failure started 15 days ago.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3706) Cluster.needsSchedulingRas always succeeds
Bipin Prasad created STORM-3706: --- Summary: Cluster.needsSchedulingRas always succeeds Key: STORM-3706 URL: https://issues.apache.org/jira/browse/STORM-3706 Project: Apache Storm Issue Type: Improvement Reporter: Bipin Prasad Assignee: Bipin Prasad Prior to refactoring of Resource Aware scheduling, Cluster.needsSchedulingRas always succeeded. Add tests to ensure this method returns correct value. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3708) ContstraintSolverConfig LOG messages should include topology id
Bipin Prasad created STORM-3708: --- Summary: ContstraintSolverConfig LOG messages should include topology id Key: STORM-3708 URL: https://issues.apache.org/jira/browse/STORM-3708 Project: Apache Storm Issue Type: Improvement Reporter: Bipin Prasad Assignee: Bipin Prasad Prior to refactoring of Resource Aware scheduling, Cluster.needsSchedulingRas always succeeded. Add tests to ensure this method returns correct value. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3708) ConstraintSolverConfig LOG messages should include topology id
[ https://issues.apache.org/jira/browse/STORM-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3708: Summary: ConstraintSolverConfig LOG messages should include topology id (was: ContstraintSolverConfig LOG messages should include topology id) > ConstraintSolverConfig LOG messages should include topology id > -- > > Key: STORM-3708 > URL: https://issues.apache.org/jira/browse/STORM-3708 > Project: Apache Storm > Issue Type: Improvement >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Prior to refactoring of Resource Aware scheduling, Cluster.needsSchedulingRas > always succeeded. Add tests to ensure this method returns correct value. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3708) ConstraintSolverConfig LOG messages should include topology id
[ https://issues.apache.org/jira/browse/STORM-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3708: Description: LOG messages in ConstraintSolverConfig class should include topology id. (was: Prior to refactoring of Resource Aware scheduling, Cluster.needsSchedulingRas always succeeded. Add tests to ensure this method returns correct value.) > ConstraintSolverConfig LOG messages should include topology id > -- > > Key: STORM-3708 > URL: https://issues.apache.org/jira/browse/STORM-3708 > Project: Apache Storm > Issue Type: Improvement >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > LOG messages in ConstraintSolverConfig class should include topology id. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3709) Reject topology submission if missing spout
Bipin Prasad created STORM-3709: --- Summary: Reject topology submission if missing spout Key: STORM-3709 URL: https://issues.apache.org/jira/browse/STORM-3709 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad Topologies without spout cannot be scheduled. These topologies should be rejected at submission time. 2020-10-28 19:40:26.608 o.a.s.s.r.s.s.BaseResourceAwareStrategy pool-21-thread-1 [ERROR] Topology test_topo_t04_01:Cannot find a Spout! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3721) Change child pom.xml reference to parent pom.xml
Bipin Prasad created STORM-3721:
---
Summary: Change child pom.xml reference to parent pom.xml
Key: STORM-3721
URL: https://issues.apache.org/jira/browse/STORM-3721
Project: Apache Storm
Issue Type: Improvement
Components: build
Reporter: Bipin Prasad
Assignee: Bipin Prasad
Several (but not all) module pom.xml files refer to the parent specifying only
the directory name. The correct syntax (as per Example 2 on
[https://maven.apache.org/guides/introduction/introduction-to-the-pom.html)] is
to include the relative path to parent pom.xml
{code:java}
com.mycompany.app
my-app
1
../parent/pom.xml
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3722) Update committer list
Bipin Prasad created STORM-3722: --- Summary: Update committer list Key: STORM-3722 URL: https://issues.apache.org/jira/browse/STORM-3722 Project: Apache Storm Issue Type: Documentation Reporter: Bipin Prasad Assignee: Bipin Prasad Update committer list -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3723) ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result
[ https://issues.apache.org/jira/browse/STORM-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3723: --- Assignee: Bipin Prasad > ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result > -- > > Key: STORM-3723 > URL: https://issues.apache.org/jira/browse/STORM-3723 > Project: Apache Storm > Issue Type: Bug > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > There is a bug in ServerUtils.isAnyPosixProcessPidDirAlive() and it returns > incorrect value when all the processIds have been reassigned/reused. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3723) ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result
Bipin Prasad created STORM-3723: --- Summary: ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result Key: STORM-3723 URL: https://issues.apache.org/jira/browse/STORM-3723 Project: Apache Storm Issue Type: Bug Components: storm-server Reporter: Bipin Prasad There is a bug in ServerUtils.isAnyPosixProcessPidDirAlive() and it returns incorrect value when all the processIds have been reassigned/reused. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3723) ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result
[ https://issues.apache.org/jira/browse/STORM-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250012#comment-17250012 ] Bipin Prasad commented on STORM-3723: - PR https://github.com/apache/storm/pull/3362 > ServerUtils.isAnyPosixProcessPidDirAlive might return wrong result > -- > > Key: STORM-3723 > URL: https://issues.apache.org/jira/browse/STORM-3723 > Project: Apache Storm > Issue Type: Bug > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > There is a bug in ServerUtils.isAnyPosixProcessPidDirAlive() and it returns > incorrect value when all the processIds have been reassigned/reused. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3730) Remove PMD Exceptions
Bipin Prasad created STORM-3730: --- Summary: Remove PMD Exceptions Key: STORM-3730 URL: https://issues.apache.org/jira/browse/STORM-3730 Project: Apache Storm Issue Type: Bug Components: storm-hdfs Reporter: Bipin Prasad Assignee: Bipin Prasad [*WARNING*] There are 5 PMD processing errors: [*WARNING*] /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/AvroGenericRecordBoltTest.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/AvroGenericRecordBoltTest.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestHdfsBolt.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestHdfsBolt.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestSequenceFileBolt.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestSequenceFileBolt.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterExtension.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterExtension.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterRule.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterRule.java -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3730) Remove PMD Exceptions
[ https://issues.apache.org/jira/browse/STORM-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3730: Description: At top level. run "mvn install -DskipTests" or "mvn pmd:pmd". Redirect to file to capture output. Approximately 200 PMDExceptions. was: [*WARNING*] There are 5 PMD processing errors: [*WARNING*] /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/AvroGenericRecordBoltTest.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/AvroGenericRecordBoltTest.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestHdfsBolt.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestHdfsBolt.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestSequenceFileBolt.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestSequenceFileBolt.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterExtension.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterExtension.java /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterRule.java: PMDException: Error while parsing /Users/bprasad/sources/community/bipinprasad/storm/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/testing/MiniDFSClusterRule.java > Remove PMD Exceptions > - > > Key: STORM-3730 > URL: https://issues.apache.org/jira/browse/STORM-3730 > Project: Apache Storm > Issue Type: Bug > Components: storm-hdfs >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > At top level. run "mvn install -DskipTests" or "mvn pmd:pmd". > Redirect to file to capture output. Approximately 200 PMDExceptions. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3730) Remove PMD Exceptions
[ https://issues.apache.org/jira/browse/STORM-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3730: Component/s: (was: storm-hdfs) storm-core > Remove PMD Exceptions > - > > Key: STORM-3730 > URL: https://issues.apache.org/jira/browse/STORM-3730 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > At top level. run "mvn install -DskipTests" or "mvn pmd:pmd". > Redirect to file to capture output. Approximately 200 PMDExceptions. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3730) Remove PMD Exceptions
[ https://issues.apache.org/jira/browse/STORM-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3730: --- Assignee: (was: Bipin Prasad) > Remove PMD Exceptions > - > > Key: STORM-3730 > URL: https://issues.apache.org/jira/browse/STORM-3730 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Reporter: Bipin Prasad >Priority: Minor > > At top level. run "mvn install -DskipTests" or "mvn pmd:pmd". > Redirect to file to capture output. Approximately 200 PMDExceptions. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3731) Remove unused nashorn import in storm-loadgen:OutputStream.java
Bipin Prasad created STORM-3731: --- Summary: Remove unused nashorn import in storm-loadgen:OutputStream.java Key: STORM-3731 URL: https://issues.apache.org/jira/browse/STORM-3731 Project: Apache Storm Issue Type: Bug Components: storm-loadgen Reporter: Bipin Prasad Assignee: Bipin Prasad Removing unused nashorn import. This allows mvn build to succeed in JDK16 EA as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3731) Remove unused nashorn import in storm-loadgen:OutputStream.java
[ https://issues.apache.org/jira/browse/STORM-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254499#comment-17254499 ] Bipin Prasad commented on STORM-3731: - https://github.com/apache/storm/pull/3369 > Remove unused nashorn import in storm-loadgen:OutputStream.java > --- > > Key: STORM-3731 > URL: https://issues.apache.org/jira/browse/STORM-3731 > Project: Apache Storm > Issue Type: Bug > Components: storm-loadgen >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Removing unused nashorn import. > This allows mvn build to succeed in JDK16 EA as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3732) Fix test test-cluster in storm-core: org.apache.storm.scheduler-test
Bipin Prasad created STORM-3732: --- Summary: Fix test test-cluster in storm-core: org.apache.storm.scheduler-test Key: STORM-3732 URL: https://issues.apache.org/jira/browse/STORM-3732 Project: Apache Storm Issue Type: Bug Components: storm-core Reporter: Bipin Prasad Assignee: Bipin Prasad Asserts in TopologyDetails.initConfigs fail the test "test-cluster" in storm-core: org.apache.storm.scheduler-test -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3732) Fix test test-cluster in storm-core: org.apache.storm.scheduler-test
[ https://issues.apache.org/jira/browse/STORM-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259761#comment-17259761 ] Bipin Prasad commented on STORM-3732: - To duplicate this: - checkout storm code on *Linux* - mvn clean ; mvn package -DskipTests - mvn test (runs Java tests, followed by Clojure tests) To run the specific Clojure test do the following - mvn test -Dtest="org.apache.storm.scheduler-test" -DfailIfNoTests=false (goes past java tests, then executes the single Clojure test class with 4 tests) Output file is generated in target/test-reports > Fix test test-cluster in storm-core: org.apache.storm.scheduler-test > - > > Key: STORM-3732 > URL: https://issues.apache.org/jira/browse/STORM-3732 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > Asserts in TopologyDetails.initConfigs fail the test "test-cluster" in > storm-core: org.apache.storm.scheduler-test -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3732) Fix test test-cluster in storm-core: org.apache.storm.scheduler-test
[ https://issues.apache.org/jira/browse/STORM-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259770#comment-17259770 ] Bipin Prasad commented on STORM-3732: - Cannot duplicate Clojure test failure. Closing this Jira (mentioned in pull request for https://issues.apache.org/jira/browse/STORM-3639 https://github.com/apache/storm/pull/3274 > Fix test test-cluster in storm-core: org.apache.storm.scheduler-test > - > > Key: STORM-3732 > URL: https://issues.apache.org/jira/browse/STORM-3732 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > Asserts in TopologyDetails.initConfigs fail the test "test-cluster" in > storm-core: org.apache.storm.scheduler-test -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (STORM-3732) Fix test test-cluster in storm-core: org.apache.storm.scheduler-test
[ https://issues.apache.org/jira/browse/STORM-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad closed STORM-3732. --- Resolution: Cannot Reproduce > Fix test test-cluster in storm-core: org.apache.storm.scheduler-test > - > > Key: STORM-3732 > URL: https://issues.apache.org/jira/browse/STORM-3732 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > Asserts in TopologyDetails.initConfigs fail the test "test-cluster" in > storm-core: org.apache.storm.scheduler-test -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3739) Scheduling should sort numa zones by host groups
Bipin Prasad created STORM-3739: --- Summary: Scheduling should sort numa zones by host groups Key: STORM-3739 URL: https://issues.apache.org/jira/browse/STORM-3739 Project: Apache Storm Issue Type: Bug Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad Sorting of supervisors should group numa zones into host groups, so selection happens on the basis of complete availability of resources on that node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3741) Maven Surefire plugin corrupts output and creates an excessive log file
Bipin Prasad created STORM-3741:
---
Summary: Maven Surefire plugin corrupts output and creates an
excessive log file
Key: STORM-3741
URL: https://issues.apache.org/jira/browse/STORM-3741
Project: Apache Storm
Issue Type: Bug
Reporter: Bipin Prasad
Assignee: Bipin Prasad
When running tests (mvn test) with "reuseForks=true", the STDOUT gets
corrupted.
You will see a message like this:
{code}
[WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM
1. See FAQ web page and the dump file […]
{code}
This problem goes away when "reuseForks=false". This problem has been fixed in
surefire versions 2.22.2 and 3.0.0-M3 as documented here:
https://issues.apache.org/jira/browse/SUREFIRE-1614
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3741) Maven Surefire plugin corrupts output and creates an excessive log file
[
https://issues.apache.org/jira/browse/STORM-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278618#comment-17278618
]
Bipin Prasad commented on STORM-3741:
-
Pull Request: https://github.com/apache/storm/pull/3375
> Maven Surefire plugin corrupts output and creates an excessive log file
> ---
>
> Key: STORM-3741
> URL: https://issues.apache.org/jira/browse/STORM-3741
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When running tests (mvn test) with "reuseForks=true", the STDOUT gets
> corrupted.
> You will see a message like this:
> {code}
> [WARNING] Corrupted STDOUT by directly writing to native stream in forked
> JVM 1. See FAQ web page and the dump file […]
> {code}
> This problem goes away when "reuseForks=false". This problem has been fixed
> in surefire versions 2.22.2 and 3.0.0-M3 as documented here:
> https://issues.apache.org/jira/browse/SUREFIRE-1614
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3743) Add new topologies for TestLargeCluster
Bipin Prasad created STORM-3743: --- Summary: Add new topologies for TestLargeCluster Key: STORM-3743 URL: https://issues.apache.org/jira/browse/STORM-3743 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad Add new set of topologies and a more nuanced Supervisor creation fri TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3743: --- Assignee: Bipin Prasad > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Add new set of topologies and a more nuanced Supervisor creation fri > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3743: Description: Add new set of topologies and a more nuanced Supervisor creation for TestLargeCluster. (was: Add new set of topologies and a more nuanced Supervisor creation fri TestLargeCluster.) > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17285851#comment-17285851 ] Bipin Prasad commented on STORM-3743: - Pull request https://github.com/apache/storm/pull/3378 > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3741) Maven Surefire plugin corrupts output and creates an excessive log file
[
https://issues.apache.org/jira/browse/STORM-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad resolved STORM-3741.
-
Resolution: Fixed
> Maven Surefire plugin corrupts output and creates an excessive log file
> ---
>
> Key: STORM-3741
> URL: https://issues.apache.org/jira/browse/STORM-3741
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When running tests (mvn test) with "reuseForks=true", the STDOUT gets
> corrupted.
> You will see a message like this:
> {code}
> [WARNING] Corrupted STDOUT by directly writing to native stream in forked
> JVM 1. See FAQ web page and the dump file […]
> {code}
> This problem goes away when "reuseForks=false". This problem has been fixed
> in surefire versions 2.22.2 and 3.0.0-M3 as documented here:
> https://issues.apache.org/jira/browse/SUREFIRE-1614
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3739) Scheduling should sort numa zones by host groups
[ https://issues.apache.org/jira/browse/STORM-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286170#comment-17286170 ] Bipin Prasad commented on STORM-3739: - Pull Request: https://github.com/apache/storm/pull/3379 > Scheduling should sort numa zones by host groups > > > Key: STORM-3739 > URL: https://issues.apache.org/jira/browse/STORM-3739 > Project: Apache Storm > Issue Type: Bug > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Sorting of supervisors should group numa zones into host groups, so selection > happens on the basis of complete availability of resources on that node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3744) IntelliJ does not find shaded classes
Bipin Prasad created STORM-3744: --- Summary: IntelliJ does not find shaded classes Key: STORM-3744 URL: https://issues.apache.org/jira/browse/STORM-3744 Project: Apache Storm Issue Type: Improvement Components: build Reporter: Bipin Prasad Assignee: Bipin Prasad When storm project is opened in IntelliJ, it does not find the shaded classes packaged under storm-shaded-deps package. This precludes compiling and debugging the project inside the IDE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3744) IntelliJ does not find shaded classes
[ https://issues.apache.org/jira/browse/STORM-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287791#comment-17287791 ] Bipin Prasad commented on STORM-3744: - Use the version that is built after the shading is done: https://youtrack.jetbrains.com/issue/IDEA-126596 > IntelliJ does not find shaded classes > - > > Key: STORM-3744 > URL: https://issues.apache.org/jira/browse/STORM-3744 > Project: Apache Storm > Issue Type: Improvement > Components: build >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > > When storm project is opened in IntelliJ, it does not find the shaded classes > packaged under storm-shaded-deps package. This precludes compiling and > debugging the project inside the IDE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3744) IntelliJ does not find shaded classes
[ https://issues.apache.org/jira/browse/STORM-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287793#comment-17287793 ] Bipin Prasad commented on STORM-3744: - Pull Request: https://github.com/apache/storm/pull/3380 > IntelliJ does not find shaded classes > - > > Key: STORM-3744 > URL: https://issues.apache.org/jira/browse/STORM-3744 > Project: Apache Storm > Issue Type: Improvement > Components: build >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > When storm project is opened in IntelliJ, it does not find the shaded classes > packaged under storm-shaded-deps package. This precludes compiling and > debugging the project inside the IDE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3745) Exclude blacklisted hosts when scheduling
Bipin Prasad created STORM-3745: --- Summary: Exclude blacklisted hosts when scheduling Key: STORM-3745 URL: https://issues.apache.org/jira/browse/STORM-3745 Project: Apache Storm Issue Type: Bug Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad Blacklisted hosts are not excluded when sorting nodes via NodeSorter[HostProximity] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (STORM-3745) Exclude blacklisted hosts when scheduling
[ https://issues.apache.org/jira/browse/STORM-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad closed STORM-3745. --- Resolution: Duplicate > Exclude blacklisted hosts when scheduling > - > > Key: STORM-3745 > URL: https://issues.apache.org/jira/browse/STORM-3745 > Project: Apache Storm > Issue Type: Bug > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Blacklisted hosts are not excluded when sorting nodes via > NodeSorter[HostProximity] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3746) Outline steps to anonymize topologies for TestLargeCluster
Bipin Prasad created STORM-3746: --- Summary: Outline steps to anonymize topologies for TestLargeCluster Key: STORM-3746 URL: https://issues.apache.org/jira/browse/STORM-3746 Project: Apache Storm Issue Type: Bug Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad TestLargeCluster can various tests against simulated large clusters. Topology names and their components are anonymized so as to not reveal proprietary information. Steps on creating these simulation should be documented so users can create them for contribution (or just for internal testing). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3746) Outline steps to anonymize topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289899#comment-17289899 ] Bipin Prasad commented on STORM-3746: - Current steps are embedded within javadocs in TestTopologyAnonymizerUtils.java and require changes to TestLargeCluster. > Outline steps to anonymize topologies for TestLargeCluster > -- > > Key: STORM-3746 > URL: https://issues.apache.org/jira/browse/STORM-3746 > Project: Apache Storm > Issue Type: Bug > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > TestLargeCluster can various tests against simulated large clusters. Topology > names and their components are anonymized so as to not reveal proprietary > information. > Steps on creating these simulation should be documented so users can create > them for contribution (or just for internal testing). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3747) Remove deprecated NodeSortTypes and related code
[ https://issues.apache.org/jira/browse/STORM-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3747: --- Assignee: Bipin Prasad > Remove deprecated NodeSortTypes and related code > > > Key: STORM-3747 > URL: https://issues.apache.org/jira/browse/STORM-3747 > Project: Apache Storm > Issue Type: Bug > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > The following classes are no longer in use and should be removed. > (1) NodeSorter > (2) DefaultResourceAwareStrategyOld > (3) GenericResourceAwareStrategyOld > (4) NodeSorterType > Unused/deprecated methods in BaseResourceAwareStrategy can be removed when > made obsolete by removal of classes above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3747) Remove deprecated NodeSortTypes and related code
Bipin Prasad created STORM-3747: --- Summary: Remove deprecated NodeSortTypes and related code Key: STORM-3747 URL: https://issues.apache.org/jira/browse/STORM-3747 Project: Apache Storm Issue Type: Bug Components: storm-server Reporter: Bipin Prasad The following classes are no longer in use and should be removed. (1) NodeSorter (2) DefaultResourceAwareStrategyOld (3) GenericResourceAwareStrategyOld (4) NodeSorterType Unused/deprecated methods in BaseResourceAwareStrategy can be removed when made obsolete by removal of classes above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad resolved STORM-3743. - Resolution: Fixed Merged commit e3df2c1 into apache:master > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3739) Scheduling should sort numa zones by host groups
[ https://issues.apache.org/jira/browse/STORM-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3739: Issue Type: Improvement (was: Bug) > Scheduling should sort numa zones by host groups > > > Key: STORM-3739 > URL: https://issues.apache.org/jira/browse/STORM-3739 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 5h 20m > Remaining Estimate: 0h > > Sorting of supervisors should group numa zones into host groups, so selection > happens on the basis of complete availability of resources on that node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3743: Issue Type: Test (was: Improvement) > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Test > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3743: Priority: Minor (was: Major) > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Test > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3743: Affects Version/s: 2.3.0 > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Test > Components: storm-server >Affects Versions: 2.3.0 >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3743) Add new topologies for TestLargeCluster
[ https://issues.apache.org/jira/browse/STORM-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3743: Fix Version/s: 2.3.0 > Add new topologies for TestLargeCluster > --- > > Key: STORM-3743 > URL: https://issues.apache.org/jira/browse/STORM-3743 > Project: Apache Storm > Issue Type: Test > Components: storm-server >Affects Versions: 2.3.0 >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add new set of topologies and a more nuanced Supervisor creation for > TestLargeCluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3741) Maven Surefire plugin corrupts output and creates an excessive log file
[
https://issues.apache.org/jira/browse/STORM-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad updated STORM-3741:
Fix Version/s: 2.3.0
> Maven Surefire plugin corrupts output and creates an excessive log file
> ---
>
> Key: STORM-3741
> URL: https://issues.apache.org/jira/browse/STORM-3741
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Fix For: 2.3.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When running tests (mvn test) with "reuseForks=true", the STDOUT gets
> corrupted.
> You will see a message like this:
> {code}
> [WARNING] Corrupted STDOUT by directly writing to native stream in forked
> JVM 1. See FAQ web page and the dump file […]
> {code}
> This problem goes away when "reuseForks=false". This problem has been fixed
> in surefire versions 2.22.2 and 3.0.0-M3 as documented here:
> https://issues.apache.org/jira/browse/SUREFIRE-1614
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (STORM-3742) Topology arguments starting with -c get removed
[ https://issues.apache.org/jira/browse/STORM-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17296297#comment-17296297 ] Bipin Prasad commented on STORM-3742: - Can you try after replacing -conf with --config > Topology arguments starting with -c get removed > --- > > Key: STORM-3742 > URL: https://issues.apache.org/jira/browse/STORM-3742 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Julien Nioche >Priority: Major > > with the following command > storm local target/stormcrawler-1.0-SNAPSHOT.jar > global.climateemergency.ESCrawlTopology -conf crawler-conf.yaml -conf > es-conf.yaml . seeds.txt > the topology class is only getting [crawler-conf.yaml, es-conf.yaml, ., > seeds.txt]and the -conf have been removed. > Anything after the class name should be left as-is. > This does not happen with Storm 1.2.3 > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3742) Topology arguments starting with -c get removed
[ https://issues.apache.org/jira/browse/STORM-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3742: --- Assignee: Bipin Prasad > Topology arguments starting with -c get removed > --- > > Key: STORM-3742 > URL: https://issues.apache.org/jira/browse/STORM-3742 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Julien Nioche >Assignee: Bipin Prasad >Priority: Major > > with the following command > storm local target/stormcrawler-1.0-SNAPSHOT.jar > global.climateemergency.ESCrawlTopology -conf crawler-conf.yaml -conf > es-conf.yaml . seeds.txt > the topology class is only getting [crawler-conf.yaml, es-conf.yaml, ., > seeds.txt]and the -conf have been removed. > Anything after the class name should be left as-is. > This does not happen with Storm 1.2.3 > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3742) Topology arguments starting with -c get removed
[
https://issues.apache.org/jira/browse/STORM-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17296322#comment-17296322
]
Bipin Prasad commented on STORM-3742:
-
Alternatively, if your intent is to have your class process the arguments: then
insert "--" to stop argument processing:
{code}
storm local target/stormcrawler-1.0-SNAPSHOT.jar
global.climateemergency.ESCrawlTopology -- -conf crawler-conf.yaml -conf
es-conf.yaml . seeds.txt
{code}
Storm 2.2.x will parse "-c" as a single override and --config accepts file
argument.
But "-conf crawler-conf.yaml" will be parsed as "-c onf crawler-conf.yaml"
> Topology arguments starting with -c get removed
> ---
>
> Key: STORM-3742
> URL: https://issues.apache.org/jira/browse/STORM-3742
> Project: Apache Storm
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Julien Nioche
>Assignee: Bipin Prasad
>Priority: Major
>
> with the following command
> storm local target/stormcrawler-1.0-SNAPSHOT.jar
> global.climateemergency.ESCrawlTopology -conf crawler-conf.yaml -conf
> es-conf.yaml . seeds.txt
> the topology class is only getting [crawler-conf.yaml, es-conf.yaml, .,
> seeds.txt]and the -conf have been removed.
> Anything after the class name should be left as-is.
> This does not happen with Storm 1.2.3
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3754) Upgrade Guava version because of security vulnerability
Bipin Prasad created STORM-3754: --- Summary: Upgrade Guava version because of security vulnerability Key: STORM-3754 URL: https://issues.apache.org/jira/browse/STORM-3754 Project: Apache Storm Issue Type: Improvement Components: storm-hdfs, storm-hive Reporter: Bipin Prasad storm-hdfs-examples and storm-hive-examples use com.google.guava:guava:16.0.1 This has know vulnerability https://nvd.nist.gov/vuln/detail/CVE-2018-10237 "Unbounded memory allocation in Google Guava 11.0 through 24.x before 24.1.1 allows remote attackers to conduct denial of service attack." The guava version downgrade was required earlier because of hadoop-hdfs 2.6.1. Since storm is now using hadoop-hdfs 2.8.5, this downgrade may not be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3739) Scheduling should sort numa zones by host groups
[ https://issues.apache.org/jira/browse/STORM-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3739: Fix Version/s: 2.3.0 > Scheduling should sort numa zones by host groups > > > Key: STORM-3739 > URL: https://issues.apache.org/jira/browse/STORM-3739 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 9h > Remaining Estimate: 0h > > Sorting of supervisors should group numa zones into host groups, so selection > happens on the basis of complete availability of resources on that node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3739) Scheduling should sort numa zones by host groups
[ https://issues.apache.org/jira/browse/STORM-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad resolved STORM-3739. - Resolution: Fixed Squash/merged commits into apache:master > Scheduling should sort numa zones by host groups > > > Key: STORM-3739 > URL: https://issues.apache.org/jira/browse/STORM-3739 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 9h > Remaining Estimate: 0h > > Sorting of supervisors should group numa zones into host groups, so selection > happens on the basis of complete availability of resources on that node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3754) Upgrade Guava version because of security vulnerability
[ https://issues.apache.org/jira/browse/STORM-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3754: Description: storm-hdfs-examples and storm-hive-examples use com.google.guava:guava:16.0.1 This has know vulnerability https://nvd.nist.gov/vuln/detail/CVE-2018-10237 "Unbounded memory allocation in Google Guava 11.0 through 24.x before 24.1.1 allows remote attackers to conduct denial of service attack." The guava version downgrade was required earlier because of hadoop-hdfs 2.6.1. Since storm is now using hadoop-hdfs 2.8.5, this downgrade may not be necessary. It is possible that the a separate jar may need to be added as dependency com.google.guava:failureaccess:1.0. See https://github.com/google/guava/releases around Oct 18, 2018 when Guava version 27.0 was released. Note that Hadoop HDFS 2.8.5 was released on Sep 8, 2018 (i.e. before the guava version 27.0). was: storm-hdfs-examples and storm-hive-examples use com.google.guava:guava:16.0.1 This has know vulnerability https://nvd.nist.gov/vuln/detail/CVE-2018-10237 "Unbounded memory allocation in Google Guava 11.0 through 24.x before 24.1.1 allows remote attackers to conduct denial of service attack." The guava version downgrade was required earlier because of hadoop-hdfs 2.6.1. Since storm is now using hadoop-hdfs 2.8.5, this downgrade may not be necessary. > Upgrade Guava version because of security vulnerability > --- > > Key: STORM-3754 > URL: https://issues.apache.org/jira/browse/STORM-3754 > Project: Apache Storm > Issue Type: Improvement > Components: storm-hdfs, storm-hive >Reporter: Bipin Prasad >Priority: Minor > > storm-hdfs-examples and storm-hive-examples use com.google.guava:guava:16.0.1 > This has know vulnerability https://nvd.nist.gov/vuln/detail/CVE-2018-10237 > "Unbounded memory allocation in Google Guava 11.0 through 24.x before 24.1.1 > allows remote attackers to conduct denial of service attack." > The guava version downgrade was required earlier because of hadoop-hdfs 2.6.1. > Since storm is now using hadoop-hdfs 2.8.5, this downgrade may not be > necessary. > It is possible that the a separate jar may need to be added as dependency > com.google.guava:failureaccess:1.0. See > https://github.com/google/guava/releases around Oct 18, 2018 when Guava > version 27.0 was released. Note that Hadoop HDFS 2.8.5 was released on Sep 8, > 2018 (i.e. before the guava version 27.0). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3755) While scheduling multiple ackers with executor use best effort basis
Bipin Prasad created STORM-3755: --- Summary: While scheduling multiple ackers with executor use best effort basis Key: STORM-3755 URL: https://issues.apache.org/jira/browse/STORM-3755 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad Fix For: 2.3.0 In the scheduling loop, if the number of ackers to be scheduled along with the executor is greater than zero, and they cannot all fit, then try to fit as many ackers as possible. To do this, fit the executor first. If this succeeds, then attempt to fit as many ackers are possible upto and including the max required as per calculation (and as low as zero). This second step should not fail. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3755) While scheduling multiple ackers with executor use best effort basis
[ https://issues.apache.org/jira/browse/STORM-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17302965#comment-17302965 ] Bipin Prasad commented on STORM-3755: - Pull Request: https://github.com/apache/storm/pull/3386 > While scheduling multiple ackers with executor use best effort basis > > > Key: STORM-3755 > URL: https://issues.apache.org/jira/browse/STORM-3755 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In the scheduling loop, if the number of ackers to be scheduled along with > the executor is greater than zero, and they cannot all fit, then try to fit > as many ackers as possible. > To do this, fit the executor first. If this succeeds, then attempt to fit as > many ackers are possible upto and including the max required as per > calculation (and as low as zero). This second step should not fail. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3758) Zookeeper reconnect events sometime cause workers to restart
Bipin Prasad created STORM-3758: --- Summary: Zookeeper reconnect events sometime cause workers to restart Key: STORM-3758 URL: https://issues.apache.org/jira/browse/STORM-3758 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad ZooKeeper reconnect even may happen when Worker.java#checkCredentialsChanged() is executed inside a timer. This Exception causes Worker to be restarted. Fix this by tolerating a constant number of consecutive exceptions in this lamba call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (STORM-3758) Zookeeper reconnect events sometime cause workers to restart
[ https://issues.apache.org/jira/browse/STORM-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306351#comment-17306351 ] Bipin Prasad commented on STORM-3758: - PR https://github.com/apache/storm/pull/3389 > Zookeeper reconnect events sometime cause workers to restart > > > Key: STORM-3758 > URL: https://issues.apache.org/jira/browse/STORM-3758 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ZooKeeper reconnect even may happen when > Worker.java#checkCredentialsChanged() is executed inside a timer. This > Exception causes Worker to be restarted. > Fix this by tolerating a constant number of consecutive exceptions in this > lamba call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3758) Zookeeper reconnect events sometime cause workers to restart
[ https://issues.apache.org/jira/browse/STORM-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3758: Fix Version/s: 2.3.0 > Zookeeper reconnect events sometime cause workers to restart > > > Key: STORM-3758 > URL: https://issues.apache.org/jira/browse/STORM-3758 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Fix For: 2.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > ZooKeeper reconnect even may happen when > Worker.java#checkCredentialsChanged() is executed inside a timer. This > Exception causes Worker to be restarted. > Fix this by tolerating a constant number of consecutive exceptions in this > lamba call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3758) Zookeeper reconnect events sometime cause workers to restart
[ https://issues.apache.org/jira/browse/STORM-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad resolved STORM-3758. - Resolution: Fixed merged commit 3f96c24 into apache:master > Zookeeper reconnect events sometime cause workers to restart > > > Key: STORM-3758 > URL: https://issues.apache.org/jira/browse/STORM-3758 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Fix For: 2.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > ZooKeeper reconnect even may happen when > Worker.java#checkCredentialsChanged() is executed inside a timer. This > Exception causes Worker to be restarted. > Fix this by tolerating a constant number of consecutive exceptions in this > lamba call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3764) While backtracking only remove the ackers that were skipped
Bipin Prasad created STORM-3764:
---
Summary: While backtracking only remove the ackers that were
skipped
Key: STORM-3764
URL: https://issues.apache.org/jira/browse/STORM-3764
Project: Apache Storm
Issue Type: Improvement
Components: storm-server
Reporter: Bipin Prasad
Assignee: Bipin Prasad
While backtracking in scheduling code, Ackers are skipped. However at the end
of this step all ackers on the workerslot are removed. This removal should be
limited to only the Ackers that were skipped in
SchedulingSearcherState.backtrack(). Otherwise, the next immediate backtrack()
call can result in an NPE:
{code}
2021-03-30 15:27:03.212 o.a.s.s.r.ResourceAwareScheduler timer [ERROR]
mx3_test-5-1614799784 Internal Error - Exception thrown when scheduling. Please
check logs for details java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
~[?:1.8.0_262]
at java.util.concurrent.FutureTask.get(FutureTask.java:206) ~[?:1.8.0_262]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.scheduleTopology(ResourceAwareScheduler.java:191)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.schedule(ResourceAwareScheduler.java:132)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.blacklist.BlacklistScheduler.schedule(BlacklistScheduler.java:131)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.computeNewSchedulerAssignments(Nimbus.java:2314)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lambda$lockingMkAssignments$44(Nimbus.java:2493)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.executeWithLock(Nimbus.java:4549)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lockingMkAssignments(Nimbus.java:2491)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2478)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2415)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$17(Nimbus.java:1420)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
[storm-client-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
[storm-client-2.3.0.y.jar:2.3.0.y]
Caused by: java.lang.NullPointerException
at
org.apache.storm.scheduler.resource.strategies.scheduling.SchedulingSearcherState.backtrack(SchedulingSearcherState.java:292)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.scheduleExecutorsOnNodes(BaseResourceAwareStrategy.java:559)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.schedule(BaseResourceAwareStrategy.java:172)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.lambda$scheduleTopology$1(ResourceAwareScheduler.java:189)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_262]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_262]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_262]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (STORM-3764) Nimbus has internal scheduling errors when backtracking
[
https://issues.apache.org/jira/browse/STORM-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad updated STORM-3764:
Summary: Nimbus has internal scheduling errors when backtracking (was:
While backtracking only remove the ackers that were skipped)
> Nimbus has internal scheduling errors when backtracking
> ---
>
> Key: STORM-3764
> URL: https://issues.apache.org/jira/browse/STORM-3764
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-server
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
>
> While backtracking in scheduling code, Ackers are skipped. However at the end
> of this step all ackers on the workerslot are removed. This removal should be
> limited to only the Ackers that were skipped in
> SchedulingSearcherState.backtrack(). Otherwise, the next immediate
> backtrack() call can result in an NPE:
> {code}
> 2021-03-30 15:27:03.212 o.a.s.s.r.ResourceAwareScheduler timer [ERROR]
> mx3_test-5-1614799784 Internal Error - Exception thrown when scheduling.
> Please check logs for details java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[?:1.8.0_262]
> at java.util.concurrent.FutureTask.get(FutureTask.java:206) ~[?:1.8.0_262]
> at
> org.apache.storm.scheduler.resource.ResourceAwareScheduler.scheduleTopology(ResourceAwareScheduler.java:191)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.ResourceAwareScheduler.schedule(ResourceAwareScheduler.java:132)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.blacklist.BlacklistScheduler.schedule(BlacklistScheduler.java:131)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.computeNewSchedulerAssignments(Nimbus.java:2314)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$lockingMkAssignments$44(Nimbus.java:2493)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.executeWithLock(Nimbus.java:4549)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lockingMkAssignments(Nimbus.java:2491)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2478)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2415)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$17(Nimbus.java:1420)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
> [storm-client-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
> [storm-client-2.3.0.y.jar:2.3.0.y]
> Caused by: java.lang.NullPointerException
> at
> org.apache.storm.scheduler.resource.strategies.scheduling.SchedulingSearcherState.backtrack(SchedulingSearcherState.java:292)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.scheduleExecutorsOnNodes(BaseResourceAwareStrategy.java:559)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.schedule(BaseResourceAwareStrategy.java:172)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.ResourceAwareScheduler.lambda$scheduleTopology$1(ResourceAwareScheduler.java:189)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_262]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_262]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_262]
> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_262]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (STORM-3764) Nimbus has internal scheduling errors when backtracking
[
https://issues.apache.org/jira/browse/STORM-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad updated STORM-3764:
Description:
While backtracking in scheduling code, Ackers are skipped. However, under
certain circumstances a null WorkerSlot is passed as parameter to
SchedulingSearcherState.backtrack(). This null key is used to retrieve an entry
from Map. The returned null map value causes NPE.
{code}
2021-03-30 15:27:03.212 o.a.s.s.r.ResourceAwareScheduler timer [ERROR]
mx3_test-5-1614799784 Internal Error - Exception thrown when scheduling. Please
check logs for details java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
~[?:1.8.0_262]
at java.util.concurrent.FutureTask.get(FutureTask.java:206) ~[?:1.8.0_262]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.scheduleTopology(ResourceAwareScheduler.java:191)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.schedule(ResourceAwareScheduler.java:132)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.blacklist.BlacklistScheduler.schedule(BlacklistScheduler.java:131)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.computeNewSchedulerAssignments(Nimbus.java:2314)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lambda$lockingMkAssignments$44(Nimbus.java:2493)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.executeWithLock(Nimbus.java:4549)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lockingMkAssignments(Nimbus.java:2491)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2478)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2415)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$17(Nimbus.java:1420)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
[storm-client-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
[storm-client-2.3.0.y.jar:2.3.0.y]
Caused by: java.lang.NullPointerException
at
org.apache.storm.scheduler.resource.strategies.scheduling.SchedulingSearcherState.backtrack(SchedulingSearcherState.java:292)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.scheduleExecutorsOnNodes(BaseResourceAwareStrategy.java:559)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.schedule(BaseResourceAwareStrategy.java:172)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.lambda$scheduleTopology$1(ResourceAwareScheduler.java:189)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_262]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_262]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_262]
{code}
was:
While backtracking in scheduling code, Ackers are skipped. However at the end
of this step all ackers on the workerslot are removed. This removal should be
limited to only the Ackers that were skipped in
SchedulingSearcherState.backtrack(). Otherwise, the next immediate backtrack()
call can result in an NPE:
{code}
2021-03-30 15:27:03.212 o.a.s.s.r.ResourceAwareScheduler timer [ERROR]
mx3_test-5-1614799784 Internal Error - Exception thrown when scheduling. Please
check logs for details java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
~[?:1.8.0_262]
at java.util.concurrent.FutureTask.get(FutureTask.java:206) ~[?:1.8.0_262]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.scheduleTopology(ResourceAwareScheduler.java:191)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.resource.ResourceAwareScheduler.schedule(ResourceAwareScheduler.java:132)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.scheduler.blacklist.BlacklistScheduler.schedule(BlacklistScheduler.java:131)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.computeNewSchedulerAssignments(Nimbus.java:2314)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.daemon.nimbus.Nimbus.lambda$lockingMkAssignments$44(Nimbus.java:2493)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at org.apache.storm.daemon.nimbus.Nimbus.ex
[jira] [Commented] (STORM-3764) Nimbus has internal scheduling errors when backtracking
[
https://issues.apache.org/jira/browse/STORM-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317915#comment-17317915
]
Bipin Prasad commented on STORM-3764:
-
Pull Request https://github.com/apache/storm/pull/3391
> Nimbus has internal scheduling errors when backtracking
> ---
>
> Key: STORM-3764
> URL: https://issues.apache.org/jira/browse/STORM-3764
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-server
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While backtracking in scheduling code, Ackers are skipped. However, under
> certain circumstances a null WorkerSlot is passed as parameter to
> SchedulingSearcherState.backtrack(). This null key is used to retrieve an
> entry from Map. The returned null map value causes NPE.
> {code}
> 2021-03-30 15:27:03.212 o.a.s.s.r.ResourceAwareScheduler timer [ERROR]
> mx3_test-5-1614799784 Internal Error - Exception thrown when scheduling.
> Please check logs for details java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[?:1.8.0_262]
> at java.util.concurrent.FutureTask.get(FutureTask.java:206) ~[?:1.8.0_262]
> at
> org.apache.storm.scheduler.resource.ResourceAwareScheduler.scheduleTopology(ResourceAwareScheduler.java:191)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.ResourceAwareScheduler.schedule(ResourceAwareScheduler.java:132)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.blacklist.BlacklistScheduler.schedule(BlacklistScheduler.java:131)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.computeNewSchedulerAssignments(Nimbus.java:2314)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$lockingMkAssignments$44(Nimbus.java:2493)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.executeWithLock(Nimbus.java:4549)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lockingMkAssignments(Nimbus.java:2491)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2478)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.daemon.nimbus.Nimbus.mkAssignments(Nimbus.java:2415)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$17(Nimbus.java:1420)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
> [storm-client-2.3.0.y.jar:2.3.0.y]
> at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
> [storm-client-2.3.0.y.jar:2.3.0.y]
> Caused by: java.lang.NullPointerException
> at
> org.apache.storm.scheduler.resource.strategies.scheduling.SchedulingSearcherState.backtrack(SchedulingSearcherState.java:292)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.scheduleExecutorsOnNodes(BaseResourceAwareStrategy.java:559)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy.schedule(BaseResourceAwareStrategy.java:172)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at
> org.apache.storm.scheduler.resource.ResourceAwareScheduler.lambda$scheduleTopology$1(ResourceAwareScheduler.java:189)
> ~[storm-server-2.3.0.y.jar:2.3.0.y]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_262]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_262]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_262]
> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_262]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3766) Update org.apache.calcite version to 1.26.0 due to security vulnerability
Bipin Prasad created STORM-3766: --- Summary: Update org.apache.calcite version to 1.26.0 due to security vulnerability Key: STORM-3766 URL: https://issues.apache.org/jira/browse/STORM-3766 Project: Apache Storm Issue Type: Improvement Components: storm-sql Reporter: Bipin Prasad Update calcite version to 1.26.0 to avoid "Missing Authentication for Critical Function in Apache Calcite" as detailed here https://github.com/advisories/GHSA-hxp5-8pgq-mgv9 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3771) Execute doCleanup in its own timer thread without lock
Bipin Prasad created STORM-3771: --- Summary: Execute doCleanup in its own timer thread without lock Key: STORM-3771 URL: https://issues.apache.org/jira/browse/STORM-3771 Project: Apache Storm Issue Type: Improvement Components: storm-server Reporter: Bipin Prasad Assignee: Bipin Prasad _Sometimes doCleanup can take a long time (several minutes) while holding submitLock. This slows down other "submitLock" dependent activities. Execute doCleanup without a lock and within its own timer thread. Ignore exception that may occur inside doCleanup() method._ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3792) Change pom.xml to use more threads without reuse
Bipin Prasad created STORM-3792: --- Summary: Change pom.xml to use more threads without reuse Key: STORM-3792 URL: https://issues.apache.org/jira/browse/STORM-3792 Project: Apache Storm Issue Type: Improvement Components: blobstore, examples, integration-test, storm-hdfs, storm-hive, storm-kafka, storm-kafka-client, storm-kafka-monitor, storm-server, storm-sql Reporter: Bipin Prasad Assignee: Bipin Prasad Maven surefire plugin configuration should be set to NOT reuse the forked JVMs for testing. Sharing JVMs can cause unintended test failures caused by singleton classes being in an inconsistent stage. Example of this class would be Time. This will cause a slowdown in tests because a new JVM would have to be created. To mitigate this slowdown, whereever the "forkCount=1", change it to "forkCount=1.0C" and add reuseForks=false if not already present. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3792) Change pom.xml to use more test JVM threads without reuse
[ https://issues.apache.org/jira/browse/STORM-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3792: Summary: Change pom.xml to use more test JVM threads without reuse (was: Change pom.xml to use more threads without reuse) > Change pom.xml to use more test JVM threads without reuse > - > > Key: STORM-3792 > URL: https://issues.apache.org/jira/browse/STORM-3792 > Project: Apache Storm > Issue Type: Improvement > Components: blobstore, examples, integration-test, storm-hdfs, > storm-hive, storm-kafka, storm-kafka-client, storm-kafka-monitor, > storm-server, storm-sql >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Maven surefire plugin configuration should be set to NOT reuse the forked > JVMs for testing. Sharing JVMs can cause unintended test failures caused by > singleton classes being in an inconsistent stage. Example of this class would > be Time. > This will cause a slowdown in tests because a new JVM would have to be > created. > To mitigate this slowdown, whereever the "forkCount=1", change it to > "forkCount=1.0C" and add reuseForks=false if not already present. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3792) Change pom.xml to use more test JVM threads without reuse
[ https://issues.apache.org/jira/browse/STORM-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3792: Description: Maven surefire plugin configuration has two important flags: (1) reuseForks - setting of true mean reuse created JVMs. (2) forkCount - number of JVMs to create for testing. Pure number is an absolute count, whereas 1.0C means same as the number of CPUs. Reusing forked JVMs can cause somewhat indeterminate failures in test when global class instances are not properly initialized or cleaned. An example of such a singleton class Time. Not reusing will cause a slowdown in tests. To mitigate this slowdown, wherever the "forkCount=1", change it to "forkCount=1.0C" and add reuseForks=false if not already present. However, some modules (example storm-hdfs) forkCount cannot be increased from 1 to 1.0C, due to either resources limits or global locking (as in hdfs tests). was: Maven surefire plugin configuration should be set to NOT reuse the forked JVMs for testing. Sharing JVMs can cause unintended test failures caused by singleton classes being in an inconsistent stage. Example of this class would be Time. This will cause a slowdown in tests because a new JVM would have to be created. To mitigate this slowdown, whereever the "forkCount=1", change it to "forkCount=1.0C" and add reuseForks=false if not already present. > Change pom.xml to use more test JVM threads without reuse > - > > Key: STORM-3792 > URL: https://issues.apache.org/jira/browse/STORM-3792 > Project: Apache Storm > Issue Type: Improvement > Components: blobstore, examples, integration-test, storm-hdfs, > storm-hive, storm-kafka, storm-kafka-client, storm-kafka-monitor, > storm-server, storm-sql >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Maven surefire plugin configuration has two important flags: > (1) reuseForks - setting of true mean reuse created JVMs. > (2) forkCount - number of JVMs to create for testing. Pure number is an > absolute count, whereas 1.0C means same as the number of CPUs. > Reusing forked JVMs can cause somewhat indeterminate failures in test when > global class instances are not properly initialized or cleaned. An example of > such a singleton class Time. > Not reusing will cause a slowdown in tests. > To mitigate this slowdown, wherever the "forkCount=1", change it to > "forkCount=1.0C" and add reuseForks=false if not already present. However, > some modules (example storm-hdfs) forkCount cannot be increased from 1 to > 1.0C, due to either resources limits or global locking (as in hdfs tests). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (STORM-3803) Format large integers in Storm UI to have with commas for readability
Bipin Prasad created STORM-3803: --- Summary: Format large integers in Storm UI to have with commas for readability Key: STORM-3803 URL: https://issues.apache.org/jira/browse/STORM-3803 Project: Apache Storm Issue Type: Improvement Components: storm-webapp Reporter: Bipin Prasad Assignee: Bipin Prasad -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3803) Format large integers in Storm UI with commas for readability
[ https://issues.apache.org/jira/browse/STORM-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3803: Summary: Format large integers in Storm UI with commas for readability (was: Format large integers in Storm UI to have with commas for readability) > Format large integers in Storm UI with commas for readability > - > > Key: STORM-3803 > URL: https://issues.apache.org/jira/browse/STORM-3803 > Project: Apache Storm > Issue Type: Improvement > Components: storm-webapp >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3803) Format large integers in Storm UI with commas for readability
[ https://issues.apache.org/jira/browse/STORM-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad resolved STORM-3803. - Resolution: Fixed Merged > Format large integers in Storm UI with commas for readability > - > > Key: STORM-3803 > URL: https://issues.apache.org/jira/browse/STORM-3803 > Project: Apache Storm > Issue Type: Improvement > Components: storm-webapp >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (STORM-3807) Topology Stats columns show NaN
Bipin Prasad created STORM-3807: --- Summary: Topology Stats columns show NaN Key: STORM-3807 URL: https://issues.apache.org/jira/browse/STORM-3807 Project: Apache Storm Issue Type: Improvement Components: storm-webapp Reporter: Bipin Prasad Assignee: Bipin Prasad Attachments: Screen Shot 2021-11-16 at 11.44.44 AM.png Some columns that should be a number (0) show up as "NaN" in the Storm UI. Example is Acked and Failed columns in the Topology Stats. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (STORM-3807) Topology Stats columns show NaN
[ https://issues.apache.org/jira/browse/STORM-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444899#comment-17444899 ] Bipin Prasad commented on STORM-3807: - Pull Request https://github.com/apache/storm/pull/3424 > Topology Stats columns show NaN > --- > > Key: STORM-3807 > URL: https://issues.apache.org/jira/browse/STORM-3807 > Project: Apache Storm > Issue Type: Improvement > Components: storm-webapp >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Attachments: Screen Shot 2021-11-16 at 11.44.44 AM.png > > Time Spent: 20m > Remaining Estimate: 0h > > Some columns that should be a number (0) show up as "NaN" in the Storm UI. > Example is Acked and Failed columns in the Topology Stats. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (STORM-3807) Topology Stats columns show NaN
[ https://issues.apache.org/jira/browse/STORM-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad updated STORM-3807: Fix Version/s: 2.4.0 > Topology Stats columns show NaN > --- > > Key: STORM-3807 > URL: https://issues.apache.org/jira/browse/STORM-3807 > Project: Apache Storm > Issue Type: Improvement > Components: storm-webapp >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.4.0 > > Attachments: Screen Shot 2021-11-16 at 11.44.44 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > Some columns that should be a number (0) show up as "NaN" in the Storm UI. > Example is Acked and Failed columns in the Topology Stats. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (STORM-3807) Topology Stats columns show NaN
[ https://issues.apache.org/jira/browse/STORM-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad resolved STORM-3807. - Resolution: Fixed Pull Request https://github.com/apache/storm/pull/3424 merged > Topology Stats columns show NaN > --- > > Key: STORM-3807 > URL: https://issues.apache.org/jira/browse/STORM-3807 > Project: Apache Storm > Issue Type: Improvement > Components: storm-webapp >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.4.0 > > Attachments: Screen Shot 2021-11-16 at 11.44.44 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > Some columns that should be a number (0) show up as "NaN" in the Storm UI. > Example is Acked and Failed columns in the Topology Stats. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (STORM-3823) Release Prep fails in storm-client/test/py/test_storm_cli.py
Bipin Prasad created STORM-3823: --- Summary: Release Prep fails in storm-client/test/py/test_storm_cli.py Key: STORM-3823 URL: https://issues.apache.org/jira/browse/STORM-3823 Project: Apache Storm Issue Type: Bug Affects Versions: 2.3.0 Reporter: Bipin Prasad Assignee: Bipin Prasad When running the release preparation script via: *mvn release:prepare -P dist,rat,externals,examples* the python test script *storm-client/test/py/test_storm_cli.py* fails because the command generated by storm.py is slightly different than what the test script expects. In particular, directories are added to java classpath with a "/*". This is correct. However, the test script does not expect this. Fix the test script storm-client/test/py/test_storm_cli.py to append "/*" to directories that contain jar files. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (STORM-3823) Release Prep fails in storm-client/test/py/test_storm_cli.py
[ https://issues.apache.org/jira/browse/STORM-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494858#comment-17494858 ] Bipin Prasad commented on STORM-3823: - Add "/*" to directory in classpath > Release Prep fails in storm-client/test/py/test_storm_cli.py > > > Key: STORM-3823 > URL: https://issues.apache.org/jira/browse/STORM-3823 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When running the release preparation script via: > *mvn release:prepare -P dist,rat,externals,examples* > the python test script *storm-client/test/py/test_storm_cli.py* fails because > the command generated by storm.py is slightly different than what the test > script expects. > In particular, directories are added to java classpath with a "/*". This is > correct. > However, the test script does not expect this. > Fix the test script storm-client/test/py/test_storm_cli.py to append "/*" to > directories that contain jar files. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (STORM-3829) Remove log4j version 1 (1.2.17) from storm-core
Bipin Prasad created STORM-3829:
---
Summary: Remove log4j version 1 (1.2.17) from storm-core
Key: STORM-3829
URL: https://issues.apache.org/jira/browse/STORM-3829
Project: Apache Storm
Issue Type: Improvement
Components: storm-core
Affects Versions: 2.3.0
Reporter: Bipin Prasad
Assignee: Bipin Prasad
Fix For: 2.4.0
storm-core/pom.xml has the following entry:
{code:java}
log4j
log4j
1.2.17
test
{code}
This dependency can be removed. "mvn test" goes to success. And "mvn
dependency:tree | grep log4j" shows no transitive dependency on log4j-1.2.17.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (STORM-3829) Remove log4j version 1 (1.2.17) from storm-core
[
https://issues.apache.org/jira/browse/STORM-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bipin Prasad resolved STORM-3829.
-
Resolution: Fixed
> Remove log4j version 1 (1.2.17) from storm-core
> ---
>
> Key: STORM-3829
> URL: https://issues.apache.org/jira/browse/STORM-3829
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
>Affects Versions: 2.3.0
>Reporter: Bipin Prasad
>Assignee: Bipin Prasad
>Priority: Major
> Fix For: 2.4.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> storm-core/pom.xml has the following entry:
> {code:java}
>
>
> log4j
> log4j
> 1.2.17
> test
>
> {code}
> This dependency can be removed. "mvn test" goes to success. And "mvn
> dependency:tree | grep log4j" shows no transitive dependency on log4j-1.2.17.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (STORM-3832) Remove python2 support
Bipin Prasad created STORM-3832: --- Summary: Remove python2 support Key: STORM-3832 URL: https://issues.apache.org/jira/browse/STORM-3832 Project: Apache Storm Issue Type: Improvement Components: build Reporter: Bipin Prasad Python version 2 has been EOL since January 1, 2020 (https://www.python.org/doc/sunset-python-2/). Remove python2 support so that code can be migrated to python3. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (STORM-3832) Remove python2 support
[ https://issues.apache.org/jira/browse/STORM-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bipin Prasad reassigned STORM-3832: --- Assignee: Bipin Prasad > Remove python2 support > -- > > Key: STORM-3832 > URL: https://issues.apache.org/jira/browse/STORM-3832 > Project: Apache Storm > Issue Type: Improvement > Components: build >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Major > > Python version 2 has been EOL since January 1, 2020 > (https://www.python.org/doc/sunset-python-2/). > Remove python2 support so that code can be migrated to python3. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (STORM-3833) Migrate to JUnit5 and remove JUnit4
Bipin Prasad created STORM-3833: --- Summary: Migrate to JUnit5 and remove JUnit4 Key: STORM-3833 URL: https://issues.apache.org/jira/browse/STORM-3833 Project: Apache Storm Issue Type: Improvement Components: build Reporter: Bipin Prasad Assignee: Bipin Prasad JUnit4 is old and works upto JDK 1.7. pom.xml contains org.junit.vintage to provide compatibility with JUnit4 classes. The tests break with running with more recent versions of JDK - for example with openjdk-17. The vintage classes cannot detect the Test methods. Remove the vintage junit, migrate fully to junit5. -- This message was sent by Atlassian Jira (v8.20.1#820001)
