[jira] [Commented] (STORM-1065) storm-kafka : kafka-partition can not find leader in zookeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160303#comment-15160303
 ] 

ASF GitHub Bot commented on STORM-1065:
---

GitHub user lujinhong opened a pull request:

https://github.com/apache/storm/pull/1146

 [STORM-1065] kafka-partition can not find leader in zookeeper

fix  [STORM-1065] kafka-partition can not find leader in zookeeper

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lujinhong/storm storm-1065

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1146.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1146


commit 556ab52fdf9cffe39994cb601f5103d1219c4052
Author: jinhong-lu 
Date:   2016-02-24T06:49:09Z

fix storm-1065: kafka-partition can not find leader in zookeeper




> storm-kafka : kafka-partition can not find leader in zookeeper 
> ---
>
> Key: STORM-1065
> URL: https://issues.apache.org/jira/browse/STORM-1065
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Reporter: dongxinwang
>Priority: Minor
>
> If the Kafka cluster is not consistent with the zookeeper data, the partition 
> can not find leader in zookeeper.In storm-kafka, it throws runtime 
> exception"No leader found for partition",it is not friendly.
> Suggestion:
> If there is no leader partition in zookeeper,don't add the partition to 
> GlobalPartitionInformation object,instead of throwing runtime exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1065] kafka-partition can not find lead...

[lujinhong]

GitHub user lujinhong opened a pull request:

https://github.com/apache/storm/pull/1146

 [STORM-1065] kafka-partition can not find leader in zookeeper

fix  [STORM-1065] kafka-partition can not find leader in zookeeper

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lujinhong/storm storm-1065

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1146.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1146


commit 556ab52fdf9cffe39994cb601f5103d1219c4052
Author: jinhong-lu 
Date:   2016-02-24T06:49:09Z

fix storm-1065: kafka-partition can not find leader in zookeeper




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1065] kafka-partition can not find lead...

[lujinhong]

Github user lujinhong commented on the pull request:

https://github.com/apache/storm/pull/1140#issuecomment-188110642
  
refresh my fork, close this PR, I will commit another one.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1065) storm-kafka : kafka-partition can not find leader in zookeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160299#comment-15160299
 ] 

ASF GitHub Bot commented on STORM-1065:
---

Github user lujinhong closed the pull request at:

https://github.com/apache/storm/pull/1140


> storm-kafka : kafka-partition can not find leader in zookeeper 
> ---
>
> Key: STORM-1065
> URL: https://issues.apache.org/jira/browse/STORM-1065
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Reporter: dongxinwang
>Priority: Minor
>
> If the Kafka cluster is not consistent with the zookeeper data, the partition 
> can not find leader in zookeeper.In storm-kafka, it throws runtime 
> exception"No leader found for partition",it is not friendly.
> Suggestion:
> If there is no leader partition in zookeeper,don't add the partition to 
> GlobalPartitionInformation object,instead of throwing runtime exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1065) storm-kafka : kafka-partition can not find leader in zookeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160298#comment-15160298
 ] 

ASF GitHub Bot commented on STORM-1065:
---

Github user lujinhong commented on the pull request:

https://github.com/apache/storm/pull/1140#issuecomment-188110642
  
refresh my fork, close this PR, I will commit another one.


> storm-kafka : kafka-partition can not find leader in zookeeper 
> ---
>
> Key: STORM-1065
> URL: https://issues.apache.org/jira/browse/STORM-1065
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Reporter: dongxinwang
>Priority: Minor
>
> If the Kafka cluster is not consistent with the zookeeper data, the partition 
> can not find leader in zookeeper.In storm-kafka, it throws runtime 
> exception"No leader found for partition",it is not friendly.
> Suggestion:
> If there is no leader partition in zookeeper,don't add the partition to 
> GlobalPartitionInformation object,instead of throwing runtime exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1065] kafka-partition can not find lead...

[lujinhong]

Github user lujinhong closed the pull request at:

https://github.com/apache/storm/pull/1140


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1569) Allowing users to specify the nimbus thrift server queue size.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160249#comment-15160249
 ] 

ASF GitHub Bot commented on STORM-1569:
---

Github user wuchong commented on the pull request:

https://github.com/apache/storm/pull/1144#issuecomment-188100464
  
+1  LGTM


> Allowing users to specify the nimbus thrift server queue size.
> --
>
> Key: STORM-1569
> URL: https://issues.apache.org/jira/browse/STORM-1569
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Affects Versions: 0.10.0
>Reporter: Parth Brahmbhatt
>Assignee: Parth Brahmbhatt
> Fix For: 1.0.0
>
>
> Currently the nimbus sever in secure mode uses 
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
>  Backed by 
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/SynchronousQueue.html,
>  Please see 
> https://github.com/apache/thrift/blob/0.9.2/lib/java/src/org/apache/thrift/server/TThreadPoolServer.java#L132.
>  This means that if all executor threads are busy serving a request and new 
> requests come in we will see RejectedExecutionExceptions in logs once they 
> have reached the retry limit. Instead we should allow the requests to be 
> queued. This patch allows the requests to be queued by replacing 
> SynchronousQueue with 
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ArrayBlockingQueue.html
>  with default size of 10 requests which should be large enough for most 
> applications. Applications can modify this default by adding the config 
> nimbus.queue.size to their storm.yaml and bouncing nimbus.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1572) throw NPE when parsing the command line arguments by CLI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160247#comment-15160247
 ] 

ASF GitHub Bot commented on STORM-1572:
---

Github user wuchong commented on the pull request:

https://github.com/apache/storm/pull/1142#issuecomment-188100054
  
+1 except a minor comment


> throw NPE when parsing the command line arguments by CLI
> 
>
> Key: STORM-1572
> URL: https://issues.apache.org/jira/browse/STORM-1572
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: John Fang
>Assignee: John Fang
>Priority: Blocker
>
> $ storm kill test
> Running: /data/nfs_share/soft/jdk1.7.0_79/bin/java -client -Ddaemon.name= 
> -Dstorm.options= -Dstorm.home=/data/nfs_share/soft/storm 
> -Dstorm.log.dir=/home/admin/storm_logs -Djava.library.path=/usr/local/lib 
> -Dstorm.conf.file= -cp 
> /data/nfs_share/soft/storm/lib/servlet-api-2.5.jar:/data/nfs_share/soft/storm/lib/slf4j-api-1.7.7.jar:/data/nfs_share/soft/storm/lib/log4j-over-slf4j-1.6.6.jar:/data/nfs_share/soft/storm/lib/reflectasm-1.07-shaded.jar:/data/nfs_share/soft/storm/lib/log4j-api-2.1.jar:/data/nfs_share/soft/storm/lib/storm-core-2.0.0-SNAPSHOT.jar:/data/nfs_share/soft/storm/lib/log4j-core-2.1.jar:/data/nfs_share/soft/storm/lib/kryo-2.21.jar:/data/nfs_share/soft/storm/lib/clojure-1.7.0.jar:/data/nfs_share/soft/storm/lib/log4j-slf4j-impl-2.1.jar:/data/nfs_share/soft/storm/lib/minlog-1.2.jar:/data/nfs_share/soft/storm/lib/disruptor-3.3.2.jar:/data/nfs_share/soft/storm/lib/asm-4.0.jar:/home/admin/.storm:/data/nfs_share/soft/storm/bin
>  org.apache.storm.command.KillTopology test
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.storm.command.CLI$CLIBuilder.parse(CLI.java:243)
> at org.apache.storm.command.KillTopology.main(KillTopology.java:33)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1569: Adding option in nimbus to specify...

[wuchong]

Github user wuchong commented on the pull request:

https://github.com/apache/storm/pull/1144#issuecomment-188100464
  
+1  LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1572] throw NPE when parsing the comman...

[wuchong]

Github user wuchong commented on the pull request:

https://github.com/apache/storm/pull/1142#issuecomment-188100054
  
+1 except a minor comment


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1571) Improvment Kafka Spout Time Metric

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160245#comment-15160245
 ] 

ASF GitHub Bot commented on STORM-1571:
---

Github user unsleepy22 commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188099708
  
I agree with @darionyaphet . Actually I think all System.nanoTime callings 
in external modules should be replaced with System.currentTimeMillis since they 
use System.nanoTime to get a ns value and then convert the time back to ms, 
which is unnecessary.

System.nanoTime should only be used in such cases that we do need high 
precision timing, e.g., timing certain code snippets in metrics system. If 
called on a critical path, this method is 5+ times slower than 
System.currentTimeMillis.


> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch, 
> 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1571]Improvment Kafka Spout Time Metric

[unsleepy22]

Github user unsleepy22 commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188099708
  
I agree with @darionyaphet . Actually I think all System.nanoTime callings 
in external modules should be replaced with System.currentTimeMillis since they 
use System.nanoTime to get a ns value and then convert the time back to ms, 
which is unnecessary.

System.nanoTime should only be used in such cases that we do need high 
precision timing, e.g., timing certain code snippets in metrics system. If 
called on a critical path, this method is 5+ times slower than 
System.currentTimeMillis.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1572) throw NPE when parsing the command line arguments by CLI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160240#comment-15160240
 ] 

ASF GitHub Bot commented on STORM-1572:
---

Github user wuchong commented on a diff in the pull request:

https://github.com/apache/storm/pull/1142#discussion_r53898826
  
--- Diff: storm-core/src/jvm/org/apache/storm/command/CLI.java ---
@@ -238,10 +238,13 @@ public CLIBuilder arg(String name, Parse parse, Assoc 
assoc) {
 DefaultParser parser = new DefaultParser();
 CommandLine cl = parser.parse(options, rawArgs);
 HashMap ret = new HashMap<>();
-for (Opt opt: opts) {
+for (Opt opt : opts) {
 Object current = null;
-for (String val: cl.getOptionValues(opt.shortName)) {
-current = opt.process(current, val);
+String[] strings = cl.getOptionValues(opt.shortName);
+if (strings != null) {
+for (String val : cl.getOptionValues(opt.shortName)) {
--- End diff --

replace `cl.getOptionValues(opt.shortName)` with `strings` to avoid 
duplicate getOptionValues


> throw NPE when parsing the command line arguments by CLI
> 
>
> Key: STORM-1572
> URL: https://issues.apache.org/jira/browse/STORM-1572
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: John Fang
>Assignee: John Fang
>Priority: Blocker
>
> $ storm kill test
> Running: /data/nfs_share/soft/jdk1.7.0_79/bin/java -client -Ddaemon.name= 
> -Dstorm.options= -Dstorm.home=/data/nfs_share/soft/storm 
> -Dstorm.log.dir=/home/admin/storm_logs -Djava.library.path=/usr/local/lib 
> -Dstorm.conf.file= -cp 
> /data/nfs_share/soft/storm/lib/servlet-api-2.5.jar:/data/nfs_share/soft/storm/lib/slf4j-api-1.7.7.jar:/data/nfs_share/soft/storm/lib/log4j-over-slf4j-1.6.6.jar:/data/nfs_share/soft/storm/lib/reflectasm-1.07-shaded.jar:/data/nfs_share/soft/storm/lib/log4j-api-2.1.jar:/data/nfs_share/soft/storm/lib/storm-core-2.0.0-SNAPSHOT.jar:/data/nfs_share/soft/storm/lib/log4j-core-2.1.jar:/data/nfs_share/soft/storm/lib/kryo-2.21.jar:/data/nfs_share/soft/storm/lib/clojure-1.7.0.jar:/data/nfs_share/soft/storm/lib/log4j-slf4j-impl-2.1.jar:/data/nfs_share/soft/storm/lib/minlog-1.2.jar:/data/nfs_share/soft/storm/lib/disruptor-3.3.2.jar:/data/nfs_share/soft/storm/lib/asm-4.0.jar:/home/admin/.storm:/data/nfs_share/soft/storm/bin
>  org.apache.storm.command.KillTopology test
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.storm.command.CLI$CLIBuilder.parse(CLI.java:243)
> at org.apache.storm.command.KillTopology.main(KillTopology.java:33)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1572] throw NPE when parsing the comman...

[wuchong]

Github user wuchong commented on a diff in the pull request:

https://github.com/apache/storm/pull/1142#discussion_r53898826
  
--- Diff: storm-core/src/jvm/org/apache/storm/command/CLI.java ---
@@ -238,10 +238,13 @@ public CLIBuilder arg(String name, Parse parse, Assoc 
assoc) {
 DefaultParser parser = new DefaultParser();
 CommandLine cl = parser.parse(options, rawArgs);
 HashMap ret = new HashMap<>();
-for (Opt opt: opts) {
+for (Opt opt : opts) {
 Object current = null;
-for (String val: cl.getOptionValues(opt.shortName)) {
-current = opt.process(current, val);
+String[] strings = cl.getOptionValues(opt.shortName);
+if (strings != null) {
+for (String val : cl.getOptionValues(opt.shortName)) {
--- End diff --

replace `cl.getOptionValues(opt.shortName)` with `strings` to avoid 
duplicate getOptionValues


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1245) port backtype.storm.daemon.acker to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160237#comment-15160237
 ] 

ASF GitHub Bot commented on STORM-1245:
---

Github user abhishekagarwal87 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1069#discussion_r53898480
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/Acker.java ---
@@ -0,0 +1,127 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon;
+
+import org.apache.storm.task.IBolt;
+import org.apache.storm.task.OutputCollector;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
+import org.apache.storm.tuple.Values;
+import org.apache.storm.utils.RotatingMap;
+import org.apache.storm.utils.TupleUtils;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Map;
+
+public class Acker implements IBolt {
+private static final Logger LOG = LoggerFactory.getLogger(Acker.class);
+
+private static final long serialVersionUID = 4430906880683183091L;
+
+public static final String ACKER_COMPONENT_ID = "__acker";
+public static final String ACKER_INIT_STREAM_ID = "__ack_init";
+public static final String ACKER_ACK_STREAM_ID = "__ack_ack";
+public static final String ACKER_FAIL_STREAM_ID = "__ack_fail";
+
+public static final int TIMEOUT_BUCKET_NUM = 3;
+
+private OutputCollector collector;
+private RotatingMap pending;
+
+private class AckObject {
+public long val = 0L;
+public Integer spoutTask = null;
+public boolean failed = false;
+
+// val xor value
+public void updateAck(Long value) {
+val = Utils.bitXor(val, value);
+}
+}
+
+@Override
+public void prepare(Map stormConf, TopologyContext context, 
OutputCollector collector) {
+this.collector = collector;
+this.pending = new RotatingMap(TIMEOUT_BUCKET_NUM);
+}
+
+@Override
+public void execute(Tuple input) {
+if (TupleUtils.isTick(input)) {
+Map tmp = pending.rotate();
+LOG.debug("Number of timeout tuples:{}", tmp.size());
+return;
+}
+
+String streamId = input.getSourceStreamId();
+Object id = input.getValue(0);
+AckObject curr = pending.get(id);
+if (ACKER_INIT_STREAM_ID.equals(streamId)) {
+if (curr == null) {
+curr = new AckObject();
+curr.val = input.getLong(1);
+curr.spoutTask = input.getInteger(2);
+pending.put(id, curr);
+} else {
+// If receiving bolt's ack before the init message from 
spout, just update the xor value.
+curr.updateAck(input.getLong(1));
+curr.spoutTask = input.getInteger(2);
+}
+} else if (ACKER_ACK_STREAM_ID.equals(streamId)) {
+if (curr != null) {
+curr.updateAck(input.getLong(1));
+} else {
+curr = new AckObject();
+curr.val = input.getLong(1);
+pending.put(id, curr);
+}
+} else if (ACKER_FAIL_STREAM_ID.equals(streamId)) {
+if (curr == null) {
+// The tuple has been already timeout or failed. So, do 
nothing
--- End diff --

That is correct @bastiliu. It would be rare but it is a possibility. 
Anyway, spout doesn't differentiate between failures and timeouts so I am fine 
with it.


> port backtype.storm.daemon.acker to java
> 
>
> Key: 

[GitHub] storm pull request: [STORM-1245] port backtype.storm.daemon.acker ...

[abhishekagarwal87]

Github user abhishekagarwal87 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1069#discussion_r53898480
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/Acker.java ---
@@ -0,0 +1,127 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon;
+
+import org.apache.storm.task.IBolt;
+import org.apache.storm.task.OutputCollector;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
+import org.apache.storm.tuple.Values;
+import org.apache.storm.utils.RotatingMap;
+import org.apache.storm.utils.TupleUtils;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Map;
+
+public class Acker implements IBolt {
+private static final Logger LOG = LoggerFactory.getLogger(Acker.class);
+
+private static final long serialVersionUID = 4430906880683183091L;
+
+public static final String ACKER_COMPONENT_ID = "__acker";
+public static final String ACKER_INIT_STREAM_ID = "__ack_init";
+public static final String ACKER_ACK_STREAM_ID = "__ack_ack";
+public static final String ACKER_FAIL_STREAM_ID = "__ack_fail";
+
+public static final int TIMEOUT_BUCKET_NUM = 3;
+
+private OutputCollector collector;
+private RotatingMap pending;
+
+private class AckObject {
+public long val = 0L;
+public Integer spoutTask = null;
+public boolean failed = false;
+
+// val xor value
+public void updateAck(Long value) {
+val = Utils.bitXor(val, value);
+}
+}
+
+@Override
+public void prepare(Map stormConf, TopologyContext context, 
OutputCollector collector) {
+this.collector = collector;
+this.pending = new RotatingMap(TIMEOUT_BUCKET_NUM);
+}
+
+@Override
+public void execute(Tuple input) {
+if (TupleUtils.isTick(input)) {
+Map tmp = pending.rotate();
+LOG.debug("Number of timeout tuples:{}", tmp.size());
+return;
+}
+
+String streamId = input.getSourceStreamId();
+Object id = input.getValue(0);
+AckObject curr = pending.get(id);
+if (ACKER_INIT_STREAM_ID.equals(streamId)) {
+if (curr == null) {
+curr = new AckObject();
+curr.val = input.getLong(1);
+curr.spoutTask = input.getInteger(2);
+pending.put(id, curr);
+} else {
+// If receiving bolt's ack before the init message from 
spout, just update the xor value.
+curr.updateAck(input.getLong(1));
+curr.spoutTask = input.getInteger(2);
+}
+} else if (ACKER_ACK_STREAM_ID.equals(streamId)) {
+if (curr != null) {
+curr.updateAck(input.getLong(1));
+} else {
+curr = new AckObject();
+curr.val = input.getLong(1);
+pending.put(id, curr);
+}
+} else if (ACKER_FAIL_STREAM_ID.equals(streamId)) {
+if (curr == null) {
+// The tuple has been already timeout or failed. So, do 
nothing
--- End diff --

That is correct @bastiliu. It would be rare but it is a possibility. 
Anyway, spout doesn't differentiate between failures and timeouts so I am fine 
with it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1245) port backtype.storm.daemon.acker to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160225#comment-15160225
 ] 

ASF GitHub Bot commented on STORM-1245:
---

Github user abhishekagarwal87 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1069#discussion_r53898379
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/Acker.java ---
@@ -0,0 +1,127 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon;
+
+import org.apache.storm.task.IBolt;
+import org.apache.storm.task.OutputCollector;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
+import org.apache.storm.tuple.Values;
+import org.apache.storm.utils.RotatingMap;
+import org.apache.storm.utils.TupleUtils;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Map;
+
+public class Acker implements IBolt {
+private static final Logger LOG = LoggerFactory.getLogger(Acker.class);
+
+private static final long serialVersionUID = 4430906880683183091L;
+
+public static final String ACKER_COMPONENT_ID = "__acker";
+public static final String ACKER_INIT_STREAM_ID = "__ack_init";
+public static final String ACKER_ACK_STREAM_ID = "__ack_ack";
+public static final String ACKER_FAIL_STREAM_ID = "__ack_fail";
+
+public static final int TIMEOUT_BUCKET_NUM = 3;
+
+private OutputCollector collector;
+private RotatingMap pending;
+
+private class AckObject {
+public long val = 0L;
--- End diff --

0 is assigned as default-value only in the update-ack call. let's say FAIL 
arrives first. `cond (= 0 (:val curr)` will return false in  the clojure code 
because val is nil. If val was zero instead, acker would have acked a failed 
message. The reason is value is checked first and failure is checked 
afterwards. It is not a problem in your changes since code just returns if fail 
comes before init. 


> port backtype.storm.daemon.acker to java
> 
>
> Key: STORM-1245
> URL: https://issues.apache.org/jira/browse/STORM-1245
> Project: Apache Storm
>  Issue Type: New Feature
>Reporter: Robert Joseph Evans
>Assignee: Basti Liu
>  Labels: java-migration, jstorm-merger
>
> use 
> https://github.com/apache/storm/blob/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/task/acker/Acker.java
>  as a guide



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1245] port backtype.storm.daemon.acker ...

[abhishekagarwal87]

Github user abhishekagarwal87 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1069#discussion_r53898379
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/Acker.java ---
@@ -0,0 +1,127 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon;
+
+import org.apache.storm.task.IBolt;
+import org.apache.storm.task.OutputCollector;
+import org.apache.storm.task.TopologyContext;
+import org.apache.storm.tuple.Tuple;
+import org.apache.storm.tuple.Values;
+import org.apache.storm.utils.RotatingMap;
+import org.apache.storm.utils.TupleUtils;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Map;
+
+public class Acker implements IBolt {
+private static final Logger LOG = LoggerFactory.getLogger(Acker.class);
+
+private static final long serialVersionUID = 4430906880683183091L;
+
+public static final String ACKER_COMPONENT_ID = "__acker";
+public static final String ACKER_INIT_STREAM_ID = "__ack_init";
+public static final String ACKER_ACK_STREAM_ID = "__ack_ack";
+public static final String ACKER_FAIL_STREAM_ID = "__ack_fail";
+
+public static final int TIMEOUT_BUCKET_NUM = 3;
+
+private OutputCollector collector;
+private RotatingMap pending;
+
+private class AckObject {
+public long val = 0L;
--- End diff --

0 is assigned as default-value only in the update-ack call. let's say FAIL 
arrives first. `cond (= 0 (:val curr)` will return false in  the clojure code 
because val is nil. If val was zero instead, acker would have acked a failed 
message. The reason is value is checked first and failure is checked 
afterwards. It is not a problem in your changes since code just returns if fail 
comes before init. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1571]Improvment Kafka Spout Time Metric

[darionyaphet]

Github user darionyaphet commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188095012
  
This question doesn't focus on performance improvement . 

Using System.currentTimeMillis to replace System.nanoTime is more reasonable


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1571) Improvment Kafka Spout Time Metric

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160224#comment-15160224
 ] 

ASF GitHub Bot commented on STORM-1571:
---

Github user darionyaphet commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188095012
  
This question doesn't focus on performance improvement . 

Using System.currentTimeMillis to replace System.nanoTime is more reasonable


> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch, 
> 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1571]Improvment Kafka Spout Time Metric

[abhishekagarwal87]

Github user abhishekagarwal87 commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188092004
  
Given that the timing is measured only in the fill call, which happens per 
batch of messages, the relative performance improvement is very low. By the 
way, call could return in 1ms if fetch size is configured at a lower value for 
better latency. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1571) Improvment Kafka Spout Time Metric

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160219#comment-15160219
 ] 

ASF GitHub Bot commented on STORM-1571:
---

Github user abhishekagarwal87 commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188092004
  
Given that the timing is measured only in the fill call, which happens per 
batch of messages, the relative performance improvement is very low. By the 
way, call could return in 1ms if fetch size is configured at a lower value for 
better latency. 


> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch, 
> 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-1304) port backtype.storm.submitter-test to java

[Jark Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu reassigned STORM-1304:
--

Assignee: Jark Wu

> port  backtype.storm.submitter-test to java
> ---
>
> Key: STORM-1304
> URL: https://issues.apache.org/jira/browse/STORM-1304
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> Test ZookeeperAuthentication payload generation that is a part of 
> StormSubmitter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-1229) port backtype.storm.metric.testing to java

[Abhishek Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Agarwal reassigned STORM-1229:
---

Assignee: Abhishek Agarwal

> port backtype.storm.metric.testing to java
> --
>
> Key: STORM-1229
> URL: https://issues.apache.org/jira/browse/STORM-1229
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Abhishek Agarwal
>  Labels: java-migration, jstorm-merger
>
> This provides a way to capture and inspect the datapoints sent to a 
> FakeMetricConsumer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-1283) port backtype.storm.MockAutoCred to java

[Abhishek Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Agarwal reassigned STORM-1283:
---

Assignee: Abhishek Agarwal

> port backtype.storm.MockAutoCred to java
> 
>
> Key: STORM-1283
> URL: https://issues.apache.org/jira/browse/STORM-1283
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Abhishek Agarwal
>  Labels: java-migration, jstorm-merger
>
> Mock implementation of INimbusCredentialPlugin,IAutoCredentials and 
> ICredentialsRenewer for testing only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-1305) port backtype.storm.supervisor-test to java

[Abhishek Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Agarwal reassigned STORM-1305:
---

Assignee: Abhishek Agarwal

> port backtype.storm.supervisor-test to java
> ---
>
> Key: STORM-1305
> URL: https://issues.apache.org/jira/browse/STORM-1305
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Abhishek Agarwal
>  Labels: java-migration, jstorm-merger
>
> Test the supervisor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-1312) port backtype.storm.worker-test to java

[Abhishek Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Agarwal reassigned STORM-1312:
---

Assignee: Abhishek Agarwal

> port backtype.storm.worker-test to java
> ---
>
> Key: STORM-1312
> URL: https://issues.apache.org/jira/browse/STORM-1312
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Abhishek Agarwal
>  Labels: java-migration, jstorm-merger
>
> Test Worker



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1571]Improvment Kafka Spout Time Metric

[darionyaphet]

Github user darionyaphet commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188062656
  
sure . I push a new commit to update TridentKafkaEmitter's time interval 
calculation . Should merge them into one commit ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1571) Improvment Kafka Spout Time Metric

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160170#comment-15160170
 ] 

ASF GitHub Bot commented on STORM-1571:
---

Github user darionyaphet commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188062656
  
sure . I push a new commit to update TridentKafkaEmitter's time interval 
calculation . Should merge them into one commit ?


> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch, 
> 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1571) Improvment Kafka Spout Time Metric

[darion yaphet (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

darion yaphet updated STORM-1571:
-
Attachment: 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch

> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch, 
> 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1567) in defaults.yaml 'topology.disable.loadaware' should be 'topology.disable.loadaware.messaging'

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160127#comment-15160127
 ] 

ASF GitHub Bot commented on STORM-1567:
---

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/1135#issuecomment-188043611
  
LGTM.


> in defaults.yaml  'topology.disable.loadaware' should be 
> 'topology.disable.loadaware.messaging'
> ---
>
> Key: STORM-1567
> URL: https://issues.apache.org/jira/browse/STORM-1567
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 2.0.0
>Reporter: caofangkun
>Assignee: caofangkun
>Priority: Trivial
>
> {code:title=defaults.yaml|borderStyle=solid}
> diff --git a/conf/defaults.yaml b/conf/defaults.yaml
> index 166b249..01821e1 100644
> --- a/conf/defaults.yaml
> +++ b/conf/defaults.yaml
> @@ -256,7 +256,7 @@ topology.bolts.outgoing.overflow.buffer.enable: false
>  topology.disruptor.wait.timeout.millis: 1000
>  topology.disruptor.batch.size: 100
>  topology.disruptor.batch.timeout.millis: 1
> -topology.disable.loadaware: false
> +topology.disable.loadaware.messaging: false
>  topology.state.checkpoint.interval.ms: 1000
>  
>  # Configs for Resource Aware Scheduler
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1567] in defaults.yaml 'topology.disabl...

[zhuoliu]

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/1135#issuecomment-188043611
  
LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160126#comment-15160126
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user hustfxj commented on the pull request:

https://github.com/apache/storm/pull/1081#issuecomment-188041296
  
@revans2  Thank you again. I have addressed your comments.


> port backtype.storm.cluster to java
> ---
>
> Key: STORM-1273
> URL: https://issues.apache.org/jira/browse/STORM-1273
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: John Fang
>  Labels: java-migration, jstorm-merger
>
> current state of the cluster (Some of this moves to java as a part of 
> heartbeat server)
> https://github.com/apache/storm/tree/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/cluster
>  as an example



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[hustfxj]

Github user hustfxj commented on the pull request:

https://github.com/apache/storm/pull/1081#issuecomment-188041296
  
@revans2  Thank you again. I have addressed your comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160124#comment-15160124
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user hustfxj commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53888948
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ClusterUtils.java ---
@@ -0,0 +1,261 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.storm.Config;
+import org.apache.storm.generated.ClusterWorkerHeartbeat;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ProfileAction;
+import org.apache.storm.utils.Utils;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.server.auth.DigestAuthenticationProvider;
+
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.io.UnsupportedEncodingException;
+import java.net.URLEncoder;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class ClusterUtils {
+
+public static final String ZK_SEPERATOR = "/";
+
+public static final String ASSIGNMENTS_ROOT = "assignments";
+public static final String CODE_ROOT = "code";
+public static final String STORMS_ROOT = "storms";
+public static final String SUPERVISORS_ROOT = "supervisors";
+public static final String WORKERBEATS_ROOT = "workerbeats";
+public static final String BACKPRESSURE_ROOT = "backpressure";
+public static final String ERRORS_ROOT = "errors";
+public static final String BLOBSTORE_ROOT = "blobstore";
+public static final String BLOBSTORE_MAX_KEY_SEQUENCE_NUMBER_ROOT = 
"blobstoremaxkeysequencenumber";
+public static final String NIMBUSES_ROOT = "nimbuses";
+public static final String CREDENTIALS_ROOT = "credentials";
+public static final String LOGCONFIG_ROOT = "logconfigs";
+public static final String PROFILERCONFIG_ROOT = "profilerconfigs";
+
+public static final String ASSIGNMENTS_SUBTREE = ZK_SEPERATOR + 
ASSIGNMENTS_ROOT;
+public static final String STORMS_SUBTREE = ZK_SEPERATOR + STORMS_ROOT;
+public static final String SUPERVISORS_SUBTREE = ZK_SEPERATOR + 
SUPERVISORS_ROOT;
+public static final String WORKERBEATS_SUBTREE = ZK_SEPERATOR + 
WORKERBEATS_ROOT;
+public static final String BACKPRESSURE_SUBTREE = ZK_SEPERATOR + 
BACKPRESSURE_ROOT;
+public static final String ERRORS_SUBTREE = ZK_SEPERATOR + ERRORS_ROOT;
+public static final String BLOBSTORE_SUBTREE = ZK_SEPERATOR + 
BLOBSTORE_ROOT;
+public static final String BLOBSTORE_MAX_KEY_SEQUENCE_NUMBER_SUBTREE = 
ZK_SEPERATOR + BLOBSTORE_MAX_KEY_SEQUENCE_NUMBER_ROOT;
+public static final String NIMBUSES_SUBTREE = ZK_SEPERATOR + 
NIMBUSES_ROOT;
+public static final String CREDENTIALS_SUBTREE = ZK_SEPERATOR + 
CREDENTIALS_ROOT;
+public static final String LOGCONFIG_SUBTREE = ZK_SEPERATOR + 
LOGCONFIG_ROOT;
+public static final String PROFILERCONFIG_SUBTREE = ZK_SEPERATOR + 
PROFILERCONFIG_ROOT;
+
+// A singleton instance allows us to mock delegated static methods in 
our
+// tests by subclassing.
+private static final ClusterUtils INSTANCE = new ClusterUtils();
+private static ClusterUtils _instance = INSTANCE;
+
+/**
+ * Provide an instance of this class for delegates to use. To mock out 
delegated methods, provide an instance of a subclass that overrides the
+ * implementation of the delegated method.
+ *
+ * @param u a Cluster instance
+ */
+public static void setInstance(ClusterUtils u) {
+  

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[hustfxj]

Github user hustfxj commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53888948
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ClusterUtils.java ---
@@ -0,0 +1,261 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.storm.Config;
+import org.apache.storm.generated.ClusterWorkerHeartbeat;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ProfileAction;
+import org.apache.storm.utils.Utils;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.server.auth.DigestAuthenticationProvider;
+
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.io.UnsupportedEncodingException;
+import java.net.URLEncoder;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class ClusterUtils {
+
+public static final String ZK_SEPERATOR = "/";
+
+public static final String ASSIGNMENTS_ROOT = "assignments";
+public static final String CODE_ROOT = "code";
+public static final String STORMS_ROOT = "storms";
+public static final String SUPERVISORS_ROOT = "supervisors";
+public static final String WORKERBEATS_ROOT = "workerbeats";
+public static final String BACKPRESSURE_ROOT = "backpressure";
+public static final String ERRORS_ROOT = "errors";
+public static final String BLOBSTORE_ROOT = "blobstore";
+public static final String BLOBSTORE_MAX_KEY_SEQUENCE_NUMBER_ROOT = 
"blobstoremaxkeysequencenumber";
+public static final String NIMBUSES_ROOT = "nimbuses";
+public static final String CREDENTIALS_ROOT = "credentials";
+public static final String LOGCONFIG_ROOT = "logconfigs";
+public static final String PROFILERCONFIG_ROOT = "profilerconfigs";
+
+public static final String ASSIGNMENTS_SUBTREE = ZK_SEPERATOR + 
ASSIGNMENTS_ROOT;
+public static final String STORMS_SUBTREE = ZK_SEPERATOR + STORMS_ROOT;
+public static final String SUPERVISORS_SUBTREE = ZK_SEPERATOR + 
SUPERVISORS_ROOT;
+public static final String WORKERBEATS_SUBTREE = ZK_SEPERATOR + 
WORKERBEATS_ROOT;
+public static final String BACKPRESSURE_SUBTREE = ZK_SEPERATOR + 
BACKPRESSURE_ROOT;
+public static final String ERRORS_SUBTREE = ZK_SEPERATOR + ERRORS_ROOT;
+public static final String BLOBSTORE_SUBTREE = ZK_SEPERATOR + 
BLOBSTORE_ROOT;
+public static final String BLOBSTORE_MAX_KEY_SEQUENCE_NUMBER_SUBTREE = 
ZK_SEPERATOR + BLOBSTORE_MAX_KEY_SEQUENCE_NUMBER_ROOT;
+public static final String NIMBUSES_SUBTREE = ZK_SEPERATOR + 
NIMBUSES_ROOT;
+public static final String CREDENTIALS_SUBTREE = ZK_SEPERATOR + 
CREDENTIALS_ROOT;
+public static final String LOGCONFIG_SUBTREE = ZK_SEPERATOR + 
LOGCONFIG_ROOT;
+public static final String PROFILERCONFIG_SUBTREE = ZK_SEPERATOR + 
PROFILERCONFIG_ROOT;
+
+// A singleton instance allows us to mock delegated static methods in 
our
+// tests by subclassing.
+private static final ClusterUtils INSTANCE = new ClusterUtils();
+private static ClusterUtils _instance = INSTANCE;
+
+/**
+ * Provide an instance of this class for delegates to use. To mock out 
delegated methods, provide an instance of a subclass that overrides the
+ * implementation of the delegated method.
+ *
+ * @param u a Cluster instance
+ */
+public static void setInstance(ClusterUtils u) {
+_instance = u;
+}
+
+/**
+ * Resets the singleton instance to the default. This is helpful to 
reset the class to its original functionality when mocking is no longer desired.
+ */
+public 

[GitHub] storm pull request: Fix Log4j2.xml config to output the the timest...

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1145#issuecomment-188035678
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1573) Add batch support for MongoInsertBolt

[Xin Wang (JIRA)]

Xin Wang created STORM-1573:
---

 Summary: Add batch support for MongoInsertBolt
 Key: STORM-1573
 URL: https://issues.apache.org/jira/browse/STORM-1573
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Xin Wang
Assignee: Xin Wang


1. Support MongoInsertBolt batch inserting _withBatch_ option
2. Specify whether to sort _withOrdered_ option



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: Fix Log4j2.xml config to output the the timest...

[zhuoliu]

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/1145#issuecomment-188035152
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1320) Port JStorm scheduler to storm

[Xin Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160105#comment-15160105
 ] 

Xin Wang commented on STORM-1320:
-

The load balance issue stated in pdf(_2 Balance load of topology and cluster_) 
has been fixed in prior release (see: 
[STORM-132|https://issues.apache.org/jira/browse/STORM-132]).

> Port JStorm scheduler to storm
> --
>
> Key: STORM-1320
> URL: https://issues.apache.org/jira/browse/STORM-1320
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: John Fang
>  Labels: jstorm-merger
> Attachments: Discussion about sheduler.pdf
>
>
> Port the JStorm scheduler to storm core.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-1250) port backtype.storm.serialization-test to java

[Jark Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu reassigned STORM-1250:
--

Assignee: Jark Wu

> port  backtype.storm.serialization-test to java
> ---
>
> Key: STORM-1250
> URL: https://issues.apache.org/jira/browse/STORM-1250
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> junit test migration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160087#comment-15160087
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user hustfxj commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53886541
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[hustfxj]

Github user hustfxj commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53886541
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.debug("Received event {} : {} : {}", 
state, type, path);
+

[jira] [Assigned] (STORM-1277) port backtype.storm.daemon.executor to java

[John Fang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Fang reassigned STORM-1277:


Assignee: John Fang

> port backtype.storm.daemon.executor to java
> ---
>
> Key: STORM-1277
> URL: https://issues.apache.org/jira/browse/STORM-1277
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: John Fang
>  Labels: java-migration, jstorm-merger
>
> https://github.com/apache/storm/tree/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/task
>  kind of.  Tasks and executors are combined in jstorm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: 答复: [DISCUSS] Java REST Framework adoption

[Prajwal Tuladhar]

+1 for DW

On Tue, Feb 23, 2016 at 9:13 PM, John Fang 
wrote:

> +1 for DropWizard
>
> -邮件原件-
> 发件人: Hugo Da Cruz Louro [mailto:hlo...@hortonworks.com]
> 发送时间: 2016年2月24日 6:36
> 收件人: dev@storm.apache.org
> 主题: Re: [DISCUSS] Java REST Framework adoption
>
> I also vote on DropWizard
>
> > On Feb 23, 2016, at 2:29 PM, Parth Brahmbhatt <
> pbrahmbh...@hortonworks.com> wrote:
> >
> > +1 on DropWizard.
> >
> > On 2/23/16, 2:02 PM, "Harsha"  wrote:
> >
> >> -1 on spring boot or anything related to spring.
> >> This api is intended to be very simple powering UI and any rest
> >> clients interested in grabbing the metrics from the same api as UI does.
> >>
> >> Jersey is good and  dropwizard (http://www.dropwizard.io/0.9.2/docs/)
> >> has been a way to go for java REST api offlate. Underneath it uses
> >> jersey and one can run jetty server as well which is what we've as
> >> the UI and logviewer server.
> >>
> >>
> >> -Harsha
> >>
> >> On Tue, Feb 23, 2016, at 08:23 AM, Ravi Sharma wrote:
> >>> spring boot +
> >>>
> >>> Ravi
> >>>
> >>> On Tue, Feb 23, 2016 at 3:16 PM, Ankur Garg 
> >>> wrote:
> >>>
>  How about using Spring Boot & Jersey for writing this .  Spring
>  Boot
> >>> will
>  give us packaged  jar which once executed will bring up its own
> >>> embedded
>  server (Jetty or Tomcat or some other ) . Although Spring Boot has
> >>> some
>  disadvantages as well , but worth investigating this option too .
> 
> 
>  Any thoughts??
> 
>  Thanks
>  Ankur
> 
>  On Tue, Feb 23, 2016 at 8:20 PM, Bobby Evans
> >>> 
>  wrote:
> 
> > Yes, we need to pick something.  I have used Jersey in the past
> > and
> >>> I
> > think it is fairly decent.  I have never used RESTEasy, but it is
> >>> more or
> > less the same API, so either one is fine with me, but Jersey is my
> >>> vote
> > just because of experience.
> >
> > You should keep in mind that we are currently on a very old
> > version
> >>> of
> > jetty, and I am not sure if newer libraries will work with it.
> > But
> >>> also
> > the old versions of ring and hiccup that we use don't support
> > newer
> >>> jetty
> > versions either.
> >
> > I personally think that now would be a good time to separate out
> >>> the UI
> > into a separate package + classpath.  This would allow us to
> >>> package the
>  UI
> > as both a war with embedded jetty as a default option to run it;
> >>> start
>  from
> > scratch with up to date versions of Jetty, Jersey/RESTEasy, and
> >>> JAXB; and
> > upgrade the different servers/components one at a time instead of
> >>> all at
> > once.  The DRPC server also uses the embedded jetty and exposes a
> >>> REST
> > interface, and that is going to be a harder one to tease out so it
> >>> should
> > probably be the last one to go.
> > - Bobby
> >
> >On Tuesday, February 23, 2016 3:40 AM, 伍翀(云邪) <
> > wuchong...@alibaba-inc.com> wrote:
> >
> >
> > Hi all, I’m planning to move UI/REST service and logviewer to
> > Java,
>  which
> > means that we need to pick some alternatives for ring and hiccup.
> > So the first thing is to pick up a REST framework.
> > For the REST APIs, I think Jersey is a good choice (RESTEasy is
> > fine
>  too).
> > It’s easy to develop and good performance.
> > Now logviewer use hiccup to return HTML we build ourselves, but
> >>> it’s hard
> > to debug and maintain. So in my opinion, it’s better to replace it
> >>> with
> > static HTML + REST like regular UI.
> > Please let me know what you think.
> > – Jark Wu
> >
> >
> >
> 
> >>
> >
>
>
>


-- 
--
Cheers,
Praj

答复: [DISCUSS] Java REST Framework adoption

[John Fang]

+1 for DropWizard

-邮件原件-
发件人: Hugo Da Cruz Louro [mailto:hlo...@hortonworks.com] 
发送时间: 2016年2月24日 6:36
收件人: dev@storm.apache.org
主题: Re: [DISCUSS] Java REST Framework adoption

I also vote on DropWizard

> On Feb 23, 2016, at 2:29 PM, Parth Brahmbhatt  
> wrote:
> 
> +1 on DropWizard.
> 
> On 2/23/16, 2:02 PM, "Harsha"  wrote:
> 
>> -1 on spring boot or anything related to spring.
>> This api is intended to be very simple powering UI and any rest 
>> clients interested in grabbing the metrics from the same api as UI does.
>> 
>> Jersey is good and  dropwizard (http://www.dropwizard.io/0.9.2/docs/)
>> has been a way to go for java REST api offlate. Underneath it uses 
>> jersey and one can run jetty server as well which is what we've as 
>> the UI and logviewer server.
>> 
>> 
>> -Harsha
>> 
>> On Tue, Feb 23, 2016, at 08:23 AM, Ravi Sharma wrote:
>>> spring boot +
>>> 
>>> Ravi
>>> 
>>> On Tue, Feb 23, 2016 at 3:16 PM, Ankur Garg 
>>> wrote:
>>> 
 How about using Spring Boot & Jersey for writing this .  Spring 
 Boot
>>> will
 give us packaged  jar which once executed will bring up its own
>>> embedded
 server (Jetty or Tomcat or some other ) . Although Spring Boot has
>>> some
 disadvantages as well , but worth investigating this option too .
 
 
 Any thoughts??
 
 Thanks
 Ankur
 
 On Tue, Feb 23, 2016 at 8:20 PM, Bobby Evans
>>> 
 wrote:
 
> Yes, we need to pick something.  I have used Jersey in the past 
> and
>>> I
> think it is fairly decent.  I have never used RESTEasy, but it is
>>> more or
> less the same API, so either one is fine with me, but Jersey is my
>>> vote
> just because of experience.
> 
> You should keep in mind that we are currently on a very old 
> version
>>> of
> jetty, and I am not sure if newer libraries will work with it.  
> But
>>> also
> the old versions of ring and hiccup that we use don't support 
> newer
>>> jetty
> versions either.
> 
> I personally think that now would be a good time to separate out
>>> the UI
> into a separate package + classpath.  This would allow us to
>>> package the
 UI
> as both a war with embedded jetty as a default option to run it;
>>> start
 from
> scratch with up to date versions of Jetty, Jersey/RESTEasy, and
>>> JAXB; and
> upgrade the different servers/components one at a time instead of
>>> all at
> once.  The DRPC server also uses the embedded jetty and exposes a
>>> REST
> interface, and that is going to be a harder one to tease out so it
>>> should
> probably be the last one to go.
> - Bobby
> 
>On Tuesday, February 23, 2016 3:40 AM, 伍翀(云邪) < 
> wuchong...@alibaba-inc.com> wrote:
> 
> 
> Hi all, I’m planning to move UI/REST service and logviewer to 
> Java,
 which
> means that we need to pick some alternatives for ring and hiccup.
> So the first thing is to pick up a REST framework.
> For the REST APIs, I think Jersey is a good choice (RESTEasy is 
> fine
 too).
> It’s easy to develop and good performance.
> Now logviewer use hiccup to return HTML we build ourselves, but
>>> it’s hard
> to debug and maintain. So in my opinion, it’s better to replace it
>>> with
> static HTML + REST like regular UI.
> Please let me know what you think.
> – Jark Wu
> 
> 
> 
 
>> 
> 

[jira] [Assigned] (STORM-1278) port backtype.storm.daemon.worker to java

[Jark Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu reassigned STORM-1278:
--

Assignee: Jark Wu

> port backtype.storm.daemon.worker to java
> -
>
> Key: STORM-1278
> URL: https://issues.apache.org/jira/browse/STORM-1278
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> https://github.com/apache/storm/tree/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/daemon/worker
>  as an example



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1571]Improvment Kafka Spout Time Metric

[unsleepy22]

Github user unsleepy22 commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188003867
  
+1, but there're 2 occurrences of System.nanoTime in storm-kafka, 
@darionyaphet would you please change the other?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1571) Improvment Kafka Spout Time Metric

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160039#comment-15160039
 ] 

ASF GitHub Bot commented on STORM-1571:
---

Github user unsleepy22 commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-188003867
  
+1, but there're 2 occurrences of System.nanoTime in storm-kafka, 
@darionyaphet would you please change the other?


> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: Fix Log4j2.xml config to output the the timest...

[Parth-Brahmbhatt]

Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/1145#issuecomment-187957776
  
+1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: Fix Log4j2.xml config to output the the timest...

[hmcl]

GitHub user hmcl opened a pull request:

https://github.com/apache/storm/pull/1145

Fix Log4j2.xml config to output the the timestamp in HH:mm:ss.SSS

Currently the -r option outputs the number of milliseconds elapsed from the 
construction of the layout until the creation of the logging event.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hmcl/storm-apache FIX_LOG4j12_Config

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1145.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1145


commit 908b2864dd403c82ab9aaee78eb5898728e8e7e0
Author: Hugo Louro 
Date:   2016-02-23T23:01:32Z

Fix Log4j2.xml config to output the the timestamp in HH:mm:ss.SSS




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159796#comment-15159796
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/1139#issuecomment-187946863
  
A lot of this looks good, But it needs a little bit of work to move the 
clojure/ring/hiccup specific parts where they need to be.


> port backtype.storm.ui.helpers to java
> --
>
> Key: STORM-1254
> URL: https://issues.apache.org/jira/browse/STORM-1254
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> Helper functions for the UI/HTTP



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/1139#issuecomment-187946863
  
A lot of this looks good, But it needs a little bit of work to move the 
clojure/ring/hiccup specific parts where they need to be.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159793#comment-15159793
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53863268
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53863268
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+public static Map unauthorizedUserJson(String user) {
+return ImmutableMap.of(
+"error", "No Authorization",
+"errorMessage", String.format("User %s is not 
authorized.", user)
+

Re: [DISCUSS] Java REST Framework adoption

[P. Taylor Goetz]

+1 for Dropwizard as well...

It includes metrics, which we're already using elsewhere. And I'm all for 
dependency reuse.

-Taylor

> On Feb 23, 2016, at 5:29 PM, Parth Brahmbhatt  
> wrote:
> 
> +1 on DropWizard.
> 
>> On 2/23/16, 2:02 PM, "Harsha"  wrote:
>> 
>> -1 on spring boot or anything related to spring.
>> This api is intended to be very simple powering UI and any rest clients
>> interested in grabbing the metrics from the same api as UI does.
>> 
>> Jersey is good and  dropwizard (http://www.dropwizard.io/0.9.2/docs/)
>> has been a way to go for java REST api offlate. Underneath it uses
>> jersey and one can run jetty server as well which is what we've as the
>> UI and logviewer server.
>> 
>> 
>> -Harsha
>> 
>>> On Tue, Feb 23, 2016, at 08:23 AM, Ravi Sharma wrote:
>>> spring boot +
>>> 
>>> Ravi
>>> 
>>> On Tue, Feb 23, 2016 at 3:16 PM, Ankur Garg 
>>> wrote:
>>> 
 How about using Spring Boot & Jersey for writing this .  Spring Boot
>>> will
 give us packaged  jar which once executed will bring up its own
>>> embedded
 server (Jetty or Tomcat or some other ) . Although Spring Boot has
>>> some
 disadvantages as well , but worth investigating this option too .
 
 
 Any thoughts??
 
 Thanks
 Ankur
 
 On Tue, Feb 23, 2016 at 8:20 PM, Bobby Evans
>>> 
 wrote:
 
> Yes, we need to pick something.  I have used Jersey in the past and
>>> I
> think it is fairly decent.  I have never used RESTEasy, but it is
>>> more or
> less the same API, so either one is fine with me, but Jersey is my
>>> vote
> just because of experience.
> 
> You should keep in mind that we are currently on a very old version
>>> of
> jetty, and I am not sure if newer libraries will work with it.  But
>>> also
> the old versions of ring and hiccup that we use don't support newer
>>> jetty
> versions either.
> 
> I personally think that now would be a good time to separate out
>>> the UI
> into a separate package + classpath.  This would allow us to
>>> package the
 UI
> as both a war with embedded jetty as a default option to run it;
>>> start
 from
> scratch with up to date versions of Jetty, Jersey/RESTEasy, and
>>> JAXB; and
> upgrade the different servers/components one at a time instead of
>>> all at
> once.  The DRPC server also uses the embedded jetty and exposes a
>>> REST
> interface, and that is going to be a harder one to tease out so it
>>> should
> probably be the last one to go.
> - Bobby
> 
>On Tuesday, February 23, 2016 3:40 AM, 伍翀(云邪) <
> wuchong...@alibaba-inc.com> wrote:
> 
> 
> Hi all, I’m planning to move UI/REST service and logviewer to Java,
 which
> means that we need to pick some alternatives for ring and hiccup.
> So the first thing is to pick up a REST framework.
> For the REST APIs, I think Jersey is a good choice (RESTEasy is fine
 too).
> It’s easy to develop and good performance.
> Now logviewer use hiccup to return HTML we build ourselves, but
>>> it’s hard
> to debug and maintain. So in my opinion, it’s better to replace it
>>> with
> static HTML + REST like regular UI.
> Please let me know what you think.
> – Jark Wu
> 
> 
> 
 
>> 
> 

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53862671
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+public static Map unauthorizedUserJson(String user) {
+return ImmutableMap.of(
+"error", "No Authorization",
+"errorMessage", String.format("User %s is not 
authorized.", user)
+

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159785#comment-15159785
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53862671
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+

Re: [DISCUSS] Java REST Framework adoption

[Hugo Da Cruz Louro]

I also vote on DropWizard

> On Feb 23, 2016, at 2:29 PM, Parth Brahmbhatt  
> wrote:
> 
> +1 on DropWizard.
> 
> On 2/23/16, 2:02 PM, "Harsha"  wrote:
> 
>> -1 on spring boot or anything related to spring.
>> This api is intended to be very simple powering UI and any rest clients
>> interested in grabbing the metrics from the same api as UI does.
>> 
>> Jersey is good and  dropwizard (http://www.dropwizard.io/0.9.2/docs/)
>> has been a way to go for java REST api offlate. Underneath it uses
>> jersey and one can run jetty server as well which is what we've as the
>> UI and logviewer server.
>> 
>> 
>> -Harsha
>> 
>> On Tue, Feb 23, 2016, at 08:23 AM, Ravi Sharma wrote:
>>> spring boot +
>>> 
>>> Ravi
>>> 
>>> On Tue, Feb 23, 2016 at 3:16 PM, Ankur Garg 
>>> wrote:
>>> 
 How about using Spring Boot & Jersey for writing this .  Spring Boot
>>> will
 give us packaged  jar which once executed will bring up its own
>>> embedded
 server (Jetty or Tomcat or some other ) . Although Spring Boot has
>>> some
 disadvantages as well , but worth investigating this option too .
 
 
 Any thoughts??
 
 Thanks
 Ankur
 
 On Tue, Feb 23, 2016 at 8:20 PM, Bobby Evans
>>> 
 wrote:
 
> Yes, we need to pick something.  I have used Jersey in the past and
>>> I
> think it is fairly decent.  I have never used RESTEasy, but it is
>>> more or
> less the same API, so either one is fine with me, but Jersey is my
>>> vote
> just because of experience.
> 
> You should keep in mind that we are currently on a very old version
>>> of
> jetty, and I am not sure if newer libraries will work with it.  But
>>> also
> the old versions of ring and hiccup that we use don't support newer
>>> jetty
> versions either.
> 
> I personally think that now would be a good time to separate out
>>> the UI
> into a separate package + classpath.  This would allow us to
>>> package the
 UI
> as both a war with embedded jetty as a default option to run it;
>>> start
 from
> scratch with up to date versions of Jetty, Jersey/RESTEasy, and
>>> JAXB; and
> upgrade the different servers/components one at a time instead of
>>> all at
> once.  The DRPC server also uses the embedded jetty and exposes a
>>> REST
> interface, and that is going to be a harder one to tease out so it
>>> should
> probably be the last one to go.
> - Bobby
> 
>On Tuesday, February 23, 2016 3:40 AM, 伍翀(云邪) <
> wuchong...@alibaba-inc.com> wrote:
> 
> 
> Hi all, I’m planning to move UI/REST service and logviewer to Java,
 which
> means that we need to pick some alternatives for ring and hiccup.
> So the first thing is to pick up a REST framework.
> For the REST APIs, I think Jersey is a good choice (RESTEasy is fine
 too).
> It’s easy to develop and good performance.
> Now logviewer use hiccup to return HTML we build ourselves, but
>>> it’s hard
> to debug and maintain. So in my opinion, it’s better to replace it
>>> with
> static HTML + REST like regular UI.
> Please let me know what you think.
> – Jark Wu
> 
> 
> 
 
>> 
> 

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159784#comment-15159784
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53862294
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53862294
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+public static Map unauthorizedUserJson(String user) {
+return ImmutableMap.of(
+"error", "No Authorization",
+"errorMessage", String.format("User %s is not 
authorized.", user)
+

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159782#comment-15159782
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53861999
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53861999
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
+tmp.addFirst(uptime % div + divider[0]);
+uptime = uptime / div;
+} else {
+tmp.addFirst(uptime + divider[0]);
+}
+}
+}
+return Joiner.on(" ").join(tmp);
+}
+
+public static String prettyUptimeSec(String sec) {
+return prettyUptimeStr(sec, PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeSec(int secs) {
+return prettyUptimeStr(String.valueOf(secs), PRETTY_SEC_DIVIDERS);
+}
+
+public static String prettyUptimeMs(String ms) {
+return prettyUptimeStr(ms, PRETTY_MS_DIVIDERS);
+}
+
+public static String prettyUptimeMs(int ms) {
+return prettyUptimeStr(String.valueOf(ms), PRETTY_MS_DIVIDERS);
+}
+
+
+public static String urlFormat(String fmt, Object... args) {
+String[] argsEncoded = new String[args.length];
+for (int i = 0; i < args.length; i++) {
+argsEncoded[i] = URLEncoder.encode(String.valueOf(args[i]));
+}
+return String.format(fmt, argsEncoded);
+}
+
+public static String prettyExecutorInfo(ExecutorInfo e) {
+return "[" + e.get_task_start() + "-" + e.get_task_end() + "]";
+}
+
+public static Map unauthorizedUserJson(String user) {
+return ImmutableMap.of(
+"error", "No Authorization",
+"errorMessage", String.format("User %s is not 
authorized.", user)
+

Re: [DISCUSS] Java REST Framework adoption

[Parth Brahmbhatt]

+1 on DropWizard.

On 2/23/16, 2:02 PM, "Harsha"  wrote:

>-1 on spring boot or anything related to spring.
>This api is intended to be very simple powering UI and any rest clients
>interested in grabbing the metrics from the same api as UI does.
>
>Jersey is good and  dropwizard (http://www.dropwizard.io/0.9.2/docs/)
>has been a way to go for java REST api offlate. Underneath it uses
>jersey and one can run jetty server as well which is what we've as the
>UI and logviewer server.
>
>
>-Harsha
>
>On Tue, Feb 23, 2016, at 08:23 AM, Ravi Sharma wrote:
>> spring boot +
>> 
>> Ravi
>> 
>> On Tue, Feb 23, 2016 at 3:16 PM, Ankur Garg 
>>wrote:
>> 
>> > How about using Spring Boot & Jersey for writing this .  Spring Boot
>>will
>> > give us packaged  jar which once executed will bring up its own
>>embedded
>> > server (Jetty or Tomcat or some other ) . Although Spring Boot has
>>some
>> > disadvantages as well , but worth investigating this option too .
>> >
>> >
>> > Any thoughts??
>> >
>> > Thanks
>> > Ankur
>> >
>> > On Tue, Feb 23, 2016 at 8:20 PM, Bobby Evans
>>
>> > wrote:
>> >
>> > > Yes, we need to pick something.  I have used Jersey in the past and
>>I
>> > > think it is fairly decent.  I have never used RESTEasy, but it is
>>more or
>> > > less the same API, so either one is fine with me, but Jersey is my
>>vote
>> > > just because of experience.
>> > >
>> > > You should keep in mind that we are currently on a very old version
>>of
>> > > jetty, and I am not sure if newer libraries will work with it.  But
>>also
>> > > the old versions of ring and hiccup that we use don't support newer
>>jetty
>> > > versions either.
>> > >
>> > > I personally think that now would be a good time to separate out
>>the UI
>> > > into a separate package + classpath.  This would allow us to
>>package the
>> > UI
>> > > as both a war with embedded jetty as a default option to run it;
>>start
>> > from
>> > > scratch with up to date versions of Jetty, Jersey/RESTEasy, and
>>JAXB; and
>> > > upgrade the different servers/components one at a time instead of
>>all at
>> > > once.  The DRPC server also uses the embedded jetty and exposes a
>>REST
>> > > interface, and that is going to be a harder one to tease out so it
>>should
>> > > probably be the last one to go.
>> > >  - Bobby
>> > >
>> > > On Tuesday, February 23, 2016 3:40 AM, 伍翀(云邪) <
>> > > wuchong...@alibaba-inc.com> wrote:
>> > >
>> > >
>> > >  Hi all, I’m planning to move UI/REST service and logviewer to Java,
>> > which
>> > > means that we need to pick some alternatives for ring and hiccup.
>> > > So the first thing is to pick up a REST framework.
>> > > For the REST APIs, I think Jersey is a good choice (RESTEasy is fine
>> > too).
>> > > It’s easy to develop and good performance.
>> > > Now logviewer use hiccup to return HTML we build ourselves, but
>>it’s hard
>> > > to debug and maintain. So in my opinion, it’s better to replace it
>>with
>> > > static HTML + REST like regular UI.
>> > > Please let me know what you think.
>> > > – Jark Wu
>> > >
>> > >
>> > >
>> >
>

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159779#comment-15159779
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53861576
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
--- End diff --

It seems really odd to parse an int out of a Constant that we are setting.  
I think this is OK, but just really odd.


> port backtype.storm.ui.helpers to java
> --
>
> Key: STORM-1254
> URL: https://issues.apache.org/jira/browse/STORM-1254
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> Helper functions for the UI/HTTP



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53861576
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import org.apache.commons.lang.StringEscapeUtils;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.logging.filters.AccessLoggingFilter;
+import org.apache.storm.utils.Utils;
+import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.DispatcherType;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.nio.SelectChannelConnector;
+import org.eclipse.jetty.server.ssl.SslSocketConnector;
+import org.eclipse.jetty.servlet.FilterHolder;
+import org.eclipse.jetty.servlet.FilterMapping;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.servlets.CrossOriginFilter;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
+import org.json.simple.JSONValue;
+
+import javax.servlet.Servlet;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.URLEncoder;
+import java.util.*;
+
+public class UIHelpers {
+
+private static final String[][] PRETTY_SEC_DIVIDERS = {
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+private static final String[][] PRETTY_MS_DIVIDERS = {
+new String[]{"ms", "1000"},
+new String[]{"s", "60"},
+new String[]{"m", "60"},
+new String[]{"h", "24"},
+new String[]{"d", null}};
+
+public static String prettyUptimeStr(String val, String[][] dividers) {
+int uptime = Integer.parseInt(val);
+LinkedList tmp = new LinkedList<>();
+for (String[] divider : dividers) {
+if (uptime > 0) {
+if (divider[1] != null) {
+int div = Integer.parseInt(divider[1]);
--- End diff --

It seems really odd to parse an int out of a Constant that we are setting.  
I think this is OK, but just really odd.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159775#comment-15159775
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53861204
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
--- End diff --

We should not depend on clojure internals.  For keywords we should have 
something that can translate clojure maps/keywords into java objects, or just 
use those java object directly.


> port backtype.storm.ui.helpers to java
> --
>
> Key: STORM-1254
> URL: https://issues.apache.org/jira/browse/STORM-1254
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> Helper functions for the UI/HTTP



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53861204
  
--- Diff: storm-core/src/jvm/org/apache/storm/ui/UIHelpers.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.ui;
+
+import clojure.lang.Keyword;
+import clojure.lang.RT;
--- End diff --

We should not depend on clojure internals.  For keywords we should have 
something that can translate clojure maps/keywords into java objects, or just 
use those java object directly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1254]: port ui.helper to java

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53860859
  
--- Diff: storm-core/src/clj/org/apache/storm/ui/helpers.clj ---
@@ -46,197 +46,3 @@
   (fn [req]
--- End diff --

Should we move this someplace else?  Or perhaps inline it into the servers 
that use it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1254) port backtype.storm.ui.helpers to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159770#comment-15159770
 ] 

ASF GitHub Bot commented on STORM-1254:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1139#discussion_r53860859
  
--- Diff: storm-core/src/clj/org/apache/storm/ui/helpers.clj ---
@@ -46,197 +46,3 @@
   (fn [req]
--- End diff --

Should we move this someplace else?  Or perhaps inline it into the servers 
that use it?


> port backtype.storm.ui.helpers to java
> --
>
> Key: STORM-1254
> URL: https://issues.apache.org/jira/browse/STORM-1254
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Jark Wu
>  Labels: java-migration, jstorm-merger
>
> Helper functions for the UI/HTTP



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSS] Java REST Framework adoption

[Harsha]

-1 on spring boot or anything related to spring. 
This api is intended to be very simple powering UI and any rest clients
interested in grabbing the metrics from the same api as UI does.

Jersey is good and  dropwizard (http://www.dropwizard.io/0.9.2/docs/)
has been a way to go for java REST api offlate. Underneath it uses
jersey and one can run jetty server as well which is what we've as the
UI and logviewer server.


-Harsha

On Tue, Feb 23, 2016, at 08:23 AM, Ravi Sharma wrote:
> spring boot +
> 
> Ravi
> 
> On Tue, Feb 23, 2016 at 3:16 PM, Ankur Garg  wrote:
> 
> > How about using Spring Boot & Jersey for writing this .  Spring Boot will
> > give us packaged  jar which once executed will bring up its own embedded
> > server (Jetty or Tomcat or some other ) . Although Spring Boot has some
> > disadvantages as well , but worth investigating this option too .
> >
> >
> > Any thoughts??
> >
> > Thanks
> > Ankur
> >
> > On Tue, Feb 23, 2016 at 8:20 PM, Bobby Evans 
> > wrote:
> >
> > > Yes, we need to pick something.  I have used Jersey in the past and I
> > > think it is fairly decent.  I have never used RESTEasy, but it is more or
> > > less the same API, so either one is fine with me, but Jersey is my vote
> > > just because of experience.
> > >
> > > You should keep in mind that we are currently on a very old version of
> > > jetty, and I am not sure if newer libraries will work with it.  But also
> > > the old versions of ring and hiccup that we use don't support newer jetty
> > > versions either.
> > >
> > > I personally think that now would be a good time to separate out the UI
> > > into a separate package + classpath.  This would allow us to package the
> > UI
> > > as both a war with embedded jetty as a default option to run it; start
> > from
> > > scratch with up to date versions of Jetty, Jersey/RESTEasy, and JAXB; and
> > > upgrade the different servers/components one at a time instead of all at
> > > once.  The DRPC server also uses the embedded jetty and exposes a REST
> > > interface, and that is going to be a harder one to tease out so it should
> > > probably be the last one to go.
> > >  - Bobby
> > >
> > > On Tuesday, February 23, 2016 3:40 AM, 伍翀(云邪) <
> > > wuchong...@alibaba-inc.com> wrote:
> > >
> > >
> > >  Hi all, I’m planning to move UI/REST service and logviewer to Java,
> > which
> > > means that we need to pick some alternatives for ring and hiccup.
> > > So the first thing is to pick up a REST framework.
> > > For the REST APIs, I think Jersey is a good choice (RESTEasy is fine
> > too).
> > > It’s easy to develop and good performance.
> > > Now logviewer use hiccup to return HTML we build ourselves, but it’s hard
> > > to debug and maintain. So in my opinion, it’s better to replace it with
> > > static HTML + REST like regular UI.
> > > Please let me know what you think.
> > > – Jark Wu
> > >
> > >
> > >
> >

[jira] [Commented] (STORM-1571) Improvment Kafka Spout Time Metric

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159647#comment-15159647
 ] 

ASF GitHub Bot commented on STORM-1571:
---

Github user roshannaik commented on the pull request:

https://github.com/apache/storm/pull/1143#issuecomment-187905337
  
Also people may not care much about numbers less than 1 ms. 


> Improvment Kafka Spout Time Metric
> --
>
> Key: STORM-1571
> URL: https://issues.apache.org/jira/browse/STORM-1571
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-kafka
>Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6
> Environment: Mac 10.11.1 JDK 1.8.0_40
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch
>
>
> Use System.currentTimeMillis() to calculation time interval is better than 
> System.nanoTime() 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1552) Fix topology event sampling log directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159631#comment-15159631
 ] 

ASF GitHub Bot commented on STORM-1552:
---

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1127


> Fix topology event sampling log directory 
> --
>
> Key: STORM-1552
> URL: https://issues.apache.org/jira/browse/STORM-1552
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0
>Reporter: Arun Mahadevan
>Assignee: Arun Mahadevan
>
> Run a topology and enable event inspection by clicking "Debug" from UI. The 
> events are logged under 
> "storm-local/workers-artifacts/{storm-id}/port/events.log". In the spout/bolt 
> details page, the "events" link does not display the log file.
> The events.log should be kept under 
> logs/workers-artifacts/{storm-id}/{port}/events.log so that its viewable via 
> logviewer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1552) Fix topology event sampling log directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159630#comment-15159630
 ] 

ASF GitHub Bot commented on STORM-1552:
---

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1127#issuecomment-187899371
  
+1


> Fix topology event sampling log directory 
> --
>
> Key: STORM-1552
> URL: https://issues.apache.org/jira/browse/STORM-1552
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0
>Reporter: Arun Mahadevan
>Assignee: Arun Mahadevan
>
> Run a topology and enable event inspection by clicking "Debug" from UI. The 
> events are logged under 
> "storm-local/workers-artifacts/{storm-id}/port/events.log". In the spout/bolt 
> details page, the "events" link does not display the log file.
> The events.log should be kept under 
> logs/workers-artifacts/{storm-id}/{port}/events.log so that its viewable via 
> logviewer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1552] Fix topology event sampling log d...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1127


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1552] Fix topology event sampling log d...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1127#issuecomment-187899371
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: MINOR: fix typo in CHANGELOG

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1039


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1479) use a simple implemention for IntSerializer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159594#comment-15159594
 ] 

ASF GitHub Bot commented on STORM-1479:
---

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1021


> use a simple implemention for IntSerializer
> ---
>
> Key: STORM-1479
> URL: https://issues.apache.org/jira/browse/STORM-1479
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-sql
>Reporter: Xin Wang
>Assignee: Xin Wang
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (STORM-1479) use a simple implemention for IntSerializer

[Robert Joseph Evans (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Joseph Evans resolved STORM-1479.

   Resolution: Fixed
Fix Version/s: 2.0.0

Thanks [~vesense],

I merged this into master.

> use a simple implemention for IntSerializer
> ---
>
> Key: STORM-1479
> URL: https://issues.apache.org/jira/browse/STORM-1479
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-sql
>Reporter: Xin Wang
>Assignee: Xin Wang
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1479] use a simple implemention for Int...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1021


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (STORM-1255) port backtype.storm.utils-test to java

[Robert Joseph Evans (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Joseph Evans resolved STORM-1255.

   Resolution: Fixed
Fix Version/s: 2.0.0

Thanks [~abellina],

I merged this into master.  Keep up the good work.

> port backtype.storm.utils-test to java
> --
>
> Key: STORM-1255
> URL: https://issues.apache.org/jira/browse/STORM-1255
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: rm-core, storm-core
>Reporter: Robert Joseph Evans
>Assignee: Alessandro Bellina
>  Labels: java-migration, jstorm-merger
> Fix For: 2.0.0
>
>
> junit test migration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1255) port backtype.storm.utils-test to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159578#comment-15159578
 ] 

ASF GitHub Bot commented on STORM-1255:
---

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1114


> port backtype.storm.utils-test to java
> --
>
> Key: STORM-1255
> URL: https://issues.apache.org/jira/browse/STORM-1255
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: rm-core, storm-core
>Reporter: Robert Joseph Evans
>Assignee: Alessandro Bellina
>  Labels: java-migration, jstorm-merger
>
> junit test migration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1255: port storm_utils.clj to java and s...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1114


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1255) port backtype.storm.utils-test to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159567#comment-15159567
 ] 

ASF GitHub Bot commented on STORM-1255:
---

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/1114#issuecomment-187881681
  
+1 looks good to me


> port backtype.storm.utils-test to java
> --
>
> Key: STORM-1255
> URL: https://issues.apache.org/jira/browse/STORM-1255
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: rm-core, storm-core
>Reporter: Robert Joseph Evans
>Assignee: Alessandro Bellina
>  Labels: java-migration, jstorm-merger
>
> junit test migration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1255: port storm_utils.clj to java and s...

[revans2]

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/1114#issuecomment-187881681
  
+1 looks good to me


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159454#comment-15159454
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/1081#issuecomment-187846619
  
OK I made it all of the way through.  Things look fairly good, but there 
are a few places that need to be updated/fixed.


> port backtype.storm.cluster to java
> ---
>
> Key: STORM-1273
> URL: https://issues.apache.org/jira/browse/STORM-1273
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: John Fang
>  Labels: java-migration, jstorm-merger
>
> current state of the cluster (Some of this moves to java as a part of 
> heartbeat server)
> https://github.com/apache/storm/tree/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/cluster
>  as an example



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/1081#issuecomment-187846619
  
OK I made it all of the way through.  Things look fairly good, but there 
are a few places that need to be updated/fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159451#comment-15159451
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53832190
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53832190
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.debug("Received event {} : {} : {}", 
state, type, path);
+

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159445#comment-15159445
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53831586
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53831586
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.debug("Received event {} : {} : {}", 
state, type, path);
+

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53831437
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.debug("Received event {} : {} : {}", 
state, type, path);
+

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159442#comment-15159442
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53831437
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159438#comment-15159438
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53830934
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
--- End diff --

It might be nice to reuse the same watcher callback instance, just so we 
don't have two identical anonymous classes.


> port backtype.storm.cluster to java
> ---
>
> Key: STORM-1273
> URL: 

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53830934
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
+}
+
+if (!type.equals(Watcher.Event.EventType.None)) {
+for (Map.Entry e : 
callbacks.entrySet()) {
+ZKStateChangedCallback fn = e.getValue();
+fn.changed(type, path);
+}
+}
+}
+}
+});
+if (isNimbus) {
+zkReader = mkZk(new WatcherCallBack() {
--- End diff --

It might be nice to reuse the same watcher callback instance, just so we 
don't have two identical anonymous classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159434#comment-15159434
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53830685
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
--- End diff --

This info message was not logged before. Could we at least make it debug.


> port backtype.storm.cluster to java
> ---
>
> Key: STORM-1273
> URL: https://issues.apache.org/jira/browse/STORM-1273
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: John Fang
>  Labels: java-migration, jstorm-merger
>
> current state of the cluster (Some of this moves to java as a part of 
> heartbeat server)
> https://github.com/apache/storm/tree/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/cluster
>  as an example



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53830685
  
--- Diff: storm-core/src/jvm/org/apache/storm/cluster/ZKStateStorage.java 
---
@@ -0,0 +1,270 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.Config;
+import org.apache.storm.callback.DefaultWatcherCallBack;
+import org.apache.storm.callback.WatcherCallBack;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+public class ZKStateStorage implements IStateStorage {
+
+private static Logger LOG = 
LoggerFactory.getLogger(ZKStateStorage.class);
+
+private ConcurrentHashMap callbacks = 
new ConcurrentHashMap();
+private CuratorFramework zkWriter;
+private CuratorFramework zkReader;
+private AtomicBoolean active;
+
+private boolean isNimbus;
+private Map authConf;
+private Map conf;
+
+public ZKStateStorage(Map conf, Map authConf, 
List acls, ClusterStateContext context) throws Exception {
+this.conf = conf;
+this.authConf = authConf;
+if (context.getDaemonType().equals(DaemonType.NIMBUS))
+this.isNimbus = true;
+
+// just mkdir STORM_ZOOKEEPER_ROOT dir
+CuratorFramework zkTemp = mkZk();
+String rootPath = 
String.valueOf(conf.get(Config.STORM_ZOOKEEPER_ROOT));
+Zookeeper.mkdirs(zkTemp, rootPath, acls);
+zkTemp.close();
+
+active = new AtomicBoolean(true);
+zkWriter = mkZk(new WatcherCallBack() {
+@Override
+public void execute(Watcher.Event.KeeperState state, 
Watcher.Event.EventType type, String path) {
+if (active.get()) {
+if 
(!(state.equals(Watcher.Event.KeeperState.SyncConnected))) {
+LOG.warn("Received event {} : {}: {} with 
disconnected Zookeeper.", state, type, path);
+} else {
+LOG.info("Received event {} : {} : {}", state, 
type, path);
--- End diff --

This info message was not logged before. Could we at least make it debug.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1273) port backtype.storm.cluster to java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159417#comment-15159417
 ] 

ASF GitHub Bot commented on STORM-1273:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53829764
  
--- Diff: 
storm-core/src/jvm/org/apache/storm/cluster/StormClusterStateImpl.java ---
@@ -0,0 +1,687 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import clojure.lang.*;
+import org.apache.commons.lang.StringUtils;
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.generated.*;
+import org.apache.storm.nimbus.NimbusInfo;
+import org.apache.storm.utils.Time;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+
+public class StormClusterStateImpl implements IStormClusterState {
+
+private static Logger LOG = 
LoggerFactory.getLogger(StormClusterStateImpl.class);
+
+private IStateStorage stateStorage;
+
+private ConcurrentHashMap assignmentInfoCallback;
+private ConcurrentHashMap 
assignmentInfoWithVersionCallback;
+private ConcurrentHashMap assignmentVersionCallback;
+private AtomicReference supervisorsCallback;
+// we want to reigister a topo directory getChildren callback for all 
workers of this dir
+private ConcurrentHashMap backPressureCallback;
+private AtomicReference assignmentsCallback;
+private ConcurrentHashMap stormBaseCallback;
+private AtomicReference blobstoreCallback;
+private ConcurrentHashMap credentialsCallback;
+private ConcurrentHashMap logConfigCallback;
+
+private List acls;
+private String stateId;
+private boolean solo;
+
+public StormClusterStateImpl(IStateStorage StateStorage, List 
acls, ClusterStateContext context, boolean solo) throws Exception {
+
+this.stateStorage = StateStorage;
+this.solo = solo;
+this.acls = acls;
+
+assignmentInfoCallback = new ConcurrentHashMap<>();
+assignmentInfoWithVersionCallback = new ConcurrentHashMap<>();
+assignmentVersionCallback = new ConcurrentHashMap<>();
+supervisorsCallback = new AtomicReference<>();
+backPressureCallback = new ConcurrentHashMap<>();
+assignmentsCallback = new AtomicReference<>();
+stormBaseCallback = new ConcurrentHashMap<>();
+credentialsCallback = new ConcurrentHashMap<>();
+logConfigCallback = new ConcurrentHashMap<>();
+blobstoreCallback = new AtomicReference<>();
+
+stateId = this.stateStorage.register(new ZKStateChangedCallback() {
+
+public void changed(Watcher.Event.EventType type, String path) 
{
+List toks = Zookeeper.tokenizePath(path);
+int size = toks.size();
+if (size >= 1) {
+String root = toks.get(0);
+if (root.equals(ClusterUtils.ASSIGNMENTS_ROOT)) {
+if (size == 1) {
+// set null and get the old value
+

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53829764
  
--- Diff: 
storm-core/src/jvm/org/apache/storm/cluster/StormClusterStateImpl.java ---
@@ -0,0 +1,687 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import clojure.lang.*;
+import org.apache.commons.lang.StringUtils;
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.generated.*;
+import org.apache.storm.nimbus.NimbusInfo;
+import org.apache.storm.utils.Time;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+
+public class StormClusterStateImpl implements IStormClusterState {
+
+private static Logger LOG = 
LoggerFactory.getLogger(StormClusterStateImpl.class);
+
+private IStateStorage stateStorage;
+
+private ConcurrentHashMap assignmentInfoCallback;
+private ConcurrentHashMap 
assignmentInfoWithVersionCallback;
+private ConcurrentHashMap assignmentVersionCallback;
+private AtomicReference supervisorsCallback;
+// we want to reigister a topo directory getChildren callback for all 
workers of this dir
+private ConcurrentHashMap backPressureCallback;
+private AtomicReference assignmentsCallback;
+private ConcurrentHashMap stormBaseCallback;
+private AtomicReference blobstoreCallback;
+private ConcurrentHashMap credentialsCallback;
+private ConcurrentHashMap logConfigCallback;
+
+private List acls;
+private String stateId;
+private boolean solo;
+
+public StormClusterStateImpl(IStateStorage StateStorage, List 
acls, ClusterStateContext context, boolean solo) throws Exception {
+
+this.stateStorage = StateStorage;
+this.solo = solo;
+this.acls = acls;
+
+assignmentInfoCallback = new ConcurrentHashMap<>();
+assignmentInfoWithVersionCallback = new ConcurrentHashMap<>();
+assignmentVersionCallback = new ConcurrentHashMap<>();
+supervisorsCallback = new AtomicReference<>();
+backPressureCallback = new ConcurrentHashMap<>();
+assignmentsCallback = new AtomicReference<>();
+stormBaseCallback = new ConcurrentHashMap<>();
+credentialsCallback = new ConcurrentHashMap<>();
+logConfigCallback = new ConcurrentHashMap<>();
+blobstoreCallback = new AtomicReference<>();
+
+stateId = this.stateStorage.register(new ZKStateChangedCallback() {
+
+public void changed(Watcher.Event.EventType type, String path) 
{
+List toks = Zookeeper.tokenizePath(path);
+int size = toks.size();
+if (size >= 1) {
+String root = toks.get(0);
+if (root.equals(ClusterUtils.ASSIGNMENTS_ROOT)) {
+if (size == 1) {
+// set null and get the old value
+issueCallback(assignmentsCallback);
+} else {
+issueMapCallback(assignmentInfoCallback, 
toks.get(1));
+issueMapCallback(assignmentVersionCallback, 
toks.get(1));
+ 

[GitHub] storm pull request: [STORM-1273] port backtype.storm.cluster to ja...

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1081#discussion_r53827548
  
--- Diff: 
storm-core/src/jvm/org/apache/storm/cluster/StormClusterStateImpl.java ---
@@ -0,0 +1,687 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.cluster;
+
+import clojure.lang.*;
+import org.apache.commons.lang.StringUtils;
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.state.*;
+import org.apache.curator.framework.state.ConnectionState;
+import org.apache.storm.callback.ZKStateChangedCallback;
+import org.apache.storm.generated.*;
+import org.apache.storm.nimbus.NimbusInfo;
+import org.apache.storm.utils.Time;
+import org.apache.storm.utils.Utils;
+import org.apache.storm.zookeeper.Zookeeper;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+
+public class StormClusterStateImpl implements IStormClusterState {
+
+private static Logger LOG = 
LoggerFactory.getLogger(StormClusterStateImpl.class);
+
+private IStateStorage stateStorage;
+
+private ConcurrentHashMap assignmentInfoCallback;
+private ConcurrentHashMap 
assignmentInfoWithVersionCallback;
+private ConcurrentHashMap assignmentVersionCallback;
+private AtomicReference supervisorsCallback;
+// we want to reigister a topo directory getChildren callback for all 
workers of this dir
+private ConcurrentHashMap backPressureCallback;
+private AtomicReference assignmentsCallback;
+private ConcurrentHashMap stormBaseCallback;
+private AtomicReference blobstoreCallback;
+private ConcurrentHashMap credentialsCallback;
+private ConcurrentHashMap logConfigCallback;
+
+private List acls;
+private String stateId;
+private boolean solo;
+
+public StormClusterStateImpl(IStateStorage StateStorage, List 
acls, ClusterStateContext context, boolean solo) throws Exception {
+
+this.stateStorage = StateStorage;
+this.solo = solo;
+this.acls = acls;
+
+assignmentInfoCallback = new ConcurrentHashMap<>();
+assignmentInfoWithVersionCallback = new ConcurrentHashMap<>();
+assignmentVersionCallback = new ConcurrentHashMap<>();
+supervisorsCallback = new AtomicReference<>();
+backPressureCallback = new ConcurrentHashMap<>();
+assignmentsCallback = new AtomicReference<>();
+stormBaseCallback = new ConcurrentHashMap<>();
+credentialsCallback = new ConcurrentHashMap<>();
+logConfigCallback = new ConcurrentHashMap<>();
+blobstoreCallback = new AtomicReference<>();
+
+stateId = this.stateStorage.register(new ZKStateChangedCallback() {
+
+public void changed(Watcher.Event.EventType type, String path) 
{
+List toks = Zookeeper.tokenizePath(path);
+int size = toks.size();
+if (size >= 1) {
+String root = toks.get(0);
+if (root.equals(ClusterUtils.ASSIGNMENTS_ROOT)) {
+if (size == 1) {
+// set null and get the old value
+issueCallback(assignmentsCallback);
+} else {
+issueMapCallback(assignmentInfoCallback, 
toks.get(1));
+issueMapCallback(assignmentVersionCallback, 
toks.get(1));
+