[GitHub] storm pull request #2061: STORM-2469: fix integration-test dependencies issu...

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2061 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #2064: STORM-1114: Handle race condition in Storm/Trident transa...

2017-04-12 Thread satishd
Github user satishd commented on the issue: https://github.com/apache/storm/pull/2064 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] storm pull request #2053: [STORM-2455] Expose the window start and end times...

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2053 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #2064: STORM-1114: Handle race condition in Storm/Trident transa...

2017-04-12 Thread vesense
Github user vesense commented on the issue: https://github.com/apache/storm/pull/2064 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread liu-zhaokun
Github user liu-zhaokun closed the pull request at: https://github.com/apache/storm/pull/2062 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] storm pull request #2066: [STORM-2472] kafkaspout should work normally in ke...

2017-04-12 Thread liu-zhaokun
GitHub user liu-zhaokun opened a pull request: https://github.com/apache/storm/pull/2066 [STORM-2472] kafkaspout should work normally in kerberos mode with kafka 0.9.x API [https://issues.apache.org/jira/browse/STORM-2472](url) storm 1.0.x-branch didn't support kafkaspout to

Re: Apache Storm/JStorm Runner(s) for Apache Beam

2017-04-12 Thread Davor Bonaci
This is a great discussion; thanks everyone. >From my perspective, the functionality to execute pipelines on both Storm and JStorm is very welcome and a big step forward for Beam. I'm not an expert on the Storm/JStorm differences, but one vs. two runners discussion sounds more like a packaging /

[GitHub] storm pull request #2030: STORM-2435: Logging levels and consistency with co...

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2030 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #2063: [STORM-2471] Add metric for thread count.

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2063 @jacobtolar please remove the line from the changelog. It is too hard to keep it in sync while the code is reviewed. We will add it back in when we merge it in. --- If your project is set up for

[GitHub] storm issue #2064: STORM-1114: Handle race condition in Storm/Trident transa...

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2064 @ptgoetz and yes lets please do port this back --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] storm pull request #2065: STORM-832: Allow config validation to be used by p...

2017-04-12 Thread revans2
GitHub user revans2 opened a pull request: https://github.com/apache/storm/pull/2065 STORM-832: Allow config validation to be used by plugins/etc. I did this because hard coding the usage of DaemonConf.java into a few places just didn't look right, and I thought it would be better

[GitHub] storm issue #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/storm/pull/2062 Hello, @harshach Yes,you are right.But I found there isn't 'sasl.jaas.config' in kafka 0.9.x, so I will create a new PR for storm 1.0.x-branch. --- If your project is set up for it, you

[GitHub] storm pull request #2024: STORM-2349: Add one RocketMQ plugin for the Apache...

2017-04-12 Thread hustfxj
Github user hustfxj commented on a diff in the pull request: https://github.com/apache/storm/pull/2024#discussion_r111294519 --- Diff: external/storm-rocketmq/src/main/java/org/apache/storm/rocketmq/spout/RocketMQSpout.java --- @@ -0,0 +1,177 @@ +/** + * Licensed to the

[GitHub] storm pull request #2024: STORM-2349: Add one RocketMQ plugin for the Apache...

2017-04-12 Thread hustfxj
Github user hustfxj commented on a diff in the pull request: https://github.com/apache/storm/pull/2024#discussion_r111292663 --- Diff: external/storm-rocketmq/src/main/java/org/apache/storm/rocketmq/spout/RocketMQSpout.java --- @@ -0,0 +1,179 @@ +/** + * Licensed to the

[GitHub] storm pull request #2057: [STORM-2465] modify storm-redis's READEME.md and u...

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2057 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm pull request #2056: [STORM-2464] update storm-mongodb.md

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2056 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #2058: [STORM-2466] The example of jaas.conf in jaas_kerberos.co...

2017-04-12 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/storm/pull/2058 Hello, @vesense Could you help me to merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] storm issue #2057: [STORM-2465] modify storm-redis's READEME.md and update s...

2017-04-12 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/storm/pull/2057 Hello, @vesense Could you help me to merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] storm issue #2056: [STORM-2464] update storm-mongodb.md

2017-04-12 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/storm/pull/2056 Hello,@vesense Could you help me to merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] storm issue #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread harshach
Github user harshach commented on the issue: https://github.com/apache/storm/pull/2062 @liu-zhaokun KafkaSpoutConfig offers this method to add security-related configs

[GitHub] storm issue #2054: STORM-2462 Adding regex mapper to KerberosPrincipalToLoca...

2017-04-12 Thread ambud
Github user ambud commented on the issue: https://github.com/apache/storm/pull/2054 @harshach got it, let me work on the implementation and update the code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] storm pull request #2064: STORM-1114: Handle race condition in Storm/Trident...

2017-04-12 Thread ptgoetz
GitHub user ptgoetz opened a pull request: https://github.com/apache/storm/pull/2064 STORM-1114: Handle race condition in Storm/Trident transactional stat… Handle race condition in Storm/Trident transactional state when ZK nodes have already been created/deleted. This impacts

[GitHub] storm issue #2029: STORM-2379: update for Elasticsearch 2.

2017-04-12 Thread harshach
Github user harshach commented on the issue: https://github.com/apache/storm/pull/2029 @hmcc can you please up-merge your patch. I'll merge it into master & 1.x-branch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] storm issue #2054: STORM-2462 Adding regex mapper to KerberosPrincipalToLoca...

2017-04-12 Thread harshach
Github user harshach commented on the issue: https://github.com/apache/storm/pull/2054 @ambud we should add the auth_to_local rules an make that as part of storm.yaml config option . Adding regex will not be helpful here. --- If your project is set up for it, you can reply to this

[GitHub] storm issue #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread harshach
Github user harshach commented on the issue: https://github.com/apache/storm/pull/2062 @liu-zhaokun More details are documented here http://kafka.apache.org/documentation.html#security_sasl . We don't need these changes on storm-kafka-client side --- If your project is set up for

[GitHub] storm issue #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread harshach
Github user harshach commented on the issue: https://github.com/apache/storm/pull/2062 @liu-zhaokun with the latest kafka consumer APIs one can pass a keytab and principal via consumer or producer properties. Even before that they can pass the jaas config via JVM param and set

[GitHub] storm issue #2063: [STORM-2471] Add metric for thread count.

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2063 +1 looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] storm pull request #2063: [STORM-2471] Add metric for thread count.

2017-04-12 Thread jacobtolar
GitHub user jacobtolar opened a pull request: https://github.com/apache/storm/pull/2063 [STORM-2471] Add metric for thread count. * Add new metric * Java8-ify two existing metrics You can merge this pull request into a Git repository by running: $ git pull

[GitHub] storm issue #2054: STORM-2462 Adding regex mapper to KerberosPrincipalToLoca...

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2054 I have been thinking about this, and I am not sure if a REGEX is really what we want. Hadoop does this in a semi-standard way using auth_to_local rules like PAM does. It might be worth

[GitHub] storm pull request #2054: STORM-2462 Adding regex mapper to KerberosPrincipa...

2017-04-12 Thread revans2
Github user revans2 commented on a diff in the pull request: https://github.com/apache/storm/pull/2054#discussion_r91074 --- Diff: storm-client/src/jvm/org/apache/storm/security/auth/RegexKerberosPrincipalToLocal.java --- @@ -0,0 +1,56 @@ +/** + * Licensed to the

[GitHub] storm pull request #2059: STORM-2463: fix DRPCTest.testDequeueAfterTimeout t...

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2059 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #2060: STORM-2468: Remove clojure from storm-client

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2060 @knusbaum please take a look at this. There were a number of changes here. Some to replace functionality being used from clojure runtime with java code. Some of it was moving/deduplicating

[GitHub] storm issue #2060: STORM-2468: Remove clojure from storm-client

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2060 Now that STORM-2447 is in I have rebased this on master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] storm issue #2059: STORM-2463: fix DRPCTest.testDequeueAfterTimeout test fai...

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2059 OK Still +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

Re: Fwd: Distribution of workers across cluster with Resource Aware Scheduler

2017-04-12 Thread Bobby Evans
No there is currently no way to do this with the default strategy.  You can write your own resource aware strategy to place the workers differently though.    You just need to write a class that implements IStrategy.  It can also be selected on a per topology basis. The main reason for doing

[GitHub] storm issue #2049: STORM-2448: Add in Storm and JDK versions when submitting...

2017-04-12 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2049 @ptgoetz @HeartSaVioR I would appreciate a review of this. I would like to know what people think about it. If it is good I am happy to add in more documentation and to rip out the rename hack

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread srdo
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2062#discussion_r53241 --- Diff: storm-client/src/jvm/org/apache/storm/Config.java --- @@ -1481,6 +1481,12 @@ @isPositiveNumber public static final String

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread srdo
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2062#discussion_r53449 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpoutSecurity.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread srdo
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2062#discussion_r53380 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpoutSecurity.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread srdo
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2062#discussion_r53122 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpoutSecurity.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread srdo
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2062#discussion_r52884 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpoutSecurity.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread srdo
Github user srdo commented on a diff in the pull request: https://github.com/apache/storm/pull/2062#discussion_r52735 --- Diff: conf/defaults.yaml --- @@ -314,4 +314,7 @@ worker.metrics: "CGroupCpu": "org.apache.storm.metric.cgroup.CGroupCpu"

[GitHub] storm issue #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/storm/pull/2062 Users should provide the path to jaas.conf which includes a section named "KafkaClient" in storm.yaml or . --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] storm pull request #2062: [STORM-2470] kafkaspout should support kerberos

2017-04-12 Thread liu-zhaokun
GitHub user liu-zhaokun opened a pull request: https://github.com/apache/storm/pull/2062 [STORM-2470] kafkaspout should support kerberos [https://issues.apache.org/jira/browse/STORM-2470](url) kafkaspout should work normally when the cluster(storm and kafka) is in kerberos

[GitHub] storm pull request #2047: STORM-2447: add in storm local to avoid having ser...

2017-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2047 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #2047: STORM-2447: add in storm local to avoid having server on ...

2017-04-12 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/2047 @revans2 Sorry I have been busy with other works. Code change looks good, and manually tested with ExclamationTopology for local mode and SQL (order_tsv_csv) for cluster mode. Great

[GitHub] storm issue #2053: [STORM-2455] Expose the window start and end timestamp in...

2017-04-12 Thread satishd
Github user satishd commented on the issue: https://github.com/apache/storm/pull/2053 +1 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] storm pull request #2061: STORM-2469: fix integration-test dependencies issu...

2017-04-12 Thread vesense
GitHub user vesense opened a pull request: https://github.com/apache/storm/pull/2061 STORM-2469: fix integration-test dependencies issue This is the hot fix for integration-test dependencies issue * add storm-server. since __the ExclamationTopology depends on