[GitHub] storm pull request: [STORM-768] Support JDK 8 compile and runtime.
Github user Lewuathe closed the pull request at: https://github.com/apache/storm/pull/519 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-768] Support JDK 8 compile and runtime.
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/519#issuecomment-173802479 Sure, I'll close this. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-768] Support JDK 8 compile and runtime.
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/519 [STORM-768] Support JDK 8 compile and runtime. - [x] Unittests (exclude storm-hive) - [x] nimbus server - [x] supervisor server - [x] drpc server - [x] Topology submit and running (storm.starter.ExclamationTopology) storm-hive unittest fails even if we use JDK 1.7. It might be due to JDK 1.8. It looks fine to support JDK 1.8 here. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm update-jdk8 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/519.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #519 commit a4032cfecd5eb3525871c361b39bf89998a04560 Author: lewuathe lewua...@me.com Date: 2015-04-13T11:24:15Z Support JDK 1.8 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-767] Add repository declaration for pen...
Github user Lewuathe closed the pull request at: https://github.com/apache/storm/pull/518 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: Add repository declaration for pentaho-aggdesi...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/518 Add repository declaration for pentaho-aggdesigner Building `storm-hive` fails due to unresolved dependencies. It is necessary to add a reference to springsource repository. ``` [ERROR] Failed to execute goal on project storm-hive: Could not resolve dependencies for project org.apache.storm:storm-hive:jar:0.11.0-SNAPSHOT: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.3-jhyde in repo.jenkins-ci.org (http://repo.jenkins-ci.org/public/) - [Help 1] ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm STORM-767 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/518.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #518 commit b69fcc9a5d9b7fe1dc3577686dc99f0252b023a5 Author: Lewuathe lewua...@me.com Date: 2015-04-12T05:21:24Z Add repository declaration for pentaho-aggdesigner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/442#issuecomment-91247461 I updated to remove unrelated lines and check the state of new metrics. Can you review it? Thank you so much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/442#issuecomment-90081039 @Parth-Brahmbhatt Thank you for advice. I merged master and updated this patch. Can you review it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/442#issuecomment-89586916 @revans2 Thank you for supporting. I'll change this metrics as optional and thrift interface. So all I have to do it 2 points, @Parth-Brahmbhatt ? If there is any other points I should care about, please let me know. * Store metrics in thrift in ZooKeeper. * Change serialization as optional metrics. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-695] storm CLI tool reports zero exit c...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/456#issuecomment-83543346 @revans2 @HeartSaVioR Thank you for reviewing. We should not distinguish error types with exit code. So JVM return code informs us the occurrence of some exception, I'll update to return exit code 1 when any exception has been occurred. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-681] Auto insert license header with ge...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/436#issuecomment-7906 @revans2 Thank you for reviewing. I exclude generated files in this patch. And I'll update. @knusbaum Yes. We should change `thrift7` to `thrift`. Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-695] storm CLI tool reports zero exit c...
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/storm/pull/456#discussion_r25820615 --- Diff: bin/storm --- @@ -183,7 +209,16 @@ def exec_storm_class(klass, jvmtype=-server, jvmopts=[], extrajars=[], args=[] os.spawnvp(os.P_WAIT, JAVA_CMD, all_args) else: # handling whitespaces in JAVA_CMD -sub.call(all_args) +try: +ret = sub.check_output(all_args, stderr=sub.STDOUT) +print(ret) --- End diff -- Because `check_output` absolutes all outout from command. It is necessary to `print` alternatively to see stdout. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-695] storm CLI tool reports zero exit c...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/456#issuecomment-77264323 @miguno Thank you for comment. I tested the error handling and exit code of current storm command. And I found when any exception has occurred `check_output` returns non-zero code(just only 1). So with this patch, storm command can detect the occurrence of exception. But error exit code always be 1, so in order to specify what exception is occurred, we must parse stdout and stderr. This is what I was trying to do with this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-695] storm CLI tool reports zero exit c...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/456#issuecomment-77156141 However the topology is submitted with `storm shell`, this cannot track exception because `spawnvp` seems to return no stdout and stderr. I don't know the reason why in this case `spawnvp` is called. Can I migrate these two cases into `subprocess.check_output`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-695] storm CLI tool reports zero exit c...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/456 [STORM-695] storm CLI tool reports zero exit code on error scenario Handling `java.lang.RuntimeException` and `NotAliveException` in storm command. This feature can be useful for handling a exit code returned from shell command. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm STORM-695 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/456.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #456 commit 7e36ce579f0dcbcf4dfe5942b906a6e94a1815e8 Author: lewuathe lewua...@me.com Date: 2015-03-04T13:14:51Z [STORM-695] storm CLI tool reports zero exit code on error scenario --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/442#issuecomment-76501848 @revans2 Thank you so much for reviewing. I updated to solve above exception and include builtin metrics. Please check it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/storm/pull/442#discussion_r25424494 --- Diff: storm-core/src/clj/backtype/storm/stats.clj --- @@ -164,13 +164,13 @@ (def COMMON-FIELDS [:emitted :transferred]) (defrecord CommonStats [emitted transferred rate]) -(def BOLT-FIELDS [:acked :failed :process-latencies :executed :execute-latencies]) +(def BOLT-FIELDS [:acked :failed :process-latencies :executed :execute-latencies :deserialize-time]) ;;acked and failed count individual tuples -(defrecord BoltExecutorStats [common acked failed process-latencies executed execute-latencies]) +(defrecord BoltExecutorStats [common acked failed process-latencies executed execute-latencies deserialize-time]) -(def SPOUT-FIELDS [:acked :failed :complete-latencies]) +(def SPOUT-FIELDS [:acked :failed :complete-latencies :deserialize-time]) --- End diff -- This might be the point I didn't fully understand. In [here](https://github.com/apache/storm/blob/master/storm-core/src/clj/backtype/storm/daemon/executor.clj#L413-L429), all tuple to even spout can go through task-receiver right? I think these values are sent as bolt ones. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/442 [STORM-671] Measure tuple serialization/deserialization latency. Measure deserialization time in spout and bolt. These values can be see on dashboard UI. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm deserialization-time Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/442.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #442 commit f5bc66f32b8bbb4d770ebb245ba6590d97ff2c1d Author: lewuathe lewua...@me.com Date: 2015-02-25T10:02:32Z [STORM-671] Measure tuple serialization/deserialization latency. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-681] Auto insert license header with ge...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/436 [STORM-681] Auto insert license header with genthrift.sh Current genthrift.sh does not insert license headers into generated source codes. These java codes and python codes should have license headers. And documentation about this command. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm auto-insert-license-declaration Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/436.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #436 commit 0ba5bc1bea44414776efa97f3e66ba08e7384260 Author: lewuathe lewua...@me.com Date: 2015-02-19T13:13:08Z [STORM-681] Auto insert license header with genthrift.sh --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-681] Auto insert license header with ge...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/436#issuecomment-75162992 @Parth-Brahmbhatt Yes, I'm not sure too. But I think it is better to include the files generated by new `genthrift.sh` script in order to keep and confirm whole source code consistency. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-669] Replace links with ones to latest ...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/425 [STORM-669] Replace links with ones to latest api document Replace links to old api documents with new ones. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm link-to-latest-javadoc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/425.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #425 commit 2be8acd69b6bd89425539231b4b79a10e831e2d7 Author: lewuathe lewua...@me.com Date: 2015-02-10T13:40:21Z [STORM-669] Replace links with ones to latest api document --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-653] missing DRPC HTTP port in SECURITY...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/413 [STORM-653] missing DRPC HTTP port in SECURITY.md There is no description about DRPC HTTP port in SECURITY.md. This port is used by DRPC Client as default 3774 You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm drpc-port-on-security Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/413.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #413 commit 5ff14142995a18a8b6fd81b126f610b1aa0bd0f4 Author: lewuathe lewua...@me.com Date: 2015-02-03T09:23:53Z [STORM-653] missing DRPC HTTP port in SECURITY.md --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-72618892 This ticket was migrated to [STORM-650](https://issues.apache.org/jira/browse/STORM-650?jql=project%20%3D%20STORM). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-639] storm-maven-plugin not found
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/403#issuecomment-72356826 It seems that there is no way to install `storm-maven-plugin` in local repository before compiling `storm-core` because with `compile` phase there is no jar files enough to install. So I think there are two options. 1. Update documents to install `storm-maven-plugin` before compiling `storm-core` project. 2. Deploy `storm-maven-plugin` on remote repository. What do you think? If someone has any other ideas, please let me know. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-639] storm-maven-plugin not found
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/403#issuecomment-72318223 @d2r Thank you for reporting. I'll fix to succeed compile with `mvn compile`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-639] storm-maven-plugin not found
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/403 [STORM-639] storm-maven-plugin not found maven compile is fail due to missing storm-maven-plugin. This plugin should be installed before compiling storm-core project. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm maven-plugin-failure Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/403.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #403 commit 715dfb9a430990835f531c316c93e0b99df626c8 Author: lewuathe lewua...@me.com Date: 2015-01-30T15:08:55Z [STORM-639] storm-maven-plugin not found --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-630] Support for Clojure 1.6.0
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/390 [STORM-630] Support for Clojure 1.6.0 Through upgrading clojure 1.6.0, there are name space conflicts in `zip-contains-dir?`. Renamed it. It looks there is no other dependencies of `some?` function and a topology, `ExclamationTopology` seems fine with on my test cluster. If there is any other concern about upgrading clojure, please let me know. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm upgrade-clojure Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/390.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #390 commit dd92c0d87e589b345ae1167e7318e43ba44556bf Author: lewuathe lewua...@me.com Date: 2015-01-21T13:04:14Z [STORM-630] Support for Clojure 1.6.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-629] Place Link to Source Code Reposito...
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/386 [STORM-629] Place Link to Source Code Repository on Webpage Attach some links to GitHub repository on download page and contributing page. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm attach-source-code-link Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/386.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #386 commit dd20c3ad91b0a8c3c824f3d3ff28636fdfda9690 Author: lewuathe lewua...@me.com Date: 2015-01-19T11:42:15Z [STORM-629] Place Link to Source Code Repository on Webpage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-624] Fix typos in SECUTIRY.md
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/379 [STORM-624] Fix typos in SECUTIRY.md Fix some typos in SECURITY.mf You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm typo-SECURITY Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/379.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #379 commit 07656bd609650f680c3326b997137a7d15d213cd Author: lewuathe lewua...@me.com Date: 2015-01-14T12:53:58Z [STORM-624] Fix typos in SECUTIRY.md --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-623] Generate latest javadocs
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/378 [STORM-623] Generate latest javadocs There is no latest javadoc on official site now. https://storm.apache.org/documentation/Home.html In addition to this, current javadocs are hosted outside of apache domain. Maven build pipeline includes generation process of javadoc. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm attach-latest-javadoc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/378.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #378 commit 0a96db477e7126bbf5c6a120c23d9176716a5d87 Author: lewuathe lewua...@me.com Date: 2015-01-13T13:21:47Z [STORM-623] Generate latest javadocs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-591] exclude logback.xml from jar
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/352#issuecomment-68525425 I found logback searches logback-test.xml first and at eventually I renamed xml file and excluded it when it is packaged. I think it is a better way because it can show this xml file is only used for test explicitly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: STORM-596: remove config topology.receiver.buf...
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/363#issuecomment-68353816 @caofangkun I think correct behaviour is restricting the max size of `batched` ArrayList within given `max-buffer-size` . But if you mean this parameter is not necessary anyway, removing at all is a option. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-67581406 @harshach Sorry, I cannot figure out what you mean. Why does storing offset info to zookeeper path break checking topic existence with `/topics`? User can access zookeeper path at will, but this does not delete specific path in zookeeper doesn't it? Overall I think we agreed KafkaSpout shouldn't depend on kafka internal in zookeeper. All parts which uses curator or zookeeper api will be replaced. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-66860765 @ptgoetz OK. I'll raise new ticket for changing I/F between kafka and kafka-spout. Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-66713313 I posted this question on kafka mailing list and found the project for improvement of kafka api now. I think it is good way to propose check-existence api to this project. https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements But anyway current storm-kafka depends on kafka metadata on zookeeper through curator. So I think it is better to solve this issue by using curator api and then raise ticket about changing the way of exchanging between kafka and storm-kafka as originally proposed by @lazyval. What do you think all? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-66392691 @lazyval +1 to your idea. Making KafkaSpout depend on kafka kafka APIs is heavier task. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-65977818 There does not seems to be consumer API that throws `UnknownTopicOrPartitionException`. [UnknownTopicOrPartitionException in Kafka](https://github.com/apache/kafka/search?utf8=%E2%9C%93q=UnknownTopicOrPartitionException) Can I use this exception for checking existence of topics from consumer? Anyway `DynamicBrokerReader` already uses [Curator](http://curator.apache.org/) API which access specific ZK path directly. I think it is not problem to lookup ZK path from curator. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/338 [STORM-391] KafkaSpout to await for the topic KafkaSpout is waiting for creation of target topic. Curator keeps watching existence of topic path for a given time length. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm kafkaspout-await-for-topic Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/338.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #338 commit f3c0523c5aec432d1de5c0ccd6cf06a6550257e9 Author: lewuathe lewua...@me.com Date: 2014-12-05T15:00:03Z [STORM-391] KafkaSpout to await for the topic KafkaSpout is waiting for creation of target topic. Curator keeps watching existence of topic path for a given time length. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-65803225 @lazyval I think all nodes in storm clusters should refer its own state through zookeeper. This is important to keep consistency of cluster. Sorry I don't know such kafka api, but does the api guarantees consistency of cluster including zookeeper? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-65881172 @harshach Catching UnknownTopicOrPartitionException consumes some network bandwidth, doesn't it? It will check whether kafka server has the topic or not. Watching ZK does not induce traffic while topic path is not created, only WATCHED event is occurred. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-391] KafkaSpout to await for the topic
Github user Lewuathe commented on the pull request: https://github.com/apache/storm/pull/338#issuecomment-65887266 @harshach I understand what you mean. I'll modify codes to check the existence of topic with Kafka api. But I have a question for confirmation. Anyway we have to access Kafka api many times to check the existence of topic until its creation? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: Typo in storm-kafka README
GitHub user Lewuathe opened a pull request: https://github.com/apache/storm/pull/320 Typo in storm-kafka README Fix typos in storm-kafka README You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/storm typo-format-storm-kafka Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/320.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #320 commit d9dc3b2a173b0124986767ff3f63b72284958467 Author: lewuathe lewua...@me.com Date: 2014-11-19T22:06:49Z Typo in storm-kafka README --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
