[GitHub] storm pull request #1579: STORM-1989: X-Frame-Options support for Storm UI
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1579 STORM-1989: X-Frame-Options support for Storm UI You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm feature/x-frame-options-support-in-ui Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1579.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1579 commit 4e2e4a990183ceee73009de60e5eafabeb11baa2 Author: Tibor Kiss Date: 2016-07-19T11:40:36Z Implement HTTP X-Frame-Options for Storm UI --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1579: STORM-1989: X-Frame-Options support for Storm UI
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1579 Thanks @ptgoetz for the review! The Travis test failure seems unrelated to me: The base version of the patch also fails with test errors. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1579: STORM-1989: X-Frame-Options support for Storm UI
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1579 @harshach : The patch is not interfering with CORS filter. It is an additional filter addressing the ClickJacking vulnerability. @ptgoetz : Super thanks for merging the PR to both branches. Next time I'll address the delivery to additional branches quicker. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1740: STORM-2158: Fix OutOfMemoryError in Nimbus' Simple...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1740 STORM-2158: Fix OutOfMemoryError in Nimbus' SimpleTransportPlugin You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm STORM-2158 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1740.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1740 commit 60d6e5ba2a2a505b5b8404c2de991d02f8cfab9a Author: Tibor Kiss Date: 2016-10-20T12:58:11Z STORM-2158: Fix OutOfMemoryError in Nimbus' SimpleTransportPlugin Change-Id: Ic01c600b7b2b36e866d10de478d7d3b5ada94183 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1740: STORM-2158: Fix OutOfMemoryError in Nimbus' SimpleTranspo...
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1740 @HeartSaVioR : Sure, will do it shortly. I'm wondering: shall make a PR for 1.0.x-branch too? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1741: STORM-2158: Fix OutOfMemoryError in Nimbus' Simple...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1741 STORM-2158: Fix OutOfMemoryError in Nimbus' SimpleTransportPlugin You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm STORM-2158-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1741.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1741 commit 7cd3f22313713abdda09e01618393b7759a05ed0 Author: Tibor Kiss Date: 2016-10-21T05:19:49Z STORM-2158: Fix OutOfMemoryError in Nimbus' SimpleTransportPlugin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1740: STORM-2158: Fix OutOfMemoryError in Nimbus' SimpleTranspo...
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1740 @HeartSaVioR : Thanks for the explanation. Created the PR for master: https://github.com/apache/storm/pull/1741 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1915: STORM-2337: Broken documentation generation for st...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1915 STORM-2337: Broken documentation generation for storm-metrics-profiling-internal-actions.md and windows-users-guide.md You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm fix/missing-generated-documentation Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1915.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1915 commit 3806ab369be8fea6529bb7acc8ebd593be3fb977 Author: Tibor Kiss Date: 2017-02-02T18:39:19Z STORM-2337: Broken documentation generation for storm-metrics-profiling-internal-actions.md and windows-users-guide.md --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1917: STORM-2338: Subprocess exception handling is broke...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1917 STORM-2338: Subprocess exception handling is broken in storm.py on Windows environment You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm fix/subrpocess_exception_handling Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1917.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1917 commit 1b43804ddcd4f578b96b90db4acd058d7514106e Author: Tibor Kiss Date: 2017-02-03T08:10:10Z STORM-2338: Subprocess exception handling is broken in storm.py on Windows environment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1918: STORM-2339: Beauty contest in storm.py
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1918 STORM-2339: Beauty contest in storm.py You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm fix/beauty_contest Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1918.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1918 commit 7d963f31533ec6d5d14b5a64343b80464ac68d2a Author: Tibor Kiss Date: 2017-02-03T11:16:58Z STORM-2339: Beauty contest in storm.py --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1923: STORM-2345: Type mismatch in ReadClusterState's Pr...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1923 STORM-2345: Type mismatch in ReadClusterState's ProfileAction processing Map You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm STORM-2345 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1923.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1923 commit d8a0957e7659825d99bdf2254de6fd297c24835e Author: Tibor Kiss Date: 2017-02-05T17:00:02Z STORM-2345: Type mismatch in ReadClusterState's ProfileAction processing Map --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1926: STORM-2350: Storm-HDFS's listFilesByModificationTi...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1926 STORM-2350: Storm-HDFS's listFilesByModificationTime is broken You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm STORM-2350 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1926.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1926 commit de70903dde47756e81859f6fe0a1b2e5190b9b1b Author: Tibor Kiss Date: 2017-02-07T05:11:32Z STORM-2350: Storm-HDFS's listFilesByModificationTime is broken --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1937: STORM-2355: Storm-HDFS: inotify support
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1937 STORM-2355: Storm-HDFS: inotify support You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm feature/inotify_in_hdfsspout-1.x-branch Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1937.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1937 commit dc9aec20b567e752adfe0c1e7c864d86a8d92e68 Author: Tibor Kiss Date: 2017-02-12T09:56:01Z STORM-2355: Storm-HDFS: inotify support (initial version) Change-Id: I5e8a81738ff59afb8073a75c604bde2c1ef5b8d1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1928: Create ZippedTextFileReader.java
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1928 Thanks for contributing, nice idea! Seems like the build has failed due to missing symbols in ZippedTextFileReader.java: https://travis-ci.org/apache/storm/jobs/199168284 Unit (and possibly integration) tests would be really beneficial so that we can ensure quality in a long run. It is a protocol that first a JIRA issue is created for the feature/fix, then a PR is posted including the JIRA Issue ID in the commit message. Could you please address these issues? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1934: STORM-2333: CGroup memory and CPU metrics
Github user tibkiss commented on a diff in the pull request:
https://github.com/apache/storm/pull/1934#discussion_r100691777
--- Diff: storm-core/src/jvm/org/apache/storm/metric/cgroup/CGroupCpu.java
---
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.metric.cgroup;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Report CPU used in the cgroup
+ */
+public class CGroupCpu extends CGroupMetricsBase> {
+long previousSystem = 0;
+long previousUser = 0;
+
+public CGroupCpu(Map conf) {
+super(conf, "cpuacct.stat");
+}
+
+public int getUserHZ() {
+return 100; // On most systems (x86) this is fine.
+// If someone really does want full support
+// we need to run `getconf CLK_TCK` and cache the result.
+}
+
+@Override
+public Map parseFileContents(String contents) {
+try {
+long systemHz = 0;
+long userHz = 0;
+for (String line: contents.split("\n")) {
+if (!line.isEmpty()) {
+String [] parts = line.toLowerCase().split("\\s+");
+if (parts[0].contains("system")) {
+systemHz = Long.parseLong(parts[1].trim());
+} else if (parts[0].contains("user")) {
+userHz = Long.parseLong(parts[1].trim());
+}
+}
+}
+long user = userHz - previousUser;
+long sys = systemHz - previousSystem;
+previousUser = userHz;
+previousSystem = systemHz;
+long hz = getUserHZ();
+HashMap ret = new HashMap<>();
+ret.put("user-ms", user * 1000/hz); //Convert to millis
--- End diff --
This is a integer division during ms conversion. A float division would
produce better resolution. It is surely not a problem with getUserHz()
returning 100, but might cause issues if 100 is changed in the future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1934: STORM-2333: CGroup memory and CPU metrics
Github user tibkiss commented on a diff in the pull request:
https://github.com/apache/storm/pull/1934#discussion_r100821485
--- Diff: storm-core/src/jvm/org/apache/storm/metric/cgroup/CGroupCpu.java
---
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.metric.cgroup;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Report CPU used in the cgroup
+ */
+public class CGroupCpu extends CGroupMetricsBase> {
+long previousSystem = 0;
+long previousUser = 0;
+
+public CGroupCpu(Map conf) {
+super(conf, "cpuacct.stat");
+}
+
+public int getUserHZ() {
+return 100; // On most systems (x86) this is fine.
+// If someone really does want full support
+// we need to run `getconf CLK_TCK` and cache the result.
+}
+
+@Override
+public Map parseFileContents(String contents) {
+try {
+long systemHz = 0;
+long userHz = 0;
+for (String line: contents.split("\n")) {
+if (!line.isEmpty()) {
+String [] parts = line.toLowerCase().split("\\s+");
+if (parts[0].contains("system")) {
+systemHz = Long.parseLong(parts[1].trim());
+} else if (parts[0].contains("user")) {
+userHz = Long.parseLong(parts[1].trim());
+}
+}
+}
+long user = userHz - previousUser;
+long sys = systemHz - previousSystem;
+previousUser = userHz;
+previousSystem = systemHz;
+long hz = getUserHZ();
+HashMap ret = new HashMap<>();
+ret.put("user-ms", user * 1000/hz); //Convert to millis
--- End diff --
Makes sense, thanks for the explanation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1938: STORM-2360: Storm-Hive: Thrift version mismatch wi...
GitHub user tibkiss opened a pull request: https://github.com/apache/storm/pull/1938 STORM-2360: Storm-Hive: Thrift version mismatch with storm-core You can merge this pull request into a Git repository by running: $ git pull https://github.com/tibkiss/storm STORM-2360 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1938.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1938 commit 28556a198e87c8f7c006699294216f78c54f9a7c Author: Tibor Kiss Date: 2017-02-14T08:13:16Z STORM-2360: Storm-Hive: Thrift version mismatch with storm-core --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1911: STORM-2336 Close Localizer and AsyncLocalizer when superv...
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1911 For future reference: This issue has been committed with a typo in the issue id: STORM-2236 was used as opposed to the correct STORM-2336 `64a9dd0 2017-02-01 Jungtaek Lim STORM-2236 Close Localizer and AsyncLocalizer when supervisor is shutting down` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1918: STORM-2339: Beauty contest in storm.py
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1918 This PR has been pending since a while now. Could someone please take a look? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1962: storm jdbc topology should log the jdbc connection...
Github user tibkiss commented on a diff in the pull request:
https://github.com/apache/storm/pull/1962#discussion_r102898864
--- Diff:
external/storm-jdbc/src/main/java/org/apache/storm/jdbc/common/HikariCPConnectionProvider.java
---
@@ -40,6 +44,10 @@ public synchronized void prepare() {
Properties properties = new Properties();
properties.putAll(configMap);
HikariConfig config = new HikariConfig(properties);
+if(properties.containsKey("dataSource.url"))
+LOG.info("DataSource Url: " +
properties.getProperty("dataSource.url"));
+else if (config.getJdbcUrl() != null)
+LOG.info("JDBC Url: " + config.getJdbcUrl());
--- End diff --
Shouldn't this printout be independent of the presence of dataSource.url?
i.e. just an {{if}} instead of {{else if}}
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1962: storm jdbc topology should log the jdbc connection string...
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1962 Thanks for this addition, looks reasonable. I'd recommend opening an issue first in jira, then posting this PR with the associated issue id. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1918: STORM-2339: Python code format cleanup in storm.py
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1918 Thanks for the review @harshach , I've changed the title in JIRA, PR & commit message accordingly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1937: STORM-2355: Storm-HDFS: inotify support
Github user tibkiss commented on a diff in the pull request: https://github.com/apache/storm/pull/1937#discussion_r103071553 --- Diff: pom.xml --- @@ -234,7 +234,7 @@ 0.2.4 3.3.2 0.9.0 -16.0.1 +14.0.1 --- End diff -- Hdfs dependency hell story: hadoop depends on 14.0.1 guava. If guava 16 is used & inotify calls are made the following exception occurs: `org.apache.hadoop.ipc.RemoteException: com.google.common.collect.ComparisonChain.compare(ZZ)Lcom/google/common/collect/ComparisonChain; at org.apache.hadoop.hdfs.server.namenode.JournalSet$1.compare(JournalSet.java:67) at org.apache.hadoop.hdfs.server.namenode.JournalSet$1.compare(JournalSet.java:62) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324) at java.util.TimSort.sort(TimSort.java:189) at java.util.TimSort.sort(TimSort.java:173) at java.util.Arrays.sort(Arrays.java:659) at java.util.Collections.sort(Collections.java:217) at org.apache.hadoop.hdfs.server.namenode.JournalSet.chainAndMakeRedundantStreams(JournalSet.java:333) at org.apache.hadoop.hdfs.server.namenode.JournalSet.selectInputStreams(JournalSet.java:284) at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1508) at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1532) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getEditsFromTxid(NameNodeRpcServer.java:1997) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getEditsFromTxid(ClientNamenodeProtocolServerSideTranslatorPB.java:1496) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy37.getEditsFromTxid(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getEditsFromTxid(ClientNamenodeProtocolTranslatorPB.java:1511) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy38.getEditsFromTxid(Unknown Source) at org.apache.hadoop.hdfs.DFSInotifyEventInputStream.poll(DFSInotifyEventInputStream.java:111) at org.apache.storm.hdfs.common.HdfsUtils.listNewFiles(HdfsUtils.java:94) at org.apache.storm.hdfs.spout.TestHdfsSpout.testListNewFiles(TestHdfsSpout.java:549)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1937: STORM-2355: Storm-HDFS: inotify support
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1937 Thanks for the comments. I'll be OoO next week, will address them beginning of March. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1937: STORM-2355: Storm-HDFS: inotify support
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1937 @roshannaik : I've replied to your questions on the JIRA by extending the description issue description. Please check it out and let me know if you have further questions. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1937: STORM-2355: Storm-HDFS: inotify support
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1937 @harshach : Updated this PR based on your review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1918: STORM-2339: Python code format cleanup in storm.py
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1918 bump --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1923: STORM-2345: Type mismatch in ReadClusterState's ProfileAc...
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1923 bump --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1937: STORM-2355: Storm-HDFS: inotify support
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1937 bump --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1918: STORM-2339: Python code format cleanup in storm.py
Github user tibkiss commented on a diff in the pull request:
https://github.com/apache/storm/pull/1918#discussion_r105094892
--- Diff: bin/storm.py ---
@@ -193,21 +205,26 @@ def parse_args(string):
args = [re.compile(r"'((?:[^'\\]|\\.)*)'").sub('\\1', x) for x in args]
return [re.compile(r'\\(.)').sub('\\1', x) for x in args]
-def exec_storm_class(klass, jvmtype="-server", jvmopts=[], extrajars=[],
args=[], fork=False, daemon=True, daemonName=""):
-global CONFFILE
-storm_log_dir = confvalue("storm.log.dir",[CLUSTER_CONF_DIR])
-if(storm_log_dir == None or storm_log_dir == "nil"):
+
+def exec_storm_class(klass, jvmtype="-server", jvmopts=None,
extrajars=None, args=None, fork=False, daemon=True,
--- End diff --
Glad you asked. Specifying empty list as default gets initialized once,
therefore multiple calls to the function will accumulate the appended values to
the list. More lengthy description of this 'feature' could be found here:
http://effbot.org/zone/default-values.htm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1918: STORM-2339: Python code format cleanup in storm.py
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1918 "At 2 months, your baby doesnât yet have the coordination to play with toys. But she may bat at a colorful object hanging in front of her. Your baby may even briefly hold a toy that you place in one of her hands." Could someone please merge this before it starts to play with toys? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1918: STORM-2339: Python code format cleanup in storm.py
Github user tibkiss commented on the issue: https://github.com/apache/storm/pull/1918 This is unfortunate. Maybe someone else can take this on: My eagerness evaporated somewhere around April. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
