Re: [DISCUSS] Accept JW Player SQE Code Donation

what happens to current storm-sql once SQE is merged into apache repository? On Thu, Aug 25, 2016 at 10:26 AM, P. Taylor Goetz wrote: > > > On Aug 24, 2016, at 5:37 PM, Jungtaek Lim wrote: > > > > While I feel Storm SQL covers (and will cover in near future) all of the > > features what SQE has

[GitHub] storm issue #838: [STORM-885] Heartbeat Server (Pacemaker)

Github user danny0405 commented on the issue: https://github.com/apache/storm/pull/838 Now pacemaker is a stand-alone service and no HA is supported. When it goes down, all the workers's heartbeats will be lost. It will take a long time to recover even if pacemaker goes up immediately

Re: [DISCUSS] Accept JW Player SQE Code Donation

> On Aug 24, 2016, at 5:37 PM, Jungtaek Lim wrote: > > While I feel Storm SQL covers (and will cover in near future) all of the > features what SQE has, I'd like to consider this as not only code donation > but also having more contributors on Storm SQL. +1 This is a great opportunity to grow

[jira] [Assigned] (STORM-1459) Allow not specifying producer properties in read-only Kafka table in StormSQL

[ https://issues.apache.org/jira/browse/STORM-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang reassigned STORM-1459: - Assignee: Manu Zhang > Allow not specifying producer properties in read-only Kafka table in Storm

[jira] [Commented] (STORM-1433) StormSQL Phase II

[ https://issues.apache.org/jira/browse/STORM-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436252#comment-15436252 ] Jungtaek Lim commented on STORM-1433: - Added you to contributor list. You can assign y

[jira] [Commented] (STORM-1433) StormSQL Phase II

[ https://issues.apache.org/jira/browse/STORM-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436248#comment-15436248 ] Manu Zhang commented on STORM-1433: --- Ok, I'd love to work on STORM-1459, could you assig

[jira] [Commented] (STORM-1433) StormSQL Phase II

[ https://issues.apache.org/jira/browse/STORM-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436173#comment-15436173 ] Jungtaek Lim commented on STORM-1433: - If you're seeing older codebase you can see tha

[jira] [Commented] (STORM-1433) StormSQL Phase II

[ https://issues.apache.org/jira/browse/STORM-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436162#comment-15436162 ] Manu Zhang commented on STORM-1433: --- Yes, I'm interested. I'm familiar with Storm/Triden

[GitHub] storm pull request #1642: DO NOT MERGE: Please review STORM-2018: Supervisor...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/1642#discussion_r76169008 --- Diff: storm-core/src/jvm/org/apache/storm/daemon/supervisor/Supervisor.java --- @@ -17,135 +17,541 @@ */ package org.apache.storm.daemon.super

[GitHub] storm issue #1642: DO NOT MERGE: Please review STORM-2018: Supervisor V2.

Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/1642 +1 on general direction. Readability is good; classes aren't too large or convoluted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] storm pull request #1642: DO NOT MERGE: Please review STORM-2018: Supervisor...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/1642#discussion_r76162679 --- Diff: storm-core/src/jvm/org/apache/storm/daemon/supervisor/Container.java --- @@ -0,0 +1,417 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] storm pull request #1642: DO NOT MERGE: Please review STORM-2018: Supervisor...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/1642#discussion_r76161402 --- Diff: storm-core/src/jvm/org/apache/storm/daemon/supervisor/Container.java --- @@ -0,0 +1,417 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] storm pull request #1642: DO NOT MERGE: Please review STORM-2018: Supervisor...

Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/1642#discussion_r76160977 --- Diff: storm-core/src/jvm/org/apache/storm/daemon/supervisor/Container.java --- @@ -0,0 +1,417 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] storm issue #1646: STORM-1870 Allow FluxShellBolt/Spout set custom "componen...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1646 @ptgoetz Could you have a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[jira] [Commented] (STORM-1433) StormSQL Phase II

[ https://issues.apache.org/jira/browse/STORM-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435879#comment-15435879 ] Jungtaek Lim commented on STORM-1433: - [~mauzhang] Manu, are you still interested to

[GitHub] storm issue #1643: STORM-2050 [storm-sql] Support User Defined Aggregate Fun...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1643 OK. Rebased with current master. @ptgoetz @harshach @haohui @arunmahadevan Please take a look. Thanks in advance! --- If your project is set up for it, you can reply to this email and have y

[GitHub] storm issue #1643: STORM-2050 [storm-sql] Support User Defined Aggregate Fun...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1643 Need to rebase again with current master (applied STORM-1434) but there're lots of conflict so would like to re-work based on current instead of resolving conflict. Will leave a comment once

[jira] [Resolved] (STORM-1434) Support the GROUP BY clause in StormSQL

[ https://issues.apache.org/jira/browse/STORM-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved STORM-1434. - Resolution: Fixed Fix Version/s: 1.1.0 2.0.0 Merged into master and 1.x-

[GitHub] storm pull request #1635: STORM-1434 Support the GROUP BY clause in StormSQL

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1635 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] storm issue #1635: STORM-1434 Support the GROUP BY clause in StormSQL

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1635 Optimizing logical / physical plan is the long term works so I would like to revisit again. (Will file an issue for analyzing logical plan for several queries. Might file additional - maybe u

Re: [DISCUSS] Accept JW Player SQE Code Donation

While I feel Storm SQL covers (and will cover in near future) all of the features what SQE has, I'd like to consider this as not only code donation but also having more contributors on Storm SQL. As the one working on storm-sql now, I think Storm SQL should have more contributors who are experienc

Re: [VOTE] Release Apache Storm 0.9.7 (RC1)

Here’s my +1 (binding). I’d appreciate if other PMC members vote on this release (for or against doesn’t matter, as long as you vote). -Taylor > On Aug 15, 2016, at 4:10 PM, P. Taylor Goetz wrote: > > This is a call to vote on releasing Apache Storm 0.9.7 (rc1). > > This release candidate ad

[VOTE] Release Apache Storm 0.10.2 (RC1)

This is a call to vote on releasing Apache Storm 0.10.2 (rc1) Full list of changes in this release: https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=3c24dce8b35527b409cae68a1ebc0e25e5a0b03f The tag/commit to be voted upon is v0.10.2: https://git-wip-us.apache.o

[GitHub] storm issue #1649: STORM-2052: Kafka Spout - New Client API - Performance Im...

Github user harshach commented on the issue: https://github.com/apache/storm/pull/1649 overall looks good. Can you squash the commits. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] storm pull request #1649: STORM-2052: Kafka Spout - New Client API - Perform...

Github user harshach commented on a diff in the pull request: https://github.com/apache/storm/pull/1649#discussion_r76124663 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java --- @@ -221,7 +224,22 @@ private boolean commit() {

Re: [DISCUSS] Release Apache Storm 0.10.2

Thanks for the feedback Jungtaek. I’ll initiate a release. -Taylor > On Aug 17, 2016, at 6:36 PM, Jungtaek Lim wrote: > > I haven't seen user requests for backporting fixes for master/1.x to 0.10 > so we maybe good to go, but I could miss some spots. > (I found an user backporting specific issu

[DISCUSS] Accept JW Player SQE Code Donation

JW Player has offered to donate SQE (Streaming Query Engine) to the Storm project. SQE is a query engine built on top of Trident that uses a SQL-like syntax (currently JSON-based) to query streams. There is obvious overlap between this and storm-sql, and this might be an opportunity to improve

[jira] [Updated] (STORM-2052) Kafka Spout - New Client API - Performance Improvements

[ https://issues.apache.org/jira/browse/STORM-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hugo Louro updated STORM-2052: -- Affects Version/s: (was: 0.10.2) > Kafka Spout - New Client API - Performance Improvements > ---

[GitHub] storm pull request #1649: STORM-2052: Kafka Spout - New Client API - Perform...

GitHub user hmcl opened a pull request: https://github.com/apache/storm/pull/1649 STORM-2052: Kafka Spout - New Client API - Performance Improvements This patch should be back-ported to 1.0.x branch and 1.x-branch. Thank you. You can merge this pull request into a Git repository by

Re: Too many machine mails

Request to create iss...@storm.apache.org is in the queue. The Storm PMC will be notified when the list is created. -Taylor > On Aug 18, 2016, at 8:46 PM, Jungtaek Lim wrote: > > FYI: Forwarding Talyor's reply. > (I need to have a habit to remove all personal r

Re: Details about the Storm Scheduler

Also your bolt may be pending on a call to an external resource (e.g. DB) and thus not consuming much CPU despite a relatively high usage. On Wed, Aug 24, 2016 at 10:09 AM, Bobby Evans wrote: > But CGroups is restricting the actual CPU usage and scheduling is taking > the CPU usage into account

Re: Details about the Storm Scheduler

But CGroups is restricting the actual CPU usage and scheduling is taking the CPU usage into account so as to not overload a box.  You can use the latency to guess how much CPU is being used, but that only works for a single threaded bolt/spout.  Not all bolts/spouts are single threaded.  Think a

[GitHub] storm issue #1635: STORM-1434 Support the GROUP BY clause in StormSQL

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1635 @arunmahadevan Yes I've seen some places to optimize. Trident groups nodes so there're many nodes but there will be only three components - one spout and two bolts. So for that situation Trident

[GitHub] storm issue #1635: STORM-1434 Support the GROUP BY clause in StormSQL

Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/1635 @HeartSaVioR I have gone through your recent changes and it looks good. In the logical plan you have posted, there are multiple processors just for the project (before the aggregation s

[jira] [Created] (STORM-2055) Exception when running topology from Maven exec with Flux

Julien Nioche created STORM-2055: Summary: Exception when running topology from Maven exec with Flux Key: STORM-2055 URL: https://issues.apache.org/jira/browse/STORM-2055 Project: Apache Storm

[GitHub] storm issue #1084: STORM-1516 Fixed issue in writing pids with distributed c...

Github user ndtreviv commented on the issue: https://github.com/apache/storm/pull/1084 @HeartSaVioR Perfect. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] storm issue #1635: STORM-1434 Support the GROUP BY clause in StormSQL

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1635 > SELECT GRPID, COUNT(*) AS CNT, MAX(AGE) AS MAX_AGE, MIN(AGE) AS MIN_AGE, AVG(AGE) AS AVG_AGE, MAX(AGE) - MIN(AGE) AS DIFF FROM FOO GROUP BY GRPID Below link is a Trident logical/physic

[GitHub] storm issue #1084: STORM-1516 Fixed issue in writing pids with distributed c...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1084 @ndtreviv This bug was from ported code which only resides on master (2.0.0) so not relate to 1.x. What you're explaining seems to [STORM-1879|https://issues.apache.org/jira/browse/

[GitHub] storm issue #1084: STORM-1516 Fixed issue in writing pids with distributed c...

Github user ndtreviv commented on the issue: https://github.com/apache/storm/pull/1084 @HeartSaVioR I'm not sure that's the one. I'm pretty sure that I'm seeing this issue. I can see the supervisor.log saying that it can't find the worker file in workers-users. As a result, the worker

[GitHub] storm issue #1084: STORM-1516 Fixed issue in writing pids with distributed c...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1084 @ndtreviv This patch is only for 2.0.0. You might hit [STORM-1934|https://issues.apache.org/jira/browse/STORM-1934] which is fixed for 1.0.2. There're lots of things fixed from 1.0.2 so you're en

[GitHub] storm issue #1084: STORM-1516 Fixed issue in writing pids with distributed c...

Github user ndtreviv commented on the issue: https://github.com/apache/storm/pull/1084 Which version was this fixed in? I'm seeing the same thing in 1.0.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] storm issue #1359: STORM-1239: port backtype.storm.scheduler.IsolationSchedu...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1359 @vesense OK no worries. I just would like to check on this. Please take your time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.