Re: Welcome new Apache Storm Committers/PMC Members

[임정택]

Congratulation! Looking forward to work with you all.

Best,
Jungtaek Lim (HeartSaVioR)

On 2015년 12월 9일 (수) at 오전 6:55 P. Taylor Goetz  wrote:

> I’m pleased to announce that the following individuals have joined as
> Apache Storm Committers/PMC Members:
>
> - Longda Feng
> - Arun Mahadevan
> - Boyang Jerry Peng
> - Matthias J. Sax
> - Aaron Dossett
>
> Longda, Arun, Jerry, Matthias, and Aaron have all demonstrated technical
> merit and dedication to Apache Storm and its community, and as PMC members
> they will help drive innovation and community development.
>
> Please join me in welcoming and congratulating Londa, Arun, Jerry,
> Matthias, and Aaron. We look forward to your continued dedication to the
> Storm community.
>
> -Taylor
>

Re: [DISCUSSION] More convenient way to maintain committer / contributor list and changelogs

[임정택]

Yeah, that would be good.
(Taylor, did you mean that we'd be better to move CHANGELOG.md to official
site?)

If we decide to maintain it from asf-site branch, I think preconditions
should be met in order to let us feel more convenient than current.

   - Pushing to asf-site branch via git repo should update our official
   site.
   - Yaml files should be updated automatically.
  - Because pushing to asf-site requires checking out asf-site branch,
  modifying yaml files, and pushing to remote, and checking out
master branch
  again.

I'm curious we can resolve preconditions, at least first one.
Any updates on this, Taylor?

Any other opinions are appreciated!

- Jungtaek Lim (HeartSaVioR)

ps. Btw, if we decide to scan jira issue and create CHANGELOG, how about
adding assignee to CHANGELOG so that anyone can see huge
contributors easily?

2015-12-03 9:35 GMT+09:00 P. Taylor Goetz <ptgo...@gmail.com>:

> IMHO we should take that out of the readme, and maintain it as part of the
> website. Better yet, automate it as haohui suggested.
>
> In the website branch in git we now have a yaml file that contains all
> that information. I'd rather maintain it there, in one place, and automate
> if we can.
>
> -Taylor
>
> > On Dec 2, 2015, at 6:06 PM, 임정택 <kabh...@gmail.com> wrote:
> >
> > Hi all,
> >
> > Apache Storm project maintains committer / contributor list to
> > README.markdown
> > <https://github.com/apache/storm/blob/master/README.markdown> and
> resolved
> > JIRA issues to CHANGELOG.md
> > <https://github.com/apache/storm/blob/master/CHANGELOG.md>.
> > And recently we're adding the page "People" to official web site,
> > http://storm.apache.org/contribute/People.html, which seems not be
> updated
> > already.
> >
> > Since we occasionally missed to add those things, it would be better to
> > have more convenient way to record.
> > For now updating "People" seems annoying since we should update site via
> > applying patch and committing to SVN repo.
> > Do we feel that is it convenient to maintain these to git repo? Are there
> > more convenient ways to maintain? Could WIKI be more convenient way?
> >
> > Thanks,
> > Jungtaek Lim (HeartSaVioR)
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Storm 2.0 plan

[임정택]

Sorry Longda, but I can't help telling that I also disagree about changing
codebase.

Feature matrix shows us how far Apache Storm and JStorm are diverged, just
in point of feature's view. We can't be safe to change although feature
matrixes are identical, because feature matrix doesn't contain the details.

I mean, users could be scared when expected behaviors are not in place
although they're small. User experience is the one of the most important
part of the project, and if UX changes are huge, barrier for upgrading
their Storm cluster to 2.0 is not far easier than migrating to Heron. It
should be the worst scenario I can imagine after merging.

The safest way to merge is applying JStorm's great features to Apache Storm.
I think porting language of Apache Storm to Java is not tightly related to
merge JStorm. I agree that merging becomes a trigger, but Apache Storm
itself can port to other languages like Java, Scala, or something else
which are more popular than Clojure.

And I'm also not scary about Flink, Heron, Spark, etc.
It doesn't mean other projects are not greater then Storm. Just I'm saying
each projects have their own strength.
For example, all conferences are saying about Spark, and as one of users of
Spark, Spark is really great. If you are a little bit familiar with Scala,
you can just apply Scala-like functional methods to RDD. Really easy to use.
But it doesn't mean that Spark can replace Storm in all kind of use cases.
Recently I've seen some articles that why Storm is more preferred in
realtime streaming processing.

Competition should give us a positive motivation. I hope that our roadmap
isn't focused to defeat competitors, but is focused to present great
features, better performance, and better UX to Storm community. It's not
commercial product, it's open source project!

tl;dr. Please don't change codebase unless we plan to release a brand new
project. It breaks UX completely which could make users leave.

I'm also open to other opinions as well.

Best,
Jungtaek Lim (HeartSaVioR)


2015-11-20 0:00 GMT+09:00 Bobby Evans :

> I disagree completely.  You claim that JStorm is compatible with storm
> 1.0.  I don't believe that it is 100% compatible.  There has been more then
> 2 years of software development happening on both sides.  Security was not
> done in a day, and porting it over to JStorm is not going to happen
> quickly, and because of the major architectural changes between storm and
> JStorm I believe we would have to make some serious enhancements to fully
> support a secure TopologyMaster, but I need to look into it more.  The blob
> store is another piece of code that has taken a very long time to develop.
> There are numberous others.  The big features are not the ones that make me
> nervous because we can plan for them, it is the hundreds of small JIRA and
> features that will result in minor incompatibilities.  If we start with
> storm itself, and follow the same process that we have been doing up until
> now, if there is a reason to stop the port add in an important feature and
> do a release, we can.  We will know that we have compatibility vs starting
> with JStorm we know from the start that we do not without adding feature X,
> Y, Z, 
>
> I personally am not scared of Flink, Heron, Spark, Apex, IBM Streams,
> etc...  We just did some major performance enhancements that will be
> released with STORM 1.0.  We now have up to 6x the throughput that we had
> before with minimal changes to the latency (20 ms vs 5 ms).  We have
> automatic back-pressure so if someone was running with acking enabled just
> for flow control they can now process close to 16x the throughput they
> could before with the same hardware.  This puts our throughput very much on
> par with flink and Spark, but with a much lower latency compared to either
> of them.  Plus from what I have heard Flink is still calling the streaming
> API beta, and their storm API compatibility is very rudimentary.  They are
> also going to have more and more problems maintaining compatibility as we
> add in new features and functionality.
>
> Spark only really works well when it is running with several seconds of
> latency. Not every one needs sub-second processing, but when your platform
> is completely unable to handle it, locks you out of a lot of use cases.
> Their throughput is decent and can scale very high when you are willing to
> tolerate similarly very high latencies.
> Who knows about Heron until they actually release their code, but it is
> missing lots of critical features, and the one they touted, better
> performance, is a moot point with storm 1.0.  The only thing we really are
> lacking is advertising, we don't have a big company really pushing storm
> and getting it in the news all the time (Sorry Hortonworks, but I really
> have not seen much about it in the news).  I am trying to do more, but
> there is only so much I can do.
> Longda I very much agree with you about 

Re: [DISCUSS] Plan for Merging JStorm Code

[임정택]

Hi Basti,

Seems like Phase 3.1 covers Phase 2.2 since Phase 3.1 is actual work for
Phase 2.2.
So in Phase 3.1, if JStorm code is fully compatible, we can just pick it.
If JStorm code is not compatible, we can port Clojure code, or implement it
based on JStorm code.
If you were talking about baseline, please read my opinion on phase 3.

All,

For phase 1, I'm also +1 for releasing 0.11.0 as 1.0.0.
Apache Storm already has various use cases, which feels me no longer beta.

For phase 3, I'm +1 for what Taylor suggests.

Precondition of my opinion is that we agree that our merged product will be
named as 'Apache Storm'.
Based on this decision, we may want to set baseline to Apache Storm, cause
users expect features based backward compatibility to newer releases.
(Brand and community is most important thing for open source product, and
our moves should avoid hurting them. Losing users lose all.)
Many features are included via 0.10.x and 0.11.0 (master branch), which
means Apache Storm and JStorm is diverged. 'Core' is no except.

Not fast but safest is, keeping Apache Storm as it is (or port Clojure code
to Java, as we start to discuss), and applying JStorm's amazing features
via pull requests, like normal contributions. I love this approach since it
is more natural.

Thanks,
Jungtaek Lim (HeartSaVioR)



2015-11-11 20:41 GMT+09:00 刘键(Basti Liu) :

> Hi Taylor,
>
>
>
> Thanks for the merge plan. I have a question about “Phase 2.2”.
>
> Do you mean community plan to create a fresh new “java core” based on
> current “clojure core” firstly, and then migrate the features from JStorm?
>
> If so, it confused me.  It is really a huge job which might require a long
> developing time to make it stable, while JStorm is already a stable version.
>
> The release planned to be release after Nov 11th has already run online
> stably several month in Alibaba.
>
> Besides this, there are many valuable internal requirements in Alibaba,
> the fast evolution of JStorm is forseeable in next few months.
>
> If the “java core” is totally fresh new, it might bring many problems for
> the coming merge.
>
> So, from the point of this view,  I think it is much better and easier to
> migrate the features of “clojure core” basing on JStorm for the “java core”.
>
> Please correct me, if any misunderstanding.
>
>
>
> Regards
>
> Basti
>
>
>
> 发件人: P. Taylor Goetz [mailto:ptgo...@gmail.com]
> 发送时间: 2015年11月11日 5:32
> 收件人: dev@storm.apache.org
> 主题: [DISCUSS] Plan for Merging JStorm Code
>
>
>
> Based on a number of discussions regarding merging the JStorm code, I’ve
> tried to distill the ideas presented and inserted some of my own. The
> result is below.
>
>
>
> I’ve divided the plan into three phases, though they are not necessarily
> sequential — obviously some tasks can take place in parallel.
>
>
>
> None of this is set in stone, just presented for discussion. Any and all
> comments are welcome.
>
>
>
> ---
>
>
>
> Phase 1 - Plan for 0.11.x Release
>
> 1. Determine feature set for 0.11.x and publish to wiki [1].
>
> 2. Announce feature-freeze for 0.11.x
>
> 3. Create 0.11.x branch from master (Phase 2.4 can begin.)
>
> 4. Release 0.11.0 (or whatever version # we want to use)
>
> 5. Bug fixes and subsequent releases from 0.11.x-branch
>
>
>
>
>
> Phase 2 - Prepare for Merge ("master" and "jstorm-import" branches)
>
> 1. Determine/document unique features in JStorm (e.g. classpath isolation,
> cgroups, etc.) and create JIRA for migrating the feature.
>
> 2. Create JIRA for migrating each clojure component (or logical group of
> components) to Java. Assumes tests will be ported as well.
>
> 3. Discuss/establish style guide for Java coding conventions. Consider
> using Oracle’s or Google’s Java conventions as a base — they are both
> pretty solid.
>
> 4. align package names (e.g backtype.storm --> org.apache.storm /
> com.alibaba.jstorm --> org.apache.storm) (Phase 3 can begin)
>
>
>
>
>
> Phase 3 - Migrate Clojure --> Java
>
> 1. Port code/tests to Java, leveraging existing JStorm code wherever
> possible (core functionality only, features distinct to JStorm migrated
> separately).
>
> 2. Port JStorm-specific features.
>
> 3. Begin releasing preview/beta versions.
>
> 4. Code cleanup (across the board) and refactoring using established
> coding conventions, and leveraging PMD/Checkstyle reports for reference.
> (Note: good oportunity for new contributors.)
>
> 5. Release 0.12.0 (or whatever version # we want to use) and lift feature
> freeze.
>
>
>
>
>
> Notes:
>
> We should consider bumping up to version 1.0 sometime soon and then
> switching to semantic versioning [3] from then on.
>
>
>
>
>
> With the exception of package name alignment, the "jstorm-import" branch
> will largely be read-only throughout the process.
>
>
>
> During migration, it's probably easiest to operate with two local clones
> of the Apache Storm repo: one for working (i.e. checked out to working
> branch) and one for reference/copying 

Re: [DISCUSS] 1.0 Release (was Re: [DISCUSS] Initial 0.11.0 Release)

[임정택]

+1

Jungtaek Lim (HeartSaVioR)

2015-11-12 7:21 GMT+09:00 P. Taylor Goetz :

> Changing subject in order to consolidate discussion of a 1.0 release in
> one thread (there was some additional discussion in the thread regarding
> the JStorm code merge).
>
> I just want to make sure I’m accurately capturing the sentiment of the
> community with regard to a 1.0 release. Please correct me if my summary
> seems off-base or jump in with an opinion.
>
> In summary:
>
> 1. What we have been calling “0.11.0” will become the Storm 1.0 release.
> 2. We will NOT be migrating package names for this release (i.e.
> “backtype.storm” —> “org.apache.storm”).
> 3. Post 1.0 release we will go into feature freeze for core functionality
> to facilitate the JStorm merge.
> 4. During the feature freeze only fixes for high priority bugs in core
> functionality will be accepted (no new features).
> 5. During the feature freeze, enhancements to “external” modules can be
> accepted.
> 6. We will stop using the “beta” flag in favor of purely numeric version
> numbers. Stable vs. non-stable (development) releases can be indicated on
> the download page.
>
> Do we all agree?
>
> -Taylor
>
> > On Nov 11, 2015, at 4:10 PM, P. Taylor Goetz  wrote:
> >
> >
> >> On Nov 11, 2015, at 9:28 AM, Bobby Evans 
> wrote:
> >>
> >> I would like to see STORM-1190, STORM-1155, STORM-1198, STORM-1196,
> STORM-885, STORM-876, STORM-1145, STORM-831, and STORM-874 added to the
> list.
> >>
> >> Some of them are more important then others but they are all things I
> would like to see in a 0.11.0 release.
> >
> >
> > Cool. Thanks for listing them out. I will add them so they get tracked.
> >
> >
> >> On a side note I don't think the beta releases have been helpful.  I
> would much rather just have a 0.11.0 go to 0.11.1 ... instead of
> 0.11.0-beta1, 0.11.0-beta2.  To me the beta label is not that helpful, but
> it is not that big of a deal for me.
> >
> > In my mind, having releases tagged as “beta” were a way for us to say to
> users “here’s a preview release that will allow you to kick the tires on
> upcoming features, but be aware that there might be bugs. Let us know if
> you find any so we can fix them.”
> >
> > I think the intent was sound, but what I didn’t really see was user
> feedback on the beta releases, which is what I hoped would happen. So I
> have no problem with dropping the use of “beta” flags.
> >
> > Another approach I’ve seen other Apache projects use is to the numbering
> scheme you describe, and just have different labels/descriptions on the
> download page — i.e. “Latest stable release” and “Latest development
> release.” The nice part of that convention is that it does not have any
> impact on the release process. For example if we’re confident that a
> particular “development” release is actually quite stable, all we would
> have to do is update the downloads page, rather than go through the whole
> release/vote process just to remove the “beta” tag.
> >
> >> I also would like to see the 0.11 release tied to the plan for the
> JStorm merger.  If we don't tie them together there can be code that does
> not make it into 0.11, but could make it into a 0.12 that will immediately
> be caught up in the merger, that could take a long time to complete.  I
> mostly want us to be very transparent about what is likely to happen after
> 0.11 is released.  So if someone has a feature that is close to getting
> something in to 0.11 that they speak up here instead of just deciding to
> wait for a 0.12 release.
> >
> >
> > I agree that the 0.11 release needs to be tied to the JStorm merger.
> Once that release goes out, we’ll be going into lockdown mode for the merge
> effort, which is likely to take a while.
> >
> > During that time it’s highly unlikely that any changes/additions to
> Storm’s core functionality will be accepted beyond high-priority bug fixes.
> Adding new features to the “external” components during that time is
> probably okay, since those components are sufficiently decoupled from
> Storm’s core functionality.
> >
> > So to reiterate Bobby’s statement:
> >
> > Please speak up now if there are any specific features or changes to
> Storm’s core functionality that you’d like to see in the next release.
> Otherwise you will have to wait.
> >
> >> - Bobby
> >
> > -Taylor
> >
> >>
> >>
> >>On Wednesday, November 11, 2015 6:32 AM, John Fang <
> xiaojian@alibaba-inc.com> wrote:
> >>
> >>
> >>  Totally agree， it's great to accelerate the release speed.  it is
> better release one official version within 3 months. The first month is for
> developing new features , .If some features hasn't been finished in this
> month, it will be put in the next release ticket. In the last two month, we
> just do test.
> >>  Storm may face the greatest challenge in a big cluster, so I am more
> concerned about ZK Optimization as jstorm did. At last, it's great for the
> next 

Re: Welcome JStorm Community!

[임정택]

Welcome, JStorm community!

I saw some developers of JStorm are already participated to review. This is
really great start!
With JStorm community I don't question that we can make Storm much better,
codebase, features, use cases, and various things.

Looking forward to co-work!

Thanks,
Jungtaek Lim (HeartSaVioR)

2015-11-05 2:48 GMT+09:00 Kishorkumar Patil :

> Welcome! Looking forward to working with integrated Storm team!
> Special Kudos to Taylor and others who are helping accomplish this.
>
> Regards,Kishor
>
>
>  On Wednesday, November 4, 2015 11:35 AM, Priyank Shah <
> ps...@hortonworks.com> wrote:
>
>
>  Welcome everyone from JStorm community. Look forward to working with you
> all.
>
>
>
>
> On 11/4/15, 8:49 AM, "Hugo Da Cruz Louro"  wrote:
>
> >Welcome to the Community! It’s exciting to have JStorm as part of the
> Apache family. Looking forward to working and getting to meet you all
> sometime during our Apache contributors journey.
> >
> >Please feel free to reach out in case you need anything.
> >
> >Best,
> >Hugo
> >
> >On Nov 3, 2015, at 2:05 PM, P. Taylor Goetz > wrote:
> >
> >For those who may not be aware, Alibaba has donated JStorm to Apache
> Storm, and we have completed the IP Clearance process. JStorm is a fork of
> Storm with the clojure components reimplemented in Java, and includes a
> number of enhancements in terms of performance and features. JStorm is used
> widely in production at Alibaba across many different areas.
> >
> >Of equal importance is the JStorm community, which has expressed a
> commitment to join forces with the Apache Storm community to bring JStorm’s
> improvements to Apache Storm and collaborate on new features and
> improvements.
> >
> >Over the coming days and weeks we will have a number of discussions on
> the dev@ list regarding how to best approach merging both the incoming
> code and associated community.
> >
> >Please join me in welcoming the JStorm contributors to the Apache Storm
> community.
> >
> >
> >JStorm community:
> >
> >Welcome abord! We’re glad you’re here.
> >
> >As you’ve probably noticed, all discussions and decision making at Apache
> takes place on the mailing list in English. I understand that might be a
> burden for non-English speakers, but there a number of Apache Storm
> contributors that speak both English and Chinese that would likely be
> willing to help translate. Google translate, while not perfect, is also
> helpful in some cases. We don’t expect perfect translation — just enough to
> get your point across is fine.
> >
> >When signing your name on an email, etc., please use the western name of
> yourself and others. This makes it easier for those of us who can't read
> Asian characters to identify individuals.
> >
> >If you’re new to Apache and the way it works, here is a good starting
> point for learning “The Apache Way”:
> >
> >https://community.apache.org/contributors/
> >
> >Feel free to introduce yourself to the Storm community by replying to
> this thread and letting us know a little more about yourself.
> >
> >Again, thank you and welcome!
> >
> >-Taylor
> >
>
>
>
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Auto-generating documentation site..

[임정택]

Kishor and Bobby,
Did you use the branch 'asf-site'?
I checked but nothing is changed in 'asf-site' branch.

Kishor,
Maybe Bobby's explanation for me could help you too, since seems like git
based publishing is not enabled yet.
https://github.com/apache/storm/pull/771#issuecomment-144726893

Best,
Jungtaek Lim (HeartSaVioR)



2015-11-05 3:20 GMT+09:00 Bobby Evans :

> I just tried to update it and the source under docs using the version of
> jekyll I have installed (2.5.2) is generating something that looks totally
> different from what we have on http://storm.apache.org/  Are there new
> instructions or a new version required for this?  - Bobby
>
>
>  On Wednesday, November 4, 2015 11:35 AM, Kishorkumar Patil
>  wrote:
>
>
>  Hello Taylor,
> I have pushed some more updates to the documentation in the git. But I am
> not sure how to update Storm Documentation site.
>
> Can you point me to steps on how to update this. Also,  I am wondering if
> we have any planned/in progress(JIRA) automation to generate and publish
> this documentation?
>
> Regards,Kishor
>
>
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Kafka exception on Toplogy start

[임정택]

Hi Abe,

Having a look into stack trace and trace some source codes, I feel that
Message itself could be broken.
(Or maybe offset is not valid. I'm just using Kafka so I couldn't know the
details.)

Message class is from Kafka, and storm-kafka just seems to fetch them from
Kafka, and doesn't modify it.
(Also Message seems to be immutable.
https://github.com/apache/kafka/blob/0.8.1.1/core/src/main/scala/kafka/message/Message.scala
)

Could you try consume same offset without KafkaSpout, and call
Message.isValid() to check its validity?

Thanks,
Jungtaek Lim (HeartSaVioR)


2015-10-30 21:47 GMT+09:00 abe oppenheim :

> I realize this is a little vague, but any advice as to how I can
> troubleshoot this would be very helpful.
>
> My Topology uses a KafkaSpout. When I start the Topology I see the below
> exception and all my Executors die. Then the Worker tries to start again,
> the exception occurs, and the Executors die. This continues with no
> resolution, the Worker's uptime never exceeds a few seconds.
>
>
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_11]
> at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:359)
> ~[na:1.7.0_11]
> at kafka.message.Message.sliceDelimited(Message.scala:229)
> ~[stormjar.jar:na]
> at kafka.message.Message.payload(Message.scala:218) ~[stormjar.jar:na]
> at storm.kafka.KafkaUtils.generateTuples(KafkaUtils.java:201)
> ~[stormjar.jar:na]
> at storm.kafka.PartitionManager.next(PartitionManager.java:131)
> ~[stormjar.jar:na]
> at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:141)
> ~[stormjar.jar:na]
> at
> backtype.storm.daemon.executor$fn__4654$fn__4669$fn__4698.invoke(executor.clj:565)
> ~[storm-core-0.9.4.jar:0.9.4]
> at backtype.storm.util$async_loop$fn__458.invoke(util.clj:463)
> ~[storm-core-0.9.4.jar:0.9.4]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: 答复: [VOTE] Release Apache Storm 0.10.0 (rc1)

[임정택]

Cody,

You may want to leave this mail to "[VOTE] Accept Alibaba JStorm Code
Donation".

Thanks,
Jungtaek Lim (HeartSaVioR)

2015-10-29 21:43 GMT+09:00 Cody Innowhere <e.neve...@gmail.com>:

> +1
>
> I think programming languages as Clojure and Java is no gap for common
> developers, but it might be a problem for industrial users, who may not be
>  willing or have time to dive into the details of the Clojure core, and
> this is the great advantage of a Java core storm and will definitely
> attract more contributors to the community and improve the development of
> storm.
> Also, as JStorm describes, it has been tested in vast industrial use cases
> for stability and efficiency, if JStorm were merged into storm, it will
> surely make storm better & stronger.
>
> On Thu, Oct 29, 2015 at 8:19 PM, 方孝健(玄弟) <xiaojian@alibaba-inc.com>
> wrote:
>
> > +1， I think it is great!
> >
> > -邮件原件-
> > 发件人: Bobby Evans [mailto:ev...@yahoo-inc.com.INVALID]
> > 发送时间: 2015年10月26日 22:24
> > 收件人: dev@storm.apache.org
> > 主题: Re: [VOTE] Release Apache Storm 0.10.0 (rc1)
> >
> > +1 looks good, built from the git tag and ran some simple test.
> >  - Bobby
> >
> >
> >  On Friday, October 23, 2015 9:47 PM, 임정택 <kabh...@gmail.com> wrote:
> >
> >
> >-
> >
> >   test passed : OK
> >   -- extract source tar
> >   -- build via "mvn clean install”
> >
> >   -
> >
> >   deploy to docker cluster using wurstmeister/storm-docker : OK
> >   -
> >
> >   topology tests via storm-starter : OK
> >   -- run WordCounts and RollingTopWords, local mode and remote mode
> >
> >   -
> >
> >   test REST APIs (retrieve / activate / deactivate / rebalance / kill) :
> OK
> >
> >
> > Looks good overall. +1.
> >
> > 2015-10-24 5:26 GMT+09:00 P. Taylor Goetz <ptgo...@gmail.com>:
> >
> > > This is a call to vote on releasing Apache Storm 0.10.0 (rc1)
> > >
> > > Full list of changes in this release:
> > >
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHA
> > > NGELOG.md;hb=d02f94268dec229d1125a24fdf53fa303cbc2b29
> > >
> > > The tag/commit to be voted upon is v0.10.0:
> > >
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=45b1b1484
> > > 01fd05f0f79cc7abdf6b5c7fc43df20;hb=d02f94268dec229d1125a24fdf53fa303cb
> > > c2b29
> > >
> > > The source archive being voted upon can be found here:
> > >
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.0/apach
> > > e-storm-0.10.0-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.0/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEY
> > > S;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > >
> > > The Nexus staging repository for this release is:
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-1025
> > >
> > > Please vote on releasing this package as Apache Storm 0.10.0.
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > This vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache Storm 0.10.0 [ ]  0 No opinion [
> > > ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > -Taylor
> > >
> >
> >
> >
> > --
> > Name : 임 정택
> > Blog : http://www.heartsavior.net / http://dev.heartsavior.net Twitter :
> > http://twitter.com/heartsavior LinkedIn :
> > http://www.linkedin.com/in/heartsavior
> >
> >
> >
> >
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Accept Alibaba JStorm Code Donation

[임정택]

+1

2015-10-28 2:48 GMT+09:00 P. Taylor Goetz :

> All,
>
> The IP Clearance process for the Alibaba JStorm code donation has
> completed.
>
> The IP Clearance Status document can be found here:
>
> http://incubator.apache.org/ip-clearance/storm-jstorm.html
>
> The source code can be found at https://github.com/alibaba/jstorm with
> the following git commit SHA: e935da91a897797dad56e24c4ffa57860ac91878
>
> This is a VOTE to accept the code donation, and import the donated code
> into the Apache Storm git repository. Discussion regarding how to proceed
> with merging the codebases can take place in separate thread.
>
> [ ] +1 Accept the Alibaba JStorm code donation.
> [ ] +0 Indifferent
> [ ] -1 Do not accept the code donation because…
>
> This VOTE will be open for at least 72 hours.
>
> -Taylor
>
>
>


-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.10.0 (rc1)

[임정택]

   -

   test passed : OK
   -- extract source tar
   -- build via "mvn clean install”

   -

   deploy to docker cluster using wurstmeister/storm-docker : OK
   -

   topology tests via storm-starter : OK
   -- run WordCounts and RollingTopWords, local mode and remote mode

   -

   test REST APIs (retrieve / activate / deactivate / rebalance / kill) : OK


Looks good overall. +1.

2015-10-24 5:26 GMT+09:00 P. Taylor Goetz :

> This is a call to vote on releasing Apache Storm 0.10.0 (rc1)
>
> Full list of changes in this release:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=d02f94268dec229d1125a24fdf53fa303cbc2b29
>
> The tag/commit to be voted upon is v0.10.0:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=45b1b148401fd05f0f79cc7abdf6b5c7fc43df20;hb=d02f94268dec229d1125a24fdf53fa303cbc2b29
>
> The source archive being voted upon can be found here:
>
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.0/apache-storm-0.10.0-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.0/
>
> The release artifacts are signed with the following key:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1025
>
> Please vote on releasing this package as Apache Storm 0.10.0.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 0.10.0
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.9.6 (rc1)

[임정택]

   -

   test passed : OK
   -- extract source tar
   -- build via "mvn clean install”

   -

   deploy to docker cluster using wurstmeister/storm-docker : OK
   -

   topology tests via storm-starter : OK
   -- run WordCounts and RollingTopWords, local mode and remote mode

   -

   test REST APIs (retrieve / activate / deactivate / kill) : OK


Looks good overall. +1.

2015-10-24 5:22 GMT+09:00 P. Taylor Goetz <ptgo...@gmail.com>:

> My bad, and sorry for not noticing this email sooner.
>
> The artifacts are there now.
>
> -Taylor
>
> > On Oct 22, 2015, at 6:04 PM, 임정택 <kabh...@gmail.com> wrote:
> >
> > Seems like you missed uploading relevant files to that directory.
> > Could you check it please? Thanks!
> >
> > Jungtaek Lim (HeartSaVioR)
> >
> > 2015-10-23 6:17 GMT+09:00 P. Taylor Goetz <ptgo...@gmail.com>:
> >
> >> This is a call to vote on releasing Apache Storm 0.9.6 (rc1).
> >>
> >> Full list of changes in this release:
> >>
> >>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=39418b20e4c2d03e133d028507bb2f8961861cf0
> >>
> >> The tag/commit to be voted upon is v0.9.6:
> >>
> >>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=a7069a058669ed49accbdd932c4fdfe8837f1e93;hb=39418b20e4c2d03e133d028507bb2f8961861cf0
> >>
> >> The source archive being voted upon can be found here:
> >>
> >>
> >>
> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/apache-storm-0.9.6-src.tar.gz
> >>
> >> Other release files, signatures and digests can be found here:
> >>
> >> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/
> >>
> >> The release artifacts are signed with the following key:
> >>
> >>
> >>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >>
> >> The Nexus staging repository for this release is:
> >>
> >> https://repository.apache.org/content/repositories/orgapachestorm-1024
> >>
> >> Please vote on releasing this package as Apache Storm 0.9.6.
> >>
> >> When voting, please list the actions taken to verify the release.
> >>
> >> This vote will be open for at least 72 hours.
> >>
> >> [ ] +1 Release this package as Apache Storm 0.9.6
> >> [ ]  0 No opinion
> >> [ ] -1 Do not release this package because...
> >>
> >> Thanks to everyone who contributed to this release.
> >>
> >> -Taylor
> >>
>
>


-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.9.6 (rc1)

[임정택]

Seems like you missed uploading relevant files to that directory.
Could you check it please? Thanks!

Jungtaek Lim (HeartSaVioR)

2015-10-23 6:17 GMT+09:00 P. Taylor Goetz :

> This is a call to vote on releasing Apache Storm 0.9.6 (rc1).
>
> Full list of changes in this release:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=39418b20e4c2d03e133d028507bb2f8961861cf0
>
> The tag/commit to be voted upon is v0.9.6:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=a7069a058669ed49accbdd932c4fdfe8837f1e93;hb=39418b20e4c2d03e133d028507bb2f8961861cf0
>
> The source archive being voted upon can be found here:
>
>
> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/apache-storm-0.9.6-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/
>
> The release artifacts are signed with the following key:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1024
>
> Please vote on releasing this package as Apache Storm 0.9.6.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 0.9.6
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor
>

Re: Storm at Stackoverflow

[임정택]

Nice work Matthias! Upvoted.

Jungtaek Lim (HeartSaVioR)

2015-10-22 1:16 GMT+09:00 Matthias J. Sax :

> Hi,
>
> currently, there are two tags (apache-storm and storm) used on SO. I
> just suggested "apache-storm" to be the main tag and "storm" to be a
> synonym for it. This enables that all questions get tagged with a unique
> tag. Old and new questions get re-tag from storm to apache-storm
> automatically if the synonym get accepted. For this to happen, at least
> 4 upvotes must be casted.
>
> If you have an SO account, please upvote here:
> https://stackoverflow.com/tags/apache-storm/synonyms
>
> Thanks for your support!
>
> -Matthias
>
>


-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.9.6

[임정택]

Yeah, message-in-order is also critical for me since our team relies on
that.
Since we fixed this for now (by backporting), it will be fine for next RC.

2015-10-20 6:49 GMT+09:00 Derek Dagit <der...@yahoo-inc.com.invalid>:

> OK, well I hadn't seen this happen before downloading the release
> candidate.
>
> For 697059e958879c4daae5f4db57ce9abc04e81bd7:
> * Verified sigs & sums
> * built the source and ran all tests
> * deployed the tarball and ran word_count
>
>
> -1: HeartSaVioR says STORM-996 does affect 0.9.x, then I agree it is a
> blocker unfortunately since it could really cause trouble especially for
> Trident topologies.
>
> The netty test must have passed the one time I ran it.
>
>
>  --
> Derek
>
>
> - Original Message -
> From: 임정택 <kabh...@gmail.com>
> To: dev@storm.apache.org
> Cc:
> Sent: Saturday, October 17, 2015 9:07 AM
> Subject: Re: [VOTE] Release Apache Storm 0.9.6
>
> Taylor,
>
> Unfortunately I found that STORM-996 also affects 0.9.x-branch while
> running test on archived source code.
>
> I merged STORM-996 to master and 0.10.x-branch, but I forgot to check
> 0.9.x-branch since its affects version(s) was set to 0.10.0.
> https://issues.apache.org/jira/browse/STORM-996
>
> I'm backporting STORM-996 to 0.9.x-branch now, so please wait a moment and
> prepare a vote again.
>
> Thanks!
>
> Jungtaek Lim (HeartSaVioR)
>
>
>
> 2015-10-17 3:39 GMT+09:00 P. Taylor Goetz <ptgo...@apache.org>:
>
> > This is a call to vote on releasing Apache Storm 0.9.6.
> >
> > Full list of changes in this release:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=697059e958879c4daae5f4db57ce9abc04e81bd7
> >
> > The tag/commit to be voted upon is v0.9.6:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=4512970b2f2720a58fa73b310acf0cb65e28fa39;hb=697059e958879c4daae5f4db57ce9abc04e81bd7
> >
> > The source archive being voted upon can be found here:
> >
> >
> >
> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/apache-storm-0.9.6-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > http://people.apache.org/~ptgoetz/apache-storm-0.9.6/
> >
> > The release artifacts are signed with the following key:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1023
> >
> > Please vote on releasing this package as Apache Storm 0.9.6.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache Storm 0.9.6
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
> >
>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.9.6

[임정택]

Taylor,

Unfortunately I found that STORM-996 also affects 0.9.x-branch while
running test on archived source code.

I merged STORM-996 to master and 0.10.x-branch, but I forgot to check
0.9.x-branch since its affects version(s) was set to 0.10.0.
https://issues.apache.org/jira/browse/STORM-996

I'm backporting STORM-996 to 0.9.x-branch now, so please wait a moment and
prepare a vote again.

Thanks!

Jungtaek Lim (HeartSaVioR)


2015-10-17 3:39 GMT+09:00 P. Taylor Goetz :

> This is a call to vote on releasing Apache Storm 0.9.6.
>
> Full list of changes in this release:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=697059e958879c4daae5f4db57ce9abc04e81bd7
>
> The tag/commit to be voted upon is v0.9.6:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=4512970b2f2720a58fa73b310acf0cb65e28fa39;hb=697059e958879c4daae5f4db57ce9abc04e81bd7
>
> The source archive being voted upon can be found here:
>
>
> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/apache-storm-0.9.6-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/
>
> The release artifacts are signed with the following key:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1023
>
> Please vote on releasing this package as Apache Storm 0.9.6.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 0.9.6
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks to everyone who contributed to this release.
>
> -Taylor
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.9.6

[임정택]

OK, it's backported now.

Btw, I think we can start new release votes about 0.9.6 and
0.10.0 simultaneously.

Thanks,
Jungtaek Lim (HeartSaVioR)

2015-10-17 23:07 GMT+09:00 임정택 <kabh...@gmail.com>:

> Taylor,
>
> Unfortunately I found that STORM-996 also affects 0.9.x-branch while
> running test on archived source code.
>
> I merged STORM-996 to master and 0.10.x-branch, but I forgot to check
> 0.9.x-branch since its affects version(s) was set to 0.10.0.
> https://issues.apache.org/jira/browse/STORM-996
>
> I'm backporting STORM-996 to 0.9.x-branch now, so please wait a moment and
> prepare a vote again.
>
> Thanks!
>
> Jungtaek Lim (HeartSaVioR)
>
>
> 2015-10-17 3:39 GMT+09:00 P. Taylor Goetz <ptgo...@apache.org>:
>
>> This is a call to vote on releasing Apache Storm 0.9.6.
>>
>> Full list of changes in this release:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=697059e958879c4daae5f4db57ce9abc04e81bd7
>>
>> The tag/commit to be voted upon is v0.9.6:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=4512970b2f2720a58fa73b310acf0cb65e28fa39;hb=697059e958879c4daae5f4db57ce9abc04e81bd7
>>
>> The source archive being voted upon can be found here:
>>
>>
>> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/apache-storm-0.9.6-src.tar.gz
>>
>> Other release files, signatures and digests can be found here:
>>
>> http://people.apache.org/~ptgoetz/apache-storm-0.9.6/
>>
>> The release artifacts are signed with the following key:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>>
>> The Nexus staging repository for this release is:
>>
>> https://repository.apache.org/content/repositories/orgapachestorm-1023
>>
>> Please vote on releasing this package as Apache Storm 0.9.6.
>>
>> When voting, please list the actions taken to verify the release.
>>
>> This vote will be open for at least 72 hours.
>>
>> [ ] +1 Release this package as Apache Storm 0.9.6
>> [ ]  0 No opinion
>> [ ] -1 Do not release this package because...
>>
>> Thanks to everyone who contributed to this release.
>>
>> -Taylor
>>
>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

[DISCUSS] How about releasing 0.9.6 and 0.10.0?

[임정택]

Hi all,

Finally there's no critical / blocker issue on 0.10.0 again.
(Thanks Bobby for resolving headache issues!)

And we pushed some bugfix commits into 0.9.x branch which can make users
suffer.

I think we can try to release 0.10.0 for current 0.10.x branch, and 0.9.6
for current 0.9.x branch.

What do you think? Did I miss some issues for 0.10.0 or 0.9.6?

Thanks,
Jungtaek Lim (HeartSaVioR)

Re: backporting minor feature for storm-on-mesos logging to 0.9.x train?

[임정택]

FYI,

Bobby gives an opinion that he's OK to backport STORM-1056
.
Now, it's backported into 0.9.6.

Jungtaek Lim (HeartSaVioR)

2015-09-22 8:24 GMT+09:00 Erik Weathers :

> hi dev list,
>
> We recently merged in a change which allows for better logging in
> storm-on-mesos (https://github.com/mesos/storm):
>
>- https://github.com/apache/storm/pull/733
>
> This fix should go out in release 0.10.0.
>
> However I'd like to get it backported to the 0.9.x release train as well.
> I realize that this runs counter to the convention that only bug fixes
> should go into the 0.9.x train.  My argument is that the change is
> particularly minor and has marked benefit for storm-on-mesos.  So I'm
> lobbying to get this change backported into 0.9.x.  HeartSaVioR has kindly
> reviewed this proposal and is open to making this one time exception.
> However he wants buy-in from at least one other PMC member before allowing
> it to happen:
>
>- https://github.com/apache/storm/pull/733#issuecomment-142127363
>
> So this email is an attempt to solicit opinion(s) about making this
> exception.
>
> Thanks!
>
> - Erik
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: does anyone else hate the verbose logging of all PR comments in the Storm JIRAs?

[임정택]

Matthias,
Personally I can't turn off github notification because of I'm also
collaborator of Jedis, which repository is belong to Github. ;(

I just wish to reduce notifications via same event (Github event &
JIRA automatic posting) from dev. mailing list.
Actually I don't have strong opinion about this. Just a wish. :)


2015-09-22 17:23 GMT+09:00 Matthias J. Sax <mj...@apache.org>:

> On Github, you can disable mail notification about each comment in your
> profile configuration (at least for you personal email address -- I
> guess it still goes over the mailing list)
>
> Profile -> Settings -> Notification Center
>
> -Matthias
>
> On 09/22/2015 03:21 AM, Erik Weathers wrote:
> > Sure, STORM-*.  ;-)
> >
> > Here's a good example:
> >
> >- https://issues.apache.org/jira/browse/STORM-329
> >
> > Compare that to this one:
> >
> >- https://issues.apache.org/jira/browse/STORM-404
> >
> > STORM-404 has a bunch of human-created comments, but it's readable since
> it
> > has no github-generated comments.  STORM-329 however intermixes the human
> > comments with the github ones.  It's really hard to read through.
> >
> > To be clear, it's not that it's *confusing* per se -- it's that the
> > behavior is *cluttering* the comments, making it harder to see any
> > human-created comments since any JIRA issue with a PR will usually end up
> > with many automated comments.
> >
> > BTW, I totally agree that linking from the JIRA issue to the github PR is
> > important!  Would be even nicer if the github PRs also directly linked
> back
> > to the JIRA issue with a clickable link.
> >
> > - Erik
> >
> > On Mon, Sep 21, 2015 at 6:03 PM, 임정택 <kabh...@gmail.com> wrote:
> >
> >> Hi Erik,
> >>
> >> I think verbose logging of PR comments could be OK. I didn't experience
> any
> >> confusing.
> >> Maybe referring sample JIRA issues could help us to understand.
> >>
> >> But I'm also open to change cause other projects already have been
> doing.
> >> (for example, https://issues.apache.org/jira/browse/SPARK-10474)
> >>
> >> In addition to SPARK has been doing, I'd like to still leave some
> events on
> >> github PR to JIRA issue, too.
> >>
> >> Btw, the thing I'm really annoyed is multiple mail notifications on each
> >> github comment.
> >>
> >> Thanks,
> >> Jungtaek Lim (HeartSaVioR)
> >>
> >> 2015-09-22 9:15 GMT+09:00 Erik Weathers <eweath...@groupon.com>:
> >>
> >>> I find that these comments majorly distract from any discussion that
> may
> >>> occur in the JIRA issues themselves.   What value are these
> providing?  I
> >>> guess just insurance against GitHub being unavailable or going away?
> But
> >>> that doesn't seem worth the distraction cost.  Is there any possibility
> >> of
> >>> removing this spamminess, or somehow putting them into attachments
> within
> >>> the JIRA issues so that they aren't directly in the comments?
> >>>
> >>> - Erik
> >>>
> >>
> >>
> >>
> >> --
> >> Name : 임 정택
> >> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> >> Twitter : http://twitter.com/heartsavior
> >> LinkedIn : http://www.linkedin.com/in/heartsavior
> >>
> >
>
>


-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: does anyone else hate the verbose logging of all PR comments in the Storm JIRAs?

[임정택]

Hi Erik,

I think verbose logging of PR comments could be OK. I didn't experience any
confusing.
Maybe referring sample JIRA issues could help us to understand.

But I'm also open to change cause other projects already have been doing.
(for example, https://issues.apache.org/jira/browse/SPARK-10474)

In addition to SPARK has been doing, I'd like to still leave some events on
github PR to JIRA issue, too.

Btw, the thing I'm really annoyed is multiple mail notifications on each
github comment.

Thanks,
Jungtaek Lim (HeartSaVioR)

2015-09-22 9:15 GMT+09:00 Erik Weathers :

> I find that these comments majorly distract from any discussion that may
> occur in the JIRA issues themselves.   What value are these providing?  I
> guess just insurance against GitHub being unavailable or going away?  But
> that doesn't seem worth the distraction cost.  Is there any possibility of
> removing this spamminess, or somehow putting them into attachments within
> the JIRA issues so that they aren't directly in the comments?
>
> - Erik
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.10.0

[임정택]

+1

- test passed : OK
-- extract source tar
-- build via "mvn clean install”

- topology tests via storm-starter : OK
-- run WordCounts and RollingTopWords, local mode and remote mode

- test REST APIs (retrieve / activate / deactivate / kill) : OK

Looks good overall.

Btw, there're some issues which seems to point to 0.10.0, STORM-912
 and STORM-1039
, STORM-1042
.
I'm not sure we should resolve these before releasing.
No issue is marked as blocker, but STORM-1042 is marked as critical, and it
seems to be regression.

2015-09-12 5:37 GMT+09:00 P. Taylor Goetz :

> This is a call to vote on releasing Apache Storm 0.10.0.
>
> Full list of changes in this release:
>
> hhttps://
> git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=f6c0abb0bd76867b48c7b7c7cfdfdc7e0e6f483a
>
> The tag/commit to be voted upon is v0.10.0:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=66004167d440d418ec6c82c6a134a9c213d632b2;hb=f6c0abb0bd76867b48c7b7c7cfdfdc7e0e6f483a
>
> The source archive being voted upon can be found here:
>
>
> http://people.apache.org/~ptgoetz/apache-storm-0.10.0/apache-storm-0.10.0-src.tar.gz
>
> Other release files, signatures and digests can be found here:
>
> http://people.apache.org/~ptgoetz/apache-storm-0.10.0/
>
> The release artifacts are signed with the following key:
>
>
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>
> The Nexus staging repository for this release is:
>
> https://repository.apache.org/content/repositories/orgapachestorm-1019
>
> Please vote on releasing this package as Apache Storm 0.10.0.
>
> When voting, please list the actions taken to verify the release.
>
> This vote will be open for at least 72 hours.
>
> [ ] +1 Release this package as Apache Storm 0.10.0
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
>
> Thanks,
> Taylor
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Storm UI does not load

[임정택]

Hi, Matthias.

Could you clear Firefox cache and try again?
I met same issue from Chrome when I switch Storm's version from
0.10.0-beta1 to 0.9.5 or opposite, and resolved that issue by clearing
cache.
If it doesn't work, how about checking any errors from Firefox developer
tool or Firebug and report to dev mailing list?

Thanks,
Jungtaek Lim (HeartSaVioR)

2015-09-02 20:44 GMT+09:00 Matthias J. Sax :

> Hi,
>
> I am running Debian Jessy in my laptop. Up to now I never had problems
> to access Storm UI via Iceweasel (ie, Debian's Firefox). But now, the UI
> does not load completely. It displays "Loading summary..." and the
> background is shaded. The cluster summary on top of the page is not
> rendered correctly. See attached screenshot.
>
> The UI loads without problems using Opera.
>
> I am building Storm from the sources using the current master version
> (0.11.0-SNAPSHOT). I just rebased. The last commit is
>
> > commit 154e9ec55deb4eea8fca8554e4d3b224bf337834
>
> I am not sure what the cause of the problem might be, but I do install
> available OS updates regularly. I guess that an update changed
> something. (Maybe an upgrade to a new version of Iceweasel?) The
> currently installed version is 31.8.0.
>
> Maybe anyone can reproduce the problem and have a look into it? For now,
> I just use Opera. So for me, it's not urgent.
>
>
> -Matthias
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Cannot run WordCountExample in Intellij

[임정택]

Hi Matthias, sorry to respond lately.

Unfortunately, AFAIK you can't run multilang feature with LocalCluster
without having packaged file.

ShellProcess relies on codeDir of TopologyContext, which is used by
supervisor. Workers are serialized to stormcode.ser, but multilang files
should extracted to outside of serialized file so that python/ruby/node/etc
can load it.

Accomplishing this with distribute mode is easy because there's always user
submitted jar, and supervisor can know it is what user submitted.

But accomplishing this with local mode is not easy cause supervisor cannot
know user submitted jar, and users can run topology to local mode without
packaging.

So, Supervisor in local mode finds resource directory (resources) from
each jars (which ends with jar) in classpath, and copy first occurrence
to codeDir.

storm jar places user topology jar to the first of classpath, so it can be
run without issue.

So normally, it's natural for ShellProcess to not find splitsentence.py.
Maybe your working directory or PYTHONPATH do the trick.

Hope this helps.

Best,
Jungtaek Lim (HeartSaVioR)
ps.
I also respond to your SO question with same content.
http://stackoverflow.com/a/32085316/3480122

2015-08-10 6:49 GMT+09:00 Matthias J. Sax mj...@informatik.hu-berlin.de:

 Hi,

 I work with Storm for a while already, but want to get started with
 development. As suggested, I am using Intellij (up to now, I was using
 Eclipse).

 I was also looking at

 https://github.com/apache/storm/tree/master/examples/storm-starter#intellij-idea

 This documentation is not complete. I was not able to run anything in
 Intellij first. I could figure out, that I need to remove the scope of
 storm-core dependency (in storm-starter pom.xml). (found here:

 https://stackoverflow.com/questions/30724424/storm-starter-with-intellij-idea-maven-project-could-not-find-class
 )

 After that I wass able to build the project. I can also run
 ExclamationTopology with no problems within Intellij. However,
 WordCountTopology fails.

 First I got the following error:

  java.lang.RuntimeException: backtype.storm.multilang.NoOutputException:
 Pipe to subprocess seems to be broken! No output read.
  Serializer Exception:
  Traceback (most recent call last):
File splitsentence.py, line 16, in module
  import storm
  ImportError: No module named storm

 I was able to resolve it via: apt-get install python-storm
 (from StackOverflow)

 However, I don't speak Python and was wondering what the problem is and
 why I could resolve it like this. Just want to get deeper into it. Maybe
 someone can explain.

 Unfortunately, I am getting a different error now:

  java.lang.RuntimeException: backtype.storm.multilang.NoOutputException:
 Pipe to subprocess seems to be broken! No output read.
  Serializer Exception:
  Traceback (most recent call last):
File splitsentence.py, line 16, in module
  import storm
  ImportError: No module named storm

 I did not find any solution on the Internet. And as I am not familiar
 with Python and never used Storm differently as low-level Java API I am
 stuck now. Because ExclamationTopology runs, I guess my basic setup is
 correct.

 What do I do wrong?

 -Matthias






-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Should spouts and bolts be stateless to guarantee fault tolerance?

[임정택]

Hi.

First of all, you may want to read Guaranteeing Message Processing
http://storm.apache.org/documentation/Guaranteeing-message-processing.html to
see how it works.

In short, Spout and Bolt should also be stateless since worker could be
killed and restarted. (at any time)
Also data source for Spout is responsible for replaying the messages when
Spout is killed and restarted.
If data source cannot support it, your spout is responsible for. One way to
support it is storing processed offset to external system.

Hope this helps.

Regards,
Jungtaek Lim (HeartSaVioR)


2015-08-14 15:59 GMT+09:00 王立 wangli1...@gmail.com:

 Hi all,

 I have a question regarding to the fault tolerance mechanism in storm.

 I believe the key idea behind fault tolerance mechanism is the
 statelessness of Nimbus and Supervisors, i.e., storing the state on
 zookeepers, such that the failure on Nimbus and Supervisors does not result
 in state loss and they can be restarted as nothing happened.

 My question is: does this means the user has to ensure their spouts and
 bolts to be stateless, in order to guarantee fault tolerance?

 Any comment will be highly appreciated! Thanks.

 Regards,
 Li






-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Backport bugfixes (to 0.10.x / 0.9.x)

[임정택]

I'm just done with backporting 4 issues to 0.10.x-branch.

- Jungtaek Lim (HeartSaVioR)

2015-08-04 10:29 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 Good catch on storm-903.

 I'll take a closer look.

 -Taylor

  On Aug 3, 2015, at 7:25 PM, 임정택 kabh...@gmail.com wrote:
 
  Thanks all.
 
  I also think that it is really painful to backport something. That's why
 I
  asked about what version lines we'll consider from other thread.
  Seems like we're sure about releasing official version of 0.10.0 and
  phasing out 0.9.x lines.
  I'll backport bugfixes to only 0.10.x-branch and let you know when I
  finished.
 
  Before start releasing 0.10.0, we may take a look at STORM-903
  https://issues.apache.org/jira/browse/STORM-903, which seems to be not
  finished.
 
  Thanks,
  Jungtaek Lim (HeartSaVioR)
 
 
  2015-08-04 5:36 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:
 
  Thanks for putting together this list Jungtaek.
 
  Back-porting is a pain, and the more the 0.9.x, 0.10.x and master lines
  diverge, the harder it gets.
 
  I propose we back-port the 4 fixes you identified for the 0.10 branch,
 and
  start discussing releasing 0.10.0 (final, not beta).
 
  Once 0.10.0 is out, I think we can start phasing out the 0.9.x line. The
  idea was to continue to support 0.9.x while 0.10.0 stabilized and allow
  early upgraders had a chance to kick the tires and report any glaring
  issues. IMO more than enough time has passed and we should move forward
  with a 0.10.0 release.
 
  In terms of the who and when of back porting, the general principle I’ve
  followed is that once a patch has been merged, it is a candidate for
  back-porting, and that any committer can do that since the patch had
  already been reviewed and accepted. I don’t think a separate pull
 request
  is necessary. In fact, I think extra pull requests for back-porting
 makes
  JIRA/Github issues a little messy and confusing.
 
  IMO the only time we need back-port pull requests is:
 
  a) A non-committer contributor is requesting a patch be applied to an
  earlier version.
  b) A committer back-ported a patch with a lot of conflicts, and feels it
  warrants further review before committing. Basically a way of saying
 “This
  merge was messy. Could others check my work?”
 
  If things go wrong at any time, there’s always “git revert”.
 
  I don’t think we need to codify any of this in our BYLAWS unless there
 is
  some sort of conflict, which for now there isn’t. If we feel the need to
  document the process I feel documenting it README/wiki entry should
  suffice. I’m more in favor of mutual trust among committers than hard
 and
  fast rules. Once a particular practice gets formalized in our bylaws, it
  can be very difficult to change.
 
  -Taylor
 
 
  On Aug 3, 2015, at 12:56 PM, Derek Dagit der...@yahoo-inc.com.INVALID
 
  wrote:
 
  Dealing with branches is a pain, and it is good we are paying attention
  to
  back-porting.  It is good to bring it up for discussion, and I agree
  checking
  with those who do releases is a reasonable thing to do.
 
  I do not think there are special restrictions on back-porting fixes to
  previous
  branches.  I would be comfortable with the normal rules for a pull
  request.
 
  Effort is one cost, and we could eventually run into some more
  challenging
  merge conflicts as well. There are multiple things to consider, and I
  think it
  is a judgment call.
 
  On the other hand, if it does become clear that clarifying principles
  helpful
  in our BYLAWS, then I am all for it.  If we commit to supporting
 specific
  branches with certain kinds of fixes, then we need to stick to such a
  commitment.
 
  --
  Derek
 
 
  - Original Message -
  From: Parth Brahmbhatt pbrahmbh...@hortonworks.com
  To: dev@storm.apache.org dev@storm.apache.org
  Cc:
  Sent: Monday, August 3, 2015 11:26 AM
  Subject: Re: [DISCUSS] Backport bugfixes (to 0.10.x / 0.9.x)
 
  Given how huge 0.10 release was I feel trying to back port all bug
 fixes
  and testing that it does not brake something else might turn out to be
 a
  huge PITA. I think going with a stable 0.10 release might be the best
  solution for now.
 
  I don’t think back porting requires confirmation however given we will
  probably have to do release for each version where back porting was
 done
  it is probably best to notify Release manager and discuss options. I
  agree
  having a rule/bylaw would help clarify things for future.
 
  Thanks
  Parth
 
 
  On 8/2/15, 4:30 PM, 임정택 kabh...@gmail.com wrote:
 
  Bump. Does anyone have opinions about this?
 
  I already did back-port some bugfixes (not in list) into 0.10.x and
  0.9.x
  lines, but I'm not 100% sure that it is preferred way.
  Seems like we don't have explicit rules about doing back-port. Only
  thing
  I
  know is Taylor was (or has been) a gatekeeper.
 
  Now I really want to know that it still need to be confirmed by Taylor
  before doing back-port.
 
  Thanks,
  Jungtaek Lim (HeartSaVioR

Re: [DISCUSS] Backport bugfixes (to 0.10.x / 0.9.x)

[임정택]

Bump. Does anyone have opinions about this?

I already did back-port some bugfixes (not in list) into 0.10.x and 0.9.x
lines, but I'm not 100% sure that it is preferred way.
Seems like we don't have explicit rules about doing back-port. Only thing I
know is Taylor was (or has been) a gatekeeper.

Now I really want to know that it still need to be confirmed by Taylor
before doing back-port.

Thanks,
Jungtaek Lim (HeartSaVioR)


2015-07-28 8:27 GMT+09:00 임정택 kabh...@gmail.com:

 Hi all,

 Recently I see many bugfixes are only merged to master, or 0.10.x-branch.

 Since 0.10.0-beta1 introduces huge changeset, and it contains a lot of
 bugfixes, I think we can consider backporting them to 0.9.x-branch before
 releasing 0.9.6.

 I create a sheet and write down bugfix issues which could be backported,
 and status of issue. (what versions it is applied, and what versions it can
 be applied)


 https://docs.google.com/spreadsheets/d/1KQrOlqk1hlE2oDmXFY34lJaY0PU7V5uxq9U1vfIhLq4/edit?usp=sharing

 Please let me know whenever you find missing spots or wrong contents.

 There seems to be other approach:
 - release stable version of 0.10.0, and drop plan to release 0.9.6 so that
 let all users who want bugfix release move to 0.10.0

 Since a lot of bugfix issues are waiting for backporting, alternative
 approach may be make sense.

 I'm open to hear any thoughts, so please share your opinions.

 Thanks,
 Jungtaek Lim (HeartSaVioR)

 to. Taylor
 I don't know I can do backport without your confirmation. (by each issue)
 If you want to decide about backporting yourself, I'll follow you.




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

[DISCUSS] Backport bugfixes (to 0.10.x / 0.9.x)

[임정택]

Hi all,

Recently I see many bugfixes are only merged to master, or 0.10.x-branch.

Since 0.10.0-beta1 introduces huge changeset, and it contains a lot of
bugfixes, I think we can consider backporting them to 0.9.x-branch before
releasing 0.9.6.

I create a sheet and write down bugfix issues which could be backported,
and status of issue. (what versions it is applied, and what versions it can
be applied)

https://docs.google.com/spreadsheets/d/1KQrOlqk1hlE2oDmXFY34lJaY0PU7V5uxq9U1vfIhLq4/edit?usp=sharing

Please let me know whenever you find missing spots or wrong contents.

There seems to be other approach:
- release stable version of 0.10.0, and drop plan to release 0.9.6 so that
let all users who want bugfix release move to 0.10.0

Since a lot of bugfix issues are waiting for backporting, alternative
approach may be make sense.

I'm open to hear any thoughts, so please share your opinions.

Thanks,
Jungtaek Lim (HeartSaVioR)

to. Taylor
I don't know I can do backport without your confirmation. (by each issue)
If you want to decide about backporting yourself, I'll follow you.

Re: 0.9.6 release for STORM-763/839?

[임정택]

In addition to Enno's mail, I'd like to know we'd like to maintain three
version lines (currently 0.9.x, 0.10.x, 0.11.x) continuously, or it is just
a period of transition.

I'm maintaining three version lines (bugfix, next minor, next major - with
respecting semver) from other project, and sometimes maintaining it is
really annoying, though most of the issues are breaking changes of next
major and current.

Since stable version of Storm is 0.9.5, and some users want to upgrade
Storm with minimized impact so releasing 0.9.6 could make sense for now.
(It may be better to collect some bugfixes and backport before releasing
0.9.6 since other version lines already has many bugfixes which can be
applied to 0.9.x.)

But I also think we should make an effort to make 0.10.0 stable and release
official version.
Storm 0.10.0-beta was released one month ago, and we don't have a plan to
release next beta, or stable version.
After releasing 0.10.0 we can choose whether we maintain 0.9.x lines or not.

If it's possible to release stable version of 0.10.0 faster, for example,
before end of this month, then we can delegate applying any bugfix issues
(except critical / blocker) to 0.10.0.

tl;dr.
We can make an effort to 0.10.0 to make it stable, and release official
version faster.
If stable version of 0.10.0 could be released shortly, I would not want to
release any new 0.9.x versions except any critical or blocker bugs.

Thanks,
Jungtaek Lim (HeartSaVioR)


2015-07-15 5:04 GMT+09:00 Enno Shioji eshi...@gmail.com:

 Hi Taylor,


 @Bobby recommended I talk to you about a potential 0.9.6 release to fix
 STORM-763/839. The fixes are already merged to master, 0.10.x and 0.9.x.

 In a nutshell, they fix the following symptoms -- but the catch is that
 these symptoms only surface when connections between bolts are lost
 frequently. So I'm not sure whether it warrants a release. The symptoms
 are:
  - Thread deadlock hazard in Netty Client (STORM-839)
  - Slow reconnection (STORM-763)
  - Verbose error log (STORM-763)

 Anyways, just thought I'll ping you. Btw thanks for your work, we are using
 Storm extensively and it's been amazing!


 Enno




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

0.9.6 release for STORM-763/839?

[임정택]

To be more clear,  I'm fine with maintaining three minor version lines
while it is period of transition.
Just a thing I'm curious is we'd like to maintain two stable version lines
continuously (with one unstable).

For now, I'm fine with releasing 0.9.6 with backporting bugfixes which are
already committed to master but not backported.
(Same things applied to 0.10.0)

Btw, 0.10.0 has its own issue, STORM-903
https://issues.apache.org/jira/browse/STORM-903, which is discussed but
not resolved. We may want to resolve it before next release.

Best,
Jungtaek Lim (HeartSaVioR)

2015-07-15 11:02 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com
javascript:_e(%7B%7D,'cvml','ptgo...@gmail.com');:

 My opinion:

 In terms of supporting the 0.9.x release line, I think that’s critical, at
 the very least until 0.10.0 stable is released. More likely for some time
 thereafter for those who don’t want to (or can’t) upgrade.

 Updates to the the 0.9.x line should be limited to bug fixes.

 I’d like to release a 0.10.0-beta2 or 0.10.0 soon. I haven’t seen much in
 terms of end user feedback on the beta. A few user/dev success stories
 would make me feel better about dropping the beta tag.

 As always, I’m open to any and all opinions.

 -Taylor


  On Jul 14, 2015, at 5:43 PM, 임정택 kabh...@gmail.com
 javascript:_e(%7B%7D,'cvml','kabh...@gmail.com'); wrote:
 
  In addition to Enno's mail, I'd like to know we'd like to maintain three
  version lines (currently 0.9.x, 0.10.x, 0.11.x) continuously, or it is
 just
  a period of transition.
 
  I'm maintaining three version lines (bugfix, next minor, next major -
 with
  respecting semver) from other project, and sometimes maintaining it is
  really annoying, though most of the issues are breaking changes of next
  major and current.
 
  Since stable version of Storm is 0.9.5, and some users want to upgrade
  Storm with minimized impact so releasing 0.9.6 could make sense for now.
  (It may be better to collect some bugfixes and backport before releasing
  0.9.6 since other version lines already has many bugfixes which can be
  applied to 0.9.x.)
 
  But I also think we should make an effort to make 0.10.0 stable and
 release
  official version.
  Storm 0.10.0-beta was released one month ago, and we don't have a plan to
  release next beta, or stable version.
  After releasing 0.10.0 we can choose whether we maintain 0.9.x lines or
 not.
 
  If it's possible to release stable version of 0.10.0 faster, for example,
  before end of this month, then we can delegate applying any bugfix issues
  (except critical / blocker) to 0.10.0.
 
  tl;dr.
  We can make an effort to 0.10.0 to make it stable, and release official
  version faster.
  If stable version of 0.10.0 could be released shortly, I would not want
 to
  release any new 0.9.x versions except any critical or blocker bugs.
 
  Thanks,
  Jungtaek Lim (HeartSaVioR)
 
 
  2015-07-15 5:04 GMT+09:00 Enno Shioji eshi...@gmail.com
 javascript:_e(%7B%7D,'cvml','eshi...@gmail.com');:
 
  Hi Taylor,
 
 
  @Bobby recommended I talk to you about a potential 0.9.6 release to fix
  STORM-763/839. The fixes are already merged to master, 0.10.x and 0.9.x.
 
  In a nutshell, they fix the following symptoms -- but the catch is that
  these symptoms only surface when connections between bolts are lost
  frequently. So I'm not sure whether it warrants a release. The symptoms
  are:
  - Thread deadlock hazard in Netty Client (STORM-839)
  - Slow reconnection (STORM-763)
  - Verbose error log (STORM-763)
 
  Anyways, just thought I'll ping you. Btw thanks for your work, we are
 using
  Storm extensively and it's been amazing!
 
 
  Enno
 
 
 
 
  --
  Name : 임 정택
  Blog : http://www.heartsavior.net / http://dev.heartsavior.net
  Twitter : http://twitter.com/heartsavior
  LinkedIn : http://www.linkedin.com/in/heartsavior




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior


-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

[Infra] Github mirror cannot catch up ASF repo now

[임정택]

Hi!

I found that Github mirror cannot catch up ASF repo now.
(Last update on mirror was 5 days ago.)

I reported this behavior to Apache INFRA,
https://issues.apache.org/jira/browse/INFRA-9978

Btw, seems like other projects also have this issue, too.

Thanks,
Jungtaek Lim (HeartSaVioR)

Re: What's the next version of Storm 0.10.0-beta1?

[임정택]

I completely forgot to check pom.xml in 0.10.x-branch, which is
0.10.0-beta2-SNAPSHOT.
I'd like to use 0.10.0-beta2 for now, and let Taylor (or PMC's consensus)
take care of removing beta label.

Sorry about the confusion.

Best,
Jungtaek Lim (HeartSaVioR)

2015-07-08 0:46 GMT+09:00 임정택 kabh...@gmail.com:

 I agree with Bobby, and Taylor.

 Only thing I'd like to point out is, since we're labeling issue (and
 modifying CHANGELOG.md) to next version, so we may always need to have next
 versions of all branch lines.
 For example, we may be better to know previously that next version of
 0.10.x branch line is 0.10.0 (not 0.10.0-beta2), and it will be changed
 only when we  agree about changing release version.

 I'll apply these changes (with Dan's change) to 0.10.0, and modify
 CHANGELOG.md.

 Thanks!

 Best,
 Jungtaek Lim (HeartSaVioR)

 2015년 7월 8일 수요일, P. Taylor Goetzptgo...@gmail.com님이 작성한 메시지:

 I agree. It would be good to reserve the 0.9.x line for bug fixes only,
 and changes 0.10.0 should be somewhat limited to minor improvements and bug
 fixes. The focus for removing the “beta” label for 0.10.0 should be
 stabilization rather than new features. That being said, it is not a hard
 rule. I’m open to pulling anything in as long as there is PMC/dev community
 consensus.

 I’m also fine with pulling in Dan’s change to 0.10.x.

 -Taylor

  On Jul 7, 2015, at 10:24 AM, Bobby Evans ev...@yahoo-inc.com.INVALID
 wrote:
 
  I personally am fine with pulling in Dan's change to 0.10.x.  From a
 documentation perspective I think we need to come to a consensus on how we
 want to manage releases.  In other projects I have worked on there is a
 release manager that is approved for a given release line.  They are the
 gate keeper for what does and does not go in.  I think we are small enough
 that we can forgo with that formality so long as we are in agreement. From
 my perspective 0.9.x is stable and closed to new features, only bug fixes
 will go in there.  0.10.x is still in beta and I am fine with small
 improvements that were missed going in, but primarily bug fixes.  Master is
 open for new features.  In general if something goes into a previous
 release line, it also needs to go into all higher numbered active lines.
 Also in general even though master is open to new features and it is a
 different version number we should avoid making changes that break binary
 compatibility.  That is not to say that it is forbidden, it is to say that
 if we can make the change without breaking compatibility we should.
  If we get to the point where there is conflict about what should go
 into a given release then we can revisit the bylaws at that point and
 resolve the issue.  we already have rules about how to merge in code.  It
 does not specify which branch that code goes into.  If you want to change
 the version number to 0.10.0 from 0.10.0-beta1 lets get the last of the
 missed features in and make the switch.
  - Bobby
 
 
 
  On Monday, July 6, 2015 11:40 PM, 임정택 kabh...@gmail.com wrote:
 
 
  Hi Bobby! Thanks for answering. :)
 
  Yes, I also think we need to keep 0.10.0's features as same, and accept
  only bugfix.
 
  First one is about documentation, so that I can feel that it's safe to
  merge to frozen branch.
  Second one is about bugfix introduced on 0.10.0-beta, so it should be
  merged to 0.10.x-branch.
 
  For now, I'd like to name version to 0.10.0.
  When we have different opinion, we can change CHANGELOG.md and relevant
  JIRA issues.
 
  Next thing I'm wondering is omitting functionality from already
 introduced
  feature.
  Dan Blanchard said he omitted one thing which he actually wanted, see
 
 https://issues.apache.org/jira/browse/STORM-789?focusedCommentId=14615630page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14615630
  .
  In this case, does Dan need to wait for 0.11.0 to add missed thing?
 
  Thanks,
  Jungtaek Lim (HeartSaVioR)
 
 
  2015-07-07 1:31 GMT+09:00 Bobby Evans ev...@yahoo-inc.com.invalid:
 
  I thought that 0.10.0 is now bug fix only, unless Taylor has a
 different
  opinion.  Once we feel that it is stable then we can drop the beta.
- Bobby
 
 
 
On Tuesday, June 30, 2015 5:03 PM, 임정택 kabh...@gmail.com
 wrote:
 
 
Hi!
 
  Since I started reviewing and merging PRs, there're some PRs which are
  suitable to master, and also 0.10.0.
  (STORM-843 https://issues.apache.org/jira/browse/STORM-843,
 STORM-866
  https://issues.apache.org/jira/browse/STORM-866)
 
  We didn't define next step of Storm 0.10.0 beta, so I cannot merge them
  into 0.10.x-branch.
 
  Maybe two questions have to be answered before merging into
 0.10.x-branch.
 
  1. What's the next version of 0.10.0-beta1? beta2 or stable?
  2. Will Taylor handle them, or it is at the discretion of the
 Committer?
 
  For now I merged them into master only.
 
  If topic was discussed, please let me know so that I can find the
 result.
 
  Thanks,
  Jungtaek Lim (HeartSaVioR

Re: What's the next version of Storm 0.10.0-beta1?

[임정택]

Hi Bobby! Thanks for answering. :)

Yes, I also think we need to keep 0.10.0's features as same, and accept
only bugfix.

First one is about documentation, so that I can feel that it's safe to
merge to frozen branch.
Second one is about bugfix introduced on 0.10.0-beta, so it should be
merged to 0.10.x-branch.

For now, I'd like to name version to 0.10.0.
When we have different opinion, we can change CHANGELOG.md and relevant
JIRA issues.

Next thing I'm wondering is omitting functionality from already introduced
feature.
Dan Blanchard said he omitted one thing which he actually wanted, see
https://issues.apache.org/jira/browse/STORM-789?focusedCommentId=14615630page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14615630
.
In this case, does Dan need to wait for 0.11.0 to add missed thing?

Thanks,
Jungtaek Lim (HeartSaVioR)


2015-07-07 1:31 GMT+09:00 Bobby Evans ev...@yahoo-inc.com.invalid:

 I thought that 0.10.0 is now bug fix only, unless Taylor has a different
 opinion.  Once we feel that it is stable then we can drop the beta.
  - Bobby



  On Tuesday, June 30, 2015 5:03 PM, 임정택 kabh...@gmail.com wrote:


  Hi!

 Since I started reviewing and merging PRs, there're some PRs which are
 suitable to master, and also 0.10.0.
 (STORM-843 https://issues.apache.org/jira/browse/STORM-843, STORM-866
 https://issues.apache.org/jira/browse/STORM-866)

 We didn't define next step of Storm 0.10.0 beta, so I cannot merge them
 into 0.10.x-branch.

 Maybe two questions have to be answered before merging into 0.10.x-branch.

 1. What's the next version of 0.10.0-beta1? beta2 or stable?
 2. Will Taylor handle them, or it is at the discretion of the Committer?

 For now I merged them into master only.

 If topic was discussed, please let me know so that I can find the result.

 Thanks,
 Jungtaek Lim (HeartSaVioR)







-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: New Committer/PMC Member: Jungtaek Lim

[임정택]

Thanks everyone!

Jungtaek Lim (HeartSaVioR)

2015-06-30 7:00 GMT+09:00 Bobby Evans ev...@yahoo-inc.com.invalid:

 Yes Congratulations well deserved.
  - Bobby



  On Monday, June 29, 2015 4:58 PM, Harsha st...@harsha.io wrote:


  Congrats  Welcome Jungtaek
 -Harsha

 On Mon, Jun 29, 2015, at 01:30 PM, Derek Dagit wrote:
  Welcome, Jungtaek!
 
 
  --
  Derek
 
 
 
  - Original Message -
  From: P. Taylor Goetz ptgo...@apache.org
  To: dev@storm.apache.org
  Cc:
  Sent: Monday, June 29, 2015 2:26 PM
  Subject: New Committer/PMC Member: Jungtaek Lim
 
  Please join me in welcoming Jungtaek Lim (AKA “HeartSaVioR) as a new
  Apache Storm Committer and PMC member.
 
  Jungtaek has demonstrated a strong commitment to the Apache Storm
  community through active participation and mentoring on the Storm mailing
  lists. Furthermore, he has authored many enhancements and bug fixes
  spanning both Storm’s core codebase, as well as a numerous integration
  components.
 
  Congratulations and welcome Jungtaek!
 
  -Taylor






-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Gathering opinion: changing multilang heartbeat mechanism

[임정택]

Hi!

Since it's about multilang feature and you can use your own implementation
of multilang (and I believe multilang library developers are subscribing
user group), I wanna get opinion about changing multilang heartbeat
mechanism.

At Storm 0.9.3, Storm introduces multilang heartbeat feature.
http://storm.apache.org/documentation/Multilang-protocol.html
If you use Storm 0.9.3 and higher, and didn't know about the change, you
may skip this mail.

Since it contains some design constraint, I'm trying my best to add
workarounds, but it cannot cover whole situation (STORM-738
https://issues.apache.org/jira/browse/STORM-738). That's why I want to
change mechanism to get rid of design constraint.

AS-IS (STORM-513 https://issues.apache.org/jira/browse/STORM-513)

- When subprocess receives heartbeat tuple, subprocess sends sync to parent.
- ShellSpout / ShellBolt updates last heartbeat timestamp when it receives
sync.
-- added workaround : ShellSpout / ShellBolt updates timestamp when it
receives any kind of message. (It doesn't applied to ShellBolt yet, but
it's ready for review. STORM-742
https://issues.apache.org/jira/browse/STORM-742)
- ShellSpout / ShellBolt checks last heartbeat timestamp periodically, and
if timestamp is not updated well, it suicides itself.

TO-BE (STORM-871 https://issues.apache.org/jira/browse/STORM-871)

- Subprocess has to update pid file's modified time periodically.
-- In default implementation, it updates pid file every 1 sec.
-- It should be handled concurrently with executing pending tuples.
-- Some languages couldn't implement this clearly, but I don't have an idea
what languages could be.
- ShellSpout / ShellBolt checks last heartbeat timestamp by reading pid
file's modified time periodically, and if timestamp is not updated well, it
suicides itself.
- Heartbeat tuple is removed.

Please let me know your opinion, especially when you're developing
multilang libraries.

Thanks,
Jungtaek Lim (HeartSaVioR)

Re: [VOTE] Release Apache Storm 0.10.0-beta

[임정택]

+1 (non-binding)

- test passed : OK
-- extract source tar
-- build via mvn clean install”
- topology tests via storm-starter : OK
-- run WordCounts and RollingTopWords, local mode and remote mode
- test REST APIs (upload / activate / deactivate / kill) : OK
-- upload / activate / deactivate / rebalance / kill works fine
-- confirmed that STORM-853 was fixed
- kill topology and restart nimbus immediately : OK
-- confirmed that STORM-856 was fixed

Looks good overall.

Thanks!
Jungtaek Lim (HeartSaVioR)


2015-06-10 11:18 GMT+09:00 P. Taylor Goetz ptgo...@apache.org:

 This is a call to vote on releasing Apache Storm 0.10.0-beta.

 Full list of changes in this release:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=d1e10ceecba4f87f1e70bf02ec63094eef4f4bb3

 The tag/commit to be voted upon is v0.10.0-beta:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=b965da3a244806eb9bfd8d7d943f72891c362cbf;hb=d1e10ceecba4f87f1e70bf02ec63094eef4f4bb3

 The source archive being voted upon can be found here:


 http://people.apache.org/~ptgoetz/apache-storm-0.10.0-beta-rc1/apache-storm-0.10.0-beta-src.tar.gz

 Other release files, signatures and digests can be found here:

 http://people.apache.org/~ptgoetz/apache-storm-0.10.0-beta-rc1/

 The release artifacts are signed with the following key:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd

 The Nexus staging repository for this release is:

 https://repository.apache.org/content/repositories/orgapachestorm-1017

 Please vote on releasing this package as Apache Storm 0.10.0-beta.

 When voting, please list the actions taken to verify the release.

 This vote will be open for at least 72 hours.

 [ ] +1 Release this package as Apache Storm 0.10.0-beta
 [ ]  0 No opinion
 [ ] -1 Do not release this package because...

 Thanks,
 Taylor




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.10.0 (rc2)

[임정택]

OK, I see your point.

I know Storm 0.10.0 is postponed several times, and it already has very
huge changeset.
I don't mean to say we have to postpone again, just want to hear
opinion that we're OK to release with know issue, or fix it ASAP and reopen
vote.

I'm +1 (non-binding) with describing known issue to release note so that
users don't meet strange behavior without information.

Thanks!
Jungtaek Lim (HeartSaVioR)


2015년 6월 9일 화요일, P. Taylor Goetzptgo...@gmail.com님이 작성한 메시지:

 I’m +1 for moving forward with the current release candidate. I think we
 can address STORM-813 in a follow-up bug fix/maintenance release (soon).

 In the big picture, the REST upload API is just one of many, many
 features/improvements added in 0.10.0, and we’ve not leveraged it in any
 way yet (i.e. in Storm UI). I don’t think it should block the release.

 -Taylor

 On Jun 6, 2015, at 2:16 AM, 임정택 kabh...@gmail.com javascript:; wrote:

  Unfortunately I found a bug from uploadTopology REST API.
  (Sorry I participated to review that feature but I didn't think about
  multiple arguments.)
 
  Here's my checklist.
 
  - test passed : OK
  -- extract source tar
  -- build via mvn clean install”
  - topology tests via storm-starter : OK
  -- run WordCounts and RollingTopWords, local mode and remote mode
  - test REST APIs (upload / activate / deactivate / kill) : FAIL
  -- activate / deactivate / kill works fine
  -- upload topology fails with multiple arguments
  -- filed issue to https://issues.apache.org/jira/browse/STORM-853
 
  Uploading topology is a new feature, so I'd like to include fixed version
  of feature to 0.10.0.
  (https://github.com/apache/storm/pull/581 is ready for the fix.)
 
  Btw, STORM-813 (correcting storm-starter's README) is related to 0.10.0
 so
  I'd like to include it, but just applying it to Github repo could be
  acceptable.
 
  Thanks!
  Jungtaek Lim (HeartSaVioR)
 
  2015-06-06 7:59 GMT+09:00 Kishorkumar Patil kpa...@yahoo-inc.com.invalid
 :
 
  +1. LGTM. I built and ran some tests.
  -Kishor
 
 
 
  On Friday, June 5, 2015 4:38 PM, P. Taylor Goetz ptgo...@gmail.com
 javascript:;
  wrote:
 
 
  The v0.10.0 tag was already updates as part of the release process. The
  release candidate was built from that tag.
 
  -Taylor
 
 
  On Jun 5, 2015, at 5:16 PM, Bobby Evans ev...@yahoo-inc.com.INVALID
  wrote:
 
  +1
  I checked out the code, built it and ran the tests.  Once the vote
  passes we need to be sure to update the v0.10.0 tag to point to the
 correct
  release version in git.
  - Bobby
 
 
 
On Friday, June 5, 2015 2:54 PM, P. Taylor Goetz ptgo...@apache.org
 javascript:;
  wrote:
 
 
  This is a call to vote on releasing Apache Storm 0.10.0 (rc2).
 
  Full list of changes in this release:
 
 
 
 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=33d6c74041c286eb8b30b26bd0d6a76aa81a667f
 
  The tag/commit to be voted upon is v0.10.0:
 
 
 
 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=b8aba80e2259f3e12493a871b1af4f69864d40c2;hb=33d6c74041c286eb8b30b26bd0d6a76aa81a667f
 
  The source archive being voted upon can be found here:
 
 
 
 http://people.apache.org/~ptgoetz/apache-storm-0.10.0-rc2/apache-storm-0.10.0-src.tar.gz
 
  Other release files, signatures and digests can be found here:
 
  http://people.apache.org/~ptgoetz/apache-storm-0.10.0-rc2/
 
  The release artifacts are signed with the following key:
 
 
 
 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
 
  The Nexus staging repository for this release is:
 
  https://repository.apache.org/content/repositories/orgapachestorm-1016
 
  Please vote on releasing this package as Apache Storm 0.10.0.
 
  When voting, please list the actions taken to verify the release.
 
  This vote will be open for at least 72 hours.
 
  [ ] +1 Release this package as Apache Storm 0.10.0
  [ ]  0 No opinion
  [ ] -1 Do not release this package because...
 
  Thanks,
  Taylor
 
 
 
 
 
 
 
  --
  Name : 임 정택
  Blog : http://www.heartsavior.net / http://dev.heartsavior.net
  Twitter : http://twitter.com/heartsavior
  LinkedIn : http://www.linkedin.com/in/heartsavior



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.10.0 (rc2)

[임정택]

Either beta or RC is great for me.
0.10.0 include many features but it seems not have time to be evaluated by
users or dog fooding.
I wanna get official version ASAP which is better for me to persuade my
team, but getting stable version is most important point, so I'd like to
test before releasing.

Thanks!
Jungtaek Lim (HeartSaVioR)

2015년 6월 9일 화요일, Bobby Evansev...@yahoo-inc.com.invalid님이 작성한 메시지:

 I think that would be great.  We expect there to be bug fixes in many
 different areas.  https://github.com/apache/storm/pull/568 is an example.
 Labeling 0.10.0 as beta seems fine, so long as we have a plan in the
 relatively near term to move from beta to stable.

 - Bobby



  On Monday, June 8, 2015 7:54 PM, P. Taylor Goetz ptgo...@gmail.com
 wrote:


  Looks like it's 0.10.x-specific.

 Perhaps we should label 0.10.0 as a beta?

 At least that way the user community could at least kick the tires on the
 new features, and know that known bugs are being addressed. And honestly,
 we probably should have switched to that model earlier.

 I think it would be good to get this version in the hands of users, even
 if it is only early adopters and beta testers (an important part of the
 community that should always be considered for committer/PMC membership,
 even if they don't make changes to the core code).

 -Taylor

  On Jun 8, 2015, at 5:32 PM, Derek Dagit der...@yahoo-inc.com.INVALID
 wrote:
 
  Found STORM-856: If nimbus goes down with a delayed kill topo cmd, it
 never comes up
 
 
  Not sure if this is particular to the 0.10.0 line, but it seems bad.
 
  --
  Derek
 
 
 
  
  From: P. Taylor Goetz ptgo...@gmail.com
  To: dev@storm.apache.org
  Sent: Monday, June 8, 2015 3:49 PM
  Subject: Re: [VOTE] Release Apache Storm 0.10.0 (rc2)
 
 
 
  I agree. I’m starting to prepare the announcement blog post (you will
 see a separate thread shortly). My plan was to not highlight the REST
 upload API because it has a known issue, and also list it as a known issue
 in the announcement.
 
  From there we should put STORM-813 in the 0.10.1 queue and start
 planning for a 0.10.1 release soon.
 
  -Taylor
 
 
 
 
  On Jun 8, 2015, at 4:32 PM, 임정택 kabh...@gmail.com wrote:
 
  OK, I see your point.
 
  I know Storm 0.10.0 is postponed several times, and it already has very
  huge changeset.
  I don't mean to say we have to postpone again, just want to hear
  opinion that we're OK to release with know issue, or fix it ASAP and
 reopen
  vote.
 
  I'm +1 (non-binding) with describing known issue to release note so that
  users don't meet strange behavior without information.
 
  Thanks!
  Jungtaek Lim (HeartSaVioR)
 
 
  2015년 6월 9일 화요일, P. Taylor Goetzptgo...@gmail.com님이 작성한 메시지:
 
 
  I’m +1 for moving forward with the current release candidate. I think we
  can address STORM-813 in a follow-up bug fix/maintenance release
 (soon).
 
  In the big picture, the REST upload API is just one of many, many
  features/improvements added in 0.10.0, and we’ve not leveraged it in
 any
  way yet (i.e. in Storm UI). I don’t think it should block the release.
 
  -Taylor
 
  On Jun 6, 2015, at 2:16 AM, 임정택 kabh...@gmail.com javascript:;
 wrote:
 
 
  Unfortunately I found a bug from uploadTopology REST API.
  (Sorry I participated to review that feature but I didn't think about
  multiple arguments.)
 
  Here's my checklist.
 
  - test passed : OK
  -- extract source tar
  -- build via mvn clean install”
  - topology tests via storm-starter : OK
  -- run WordCounts and RollingTopWords, local mode and remote mode
  - test REST APIs (upload / activate / deactivate / kill) : FAIL
  -- activate / deactivate / kill works fine
  -- upload topology fails with multiple arguments
  -- filed issue to https://issues.apache.org/jira/browse/STORM-853
 
  Uploading topology is a new feature, so I'd like to include fixed
 version
  of feature to 0.10.0.
  (https://github.com/apache/storm/pull/581 is ready for the fix.)
 
  Btw, STORM-813 (correcting storm-starter's README) is related to
 0.10.0
  so
 
  I'd like to include it, but just applying it to Github repo could be
  acceptable.
 
  Thanks!
  Jungtaek Lim (HeartSaVioR)
 
  2015-06-06 7:59 GMT+09:00 Kishorkumar Patil
 kpa...@yahoo-inc.com.invalid
  :
 
 
  +1. LGTM. I built and ran some tests.
  -Kishor
 
 
 
   On Friday, June 5, 2015 4:38 PM, P. Taylor Goetz ptgo...@gmail.com
  javascript:;
 
  wrote:
 
 
  The v0.10.0 tag was already updates as part of the release process.
 The
  release candidate was built from that tag.
 
  -Taylor
 
 
 
  On Jun 5, 2015, at 5:16 PM, Bobby Evans ev...@yahoo-inc.com.INVALID
 
  wrote:
 
 
  +1
  I checked out the code, built it and ran the tests.  Once the vote
  passes we need to be sure to update the v0.10.0 tag to point to the
  correct
 
  release version in git.
 
  - Bobby
 
 
 
  On Friday, June 5, 2015 2:54 PM, P. Taylor Goetz 
 ptgo...@apache.org
  javascript:;
 
  wrote:
 
 
 
  This is a call to vote

Re: Question about the stateless of Storm

[임정택]

Hi!

It requires that data source of spout can replay datas which are
from offset or read but not acked.
For example, ack mode of rabbitmq holds datas which are read from
subscriber, and rabbitmq removes data when data is acked.
But when subscriber is disconnected, any datas which are not acked are
available for read, which makes spout fault tolerant.
If spout can handle data source offset instead of ack mode, spout may use
ack / fail to adjust offset to achieve similar thing.

Hope this helps.

Thanks!
Jungtaek Lim (HeartSaVioR)

2015년 6월 7일 일요일, Chuanlei Ninichuan...@gmail.com님이 작성한 메시지:

 Hi,
Storm claims that it is robust even the node crushes. It is ok that
 Nimbus can just restart to control the system, and the node just holds Bolt
 can also ignore the issues. But if the spout crushes, and the `pending` map
 data is lost. So the data lost. And some calculation would be not correct.

   Is there any mis-understand above? Just for discussion.

 Thanks

 Happy Weekends

 Chuanlei Ni



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [VOTE] Release Apache Storm 0.10.0 (rc2)

[임정택]

Unfortunately I found a bug from uploadTopology REST API.
(Sorry I participated to review that feature but I didn't think about
multiple arguments.)

Here's my checklist.

- test passed : OK
-- extract source tar
-- build via mvn clean install”
- topology tests via storm-starter : OK
-- run WordCounts and RollingTopWords, local mode and remote mode
- test REST APIs (upload / activate / deactivate / kill) : FAIL
-- activate / deactivate / kill works fine
-- upload topology fails with multiple arguments
-- filed issue to https://issues.apache.org/jira/browse/STORM-853

Uploading topology is a new feature, so I'd like to include fixed version
of feature to 0.10.0.
(https://github.com/apache/storm/pull/581 is ready for the fix.)

Btw, STORM-813 (correcting storm-starter's README) is related to 0.10.0 so
I'd like to include it, but just applying it to Github repo could be
acceptable.

Thanks!
Jungtaek Lim (HeartSaVioR)

2015-06-06 7:59 GMT+09:00 Kishorkumar Patil kpa...@yahoo-inc.com.invalid:

 +1. LGTM. I built and ran some tests.
 -Kishor



  On Friday, June 5, 2015 4:38 PM, P. Taylor Goetz ptgo...@gmail.com
 wrote:


  The v0.10.0 tag was already updates as part of the release process. The
 release candidate was built from that tag.

 -Taylor


  On Jun 5, 2015, at 5:16 PM, Bobby Evans ev...@yahoo-inc.com.INVALID
 wrote:
 
  +1
  I checked out the code, built it and ran the tests.  Once the vote
 passes we need to be sure to update the v0.10.0 tag to point to the correct
 release version in git.
   - Bobby
 
 
 
 On Friday, June 5, 2015 2:54 PM, P. Taylor Goetz ptgo...@apache.org
 wrote:
 
 
  This is a call to vote on releasing Apache Storm 0.10.0 (rc2).
 
  Full list of changes in this release:
 
 
 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=33d6c74041c286eb8b30b26bd0d6a76aa81a667f
 
  The tag/commit to be voted upon is v0.10.0:
 
 
 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=b8aba80e2259f3e12493a871b1af4f69864d40c2;hb=33d6c74041c286eb8b30b26bd0d6a76aa81a667f
 
  The source archive being voted upon can be found here:
 
 
 http://people.apache.org/~ptgoetz/apache-storm-0.10.0-rc2/apache-storm-0.10.0-src.tar.gz
 
  Other release files, signatures and digests can be found here:
 
  http://people.apache.org/~ptgoetz/apache-storm-0.10.0-rc2/
 
  The release artifacts are signed with the following key:
 
 
 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
 
  The Nexus staging repository for this release is:
 
  https://repository.apache.org/content/repositories/orgapachestorm-1016
 
  Please vote on releasing this package as Apache Storm 0.10.0.
 
  When voting, please list the actions taken to verify the release.
 
  This vote will be open for at least 72 hours.
 
  [ ] +1 Release this package as Apache Storm 0.10.0
  [ ]  0 No opinion
  [ ] -1 Do not release this package because...
 
  Thanks,
  Taylor
 






-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Release Storm 0.10.0

[임정택]

Great! I'm looking forward to see 0.10.0 officially.

Our project needs feature about submitting / killing topology dynamically,
and we have been done with scheduling job which runs bash script with mesos
+ chronos but I really wanna get rid of mesos + chronos.
I hope that new features on REST API help us a lot.

Thanks!

2015-06-04 8:08 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 Thanks Jungtaek,

 I'll verify and incorporate it into the build.

 I hope to have a release candidate ready for vote tomorrow.

 -Taylor


  On Jun 3, 2015, at 6:47 PM, 임정택 kabh...@gmail.com wrote:
 
  I realized that storm-redis is not on storm binary distribution, so I
 just
  created a PR which fixes it.
  https://github.com/apache/storm/pull/574
 
  2015-06-03 7:39 GMT+09:00 임정택 kabh...@gmail.com:
 
  Taylor merged STORM-761 to 0.10.x-branch with fixing conflicts.
  (It's not have been merged to master yet.)
 
  At 0.10.x-branch we merged all of PRs about storm-redis. Amazing works!
 
 
  2015-06-03 0:55 GMT+09:00 Bobby Evans ev...@yahoo-inc.com.invalid:
 
  I took a look at both of them. I just merged in the first one so you
  could look at upmerging the second one now.
  - Bobby
 
 
 
  On Monday, June 1, 2015 6:48 PM, 임정택 kabh...@gmail.com wrote:
 
 
  Taylor,
  There're two PRs about storm-redis waiting for review, STORM-753
  https://github.com/apache/storm/pull/504, STORM-761
  https://github.com/apache/storm/pull/514.
 
  Bobby reviewed STORM-753 https://github.com/apache/storm/pull/504
 and
  left some comments, and I addressed all things.
 
  I reviewed STORM-761 https://github.com/apache/storm/pull/514 and it
  looks good but Bobby didn't take a look yet.
  It needs to be upmerged, and since I'm not author of PR and PR seems to
  have long inactive time so it could take some time to reflect.
  (It could conflict with STORM-753 so we should merge STORM-753 first.)
 
  I can make a new PR upmerging STORM-761 to master when author doesn't
  respond.
 
  Thanks!
 
 
  2015-06-02 8:06 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:
 
  The 0.10.x branch is now up to date with master.
 
  Jungtaek-
 
  Could you look at the remaining storm-redis JIRAs and update, comment,
  etc. so they merge cleanly with master?
 
  Thanks in advance!
 
  -Taylor
 
 
  On May 18, 2015, at 4:59 PM, P. Taylor Goetz ptgo...@gmail.com
  wrote:
 
  Thanks Jungtaek,
 
  We can pull all the storm-redis fixes into 0.10.0.
 
  So if you just post a list of the outstanding pull requests that are
  waiting for review, that should suffice.
 
  -Taylor
 
  On May 18, 2015, at 4:34 PM, 임정택 kabh...@gmail.com wrote:
 
  Yes, actually I mailed to Bobby personally regarding this.
 
  I'll make a list of storm-redis PRs which ones are only merged to
  master
  (not applied to 0.10.0), or other ones are waiting for reviewing,
 and
  share
  to this thread.
 
  2015-05-19 5:14 GMT+09:00 Bobby Evans ev...@yahoo-inc.com:
 
  I have had several people ask me about getting several storm-redis
  changes
  in, I just haven't had time to review all of them, butI would like
  to
  see
  0.10 released sooner rather then later, so I am willing to make the
  time if
  I need to.
 
  HeartSavior, I think you had a list of storm-redis JIRA that you
  would
  like to see pulled in?
 
  - Bobby
 
 
 
  On Monday, May 18, 2015 2:57 PM, P. Taylor Goetz 
 ptgo...@gmail.com
 
  wrote:
 
 
  I’d like to get everyone’s opinion on releasing Storm 0.10.0.
 
  Attached is the change log for everything that is currently in the
  0.10.x-branch.
 
  Is there anything glaring missing from that list that I might have
  missed?
 
  -Taylor
 
 
  ## 0.10.0
  * STORM-786: KafkaBolt should ack tick tuples
  * STORM-512: KafkaBolt doesn't handle ticks properly
  * STORM-788: Fix key for process latencies
  * STORM-748: Package Multi-Lang scripts so they don't have to be
  duplicated
  * STORM-563. Kafka Spout doesn't pick up from the beginning of the
  queue
  unless forceFromStart specified.
  * STORM-615: Add REST API to upload topology.
  * STORM-807: quote args correctly in /bin/storm
  * STORM-686: Add worker-launcher to storm-dist.
  * STORM-789: Send more topology context to Multi-Lang components
 via
  initial handshake
  * STORM-764: Have option to compress thrift heartbeat
  * STORM-766 (Include version info in the service page)
  * STORM-765: Thrift serialization for local state.
  * STORM-762: uptime for worker heartbeats is lost when converted to
  thrift
  * STORM-750: Set Config serialVersionUID
  * STORM-714: Make CSS more consistent with self, prev release
  * STORM-796: Add support for error command in ShellSpout
  * STORM-745: fix storm.cmd to evaluate 'shift' correctly with
 'storm
  jar'
  * STORM-681: Auto insert license header with genthrift.sh
  * STORM-707: Client (Netty): improve logging to help
 troubleshooting
  connection woes
  * STORM-699: storm-jdbc should support custom insert queries.
  * STORM-625: Don't leak netty clients when worker moves or reuse

Re: [DISCUSS] Release Storm 0.10.0

[임정택]

I realized that storm-redis is not on storm binary distribution, so I just
created a PR which fixes it.
https://github.com/apache/storm/pull/574

2015-06-03 7:39 GMT+09:00 임정택 kabh...@gmail.com:

 Taylor merged STORM-761 to 0.10.x-branch with fixing conflicts.
 (It's not have been merged to master yet.)

 At 0.10.x-branch we merged all of PRs about storm-redis. Amazing works!


 2015-06-03 0:55 GMT+09:00 Bobby Evans ev...@yahoo-inc.com.invalid:

 I took a look at both of them. I just merged in the first one so you
 could look at upmerging the second one now.
  - Bobby



  On Monday, June 1, 2015 6:48 PM, 임정택 kabh...@gmail.com wrote:


  Taylor,
 There're two PRs about storm-redis waiting for review, STORM-753
 https://github.com/apache/storm/pull/504, STORM-761
 https://github.com/apache/storm/pull/514.

 Bobby reviewed STORM-753 https://github.com/apache/storm/pull/504 and
 left some comments, and I addressed all things.

 I reviewed STORM-761 https://github.com/apache/storm/pull/514 and it
 looks good but Bobby didn't take a look yet.
 It needs to be upmerged, and since I'm not author of PR and PR seems to
 have long inactive time so it could take some time to reflect.
 (It could conflict with STORM-753 so we should merge STORM-753 first.)

 I can make a new PR upmerging STORM-761 to master when author doesn't
 respond.

 Thanks!


 2015-06-02 8:06 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

  The 0.10.x branch is now up to date with master.
 
  Jungtaek-
 
  Could you look at the remaining storm-redis JIRAs and update, comment,
  etc. so they merge cleanly with master?
 
  Thanks in advance!
 
  -Taylor
 
 
   On May 18, 2015, at 4:59 PM, P. Taylor Goetz ptgo...@gmail.com
 wrote:
  
   Thanks Jungtaek,
  
   We can pull all the storm-redis fixes into 0.10.0.
  
   So if you just post a list of the outstanding pull requests that are
  waiting for review, that should suffice.
  
   -Taylor
  
   On May 18, 2015, at 4:34 PM, 임정택 kabh...@gmail.com wrote:
  
   Yes, actually I mailed to Bobby personally regarding this.
  
   I'll make a list of storm-redis PRs which ones are only merged to
 master
   (not applied to 0.10.0), or other ones are waiting for reviewing, and
  share
   to this thread.
  
   2015-05-19 5:14 GMT+09:00 Bobby Evans ev...@yahoo-inc.com:
  
   I have had several people ask me about getting several storm-redis
  changes
   in, I just haven't had time to review all of them, butI would like
 to
  see
   0.10 released sooner rather then later, so I am willing to make the
  time if
   I need to.
  
   HeartSavior, I think you had a list of storm-redis JIRA that you
 would
   like to see pulled in?
  
   - Bobby
  
  
  
   On Monday, May 18, 2015 2:57 PM, P. Taylor Goetz ptgo...@gmail.com
 
   wrote:
  
  
   I’d like to get everyone’s opinion on releasing Storm 0.10.0.
  
   Attached is the change log for everything that is currently in the
   0.10.x-branch.
  
   Is there anything glaring missing from that list that I might have
  missed?
  
   -Taylor
  
  
   ## 0.10.0
   * STORM-786: KafkaBolt should ack tick tuples
   * STORM-512: KafkaBolt doesn't handle ticks properly
   * STORM-788: Fix key for process latencies
   * STORM-748: Package Multi-Lang scripts so they don't have to be
  duplicated
   * STORM-563. Kafka Spout doesn't pick up from the beginning of the
  queue
   unless forceFromStart specified.
   * STORM-615: Add REST API to upload topology.
   * STORM-807: quote args correctly in /bin/storm
   * STORM-686: Add worker-launcher to storm-dist.
   * STORM-789: Send more topology context to Multi-Lang components via
   initial handshake
   * STORM-764: Have option to compress thrift heartbeat
   * STORM-766 (Include version info in the service page)
   * STORM-765: Thrift serialization for local state.
   * STORM-762: uptime for worker heartbeats is lost when converted to
  thrift
   * STORM-750: Set Config serialVersionUID
   * STORM-714: Make CSS more consistent with self, prev release
   * STORM-796: Add support for error command in ShellSpout
   * STORM-745: fix storm.cmd to evaluate 'shift' correctly with 'storm
  jar'
   * STORM-681: Auto insert license header with genthrift.sh
   * STORM-707: Client (Netty): improve logging to help troubleshooting
   connection woes
   * STORM-699: storm-jdbc should support custom insert queries.
   * STORM-625: Don't leak netty clients when worker moves or reuse
 netty
   client.
   * STORM-682: supervisor should handle worker state corruption
  gracefully.
   * STORM-446: Allow superusers to impersonate other users in secure
  mode.
   * STORM-659: return grep matches each on its own line.
   * STORM-693: KafkaBolt exception handling improvement.
   * STORM-675: Allow users to have storm-env.sh under config dir to
 set
   custom JAVA_HOME and other env variables.
   * STORM-539: Storm Hive Connector.
   * STORM-616: Storm JDBC Connector.
   * STORM-329: fix cascading Storm failure by improving reconnection
   strategy

Re: [DISCUSS] Release Storm 0.10.0

[임정택]

Taylor merged STORM-761 to 0.10.x-branch with fixing conflicts.
(It's not have been merged to master yet.)

At 0.10.x-branch we merged all of PRs about storm-redis. Amazing works!


2015-06-03 0:55 GMT+09:00 Bobby Evans ev...@yahoo-inc.com.invalid:

 I took a look at both of them. I just merged in the first one so you could
 look at upmerging the second one now.
  - Bobby



  On Monday, June 1, 2015 6:48 PM, 임정택 kabh...@gmail.com wrote:


  Taylor,
 There're two PRs about storm-redis waiting for review, STORM-753
 https://github.com/apache/storm/pull/504, STORM-761
 https://github.com/apache/storm/pull/514.

 Bobby reviewed STORM-753 https://github.com/apache/storm/pull/504 and
 left some comments, and I addressed all things.

 I reviewed STORM-761 https://github.com/apache/storm/pull/514 and it
 looks good but Bobby didn't take a look yet.
 It needs to be upmerged, and since I'm not author of PR and PR seems to
 have long inactive time so it could take some time to reflect.
 (It could conflict with STORM-753 so we should merge STORM-753 first.)

 I can make a new PR upmerging STORM-761 to master when author doesn't
 respond.

 Thanks!


 2015-06-02 8:06 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

  The 0.10.x branch is now up to date with master.
 
  Jungtaek-
 
  Could you look at the remaining storm-redis JIRAs and update, comment,
  etc. so they merge cleanly with master?
 
  Thanks in advance!
 
  -Taylor
 
 
   On May 18, 2015, at 4:59 PM, P. Taylor Goetz ptgo...@gmail.com
 wrote:
  
   Thanks Jungtaek,
  
   We can pull all the storm-redis fixes into 0.10.0.
  
   So if you just post a list of the outstanding pull requests that are
  waiting for review, that should suffice.
  
   -Taylor
  
   On May 18, 2015, at 4:34 PM, 임정택 kabh...@gmail.com wrote:
  
   Yes, actually I mailed to Bobby personally regarding this.
  
   I'll make a list of storm-redis PRs which ones are only merged to
 master
   (not applied to 0.10.0), or other ones are waiting for reviewing, and
  share
   to this thread.
  
   2015-05-19 5:14 GMT+09:00 Bobby Evans ev...@yahoo-inc.com:
  
   I have had several people ask me about getting several storm-redis
  changes
   in, I just haven't had time to review all of them, butI would like to
  see
   0.10 released sooner rather then later, so I am willing to make the
  time if
   I need to.
  
   HeartSavior, I think you had a list of storm-redis JIRA that you
 would
   like to see pulled in?
  
   - Bobby
  
  
  
   On Monday, May 18, 2015 2:57 PM, P. Taylor Goetz ptgo...@gmail.com
   wrote:
  
  
   I’d like to get everyone’s opinion on releasing Storm 0.10.0.
  
   Attached is the change log for everything that is currently in the
   0.10.x-branch.
  
   Is there anything glaring missing from that list that I might have
  missed?
  
   -Taylor
  
  
   ## 0.10.0
   * STORM-786: KafkaBolt should ack tick tuples
   * STORM-512: KafkaBolt doesn't handle ticks properly
   * STORM-788: Fix key for process latencies
   * STORM-748: Package Multi-Lang scripts so they don't have to be
  duplicated
   * STORM-563. Kafka Spout doesn't pick up from the beginning of the
  queue
   unless forceFromStart specified.
   * STORM-615: Add REST API to upload topology.
   * STORM-807: quote args correctly in /bin/storm
   * STORM-686: Add worker-launcher to storm-dist.
   * STORM-789: Send more topology context to Multi-Lang components via
   initial handshake
   * STORM-764: Have option to compress thrift heartbeat
   * STORM-766 (Include version info in the service page)
   * STORM-765: Thrift serialization for local state.
   * STORM-762: uptime for worker heartbeats is lost when converted to
  thrift
   * STORM-750: Set Config serialVersionUID
   * STORM-714: Make CSS more consistent with self, prev release
   * STORM-796: Add support for error command in ShellSpout
   * STORM-745: fix storm.cmd to evaluate 'shift' correctly with 'storm
  jar'
   * STORM-681: Auto insert license header with genthrift.sh
   * STORM-707: Client (Netty): improve logging to help troubleshooting
   connection woes
   * STORM-699: storm-jdbc should support custom insert queries.
   * STORM-625: Don't leak netty clients when worker moves or reuse
 netty
   client.
   * STORM-682: supervisor should handle worker state corruption
  gracefully.
   * STORM-446: Allow superusers to impersonate other users in secure
  mode.
   * STORM-659: return grep matches each on its own line.
   * STORM-693: KafkaBolt exception handling improvement.
   * STORM-675: Allow users to have storm-env.sh under config dir to set
   custom JAVA_HOME and other env variables.
   * STORM-539: Storm Hive Connector.
   * STORM-616: Storm JDBC Connector.
   * STORM-329: fix cascading Storm failure by improving reconnection
   strategy and buffering messages (thanks tedxia)
   * STORM-641: Add total number of topologies to
 api/v1/cluster/summary.
   * STORM-640: Storm UI vulnerable to poodle attack.
   * STORM-651: improvements to storm.cmd

Re: [DISCUSS] Release Storm 0.10.0

[임정택]

Taylor,
There're two PRs about storm-redis waiting for review, STORM-753
https://github.com/apache/storm/pull/504, STORM-761
https://github.com/apache/storm/pull/514.

Bobby reviewed STORM-753 https://github.com/apache/storm/pull/504 and
left some comments, and I addressed all things.

I reviewed STORM-761 https://github.com/apache/storm/pull/514 and it
looks good but Bobby didn't take a look yet.
It needs to be upmerged, and since I'm not author of PR and PR seems to
have long inactive time so it could take some time to reflect.
(It could conflict with STORM-753 so we should merge STORM-753 first.)

I can make a new PR upmerging STORM-761 to master when author doesn't
respond.

Thanks!


2015-06-02 8:06 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 The 0.10.x branch is now up to date with master.

 Jungtaek-

 Could you look at the remaining storm-redis JIRAs and update, comment,
 etc. so they merge cleanly with master?

 Thanks in advance!

 -Taylor


  On May 18, 2015, at 4:59 PM, P. Taylor Goetz ptgo...@gmail.com wrote:
 
  Thanks Jungtaek,
 
  We can pull all the storm-redis fixes into 0.10.0.
 
  So if you just post a list of the outstanding pull requests that are
 waiting for review, that should suffice.
 
  -Taylor
 
  On May 18, 2015, at 4:34 PM, 임정택 kabh...@gmail.com wrote:
 
  Yes, actually I mailed to Bobby personally regarding this.
 
  I'll make a list of storm-redis PRs which ones are only merged to master
  (not applied to 0.10.0), or other ones are waiting for reviewing, and
 share
  to this thread.
 
  2015-05-19 5:14 GMT+09:00 Bobby Evans ev...@yahoo-inc.com:
 
  I have had several people ask me about getting several storm-redis
 changes
  in, I just haven't had time to review all of them, butI would like to
 see
  0.10 released sooner rather then later, so I am willing to make the
 time if
  I need to.
 
  HeartSavior, I think you had a list of storm-redis JIRA that you would
  like to see pulled in?
 
  - Bobby
 
 
 
  On Monday, May 18, 2015 2:57 PM, P. Taylor Goetz ptgo...@gmail.com
  wrote:
 
 
  I’d like to get everyone’s opinion on releasing Storm 0.10.0.
 
  Attached is the change log for everything that is currently in the
  0.10.x-branch.
 
  Is there anything glaring missing from that list that I might have
 missed?
 
  -Taylor
 
 
  ## 0.10.0
  * STORM-786: KafkaBolt should ack tick tuples
  * STORM-512: KafkaBolt doesn't handle ticks properly
  * STORM-788: Fix key for process latencies
  * STORM-748: Package Multi-Lang scripts so they don't have to be
 duplicated
  * STORM-563. Kafka Spout doesn't pick up from the beginning of the
 queue
  unless forceFromStart specified.
  * STORM-615: Add REST API to upload topology.
  * STORM-807: quote args correctly in /bin/storm
  * STORM-686: Add worker-launcher to storm-dist.
  * STORM-789: Send more topology context to Multi-Lang components via
  initial handshake
  * STORM-764: Have option to compress thrift heartbeat
  * STORM-766 (Include version info in the service page)
  * STORM-765: Thrift serialization for local state.
  * STORM-762: uptime for worker heartbeats is lost when converted to
 thrift
  * STORM-750: Set Config serialVersionUID
  * STORM-714: Make CSS more consistent with self, prev release
  * STORM-796: Add support for error command in ShellSpout
  * STORM-745: fix storm.cmd to evaluate 'shift' correctly with 'storm
 jar'
  * STORM-681: Auto insert license header with genthrift.sh
  * STORM-707: Client (Netty): improve logging to help troubleshooting
  connection woes
  * STORM-699: storm-jdbc should support custom insert queries.
  * STORM-625: Don't leak netty clients when worker moves or reuse netty
  client.
  * STORM-682: supervisor should handle worker state corruption
 gracefully.
  * STORM-446: Allow superusers to impersonate other users in secure
 mode.
  * STORM-659: return grep matches each on its own line.
  * STORM-693: KafkaBolt exception handling improvement.
  * STORM-675: Allow users to have storm-env.sh under config dir to set
  custom JAVA_HOME and other env variables.
  * STORM-539: Storm Hive Connector.
  * STORM-616: Storm JDBC Connector.
  * STORM-329: fix cascading Storm failure by improving reconnection
  strategy and buffering messages (thanks tedxia)
  * STORM-641: Add total number of topologies to api/v1/cluster/summary.
  * STORM-640: Storm UI vulnerable to poodle attack.
  * STORM-651: improvements to storm.cmd
  * STORM-456: Storm UI: cannot navigate to topology page when name
 contains
  spaces.
  * STORM-627: Storm-hbase configuration error.
  * STORM-248: cluster.xml location is hardcoded for workers
  * STORM-322: Windows script do not handle spaces in JAVA_HOME path
  * STORM-586: Trident kafka spout fails instead of updating offset when
  kafka offset is out of range.
  * STORM-595: storm-hdfs can only work with sequence files that use
  Writables.
  * STORM-487: Remove storm.cmd, no need to duplicate work python runs on
  windows too.
  * STORM-585: Performance issue

Re: [VOTE] Release Apache Storm 0.9.5

[임정택]

   - test passed : OK
  - extract source tar : OK
  - build via mvn clean install” : OK
- verify STORM-745 : OK
  - install binary version of storm 0.9.5 to Windows 7 : OK
 - use apache-storm-0.9.5.zip
  - run dev-zookeeper, nimbus, supervisor with default setting via
  bin/storm.cmd : OK
  - run RollingTopWords from storm-starter in remote/cluster mode via
  bin/storm.cmd : OK
 - it confirms STORM-745 is fine cause it receives two argument and
 if second parameter is not presented it runs locally
   - some tests that Storm 0.9.5 works with OSX : OK
  - install binary version of storm 0.9.5 to OSX : OK
 - use apache-storm-0.9.5.tar.gz
  - run dev-zookeeper, nimbus, supervisor with default setting via
  bin/storm : OK
  - run RollingTopWords from storm-starter in local cluster mode via
  bin/storm : OK
  - run RollingTopWords from storm-starter in remote/cluster mode via
  bin/storm : OK

+1 (non-binding)

Thanks!

Jungtaek Lim (HeartSaVioR)

2015-05-30 0:34 GMT+09:00 P. Taylor Goetz ptgo...@apache.org:

 This is a call to vote on releasing Apache Storm 0.9.5.

 Full list of changes in this release:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=c9c47b36071c02488ee1e75eb3851e58191b7534

 The tag/commit to be voted upon is v0.9.5:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=774b229d709af60241c6a239b7c0990ed63ad32a;hb=c9c47b36071c02488ee1e75eb3851e58191b7534

 The source archive being voted upon can be found here:


 http://people.apache.org/~ptgoetz/apache-storm-0.9.5/apache-storm-0.9.5-src.tar.gz

 Other release files, signatures and digests can be found here:

 http://people.apache.org/~ptgoetz/apache-storm-0.9.5/

 The release artifacts are signed with the following key:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd

 The Nexus staging repository for this release is:

 https://repository.apache.org/content/repositories/orgapachestorm-1014/

 Please vote on releasing this package as Apache Storm 0.9.5.

 When voting, please list the actions taken to verify the release.

 This vote will be open for at least 72 hours.

 [ ] +1 Release this package as Apache Storm 0.9.5
 [ ]  0 No opinion
 [ ] -1 Do not release this package because...

 Thanks,
 Taylor




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Release Storm 0.10.0

[임정택]

Yes, actually I mailed to Bobby personally regarding this.

I'll make a list of storm-redis PRs which ones are only merged to master
(not applied to 0.10.0), or other ones are waiting for reviewing, and share
to this thread.

2015-05-19 5:14 GMT+09:00 Bobby Evans ev...@yahoo-inc.com:

 I have had several people ask me about getting several storm-redis changes
 in, I just haven't had time to review all of them, butI would like to see
 0.10 released sooner rather then later, so I am willing to make the time if
 I need to.

 HeartSavior, I think you had a list of storm-redis JIRA that you would
 like to see pulled in?

 - Bobby



   On Monday, May 18, 2015 2:57 PM, P. Taylor Goetz ptgo...@gmail.com
 wrote:


 I’d like to get everyone’s opinion on releasing Storm 0.10.0.

 Attached is the change log for everything that is currently in the
 0.10.x-branch.

 Is there anything glaring missing from that list that I might have missed?

 -Taylor


 ## 0.10.0
 * STORM-786: KafkaBolt should ack tick tuples
 * STORM-512: KafkaBolt doesn't handle ticks properly
 * STORM-788: Fix key for process latencies
 * STORM-748: Package Multi-Lang scripts so they don't have to be duplicated
 * STORM-563. Kafka Spout doesn't pick up from the beginning of the queue
 unless forceFromStart specified.
 * STORM-615: Add REST API to upload topology.
 * STORM-807: quote args correctly in /bin/storm
 * STORM-686: Add worker-launcher to storm-dist.
 * STORM-789: Send more topology context to Multi-Lang components via
 initial handshake
 * STORM-764: Have option to compress thrift heartbeat
 * STORM-766 (Include version info in the service page)
 * STORM-765: Thrift serialization for local state.
 * STORM-762: uptime for worker heartbeats is lost when converted to thrift
 * STORM-750: Set Config serialVersionUID
 * STORM-714: Make CSS more consistent with self, prev release
 * STORM-796: Add support for error command in ShellSpout
 * STORM-745: fix storm.cmd to evaluate 'shift' correctly with 'storm jar'
 * STORM-681: Auto insert license header with genthrift.sh
 * STORM-707: Client (Netty): improve logging to help troubleshooting
 connection woes
 * STORM-699: storm-jdbc should support custom insert queries.
 * STORM-625: Don't leak netty clients when worker moves or reuse netty
 client.
 * STORM-682: supervisor should handle worker state corruption gracefully.
 * STORM-446: Allow superusers to impersonate other users in secure mode.
 * STORM-659: return grep matches each on its own line.
 * STORM-693: KafkaBolt exception handling improvement.
 * STORM-675: Allow users to have storm-env.sh under config dir to set
 custom JAVA_HOME and other env variables.
 * STORM-539: Storm Hive Connector.
 * STORM-616: Storm JDBC Connector.
 * STORM-329: fix cascading Storm failure by improving reconnection
 strategy and buffering messages (thanks tedxia)
 * STORM-641: Add total number of topologies to api/v1/cluster/summary.
 * STORM-640: Storm UI vulnerable to poodle attack.
 * STORM-651: improvements to storm.cmd
 * STORM-456: Storm UI: cannot navigate to topology page when name contains
 spaces.
 * STORM-627: Storm-hbase configuration error.
 * STORM-248: cluster.xml location is hardcoded for workers
 * STORM-322: Windows script do not handle spaces in JAVA_HOME path
 * STORM-586: Trident kafka spout fails instead of updating offset when
 kafka offset is out of range.
 * STORM-595: storm-hdfs can only work with sequence files that use
 Writables.
 * STORM-487: Remove storm.cmd, no need to duplicate work python runs on
 windows too.
 * STORM-585: Performance issue in none grouping
 * STORM-525: Add time sorting function to the 2nd col of bolt exec table
 * STORM-548: Receive Thread Shutdown hook should connect to local hostname
 but not Localhost
 * STORM-567: Move Storm Documentation/Website from SVN to git
 * STORM-533: Add in client and server IConnection metrics.
 * STORM-572: Storm UI 'favicon.ico'
 * STORM-572: Allow Users to pass TEST-TIMEOUT-MS for java
 * STORM-571: upgrade clj-time.
 * STORM-569: Add Conf for bolt's outgoing overflow-buffer.
 * STORM-565: Fix NPE when topology.groups is null.
 * STORM-575: Ability to specify Jetty host to bind to
 * STORM-577: long time launch worker will block supervisor heartbeat
 * STORM-505: Fix debug string construction
 * STORM-613: Fix wrong getOffset return value
 * STORM-612: Update the contact address in configure.ac
 * STORM-611: Remove extra breaks
 * STORM-610: Check the return value of fts_close()
 * STORM-442: multilang ShellBolt/ShellSpout die() can be hang when
 Exception happened
 * STORM-410: Add groups support to log-viewer
 * STORM-444: Add AutoHDFS like credential fetching for HBase
 * STORM-552: Add netty socket backlog config
 * STORM-578: Calls to submit-mocked-assignment in supervisor-test use
 invalid executor-id format
 * STORM-600: upgrade jacoco plugin to support jdk8
 * STORM-495: KafkaSpout retries with exponential backoff
 * STORM-620: Duplicate maven plugin declaration
 * 

Fwd: [DISCUSS] Release Storm 0.10.0

[임정택]

We can refer STORM-737 to https://github.com/apache/storm/pull/557 since it
seems to be more cleaner.

-- Forwarded message --
From: 임정택 kabh...@gmail.com
Date: 2015-05-19 6:06 GMT+09:00
Subject: Re: [DISCUSS] Release Storm 0.10.0
To: dev@storm.apache.org


Yes, I have some PRs which are waiting for reviewing but better to include
next release (I mean 0.10.0).

Note: JIRA's links are linked to actual PRs.

1. storm-redis

- Waiting for reviewing: STORM-753
https://github.com/apache/storm/pull/504, STORM-761
https://github.com/apache/storm/pull/514, STORM-752
https://github.com/apache/storm/pull/488
- Already merged to master but not 0.10.0: STORM-691
https://github.com/apache/storm/pull/451, STORM-703
https://github.com/apache/storm/pull/462, STORM-724
https://github.com/apache/storm/pull/481, STORM-735
https://github.com/apache/storm/pull/491

Some users (including me) want storm-redis to be fully functional so I'd
like to get all things released via 0.10.0.

2. bugfix
- STORM-737 https://github.com/apache/storm/pull/521: Derek already
stated it. There's a PR regarding this, but I have other branch which Derek
says it's cleaner. I'll post a new PR which points to other branch.
- STORM-790 https://github.com/apache/storm/pull/527: It is already two
+1 but not merged yet.
- STORM-742 https://github.com/apache/storm/pull/497: Without this, if
there's very busy bolt with multilang, ACK mode, it can not respond
heartbeat in time so it could be suicide itself.
- STORM-794 https://github.com/apache/storm/pull/542: Trident topology
which has long complete latency may not have a change to treat
deactivate/reactivate. It was serious issue for me, and we changed all
trident topologies to normal Spout-Bolt because of this.
- STORM-813 https://github.com/apache/storm/pull/548: Packaging multilang
breaks storm-starter's one of documented feature, so we may want to clear
it.

Thanks!


2015-05-19 4:55 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 I’d like to get everyone’s opinion on releasing Storm 0.10.0.

 Attached is the change log for everything that is currently in the
 0.10.x-branch.

 Is there anything glaring missing from that list that I might have missed?

 -Taylor


 ## 0.10.0
  * STORM-786: KafkaBolt should ack tick tuples
  * STORM-512: KafkaBolt doesn't handle ticks properly
  * STORM-788: Fix key for process latencies
  * STORM-748: Package Multi-Lang scripts so they don't have to be
 duplicated
  * STORM-563. Kafka Spout doesn't pick up from the beginning of the queue
 unless forceFromStart specified.
  * STORM-615: Add REST API to upload topology.
  * STORM-807: quote args correctly in /bin/storm
  * STORM-686: Add worker-launcher to storm-dist.
  * STORM-789: Send more topology context to Multi-Lang components via
 initial handshake
  * STORM-764: Have option to compress thrift heartbeat
  * STORM-766 (Include version info in the service page)
  * STORM-765: Thrift serialization for local state.
  * STORM-762: uptime for worker heartbeats is lost when converted to thrift
  * STORM-750: Set Config serialVersionUID
  * STORM-714: Make CSS more consistent with self, prev release
  * STORM-796: Add support for error command in ShellSpout
  * STORM-745: fix storm.cmd to evaluate 'shift' correctly with 'storm jar'
  * STORM-681: Auto insert license header with genthrift.sh
  * STORM-707: Client (Netty): improve logging to help troubleshooting
 connection woes
  * STORM-699: storm-jdbc should support custom insert queries.
  * STORM-625: Don't leak netty clients when worker moves or reuse netty
 client.
  * STORM-682: supervisor should handle worker state corruption gracefully.
  * STORM-446: Allow superusers to impersonate other users in secure mode.
  * STORM-659: return grep matches each on its own line.
  * STORM-693: KafkaBolt exception handling improvement.
  * STORM-675: Allow users to have storm-env.sh under config dir to set
 custom JAVA_HOME and other env variables.
  * STORM-539: Storm Hive Connector.
  * STORM-616: Storm JDBC Connector.
  * STORM-329: fix cascading Storm failure by improving reconnection
 strategy and buffering messages (thanks tedxia)
  * STORM-641: Add total number of topologies to api/v1/cluster/summary.
  * STORM-640: Storm UI vulnerable to poodle attack.
  * STORM-651: improvements to storm.cmd
  * STORM-456: Storm UI: cannot navigate to topology page when name
 contains spaces.
  * STORM-627: Storm-hbase configuration error.
  * STORM-248: cluster.xml location is hardcoded for workers
  * STORM-322: Windows script do not handle spaces in JAVA_HOME path
  * STORM-586: Trident kafka spout fails instead of updating offset when
 kafka offset is out of range.
  * STORM-595: storm-hdfs can only work with sequence files that use
 Writables.
  * STORM-487: Remove storm.cmd, no need to duplicate work python runs on
 windows too.
  * STORM-585: Performance issue in none grouping
  * STORM-525: Add time sorting function to the 2nd col of bolt exec table
  * STORM-548

Re: [DISCUSSION] Squashing commits before merging in.

[임정택]

Hi.

FYI, Spring Data Redis used your approach now, so we can take a look.
https://github.com/spring-projects/spring-data-redis

At first, I like your approach.

Major point is that commit's changeset size will be completely relied on
JIRA issue's size, and we can have another pain point when changeset
becomes huge.
I sought some of merged PRs from Storm repo, and their changesets seems
small enough to leave just one commit per PR.

Additional point is how we leave multiple commits' log messages into one.
Headline of log can contain useful information which describes code change,
and applying this approach improperly we can lose them.

Furthermore, we're also having huge PR. (for example, STORM-561
https://github.com/apache/storm/pull/546) I wonder we can apply same
approach to this, or go with exception.

Btw, real pain point from cherry-picking is conflicts.
Jedis uses same approach, and we have to believe collaborators and unit
tests, and we did some mistakes while cherry-picking and unit tests
couldn't catch it.
It would be not resolved though we squash commits into one.
Have to make cherry-picking branch and post a PR related to merging
cherry-picked branch into specific version?

Sincerely,
Jungtaek Lim (HeartSaVioR)


2015-05-09 6:14 GMT+09:00 Harsha st...@harsha.io:

 Hi,
  Today we are merging into storm master branch using PRs. These PRs
 are usually contain multiple commits on different time lines .
 There are few issues with this approach , the major pain point is if we
 want cherry-pick a single JIRA current approach makes it harder as we have
 track down all the commits that went in for a particular JIRA.
 I am proposing we should squash these multiple commits into one singe
 commit before merging into trunk and associate the STORM JIRA in commit
 message.
 Let me know what you think.

 Thanks,
 Harsha







-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Refer to Trident as Microbatch API in docs?

[임정택]

+1 to explaining Trident is not just a transactional thing, it's a
micro-batch.

Best.
Jungtaek Lim (HeartSaVioR)

2015-04-07 11:13 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 Trident is widely misunderstood. Very few people seem to understand what
 it is or sometimes even know about it.

 One step toward correcting that would be to simply update the docs to
 replace “Trident” with “Storm Microbatch API (Trident)”.

 Would anyone be opposed to updating the docs along those lines?

 -Taylor




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Discussion about multi-lang heartbeat constraint

[임정택]

Hi, Storm community!

Since we introduce heartbeat to multi-lang feature with design constraint
at 0.9.3, we met some issues regarding design constraint.
(Please refer https://github.com/apache/storm/pull/286#issuecomment-58366793
for see constraint.)

I'm trying to make some workarounds to avoid it, but some scenarios it
doesn't work, too.
(Please refer https://github.com/apache/storm/pull/497#issuecomment-90392638
)

Actually I introduced this constraint cause it's 'multi-lang' and I can't
sure all languages can write heartbeat response concurrently with
synchronized.
But if we're not satisfied with constraint, we may decide to drop
supporting of some languages which cannot ensure this.
(Sure we should change python/ruby/node.js implementation to support it.)

Wish to hear everyone's opinion.
Thanks!

FYI
- HeartBeat feature : https://issues.apache.org/jira/browse/STORM-513
- HeartBeat timeout issue : https://issues.apache.org/jira/browse/STORM-738
-- workaround (cannot cover all scenarios) :
https://issues.apache.org/jira/browse/STORM-742

Best.
Jungtaek Lim (HeartSaVioR)

ps. Please let me know if you think it's better to post to user group, too.

Re: Disruptor upgrade

[임정택]

Hi!

It was merged to master, but Sean Zhong found failing tuples, and it was
rollbacked.
https://issues.apache.org/jira/browse/STORM-350

Actually I dug about this issue but couldn't find a solution.
I wrote a post to Disruptor google groups for help, but since I don't have
unit tests to reproduce so it's blocked.
https://groups.google.com/d/msg/lmax-disruptor/Qt2tOiWgFtA/7qqxZOLrBfAJ

Hope this helps.

Regards.
Jungtaek Lim (HeartSaVioR)


2015-03-28 2:00 GMT+09:00 Scott Sue scott@celer-tech.com:

 Hi there,

 Are there any efforts to move Storm’s usage of the Disruptor up to 3.x?
 If not, I’m looking into upgrading this…


 Regards,
 Scott


 SCOTT SUE
 CHIEF TECHNOLOGY OFFICER

 T  : +44(0) 2031 371 603
 M : +852 9611 3969

 23/F, 1 Great George Street, Causeway Bay, Hong Kong
 www.celer-tech.com http://www.celer-tech.com/









-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Wish to hear your opinion on adopting public CI

[임정택]

Hi all!

I'd like to hear your opinion on adopting public CI.
Now Apache Storm takes advantage of Github, we can apply Travis CI to some
more advantages.
Detailed informations are on JIRA issue (
https://issues.apache.org/jira/browse/STORM-704).

If you don't know Travis CI, it's better to take a look at.
https://travis-ci.org/xetorthio/jedis

Actually I'm trying to integrate it, and resolved many hurdles but got
stuck.

If you're interested in, please participate to resolve issue. :)

Important question on this, I don't know why Storm didn't have CI.
Is there some kind of decision about not having CI?
If then this issue should be resolved as invalid.

Thanks!

Regards.
Jungtaek Lim (HeartSaVioR)

Re: [DISCUSS] Release Storm 0.9.4 / 0.10.0

[임정택]

storm-redis (scheduled to be released at 0.10.0) has one bugfix and one
essential feature PRs.

- bugfix: https://issues.apache.org/jira/browse/STORM-690
-- It fixes connection pool issue.
- feature: https://issues.apache.org/jira/browse/STORM-691
-- It provides basic lookup / persist bolts so I believe it's necessary.

Furthermore, I'd like to continue to support various data types with
storm-redis Trident, after STORM-691 is merged to master.

Thanks!

Regards
Jungtaek Lim (HeartSaVioR)


2015-03-06 2:37 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 I’d like to start a discussion for releasing 0.9.4 (maintenance release)
 and 0.10.0 (security release).

 0.9.4 is basically a branch of 0.9.3 with two important bug fixes:

 STORM-329: fix cascading Storm failure by improving reconnection
 strategy and buffering messages
 STORM-130: Supervisor getting killed due to
 java.io.FileNotFoundException: File '../stormconf.ser' does not exist.

 Both are long-standing bugs that have proven problematic for many users.
 I’d be in favor of releasing 0.9.4 with just those two fixes, but I’m
 interested in finding out if anyone thinks there are additional patches to
 master that should be considered for 0.9.4.

 0.10.0 is a much larger release in terms of changes. In addition to the
 changes above, it includes all the new security features and numerous fixes
 and enhancements (see the CHANGELOG in the master branch for a full list).

 Do we feel 0.10.0 is ready for release? If not what outstanding
 bugs/patches should we consider before releasing?

 I’m fine holding off on a 0.10.0 release if we feel there is additional
 work to be done, but I’d like to at least move forward with 0.9.4 release.

 Thoughts?

 -Taylor




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: ShellSpout hangs on reportError?

[임정택]

Hello.

I haven't seen this behavior.
But at a glimpse, die() method should ensure that subprocess will be
destroyed (with try-finally), because worker process is going to be killed
whether reportError() throws RuntimeException or not.

You can apply my suggestion to verify it works, and file a JIRA.

Hope this helps.

Regards.
Jungtaek Lim (HeartSaVioR)

2015-02-13 5:08 GMT+09:00 William Oberman ober...@civicscience.com:

 Sorry for the cross post to dev, but I think this thread has veered into
 actual dev questions.  I still don't know if there is something
 fundamentally wrong about my use case, or if this is a bug.  For a dev
 reading this for the first time, the main correction I'd make is to my
 email subject.  reportError isn't hanging, it's throwing a runtime
 exception (wrapping an interrupted exception).  As for what is throwing the
 interrupted exception, I think it's Zookeeper itself.

 Both ShellSpout and ShellBolt's die() has a
 _collector.reportError(exception); line.  I changed both to:
 
 try {
 _collector.reportError(exception);
 } catch (RuntimeException e) {
 if(e.getCause() instanceof InterruptedException) {
 //zookeeper.clj wraps zk InterruptedException with runtime
 exception
 } else {
 throw e;
 }
 }
 ==
 and now everything starts to work as I expected.

 Does this patch make any sense?  Or is it a bandaid over a deeper issue?

 will

 On Thu, Feb 12, 2015 at 2:15 PM, William Oberman ober...@civicscience.com
 
 wrote:

  Ok, I realized that I did NOT check if ShellSpout.die() was throwing a
  RuntimeException.   I added a try/catch block, and it is!   The
  RuntimeException is preventing _process.destroy and System.exit() from
  happening, both of which need to happen to make topology recovery happen.
 
  But, I'm not sure *why* this exception is happening yet, since it's an
  interrupted exception and I don't think the exception tells me *who*
  interrupted my thread...
 
  2015-02-12T14:12:35.581-0500 b.s.s.ShellSpout [ERROR] die exception!
  java.lang.RuntimeException: java.lang.InterruptedException
  at backtype.storm.util$wrap_in_runtime.invoke(util.clj:44)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.zookeeper$exists_node_QMARK_$fn__3279.invoke(zookeeper.clj:102)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:98)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at backtype.storm.zookeeper$mkdirs.invoke(zookeeper.clj:114)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.cluster$mk_distributed_cluster_state$reify__3533.mkdirs(cluster.clj:119)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.cluster$mk_storm_cluster_state$reify__3990.report_error(cluster.clj:400)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.daemon.executor$throttled_report_error_fn$fn__5565.invoke(executor.clj:180)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.daemon.executor$fn__5717$fn$reify__5759.reportError(executor.clj:533)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.spout.SpoutOutputCollector.reportError(SpoutOutputCollector.java:132)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at backtype.storm.spout.ShellSpout.die(ShellSpout.java:235)
  [storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at backtype.storm.spout.ShellSpout.access$200(ShellSpout.java:42)
  [storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 backtype.storm.spout.ShellSpout$SpoutHeartbeatTimerTask.run(ShellSpout.java:261)
  [storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  [na:1.7.0_71]
  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
  [na:1.7.0_71]
  at
 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_71]
  at
 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_71]
  at
 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
  at
 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
  at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
  Caused by: java.lang.InterruptedException: null
  at java.lang.Object.wait(Native Method) ~[na:1.7.0_71]
  at java.lang.Object.wait(Object.java:503) ~[na:1.7.0_71]
  at
  org.apache.storm.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at org.apache.storm.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 org.apache.storm.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)
  ~[storm-core-0.9.3.jar:0.9.4-SNAPSHOT]
  at
 
 

Re: [VOTE] Release Apache Storm 0.9.3

[임정택]

+1 (non-binding)

Thanks all for amazing works and I can't stand it to use new release!

2014-11-19 16:24 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

This is a call to vote on releasing Apache Storm 0.9.3. This will be the
 first Apache Storm release as a TLP.

 Full list of changes in this release:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=20ce2bb9088e793b3b019110d07e77896bb642e8

 The tag/commit to be voted upon is v0.9.3:


 https://git-wip-us.apache.org/repos/asf?p=storm.git;a=commit;h=20ce2bb9088e793b3b019110d07e77896bb642e8

 The source archive being voted upon can be found here:


 http://people.apache.org/~ptgoetz/apache-storm-0.9.3/apache-storm-0.9.3-src.tar.gz

 Other release files, signatures and digests can be found here:

 http://people.apache.org/~ptgoetz/apache-storm-0.9.3/

 The release artifacts are signed with the following key:


 https://git-wip-us.apache.org/repos/asf?p=incubator-storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd

 The Nexus staging repository for this release is:

 https://repository.apache.org/content/repositories/orgapachestorm-1010

 The generated reports and documentation for this release can be found here:

 http://people.apache.org/~ptgoetz/apache-storm-0.9.3/site/

 Please vote on releasing this package as Apache Storm 0.9.3.

 This vote will be open for at least 72 hours.

 [ ] +1 Release this package as Apache Storm 0.9.3
 [ ]  0 No opinion
 [ ] -1 Do not release this package because …

 Thanks,
 Taylor




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Release Apache Storm 0.9.3

[임정택]

Agree with Nathan.
Seems like it doesn't prepared because we should take care of STORM-350.

2014-11-15 10:54 GMT+09:00 Nathan Marz nat...@nathanmarz.com:

 -1. Looking at https://issues.apache.org/jira/browse/STORM-350 it seems
 the
 upgrade to disruptor caused message loss issues. That upgrade should be
 reverted, or I'd like @clockfly to provide more insight.

 On Fri, Nov 14, 2014 at 5:19 PM, Harsha st...@harsha.io wrote:

  I am +1 releasing 0.9.3
  +1 on including STORM-555.
  Thanks,
  Harsha
 
  On Fri, Nov 14, 2014, at 02:29 PM, P. Taylor Goetz wrote:
   I’d like to get the community’s opinion on releasing 0.9.3 with what is
   currently in the 0.9.3 branch. This would be the official release
   (skipping the unofficial rc2).
  
   The only addition I’d like to include is STORM-555, which should be
   eligible for merging early next week.
  
   Thoughts?
  
   -Taylor
  
  
   Email had 1 attachment:
   + signature.asc
 1k (application/pgp-signature)
 



 --
 Twitter: @nathanmarz
 http://nathanmarz.com




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Enquiry regarding a project on @stormprocessor

[임정택]

I skimmed some files of storm-redis, but I'm still curious that what's the
key point from storm-redis.
So I have some questions.

- Is Redis for skipping completed tuples when root tuple is replayed, or
continue processing tuple from last state?
- Do users need to handle states it by his/her hand?
- Does it still beat Storm if failing tuple rate is very low or none?
- Can we still get a higher throughput when we drop Redis? (In case of not
collaborating with Redis with some situation, for example, license issue)

In addition, Calculator.java seems to a user-defined bolt, but it
communicates to Redis directly to store states of tuple, which it should be
hidden.

It seems to lack something to use it for general purposes.

ps. I encourage you to use latest Jedis, which has fixed many bugs since
2.2.1. Latest version is 2.6.0, and hopefully we'll release 2.6.1 in a few
days.


2014-11-04 5:31 GMT+09:00 Abhishek Bhattacharjee 
abhishek.bhattacharje...@gmail.com:

 Hi,
 We ( me and the CCed persons) are maintaining a project on github
 which manages to implement stateful-ness in storm.

 We have got some encouraging results through our benchmarking.
 We would like to know what more could be done so that the project could
 be inculcated into @stormprocessor

 Github link for our project: https://github.com/pict2014/storm-redis

 We are hoping for a reply.

 Cheers,
 *Abhishek Bhattacharjee*
 *Pune Institute of Computer Technology*




-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Could JStorm project collaborate with Storm?

[임정택]

As a small contributor but willing to contribute more, let me give just 2
cents to Storm community.

I think it's essential to keep Storm's originality, so merging with JStorm
should be treated seriously. And let JStorm developers learn clojure and
contribute to Storm may be faster than merging.

But at this time, we can think about why JStorm is born. It's due to
learning curve of Clojure.
I agree Taylor's comment, developers need to be polyglot programmer.
But I don't agree we can/should learn all of languages whenever we need to.
It should be a big wall.

Think about a newbie Storm contributors that know only Java.
Not intended to make a flame between languages, but if he/she need to learn
a language runs with JVM but not Java, he/she will find which language is
widely used, with regarding trends.

And it's github trending repositories, clojure and scala.
https://github.com/trending?l=clojuresince=monthly
https://github.com/trending?l=scalasince=monthly

Since I don't know Scala well either, but I feel sometimes I need to learn
Scala because many active big projects are relied on. But Clojure seems not.
(Maybe I cannot find right things, so please correct me if I'm wrong.)

And Scala has similar looks with OOP language, especially Java, so Java
programmer can implement something like Java with small learning, though it
would not beautiful and someone blames it's not Scala. But Clojure is not.
He/She cannot read/write if he/she is familiar with Clojure, at least Lisp.
It's not that simple hurdle to newbies about to learn new language.

Btw, recently I read yahoo's storm contribution doc, 'The Evolution of
Storm at Yahoo and Apache'. They (including committers of Storm) learned
unfamiliar language, Clojure.
So I have a question regarding it.
What days/hours did you spent to get familiar with Clojure? What materials
did you use to learn Clojure?

Actually I'm committer of Jedis project, and I feel Jedis can be improved
with advanced language but I think we can't move Jedis because Java is
widely-used language so it's better to collect contributors.
Seems like Storm project has same question about this. So it's interesting
to look into discussion.

Just my 2 cents.

Regards.
Jungtaek Lim (HeartSaVioR)


2014-10-30 5:17 GMT+09:00 P. Taylor Goetz ptgo...@gmail.com:

 After taking some time to review the JStorm code and history of both
 projects, here are my thoughts:

 Apache Storm is largely implemented in Java (~68%) and Clojure (~18%). The
 disparity in LOC is primarily due to the conciseness of Clojure compared to
 Java. Apache Storm's Clojure code consists mainly of implementations of
 interfaces defined in Java. This fact makes it possible to provide an
 implementation in Java itself, which is exactly what the JStorm project
 seems to have done. JStorm is a copy of the Apache Storm codebase, with the
 Clojure code being replaced with a Java implementation, with some
 additional changes.

 The justification of the project centers around a distaste for Clojure,
 and a seeming lack of developers with Clojure skills [1]:

 Storm is wonderful product, but it is implemented with Clojure. We don't
 like clojure. it isn't a populate language, so in Alibaba, few people can
 fix the bug of storm. We are the first users of Storm since it has been
 open source. During using, we found several problem such as zeromq,
 zookeeper, performance.”


 While I agree that to someone experienced with Java development, the first
 exposure to Clojure can be a bit jarring, I don’t think it is much of a
 barrier to entry. In my opinion, learning Clojure is a valuable investment
 in making oneself a more well-rounded developer. The technology ecosystem
 to which Storm belongs is becoming increasingly polyglot with respect to
 JVM languages (Java, Scala, Clojure, etc.), and developers need to adapt to
 that fact. As an example (perhaps not the best example), lets look at
 Kafka. If I didn’t like or want to learn Scala, would it make sense to
 write a clone in Java and attempt to keep feature parity and compatibility,
 or would it be better to bite the bullet and learn Scala? (Yes, I know it’s
 not the best example since Storm’s Clojure code is a subset of the
 codebase, and the way it’s designed makes it possible for alternative
 implementations). And in reality, someone from Alibaba had to at least
 learn enough Clojure to be able to replicate Storm’s Clojure implementation
 in java.

 Looking at the history of JStorm [2][3], it appears a lot of work has gone
 into keeping feature parity with Apache Storm by copying in new code or
 reimplementing it. For example a kafka spout was added about two weeks ago
 [4], and Yahoo’s storm-on-yarn project was also recently copied in [5].

 To me, this kind of work is rather counter-productive. If the Apache Storm
 community feels strongly that Storm should support multiple “core”
 implementations (i.e. one implemented in Clojure, and one implemented in
 Java), then it could be accommodated