+1 to start the release process once STORM-2153 is merged. If STORM-2860 can be merged soon we can include that as well.
Thanks, Arun On 1/7/18, 4:14 PM, "Jungtaek Lim" <kabh...@gmail.com> wrote: >Now we merged STORM-2867 and STORM-2869. > >Remaining issues are STORM-2153 and STORM-2860, and STORM-2860 doesn't seem >to bring benefit on 1.x version line hence unless I'm missing here, we just >need to make sure STORM-2153 is resolved so that we could start release >phase for Storm 1.2.0. > >- Jungtaek Lim (HeartSaVioR) > >2017년 12월 29일 (금) 오전 4:51, Arun Mahadevan <ar...@apache.org>님이 작성: > >> >STORM-2869: KafkaSpout discards all pending record when adjusting the >> >consumer position after a commit [1] >> >> Hope we could get it merged this week or early next week. >> >> >> >>New Feature >> >STORM-2153: New Metrics Reporting API [2] >> >> I think this is waiting for a final +1 from revans2. >> >> >> >STORM-2867: Add consumer lag metrics to KafkaSpout [3] >> >> >> If required we can call out the kafka dependency since its a minor version >> change. It may not be an issue if we use the reflection workaround proposed >> in the PR ? >> >> IMO, it will be ideal to start the release process for 1.2.0 in the first >> week of Jan after the above three are addressed. >> >> Thanks, >> Arun >> >> >> >> On 12/27/17, 11:46 PM, "Jungtaek Lim" <kabh...@gmail.com> wrote: >> >> >Looks like we got lost the chance to make release phase be started in >> 2017, >> >but I think we are really close to be sure we could start the process in >> >early Jan. 2018. >> > >> >We haven't had "feature freeze" before releasing, so typically we still >> >have a chance to get more features in Storm 1.2.0. >> > >> >So far, what we have remaining issues for Storm 1.2.0: >> > >> >> Bug >> >STORM-2869: KafkaSpout discards all pending record when adjusting the >> >consumer position after a commit [1] >> > >> >The PR for master got +1, so once we have PR for 1.x-branch, we could go >> on >> >merging. I expect this will be done in several days, unless Stig is going >> >for long vacation. >> > >> >> New Feature >> >STORM-2153: New Metrics Reporting API [2] >> > >> >This is likely waiting for final review, so I expect this patch to be >> >finished within couple of weeks (early Jan. 2018), and if not I'd like to >> >propose moving out of 1.2.0. >> > >> >STORM-2867: Add consumer lag metrics to KafkaSpout [3] >> > >> >The patch looks good, but it requires Kafka dependency to be updated from >> >0.10.0.0 to 0.10.1.0 which might make Kafka 0.10.0.x user unable to use >> >storm-kafka-client in Storm 1.2.0. Do we want to have a poll, or would it >> >be not a big deal? >> > >> >STORM-2860: Add Kerberos support to Solr bolt [4] >> > >> >This patch breaks backward compatibility and we are discussing about >> >alternative way to not break backward compatibility for 1.2.0. If we can >> >get alternative in time, we could bring it to Storm 1.2.0, but if not, it >> >should not block the release. >> > >> >Please participate reviewing, or provide any missing issues for Storm >> >1.2.0, or give opinions on Storm 1.2.0. >> > >> >Thanks, >> >Jungtaek Lim (HeartSaVioR) >> > >> >1. https://issues.apache.org/jira/browse/STORM-2869 >> >2. https://issues.apache.org/jira/browse/STORM-2153 >> >3. https://issues.apache.org/jira/browse/STORM-2867 >> >4. https://issues.apache.org/jira/browse/STORM-2860 >> > >> >2017년 12월 18일 (월) 오전 3:50, Stig Rohde Døssing <stigdoess...@gmail.com>님이 >> 작성: >> > >> >> Alexandre, >> >> >> >> There are a couple more issues pending, see >> >> https://issues.apache.org/jira/browse/STORM-2710. It might be easier if >> >> you >> >> build the code yourself. There's a guide at >> >> >> >> >> https://github.com/apache/storm/blob/master/DEVELOPER.md#create-a-storm-distribution-packaging >> >> . >> >> The only change you should need to make is to run "mvn package >> -Dgpg.skip" >> >> instead of "mvn package" in the storm-dist directory to skip the GPG >> >> signature part. >> >> >> >> 2017-12-17 15:09 GMT+01:00 Alexandre Vermeerbergen < >> >> avermeerber...@gmail.com >> >> >: >> >> >> >> > Hello Storm developers, >> >> > >> >> > Now that I see that everything planned for Storm 1.2.0 release is done >> >> (as >> >> > I see at >> https://issues.apache.org/jira/projects/STORM/versions/12341047 >> >> ), >> >> > would it be possible to have new binaries for us to assess this new >> >> release >> >> > non-regression? >> >> > >> >> > In particular, I would like to check whether or not the bizarre >> >> capacities >> >> > metrics I get with the 1+ month-old Storm 1.2.0 preview are still >> there. >> >> > >> >> > Best regards, >> >> > Alexandre >> >> > >> >> > >> >> > 2017-12-09 10:38 GMT+01:00 Alexandre Vermeerbergen < >> >> > avermeerber...@gmail.com >> >> > >: >> >> > >> >> > > Hello Storm developers, >> >> > > >> >> > > It's been about 2 weeks that I running Storm 1.2.0 preview with 15 >> >> > > topologies, 6 Supervisors, 1 Nimbus, 3 Zookeepers, and Kafka 0.10 >> libs >> >> > all >> >> > > with Storm Kafka client spout instead of our own Kafka 0.10 spout. >> >> > > >> >> > > I noticed that statistics are going a bit nuts on bolts, with >> >> capacities >> >> > > reaching hundreds or more while everything seem to be running fine. >> >> > > This look like this is tied to topologies relying on tick tuple - >> not >> >> > sure >> >> > > but just a guess. >> >> > > >> >> > > See what kind of ridiculous capacities I can reach: >> >> > > >> >> > > Id Executors Tasks Emitted Transferred Capacity (last >> >> > > 10m) Execute latency (ms) Executed Process latency (ms) >> >> > > Acked Failed Error Host Error Port Last error Error >> Time >> >> > > aggregate 2 2 130660 130660 0.000 0.027 130620 >> >> > > 0.031 130580 0 >> >> > > alertsToKafka 3 3 0 0 0.010 0.054 2238580 >> >> > > 337.080 2238420 0 >> >> > > checkUnknown 20 20 145840 145840 120.391 52060.445 >> >> > > 19060 15062.337 18780 0 >> >> > > evaluateTriggers 17 17 2815520 2815520 130.115 >> >> > > 112.998 4755580 28.311 4756320 0 >> >> > > eventTimeFilter 1 1 13983880 13983880 51.324 >> 21.711 >> >> > > 13989060 37.980 13989260 0 >> >> > > filter 6 6 9103880 9091640 249.974 67.828 >> 13990120 >> >> > > 0.063 13989800 0 >> >> > > filterEventsWithDimensions 3 3 4754520 4754520 >> 233.222 >> >> > > 143.596 8958840 234.242 8960180 0 >> >> > > flappingDetection 8 8 2229860 2229860 0.003 0.020 >> >> > > 2229820 0.017 2229880 0 >> >> > > >> >> > > (sorry for the lost formating) >> >> > > >> >> > > we have capacity at 249.974 for filter? >> >> > > >> >> > > >> >> > > Is it a known issue? >> >> > > >> >> > > Best regards, >> >> > > Alexandre >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > 2017-12-01 22:38 GMT+01:00 Alexandre Vermeerbergen < >> >> > > avermeerber...@gmail.com>: >> >> > > >> >> > >> Hello Junktaek, >> >> > >> >> >> > >> Thanks for your answer. >> >> > >> >> >> > >> Actually the sooner Storm 1.2.0 could be released (now that we >> have a >> >> > >> working storm-kafka-client with an impressible list of fixed >> issue), >> >> the >> >> > >> better. >> >> > >> >> >> > >> Is it realistic to have some maximum date for starting the RC on >> 15th >> >> of >> >> > >> December? >> >> > >> >> >> > >> Best regards, >> >> > >> Alexandre Vermeerbergen >> >> > >> >> >> > >> >> >> > >> 2017-11-30 0:33 GMT+01:00 Jungtaek Lim <kabh...@gmail.com>: >> >> > >> >> >> > >>> We're getting closer to merge Metrics V2 in, so unless it will >> take >> >> > more >> >> > >>> than couple of weeks, we will include to 1.2.0 as well. >> >> > >>> I think STORM-2835 could come in couple of days, so that's not the >> >> > issue >> >> > >>> for releasing 1.2.0. >> >> > >>> >> >> > >>> -Jungtaek Lim (HeartSaVioR) >> >> > >>> >> >> > >>> 2017년 11월 30일 (목) 오전 1:01, Alexandre Vermeerbergen < >> >> > >>> avermeerber...@gmail.com>님이 >> >> > >>> 작성: >> >> > >>> >> >> > >>> > Hello Storm Dev team, >> >> > >>> > >> >> > >>> > Our tests with Storm 1.2.0 preview on our (large) preproduction >> has >> >> > >>> been >> >> > >>> > running fine for 1 week. >> >> > >>> > >> >> > >>> > This sound like a good opportunity to ask you if a Storm 1.2.0 >> >> > release >> >> > >>> > could come soon so that we could target it for our next >> production >> >> > >>> upgrade. >> >> > >>> > >> >> > >>> > Yet I noticed this new JIRA: >> >> > >>> > https://issues.apache.org/jira/browse/STORM-2835 >> >> > >>> > >> >> > >>> > Could it be important enough that we need it to be included in >> >> Storm >> >> > >>> 1.2.0 >> >> > >>> > ? >> >> > >>> > >> >> > >>> > best regards, >> >> > >>> > Alexandre Vermeerbergen >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > 2017-11-22 17:43 GMT+01:00 Alexandre Vermeerbergen < >> >> > >>> > avermeerber...@gmail.com >> >> > >>> > >: >> >> > >>> > >> >> > >>> > > Hello All, >> >> > >>> > > >> >> > >>> > > I seems that the "[Discuss] Release Storm 1.2.0" thread >> reached >> >> > some >> >> > >>> > > limits as my latest posts on it aren't being received. >> >> > >>> > > >> >> > >>> > > Whatever, here's the follow-up of my tests with storm-1.2.0 >> >> preview >> >> > >>> with >> >> > >>> > > latest version of Stig's storm-kafka-client: >> >> > >>> > > >> >> > >>> > > It works ! >> >> > >>> > > >> >> > >>> > > But... to make my topologies compatible with both 1.1.0 and >> >> 1.2.0, >> >> > I >> >> > >>> had >> >> > >>> > > to write an ugly method based on reflection so as to activate >> >> > "acks" >> >> > >>> on >> >> > >>> > the >> >> > >>> > > official Kafka spout when used in autocommit mode: >> >> > >>> > > >> >> > >>> > > /** >> >> > >>> > > * Toggle the mode that makes Kafka spout able to send >> acks, >> >> if >> >> > >>> it's >> >> > >>> > > supported. >> >> > >>> > > * The support for this toggle has been introduced in >> Storm >> >> > >>> 1.2.0, so >> >> > >>> > > we >> >> > >>> > > * internal use reflection in order to keep our topologies >> >> > >>> compatible >> >> > >>> > > with pre-1.2.0 >> >> > >>> > > * @param builder A kafka spout config builder. >> >> > >>> > > * @return The spout config builder that was passed in >> >> > argument, >> >> > >>> with >> >> > >>> > > modified toggle if it's supported. >> >> > >>> > > */ >> >> > >>> > > public static Builder<?, ?> >> fixKafkaSpoutAcking(Builder<?, >> >> > ?> >> >> > >>> > > builder) { >> >> > >>> > > try { >> >> > >>> > > Method m = >> >> > >>> > builder.getClass().getMethod("setTupleTrackingEnforced", >> >> > >>> > > boolean.class); >> >> > >>> > > m.invoke(builder, true); >> >> > >>> > > } catch (NoSuchMethodException | SecurityException | >> >> > >>> > > IllegalAccessException | IllegalArgumentException | >> >> > >>> > > InvocationTargetException e) { >> >> > >>> > > // Assume we're running with storm-kafka-client >> 1.1.0 >> >> > >>> > > } >> >> > >>> > > return builder; >> >> > >>> > > >> >> > >>> > > Next step for me: now that basic functionnal tests are OK, I >> want >> >> > to >> >> > >>> run >> >> > >>> > > the tests on my PPD environments having real big data rate and >> >> > >>> compare it >> >> > >>> > > with my PRD one with same volume... stay tuned! >> >> > >>> > > >> >> > >>> > > Thanks a lot to Stig for his quite reactive fixes on >> >> > >>> storm-kafka-client ! >> >> > >>> > > >> >> > >>> > > Best regards, >> >> > >>> > > Alexandre >> >> > >>> > > >> >> > >>> > >> >> > >>> >> >> > >> >> >> > >> >> >> > > >> >> > >> >> >> >>
