The blockers are once again resolved. Building RC1 has failed 4 times in a row due to flaky tests. Also, the last 4 runs of "Solr-Check-9.0" in Jenkins have failed: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.0/
These are the tests failing in recent Jenkins runs: - org.apache.solr.search.TestCancellableCollector.classMethod - org.apache.solr.cloud.OverseerTest.testShardLeaderChange - org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod - org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots - org.apache.solr.client.solrj.impl.CloudSolrClientTest.testPerReplicaStateCollection - org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod - org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnLeaderRestart - org.apache.solr.client.solrj.impl.CloudSolrClientCacheTest.classMethod I BadApple'd TestLeaderElectionZkExpiry (SOLR-16122), and will try build the RC again. Appreciate help with fixing some of these other tests. Jan > 26. mar. 2022 kl. 13:41 skrev Jan Høydahl <[email protected]>: > > Hi, > > I filed these new blockers for build bugs, with attached PRs. Please do a > quick review: > > https://issues.apache.org/jira/browse/SOLR-16117 > <https://issues.apache.org/jira/browse/SOLR-16117> Do not publish an empty > solr-server.jar to maven > https://issues.apache.org/jira/browse/SOLR-16118 > <https://issues.apache.org/jira/browse/SOLR-16118> Solr source jars should > contain NOTICE, LICENSE files > > Also some more smokeTester changes were necessary: > > https://github.com/apache/solr/pull/763 > <https://github.com/apache/solr/pull/763> (SOLR-15852: Smoketester fixes for > maven artifact comparison) > > > > I just built a release with these three PRs applied, and then the smoke > tester passes for me. So once I get at least one thumbs up for each of them, > I'll merge and build RC1. > > Jan > > >> 26. mar. 2022 kl. 04:25 skrev Houston Putman <[email protected] >> <mailto:[email protected]>>: >> >> Seriously monumental effort everyone! >> >> Thanks for keeping this on track Jan! >> >> - Houston >> >> On Fri, Mar 25, 2022 at 10:36 PM David Smiley <[email protected] >> <mailto:[email protected]>> wrote: >> Woohoo! >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> <http://www.linkedin.com/in/davidwsmiley> >> >> On Fri, Mar 25, 2022 at 4:16 PM Jan Høydahl <[email protected] >> <mailto:[email protected]>> wrote: >> Hi, >> >> All code blockers are now cleared: >> https://issues.apache.org/jira/issues/?filter=12351219 >> <https://issues.apache.org/jira/issues/?filter=12351219> >> Some work remains for RefGuide and docker release procedures, we can >> continue on those in parallell with the RCs. >> >> I'll proceed with RC1. >> >> Jan >> >>> 4. mar. 2022 kl. 21:52 skrev Houston Putman <[email protected] >>> <mailto:[email protected]>>: >>> >>> I think we have another blocker for 9.0. Basically there is a bug in the >>> updated version of commons-io that causes index files to be set to >>> read-only in the filesystem occasionally. The solution is to upgrade >>> commons-io, or find a workaround in Solr, but we can have that discussion >>> on the JIRA. >>> >>> More info here: https://issues.apache.org/jira/browse/SOLR-16074 >>> <https://issues.apache.org/jira/browse/SOLR-16074> >>> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <[email protected] >>> <mailto:[email protected]>> wrote: >>> I suppose the biggest spots for peer review are: >>> * use of brackets [ ] in the metric name where the request handler is. >>> Thus "/select[shard]" >>> * There is a fundamental difference in how the metrics are tracked on a >>> handler. Previously, there were metrics for all of /select (no matter how >>> it was invoked), and a few for .distrib. & .shard. depending on how it was >>> invoked. Now, the request is classified to be a shard request, or not a >>> shard request, after which separate metrics (same type/semantics) are >>> manipulated based on that classification, kind of as if there are two >>> distinct request handlers even though just one is registered. I think the >>> PR makes this clear. While I like it, the main trade-off is that a user >>> would be forced to aggregate metrics if they wanted a single metric for the >>> handler. I think the isShard=true request changes the personality/mode of >>> the handler so much that I prefer to present it as its own identity from a >>> metrics standpoint. >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> <http://www.linkedin.com/in/davidwsmiley> >>> >>> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hi David, >>> >>> I read your note about SOLR-14401 but not clear what you need from me? >>> Seems like you're renaming existing metrics and removing "distrib" >>> from handlers that don't support a distrib mode, seems right to me. >>> >>> I actually haven't done much work on the metrics backend. For Grafana, >>> it's a JSON file so search / replace the metrics you're changing. The >>> Solr operator makes it really easy to set up Solr + ZK + Grafana + >>> Prometheus + Exporter to test out your changes. It'll be pretty >>> obvious if the dashboard is broken. >>> >>> Tim >>> >>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <[email protected] >>> <mailto:[email protected]>> wrote: >>> > >>> > >>> > >>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <[email protected] >>> > <mailto:[email protected]>> wrote: >>> >> >>> >> Hi, and welcome to March! >>> >> >>> >> Our initial goal of a RC1 within February slipped, but we are still in a >>> >> good position. >>> >> I'll try to summarize the current code blockers: >>> >> >>> >> >>> >> SOLR-16061 Decouple CloudSolrClient from ZkStateReader >>> >> >>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj >>> >> modularization. There is already a draft PR. Hope there will be progress >>> >> on this so we don't have to delay solrj modularization until 10.0 >>> > >>> > >>> > I'm working with Haythem on this (a colleague). I think it's close; it's >>> > "just" a refactoring. The main constraint on this is Haythem's time. >>> > >>> >> >>> >> >>> >> SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage for >>> >> master/9.x >>> >> >>> >> This has not seen any movement despite repeated reminders, so unless >>> >> there is progress within a few days I'll remove it as blocker and add a >>> >> note to the release notes that users relying on running test framework >>> >> locally should wait for a later release. >>> > >>> > >>> > I'm interested in looking but not until I get through the other two. >>> > >>> >> >>> >> SOLR-14401 "distrib" request handler metrics should only be tracked >>> >> on... >>> >> >>> >> There is a PR, not sure how close to merge it is though. >>> > >>> > >>> > I think the core of the change is fine but there are downstream changes >>> > needed. First level is the prometheus exporter configuration to not look >>> > for ".distrib." vs ".local."; it's different now. Next level is the >>> > Grafana dashboard. I don't normally play with JQ, Prometheus or Grafana >>> > so it's taking me some time this week. I'd appreciate any feedback on >>> > the choices here; so far only Houston has weigh'ed in. I think Tim >>> > Potter, if you're reading this, would be useful given you did major work >>> > here. >>> > >>> >> >>> >> >>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics >>> >> group is missing in 9.x. There will likely be another blocker due to >>> >> this. >>> >> >>> >> >>> >> Appreciate an update in this thread on the ETA for each of these. >>> >> >>> >> Jan >>> >> >>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <[email protected] >>> >> <mailto:[email protected]>>: >>> >> >>> >> I created a new blocker >>> >> >>> >> SOLR-16040 Fix split packages in hdfs module >>> >> >>> >> Not sure if it needs to be a blocker though, but we should try to avoid >>> >> split packages as far as we can, and this cannot be done in 9.x. >>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review phase. >>> >> Jenkins is now mostly green after some turmoil! >>> >> >>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can >>> >> lend a hand there? >>> >> >>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient to >>> >> untangle ZkStateProvider should be done now, and also rename solr-solrj >>> >> as solr-solrj-all so that we can continue with the solrj modularization >>> >> in 9.x without back-compat breaks. >>> >> It seems worthy of a blocker to me, but we need someone willing to do >>> >> the work in the next few days. Anyone? >>> >> >>> >> I also created SOLR-16041 (not blocker) to try to setup nightly >>> >> smoketestRelease Jenkins jobs, I may try to give it a go. >>> >> >>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the >>> >> timeframe of next week. >>> >> >>> >> Jan >>> >> >>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <[email protected] >>> >> <mailto:[email protected]>>: >>> >> >>> >> These are the three main code-blockers for doing 9.0.0 RC1: >>> >> >>> >> (P) SOLR-15064 Atomic/partial updates to nested docs should not assume >>> >> _route_ dsmiley dsmiley >>> >> (S) SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage for >>> >> master/9.x gus gus >>> >> (S) SOLR-14401 """distrib"" request handler metrics should only be >>> >> tracked on pe dsmiley dsmiley >>> >> >>> >> The other blockers are mostly about the release process itself, >>> >> including docker and refguide. I'm doing a clean-up of 9.0 CHANGES too. >>> >> When these are resolved, I'll prepare RC1. That means we're really close >>> >> now!! >>> >> >>> >> Anshum is preparing a release notes draft, and we also need to complete >>> >> "Major Changes in 9.0" and "Upgrade Notes" in ref-guide before >>> >> publishing the guide. >>> >> >>> >> Jan >>> >> >>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <[email protected] >>> >> <mailto:[email protected]>>: >>> >> >>> >> Congrats on HDFS as a package! Huge win! Also some other blockers have >>> >> been closed recently. >>> >> >>> >> Status on the 9.0 release, one week into February. >>> >> >>> >> - I have done a dry-run of an RC and the smoketester. Think the release >>> >> scripts are ready! >>> >> - 11 open blockers: >>> >> >>> >> (P) SOLR-15587 Replicas end up with base_url as http on client side >>> >> even if clus thelabdude thelabdude >>> >> (P) SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll >>> >> ctargett ctargett >>> >> (P) SOLR-15557 Figure out how to handle ref guide page >>> >> renames/redirects janhoy ctargett >>> >> (A) SOLR-15064 Atomic/partial updates to nested docs should not assume >>> >> _route_ dsmiley dsmiley >>> >> (A) SOLR-15949 Use Java 17 in docker >>> >> dsmiley dsmiley >>> >> (S) SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage for >>> >> master/9.x gus gus >>> >> (S) SOLR-14401 """distrib"" request handler metrics should only be >>> >> tracked on pe dsmiley dsmiley >>> >> (S) SOLR-15321 "Flesh out process for managing/storing ""official"" >>> >> Dockerfiles houston hossman >>> >> (U) SOLR-15941 Wire Antora refGuide into the release scripts >>> >> janhoy >>> >> (U) SOLR-15898 Complete Major changes and Upgrade Notes in RefGudie for >>> >> 9.0.0 janhoy >>> >> (D) SOLR-13138 Remove deprecated code prior to 9.0 >>> >> romseygeek >>> >> >>> >> Based on a quick look, I assigned a readiness-code for each: >>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be closed? >>> >> (S): Stalled. (U): Unassigned >>> >> >>> >> Would be great to have a status and ETA for the (S)talled issues. >>> >> Perhaps we can push some to 9.0.1 or 9.1? >>> >> >>> >> Wrt publishing the ref-guide and docker image, those could be done >>> >> semi-manual for 9.0.0 if need be. >>> >> >>> >> Jan >>> >> >>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <[email protected] >>> >> <mailto:[email protected]>>: >>> >> >>> >>> SOLR-14660 Migrating HDFS into a module >>> >>> krisden ichattopadhyaya >>> >> >>> >> >>> >> This was just merged. Big shout out to Istvan Farkas for doing 99% of >>> >> the heavy lifting here! >>> >> >>> >> Kevin Risden >>> >> >>> >> >>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <[email protected] >>> >> <mailto:[email protected]>> wrote: >>> >>> >>> >>> Hi, >>> >>> >>> >>> Nine days ago we had 24 blockers. Several blockers have since been >>> >>> added and several resolved, and today we are at 19. >>> >>> >>> >>> I notified in the previous email that I'd remove inactive blockers. >>> >>> Here is the list of JIRAs for which I intend to remove the blocker flag >>> >>> on February 1st: >>> >>> >>> >>> SOLR-14097 coreRootDirectory should be solr_home/cores >>> >>> SOLR-15064 Atomic/partial updates to nested docs should not assume >>> >>> _route_ param is the root ID >>> >>> SOLR-15242 Consolidate README.md with solr/README.md >>> >>> SOLR-15096 [REGRESSION] Collection Delete Performance significantly >>> >>> degraded in Java 11 v 8 >>> >>> SOLR-15223 Deprecate HttpSolrClient, mark httpcomponents dep as >>> >>> "optional" in SolrJ >>> >>> SOLR-15835 Collection creation failing with https >>> >>> >>> >>> Shout out if you are working on one of these and expect it to be >>> >>> finished soon-ish. >>> >>> >>> >>> That leaves 13 blockers: >>> >>> >>> >>> Issue key Summary >>> >>> Assignee Reporter >>> >>> SOLR-14660 Migrating HDFS into a module >>> >>> krisden ichattopadhyaya >>> >>> SOLR-15956 Add documentation for creating a docker image from the >>> >>> binary dis houston houston >>> >>> SOLR-13138 Remove deprecated code prior to 9.0 >>> >>> romseygeek >>> >>> SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll >>> >>> ctargett ctargett >>> >>> SOLR-15949 Use Java 17 in docker >>> >>> dsmiley dsmiley >>> >>> SOLR-15926 Fix version specification in the Solr Ref Guide >>> >>> houston >>> >>> SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage for >>> >>> master/9.x gus gus >>> >>> SOLR-12901 Make UnifiedHighlighter the default >>> >>> dsmiley dsmiley >>> >>> SOLR-14401 """distrib"" request handler metrics should only be tracked >>> >>> on pe dsmiley dsmiley >>> >>> SOLR-15587 Replicas end up with base_url as http on client side even >>> >>> if clus thelabdude thelabdude >>> >>> SOLR-15557 Figure out how to handle ref guide page renames/redirects >>> >>> ctargett >>> >>> SOLR-15898 Complete Major changes and Upgrade Notes in RefGudie for >>> >>> 9.0.0 janhoy >>> >>> SOLR-15321 "Flesh out process for managing/storing ""official"" >>> >>> Dockerfiles houston hossman >>> >>> >>> >>> It seems like most of these have had some recent activity. >>> >>> Please everyone, have a look if you can lend a hand with any of these, >>> >>> so we can get the list to zero early in February and do the first RC. >>> >>> I'd appreciate some help on SOLR-15898, consolidating and structuring >>> >>> the "Major changes" chapter of the reference guide. >>> >>> >>> >>> Jan >>> >>> >>> >>> >>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <[email protected] >>> >>> <mailto:[email protected]>>: >>> >>> >>> >>> Hi, >>> >>> >>> >>> The list of release blockers can be seen with this JIRA filter: >>> >>> https://issues.apache.org/jira/issues/?filter=12351219 >>> >>> <https://issues.apache.org/jira/issues/?filter=12351219> >>> >>> >>> >>> After the committer's meeting today we decided to add these to the >>> >>> blockers list: >>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll >>> >>> - SOLR-15917 Rename 'contrib' as 'module' >>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search >>> >>> - SOLR-14660 Migrating HDFS into a package >>> >>> - SOLR-12901 Make UnifiedHighlighter the default >>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared >>> >>> classpath >>> >>> - And probably some minor ones too >>> >>> >>> >>> Each of these are in-flight and are expected to be ready really soon™. >>> >>> >>> >>> There are currently 24 blockers, but some of those are Unsassigned >>> >>> and/or have not been given any attention for some time. >>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those that >>> >>> have not moved anywhere since. >>> >>> If you want to own one of them, please assign yourself and communicate >>> >>> progress and an ETA. >>> >>> >>> >>> Jan >>> >>> >>> >>> >>> >> >>> >> >>> >> >>> >> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> <mailto:[email protected]> >>> For additional commands, e-mail: [email protected] >>> <mailto:[email protected]> >>> >> >
