Seriously monumental effort everyone! Thanks for keeping this on track Jan!
- Houston On Fri, Mar 25, 2022 at 10:36 PM David Smiley <[email protected]> wrote: > Woohoo! > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Fri, Mar 25, 2022 at 4:16 PM Jan Høydahl <[email protected]> wrote: > >> Hi, >> >> All code blockers are now cleared: >> https://issues.apache.org/jira/issues/?filter=12351219 >> Some work remains for RefGuide and docker release procedures, we can >> continue on those in parallell with the RCs. >> >> I'll proceed with RC1. >> >> Jan >> >> 4. mar. 2022 kl. 21:52 skrev Houston Putman <[email protected]>: >> >> I think we have another blocker for 9.0. Basically there is a bug in the >> updated version of commons-io that causes index files to be set to >> read-only in the filesystem occasionally. The solution is to upgrade >> commons-io, or find a workaround in Solr, but we can have that discussion >> on the JIRA. >> >> More info here: https://issues.apache.org/jira/browse/SOLR-16074 >> >> On Tue, Mar 1, 2022 at 5:02 PM David Smiley <[email protected]> wrote: >> >>> I suppose the biggest spots for peer review are: >>> * use of brackets [ ] in the metric name where the request handler is. >>> Thus "/select[shard]" >>> * There is a fundamental difference in how the metrics are tracked on a >>> handler. Previously, there were metrics for all of /select (no matter how >>> it was invoked), and a few for .distrib. & .shard. depending on how it was >>> invoked. Now, the request is classified to be a shard request, or not a >>> shard request, after which separate metrics (same type/semantics) are >>> manipulated based on that classification, kind of as if there are two >>> distinct request handlers even though just one is registered. I think >>> the PR makes this clear. While I like it, the main trade-off is that a >>> user would be forced to aggregate metrics if they wanted a single metric >>> for the handler. I think the isShard=true request changes the >>> personality/mode of the handler so much that I prefer to present it as its >>> own identity from a metrics standpoint. >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> On Tue, Mar 1, 2022 at 4:19 PM Timothy Potter <[email protected]> >>> wrote: >>> >>>> Hi David, >>>> >>>> I read your note about SOLR-14401 but not clear what you need from me? >>>> Seems like you're renaming existing metrics and removing "distrib" >>>> from handlers that don't support a distrib mode, seems right to me. >>>> >>>> I actually haven't done much work on the metrics backend. For Grafana, >>>> it's a JSON file so search / replace the metrics you're changing. The >>>> Solr operator makes it really easy to set up Solr + ZK + Grafana + >>>> Prometheus + Exporter to test out your changes. It'll be pretty >>>> obvious if the dashboard is broken. >>>> >>>> Tim >>>> >>>> On Tue, Mar 1, 2022 at 7:01 AM David Smiley <[email protected]> wrote: >>>> > >>>> > >>>> > >>>> > On Tue, Mar 1, 2022 at 4:46 AM Jan Høydahl <[email protected]> >>>> wrote: >>>> >> >>>> >> Hi, and welcome to March! >>>> >> >>>> >> Our initial goal of a RC1 within February slipped, but we are still >>>> in a good position. >>>> >> I'll try to summarize the current code blockers: >>>> >> >>>> >> >>>> >> SOLR-16061 Decouple CloudSolrClient from ZkStateReader >>>> >> >>>> >> This is new, a spin-off from SOLR-15342 to prepare for solrj >>>> modularization. There is already a draft PR. Hope there will be progress on >>>> this so we don't have to delay solrj modularization until 10.0 >>>> > >>>> > >>>> > I'm working with Haythem on this (a colleague). I think it's close; >>>> it's "just" a refactoring. The main constraint on this is Haythem's time. >>>> > >>>> >> >>>> >> >>>> >> SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage for >>>> master/9.x >>>> >> >>>> >> This has not seen any movement despite repeated reminders, so unless >>>> there is progress within a few days I'll remove it as blocker and add a >>>> note to the release notes that users relying on running test framework >>>> locally should wait for a later release. >>>> > >>>> > >>>> > I'm interested in looking but not until I get through the other two. >>>> > >>>> >> >>>> >> SOLR-14401 "distrib" request handler metrics should only be tracked >>>> on... >>>> >> >>>> >> There is a PR, not sure how close to merge it is though. >>>> > >>>> > >>>> > I think the core of the change is fine but there are downstream >>>> changes needed. First level is the prometheus exporter configuration to >>>> not look for ".distrib." vs ".local."; it's different now. Next level is >>>> the Grafana dashboard. I don't normally play with JQ, Prometheus or >>>> Grafana so it's taking me some time this week. I'd appreciate any feedback >>>> on the choices here; so far only Houston has weigh'ed in. I think Tim >>>> Potter, if you're reading this, would be useful given you did major work >>>> here. >>>> > >>>> >> >>>> >> >>>> >> Also, David found a new blocker bug yesterday - the "jetty" metrics >>>> group is missing in 9.x. There will likely be another blocker due to this. >>>> >> >>>> >> >>>> >> Appreciate an update in this thread on the ETA for each of these. >>>> >> >>>> >> Jan >>>> >> >>>> >> 22. feb. 2022 kl. 12:20 skrev Jan Høydahl <[email protected]>: >>>> >> >>>> >> I created a new blocker >>>> >> >>>> >> SOLR-16040 Fix split packages in hdfs module >>>> >> >>>> >> Not sure if it needs to be a blocker though, but we should try to >>>> avoid split packages as far as we can, and this cannot be done in 9.x. >>>> >> Meanwhile, SOLR-15064 is resolved and SOLR-14401 is in PR review >>>> phase. >>>> >> Jenkins is now mostly green after some turmoil! >>>> >> >>>> >> SOLR-14290 (SolrTestCaseJ4 NPE) seems to be stalled - anyone who can >>>> lend a hand there? >>>> >> >>>> >> We also discussed in SOLR-15342 whether refactoring CloudSolrClient >>>> to untangle ZkStateProvider should be done now, and also rename solr-solrj >>>> as solr-solrj-all so that we can continue with the solrj modularization in >>>> 9.x without back-compat breaks. >>>> >> It seems worthy of a blocker to me, but we need someone willing to >>>> do the work in the next few days. Anyone? >>>> >> >>>> >> I also created SOLR-16041 (not blocker) to try to setup nightly >>>> smoketestRelease Jenkins jobs, I may try to give it a go. >>>> >> >>>> >> Assuming progress on the above, I'm still hopeful for an RC1 in the >>>> timeframe of next week. >>>> >> >>>> >> Jan >>>> >> >>>> >> 16. feb. 2022 kl. 17:05 skrev Jan Høydahl <[email protected]>: >>>> >> >>>> >> These are the three main code-blockers for doing 9.0.0 RC1: >>>> >> >>>> >> (P) SOLR-15064 Atomic/partial updates to nested docs should not >>>> assume _route_ dsmiley dsmiley >>>> >> (S) SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage >>>> for master/9.x gus gus >>>> >> (S) SOLR-14401 """distrib"" request handler metrics should only be >>>> tracked on pe dsmiley dsmiley >>>> >> >>>> >> The other blockers are mostly about the release process itself, >>>> including docker and refguide. I'm doing a clean-up of 9.0 CHANGES too. >>>> >> When these are resolved, I'll prepare RC1. That means we're really >>>> close now!! >>>> >> >>>> >> Anshum is preparing a release notes draft, and we also need to >>>> complete "Major Changes in 9.0" and "Upgrade Notes" in ref-guide before >>>> publishing the guide. >>>> >> >>>> >> Jan >>>> >> >>>> >> 7. feb. 2022 kl. 14:52 skrev Jan Høydahl <[email protected]>: >>>> >> >>>> >> Congrats on HDFS as a package! Huge win! Also some other blockers >>>> have been closed recently. >>>> >> >>>> >> Status on the 9.0 release, one week into February. >>>> >> >>>> >> - I have done a dry-run of an RC and the smoketester. Think the >>>> release scripts are ready! >>>> >> - 11 open blockers: >>>> >> >>>> >> (P) SOLR-15587 Replicas end up with base_url as http on client side >>>> even if clus thelabdude thelabdude >>>> >> (P) SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll >>>> ctargett ctargett >>>> >> (P) SOLR-15557 Figure out how to handle ref guide page >>>> renames/redirects janhoy ctargett >>>> >> (A) SOLR-15064 Atomic/partial updates to nested docs should not >>>> assume _route_ dsmiley dsmiley >>>> >> (A) SOLR-15949 Use Java 17 in docker >>>> dsmiley dsmiley >>>> >> (S) SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage >>>> for master/9.x gus gus >>>> >> (S) SOLR-14401 """distrib"" request handler metrics should only be >>>> tracked on pe dsmiley dsmiley >>>> >> (S) SOLR-15321 "Flesh out process for managing/storing ""official"" >>>> Dockerfiles houston hossman >>>> >> (U) SOLR-15941 Wire Antora refGuide into the release scripts >>>> janhoy >>>> >> (U) SOLR-15898 Complete Major changes and Upgrade Notes in RefGudie >>>> for 9.0.0 janhoy >>>> >> (D) SOLR-13138 Remove deprecated code prior to 9.0 >>>> romseygeek >>>> >> >>>> >> Based on a quick look, I assigned a readiness-code for each: >>>> >> (A): Active, pending PR (P): PR in progress, (D): Done, can be >>>> closed? (S): Stalled. (U): Unassigned >>>> >> >>>> >> Would be great to have a status and ETA for the (S)talled issues. >>>> Perhaps we can push some to 9.0.1 or 9.1? >>>> >> >>>> >> Wrt publishing the ref-guide and docker image, those could be done >>>> semi-manual for 9.0.0 if need be. >>>> >> >>>> >> Jan >>>> >> >>>> >> 31. jan. 2022 kl. 21:54 skrev Kevin Risden <[email protected] >>>> >: >>>> >> >>>> >>> SOLR-14660 Migrating HDFS into a module >>>> krisden ichattopadhyaya >>>> >> >>>> >> >>>> >> This was just merged. Big shout out to Istvan Farkas for doing 99% >>>> of the heavy lifting here! >>>> >> >>>> >> Kevin Risden >>>> >> >>>> >> >>>> >> On Sat, Jan 29, 2022 at 5:11 PM Jan Høydahl <[email protected]> >>>> wrote: >>>> >>> >>>> >>> Hi, >>>> >>> >>>> >>> Nine days ago we had 24 blockers. Several blockers have since been >>>> added and several resolved, and today we are at 19. >>>> >>> >>>> >>> I notified in the previous email that I'd remove inactive blockers. >>>> Here is the list of JIRAs for which I intend to remove the blocker flag on >>>> February 1st: >>>> >>> >>>> >>> SOLR-14097 coreRootDirectory should be solr_home/cores >>>> >>> SOLR-15064 Atomic/partial updates to nested docs should not assume >>>> _route_ param is the root ID >>>> >>> SOLR-15242 Consolidate README.md with solr/README.md >>>> >>> SOLR-15096 [REGRESSION] Collection Delete Performance >>>> significantly degraded in Java 11 v 8 >>>> >>> SOLR-15223 Deprecate HttpSolrClient, mark httpcomponents dep as >>>> "optional" in SolrJ >>>> >>> SOLR-15835 Collection creation failing with https >>>> >>> >>>> >>> Shout out if you are working on one of these and expect it to be >>>> finished soon-ish. >>>> >>> >>>> >>> That leaves 13 blockers: >>>> >>> >>>> >>> Issue key Summary >>>> Assignee Reporter >>>> >>> SOLR-14660 Migrating HDFS into a module >>>> krisden ichattopadhyaya >>>> >>> SOLR-15956 Add documentation for creating a docker image from the >>>> binary dis houston houston >>>> >>> SOLR-13138 Remove deprecated code prior to 9.0 >>>> romseygeek >>>> >>> SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll >>>> ctargett ctargett >>>> >>> SOLR-15949 Use Java 17 in docker >>>> dsmiley dsmiley >>>> >>> SOLR-15926 Fix version specification in the Solr Ref Guide >>>> houston >>>> >>> SOLR-14290 Fix NPE in SolrTestCaseJ4 breaking external usage for >>>> master/9.x gus gus >>>> >>> SOLR-12901 Make UnifiedHighlighter the default >>>> dsmiley dsmiley >>>> >>> SOLR-14401 """distrib"" request handler metrics should only be >>>> tracked on pe dsmiley dsmiley >>>> >>> SOLR-15587 Replicas end up with base_url as http on client side >>>> even if clus thelabdude thelabdude >>>> >>> SOLR-15557 Figure out how to handle ref guide page >>>> renames/redirects ctargett >>>> >>> SOLR-15898 Complete Major changes and Upgrade Notes in RefGudie >>>> for 9.0.0 janhoy >>>> >>> SOLR-15321 "Flesh out process for managing/storing ""official"" >>>> Dockerfiles houston hossman >>>> >>> >>>> >>> It seems like most of these have had some recent activity. >>>> >>> Please everyone, have a look if you can lend a hand with any of >>>> these, so we can get the list to zero early in February and do the first >>>> RC. >>>> >>> I'd appreciate some help on SOLR-15898, consolidating and >>>> structuring the "Major changes" chapter of the reference guide. >>>> >>> >>>> >>> Jan >>>> >>> >>>> >>> >>>> >>> 20. jan. 2022 kl. 19:48 skrev Jan Høydahl <[email protected]>: >>>> >>> >>>> >>> Hi, >>>> >>> >>>> >>> The list of release blockers can be seen with this JIRA filter: >>>> https://issues.apache.org/jira/issues/?filter=12351219 >>>> >>> >>>> >>> After the committer's meeting today we decided to add these to the >>>> blockers list: >>>> >>> - SOLR-15556 Ref Guide Redesign Phase 3: Replace Jekyll >>>> >>> - SOLR-15917 Rename 'contrib' as 'module' >>>> >>> - SOLR-15880 Introduce Support to K Nearest Neighbors Search >>>> >>> - SOLR-14660 Migrating HDFS into a package >>>> >>> - SOLR-12901 Make UnifiedHighlighter the default >>>> >>> - SOLR-15914 Make it super simple to add a contrib module to shared >>>> classpath >>>> >>> - And probably some minor ones too >>>> >>> >>>> >>> Each of these are in-flight and are expected to be ready really >>>> soon™. >>>> >>> >>>> >>> There are currently 24 blockers, but some of those are Unsassigned >>>> and/or have not been given any attention for some time. >>>> >>> On Feb 1st I'll take the freedom to remove blocker flag for those >>>> that have not moved anywhere since. >>>> >>> If you want to own one of them, please assign yourself and >>>> communicate progress and an ETA. >>>> >>> >>>> >>> Jan >>>> >>> >>>> >>> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>
