+1 renaming for contribs -> plugins +1 for re-structuring dist I also would like to raise some concern over the 'server' directory structure:
README.txt lib resources solr-webapp contexts logs scripts start.jar etc modules solr I have been using it for years, and still, sometimes I get lost, some of them are not really clear: *resources* -> very generic, resources for what? currently, it contains logging configurations *etc* -> even more generic, currently it contains a lot of jetty stuff? *contexts* -> also in this case, not sure what to expect in here (web app context I suppose?) *solr* -> we are in the solr binary directory, so 'solr' is again very generic, this feels like the solr-home to me? *modules* -> also in this case, not sure what to expect *solr-webapp* -> is it just the UI? also in this case, it sounds a bit generic *start.jar* -> It's clear it's the entrypoint jar, but what does it contain? Cheers On Thu, 13 Jan 2022 at 02:42, David Smiley <dsmi...@apache.org> wrote: > (now to dev@solr.apache.org not lucene) > I'm just re-surfacing an older conversation where it is time for us to > take action. > Some JIRAs were recently created, and a separate thread about the > prometheus-exporter > > Full thread in ponymail: > https://lists.apache.org/thread/jbs4ds0w3r3v1hto9rqhs4qq1xfk5z61 > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Sun, Jun 13, 2021 at 7:31 AM Jan Høydahl <jan....@cominvent.com> wrote: > >> When we collapse the solr/solr structure I hope we can keep the number of >> top-level folders in git to a minimum. There are too many already, so >> adding all contribs don’t help. >> >> I hope the cobtribs will be converted to 1st party packages soon, so >> perhaps “plugins” or “packages” is a good top level folder name? >> >> Those that are not yet converted can stay in “contribs”? >> >> Can we make solr-exporter a separate git repo? With separate artifacts >> and separate docker image. Don’t know if that means it must also be a full >> sub project? >> >> Jan Høydahl >> >> 11. jun. 2021 kl. 22:46 skrev David Smiley <dsmi...@apache.org>: >> >> >> We (all?) agree to do away with "contrib" :-). >> I think a folder grouping the modules (that which can plug inside Solr) >> is useful as there are a number of them -- as such this is a nice >> organization IMO. There's a bunch of other stuff at the top level and I'd >> rather not intermix all our modules at this layer. >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Fri, Jun 11, 2021 at 4:41 PM Mike Drob <md...@mdrob.com> wrote: >> >>> We can have modules, but why do they need to be in an additional folder >>> deep? Why not just have langid next to solrj and core? Contrib to me >>> connotes experimental or unsupported, which these things are decidedly not. >>> >>> On Fri, Jun 11, 2021 at 2:59 PM David Smiley <dsmi...@apache.org> wrote: >>> >>>> The contrib folder is just a folder of modules -- optional plugins for >>>> solr-core. IMO we should simply rename "contrib" to "modules". I think >>>> the only non-module in there is the prometheus exporter which could move >>>> out. Mike, I'm not sure if you have a different notion of what "module" >>>> is? I believe most of us would be happy to move away from "contrib" >>>> wording, anyway. >>>> >>>> ~ David Smiley >>>> Apache Lucene/Solr Search Developer >>>> http://www.linkedin.com/in/davidwsmiley >>>> >>>> >>>> On Fri, Jun 11, 2021 at 3:03 PM Mike Drob <md...@mdrob.com> wrote: >>>> >>>>> I think related to this, I would like to see some "contibs" moved out >>>>> from the contrib folder and into proper modules. Right now the >>>>> definition of contrib seems to be anything that isn't core or solrj, >>>>> but maybe there is room for a backup module that has gcs and s3 and >>>>> hdfs all under it. LangId is already mentioned in our ref guide but we >>>>> pretend like it is always present and don't think of it as a contrib. >>>>> We kind of think of contrib as optional extra stuff, so maybe we call >>>>> the things what they are - plugins and extensions? Then we don't have >>>>> to think as hard about why certain things are showing up in which lib >>>>> folders. >>>>> >>>>> Also, minor benefit, I would then be able to type c<tab> instead of >>>>> having to type cor<tab> to disambiguate from con<tab> in my terminal. >>>>> >>>>> On Fri, Jun 11, 2021 at 8:09 AM David Smiley <dsmi...@apache.org> >>>>> wrote: >>>>> > >>>>> > I believe we can do a fair amount of re-organization pertaining to >>>>> Jetty without losing the Jetty configuration that I think is valuable to >>>>> users who want to tweak something. >>>>> > >>>>> > ~ David Smiley >>>>> > Apache Lucene/Solr Search Developer >>>>> > http://www.linkedin.com/in/davidwsmiley >>>>> > >>>>> > >>>>> > On Fri, Jun 11, 2021 at 8:01 AM Jan Høydahl <jan....@cominvent.com> >>>>> wrote: >>>>> >> >>>>> >> +1 to a cleanup here for 9.0. As clean and neat organization as >>>>> possible. Perhaps rename "dist" -> "lib"? >>>>> >> >>>>> >> I wish we could get rid of the server (jetty) folder altogether, >>>>> and move everything from server/solr-webapp/webapp/WEB-INF/lib to >>>>> "lib/deps/". But that ties into custom boot-class, getting rid of web.xml >>>>> and building Jetty context in Java code.. I'm willing to help here if >>>>> others also want to go this direction. This would further hide Jetty as an >>>>> impl detail and let us organize stuff more freely. >>>>> >> >>>>> >> Jan >>>>> >> >>>>> >> 11. jun. 2021 kl. 13:29 skrev David Smiley <dsmi...@apache.org>: >>>>> >> >>>>> >> Bumping this conversation up, based on recent communication. I >>>>> have yet to take action but really any of us can. >>>>> >> >>>>> >> ~ David Smiley >>>>> >> Apache Lucene/Solr Search Developer >>>>> >> http://www.linkedin.com/in/davidwsmiley >>>>> >> >>>>> >> >>>>> >> On Mon, Nov 23, 2020 at 8:48 AM David Smiley <dsmi...@apache.org> >>>>> wrote: >>>>> >>> >>>>> >>> I'll proceed on this with lazy consensus. I suspect most of us >>>>> don't care, unsurprisingly since I doubt anyone has any fondness for the >>>>> "dist" folder. >>>>> >>> >>>>> >>> ~ David Smiley >>>>> >>> Apache Lucene/Solr Search Developer >>>>> >>> http://www.linkedin.com/in/davidwsmiley >>>>> >>> >>>>> >>> >>>>> >>> On Sun, Nov 15, 2020 at 7:31 AM Erick Erickson < >>>>> erickerick...@gmail.com> wrote: >>>>> >>>> >>>>> >>>> Well, Solr has grown “organically” so some things just _are_, >>>>> like sunrises and plagues ;) >>>>> >>>> >>>>> >>>> On a serious note, AFAIC rearrange as you see fit. I wonder how >>>>> much of this is left over from the war days? Anything that’s lasted >>>>> through >>>>> all the transformations Solr has is bound to need cleaning up betimes. >>>>> >>>> >>>>> >>>> How would it relate to splitting Solr off into its own TLP? On >>>>> the surface, I’d guess the two efforts would be orthogonal, I mention it >>>>> just in case rearranging the layout would make that task easier or >>>>> harder... >>>>> >>>> >>>>> >>>> > On Nov 15, 2020, at 12:18 AM, David Smiley <dsmi...@apache.org> >>>>> wrote: >>>>> >>>> > >>>>> >>>> > I've been doing a bit of dependency work in one of our >>>>> contribs, and observing more closely than usual exactly what we produce in >>>>> the distribution layout (result of gradlew assemble). There are some >>>>> tricks Dawid did in gradle/solr/packaging.gradle to pull off this stunt to >>>>> keep things as they have been for many years. The distribution layout is >>>>> awkward, I think. We produce this "dist" folder at the top level that has >>>>> every JAR this project produces, *even contribs*. But why? I think >>>>> contribs should keep to themselves. It's ridiculous that /contribs/ltr/ >>>>> is >>>>> empty except for a README.txt... IMO it ought to have the JAR in a "lib" >>>>> subdirectory there mixed with its dependencies (LTR has none but others >>>>> sure do). Today, each contrib's JAR is in "/dist". And what about SolrJ? >>>>> I think SolrJ is important enough that it deserves its very own top-level >>>>> directory "solrj", and like the contribs, with a "lib" alongside it. >>>>> Maybe >>>>> Solrj's optional dependencies could be in a lib-optional dir next to it or >>>>> lib/opt/ (beneath it). Then... we don't need "dist" at all. It contains >>>>> the solr-core JAR but this is redundant. Furthermore, the server webapp >>>>> could be configured to add the SolrJ libs so that we don't need to >>>>> redundantly put any of them in the distribution. There might be some >>>>> duplicated jars overall, but not many. Logging libs might be explicitly >>>>> excluded so that they are only in one spot. (Logging in Java is a mess) >>>>> >>>> > >>>>> >>>> > WDYT? >>>>> >>>> > >>>>> >>>> > ~ David Smiley >>>>> >>>> > Apache Lucene/Solr Search Developer >>>>> >>>> > http://www.linkedin.com/in/davidwsmiley >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> --------------------------------------------------------------------- >>>>> >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> >>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >>>> >>>>> >> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>> >>>>>
