I think your proposal is basically a competitive/alternative proposal to my proposal for SOLR_VAR in https://lists.apache.org/thread/3vvld3xnndtthtl7sfgdbsgkbtpm55b0 which you completely agreed with. In addition to my proposal, we could move SOLR_HOME from $SOLR_TIP/server/solr to $SOLR_TIP/home since that's what Solr knows this as and there's no reason, I think, for it to be under Jetty's "server". I know "solr home" / "home" is maybe ambiguous as to its purpose to a new user but It's been a Solr concept since forever and it is possible for it to not just have configuration but data as well depending on how someone configures/uses it.
~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Jan 13, 2022 at 11:27 AM Houston Putman <[email protected]> wrote: > So would make sense to move configsets out to $SOLR_TIP/configsets and >>> have the default location of SOLR_HOME be not "./solr" but $TEMP/solr or >>> something? >>> >> >> Regardless of the rest of this discussion I think this would be a big >> win. Currently it's a pain (especially for new users/contributors) to find >> the default configsets that solr ships with. This would make it a lot >> easier. >> > > Looking at this again, maybe we just move solr/server/solr to > solr/defaults or something else. This directory can be copied to $SOLR_HOME > when starting solr, but I think it's nice to have all of those defaults > (configsets, solr.xml, zoo.cfg) together. > > On Thu, Jan 13, 2022 at 11:08 AM Houston Putman <[email protected]> > wrote: > >> So would make sense to move configsets out to $SOLR_TIP/configsets and >>> have the default location of SOLR_HOME be not "./solr" but $TEMP/solr or >>> something? >>> >> >> Regardless of the rest of this discussion I think this would be a big >> win. Currently it's a pain (especially for new users/contributors) to find >> the default configsets that solr ships with. This would make it a lot >> easier. >> >> That would also have the benefit of never polluting the git area with >>> test data? >>> >> >> Spent a few hours chasing down an issue that turned out to be this, so I >> am very much +1 here >> >> +1 renaming for contribs -> *modules* >> +1 for re-structuring dist >> >> On Thu, Jan 13, 2022 at 8:12 AM Jan Høydahl <[email protected]> >> wrote: >> >>> See https://issues.apache.org/jira/browse/SOLR-15917 for renaming of >>> "contribs" discussion. >>> >>> Wrt server/ folder that is in reality the jetty distribution with an >>> added "solr" folder for historic reasons. Yea, it is confusing. The "solr" >>> folder will become $SOLR_HOME if you start Solr without an explicit >>> $SOLR_HOME var. But it is also the authoritative location of our >>> config-sets, even if you have $SOLR_HOME somewhere else (I believe?). So >>> would make sense to move configsets out to $SOLR_TIP/configsets and have >>> the default location of SOLR_HOME be not "./solr" but $TEMP/solr or >>> something? That would also have the benefit of never polluting the git area >>> with test data? >>> >>> Jan >>> >>> 13. jan. 2022 kl. 12:00 skrev Alessandro Benedetti < >>> [email protected]>: >>> >>> +1 renaming for contribs -> plugins >>> +1 for re-structuring dist >>> >>> I also would like to raise some concern over the 'server' directory >>> structure: >>> README.txt lib resources solr-webapp >>> contexts logs scripts start.jar >>> etc modules solr >>> >>> I have been using it for years, and still, sometimes I get lost, some of >>> them are not really clear: >>> *resources* -> very generic, resources for what? currently, it contains >>> logging configurations >>> *etc* -> even more generic, currently it contains a lot of jetty stuff? >>> *contexts* -> also in this case, not sure what to expect in here (web >>> app context I suppose?) >>> *solr* -> we are in the solr binary directory, so 'solr' is again very >>> generic, this feels like the solr-home to me? >>> *modules* -> also in this case, not sure what to expect >>> *solr-webapp* -> is it just the UI? also in this case, it sounds a bit >>> generic >>> *start.jar* -> It's clear it's the entrypoint jar, but what does it >>> contain? >>> >>> Cheers >>> >>> On Thu, 13 Jan 2022 at 02:42, David Smiley <[email protected]> wrote: >>> >>>> (now to [email protected] not lucene) >>>> I'm just re-surfacing an older conversation where it is time for us to >>>> take action. >>>> Some JIRAs were recently created, and a separate thread about the >>>> prometheus-exporter >>>> >>>> Full thread in ponymail: >>>> https://lists.apache.org/thread/jbs4ds0w3r3v1hto9rqhs4qq1xfk5z61 >>>> >>>> ~ David Smiley >>>> Apache Lucene/Solr Search Developer >>>> http://www.linkedin.com/in/davidwsmiley >>>> >>>> >>>> On Sun, Jun 13, 2021 at 7:31 AM Jan Høydahl <[email protected]> >>>> wrote: >>>> >>>>> When we collapse the solr/solr structure I hope we can keep the number >>>>> of top-level folders in git to a minimum. There are too many already, so >>>>> adding all contribs don’t help. >>>>> >>>>> I hope the cobtribs will be converted to 1st party packages soon, so >>>>> perhaps “plugins” or “packages” is a good top level folder name? >>>>> >>>>> Those that are not yet converted can stay in “contribs”? >>>>> >>>>> Can we make solr-exporter a separate git repo? With separate artifacts >>>>> and separate docker image. Don’t know if that means it must also be a full >>>>> sub project? >>>>> >>>>> Jan Høydahl >>>>> >>>>> 11. jun. 2021 kl. 22:46 skrev David Smiley <[email protected]>: >>>>> >>>>> >>>>> We (all?) agree to do away with "contrib" :-). >>>>> I think a folder grouping the modules (that which can plug inside >>>>> Solr) is useful as there are a number of them -- as such this is a nice >>>>> organization IMO. There's a bunch of other stuff at the top level and I'd >>>>> rather not intermix all our modules at this layer. >>>>> >>>>> ~ David Smiley >>>>> Apache Lucene/Solr Search Developer >>>>> http://www.linkedin.com/in/davidwsmiley >>>>> >>>>> >>>>> On Fri, Jun 11, 2021 at 4:41 PM Mike Drob <[email protected]> wrote: >>>>> >>>>>> We can have modules, but why do they need to be in an additional >>>>>> folder deep? Why not just have langid next to solrj and core? Contrib to >>>>>> me >>>>>> connotes experimental or unsupported, which these things are decidedly >>>>>> not. >>>>>> >>>>>> On Fri, Jun 11, 2021 at 2:59 PM David Smiley <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> The contrib folder is just a folder of modules -- optional plugins >>>>>>> for solr-core. IMO we should simply rename "contrib" to "modules". I >>>>>>> think the only non-module in there is the prometheus exporter which >>>>>>> could >>>>>>> move out. Mike, I'm not sure if you have a different notion of what >>>>>>> "module" is? I believe most of us would be happy to move away from >>>>>>> "contrib" wording, anyway. >>>>>>> >>>>>>> ~ David Smiley >>>>>>> Apache Lucene/Solr Search Developer >>>>>>> http://www.linkedin.com/in/davidwsmiley >>>>>>> >>>>>>> >>>>>>> On Fri, Jun 11, 2021 at 3:03 PM Mike Drob <[email protected]> wrote: >>>>>>> >>>>>>>> I think related to this, I would like to see some "contibs" moved >>>>>>>> out >>>>>>>> from the contrib folder and into proper modules. Right now the >>>>>>>> definition of contrib seems to be anything that isn't core or solrj, >>>>>>>> but maybe there is room for a backup module that has gcs and s3 and >>>>>>>> hdfs all under it. LangId is already mentioned in our ref guide but >>>>>>>> we >>>>>>>> pretend like it is always present and don't think of it as a >>>>>>>> contrib. >>>>>>>> We kind of think of contrib as optional extra stuff, so maybe we >>>>>>>> call >>>>>>>> the things what they are - plugins and extensions? Then we don't >>>>>>>> have >>>>>>>> to think as hard about why certain things are showing up in which >>>>>>>> lib >>>>>>>> folders. >>>>>>>> >>>>>>>> Also, minor benefit, I would then be able to type c<tab> instead of >>>>>>>> having to type cor<tab> to disambiguate from con<tab> in my >>>>>>>> terminal. >>>>>>>> >>>>>>>> On Fri, Jun 11, 2021 at 8:09 AM David Smiley <[email protected]> >>>>>>>> wrote: >>>>>>>> > >>>>>>>> > I believe we can do a fair amount of re-organization pertaining >>>>>>>> to Jetty without losing the Jetty configuration that I think is >>>>>>>> valuable to >>>>>>>> users who want to tweak something. >>>>>>>> > >>>>>>>> > ~ David Smiley >>>>>>>> > Apache Lucene/Solr Search Developer >>>>>>>> > http://www.linkedin.com/in/davidwsmiley >>>>>>>> > >>>>>>>> > >>>>>>>> > On Fri, Jun 11, 2021 at 8:01 AM Jan Høydahl < >>>>>>>> [email protected]> wrote: >>>>>>>> >> >>>>>>>> >> +1 to a cleanup here for 9.0. As clean and neat organization as >>>>>>>> possible. Perhaps rename "dist" -> "lib"? >>>>>>>> >> >>>>>>>> >> I wish we could get rid of the server (jetty) folder altogether, >>>>>>>> and move everything from server/solr-webapp/webapp/WEB-INF/lib to >>>>>>>> "lib/deps/". But that ties into custom boot-class, getting rid of >>>>>>>> web.xml >>>>>>>> and building Jetty context in Java code.. I'm willing to help here if >>>>>>>> others also want to go this direction. This would further hide Jetty >>>>>>>> as an >>>>>>>> impl detail and let us organize stuff more freely. >>>>>>>> >> >>>>>>>> >> Jan >>>>>>>> >> >>>>>>>> >> 11. jun. 2021 kl. 13:29 skrev David Smiley <[email protected]>: >>>>>>>> >> >>>>>>>> >> Bumping this conversation up, based on recent communication. I >>>>>>>> have yet to take action but really any of us can. >>>>>>>> >> >>>>>>>> >> ~ David Smiley >>>>>>>> >> Apache Lucene/Solr Search Developer >>>>>>>> >> http://www.linkedin.com/in/davidwsmiley >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> On Mon, Nov 23, 2020 at 8:48 AM David Smiley <[email protected]> >>>>>>>> wrote: >>>>>>>> >>> >>>>>>>> >>> I'll proceed on this with lazy consensus. I suspect most of us >>>>>>>> don't care, unsurprisingly since I doubt anyone has any fondness for >>>>>>>> the >>>>>>>> "dist" folder. >>>>>>>> >>> >>>>>>>> >>> ~ David Smiley >>>>>>>> >>> Apache Lucene/Solr Search Developer >>>>>>>> >>> http://www.linkedin.com/in/davidwsmiley >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> On Sun, Nov 15, 2020 at 7:31 AM Erick Erickson < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>> >>>>>>>> >>>> Well, Solr has grown “organically” so some things just _are_, >>>>>>>> like sunrises and plagues ;) >>>>>>>> >>>> >>>>>>>> >>>> On a serious note, AFAIC rearrange as you see fit. I wonder >>>>>>>> how much of this is left over from the war days? Anything that’s lasted >>>>>>>> through all the transformations Solr has is bound to need cleaning up >>>>>>>> betimes. >>>>>>>> >>>> >>>>>>>> >>>> How would it relate to splitting Solr off into its own TLP? On >>>>>>>> the surface, I’d guess the two efforts would be orthogonal, I mention >>>>>>>> it >>>>>>>> just in case rearranging the layout would make that task easier or >>>>>>>> harder... >>>>>>>> >>>> >>>>>>>> >>>> > On Nov 15, 2020, at 12:18 AM, David Smiley < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>> > >>>>>>>> >>>> > I've been doing a bit of dependency work in one of our >>>>>>>> contribs, and observing more closely than usual exactly what we >>>>>>>> produce in >>>>>>>> the distribution layout (result of gradlew assemble). There are some >>>>>>>> tricks Dawid did in gradle/solr/packaging.gradle to pull off this >>>>>>>> stunt to >>>>>>>> keep things as they have been for many years. The distribution layout >>>>>>>> is >>>>>>>> awkward, I think. We produce this "dist" folder at the top level that >>>>>>>> has >>>>>>>> every JAR this project produces, *even contribs*. But why? I think >>>>>>>> contribs should keep to themselves. It's ridiculous that >>>>>>>> /contribs/ltr/ is >>>>>>>> empty except for a README.txt... IMO it ought to have the JAR in a >>>>>>>> "lib" >>>>>>>> subdirectory there mixed with its dependencies (LTR has none but others >>>>>>>> sure do). Today, each contrib's JAR is in "/dist". And what about >>>>>>>> SolrJ? >>>>>>>> I think SolrJ is important enough that it deserves its very own >>>>>>>> top-level >>>>>>>> directory "solrj", and like the contribs, with a "lib" alongside it. >>>>>>>> Maybe >>>>>>>> Solrj's optional dependencies could be in a lib-optional dir next to >>>>>>>> it or >>>>>>>> lib/opt/ (beneath it). Then... we don't need "dist" at all. It >>>>>>>> contains >>>>>>>> the solr-core JAR but this is redundant. Furthermore, the server >>>>>>>> webapp >>>>>>>> could be configured to add the SolrJ libs so that we don't need to >>>>>>>> redundantly put any of them in the distribution. There might be some >>>>>>>> duplicated jars overall, but not many. Logging libs might be >>>>>>>> explicitly >>>>>>>> excluded so that they are only in one spot. (Logging in Java is a >>>>>>>> mess) >>>>>>>> >>>> > >>>>>>>> >>>> > WDYT? >>>>>>>> >>>> > >>>>>>>> >>>> > ~ David Smiley >>>>>>>> >>>> > Apache Lucene/Solr Search Developer >>>>>>>> >>>> > http://www.linkedin.com/in/davidwsmiley >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>>> >>>
