Feel free to commit your proposed changes to a branch for evaluation. Karl On Thu, Dec 1, 2022 at 9:34 AM Julien Massiera < julien.massi...@francelabs.com> wrote:
> Hi Karl, > > I did a quick alpha version of a Solr 9 connector to test: I can confirm > that it works with older Solr versions ! > > HOWEVER, in SolrJ 9, the new Solr client has been reimplemented: it now > prevents to easily customize the httpClient and the way it performs > requests. This makes it very challenging - at least for me - to port all > of the custom code concerning the multipart post requests, as well as > the basic and preemptive auth of the current Solr connector! Who knows, > maybe with this new SolrJ client, those custom codes have become useless > and now the multipart/basic/preemptive auth work OOTB... Unfortunatly, I > don't have time to test whether those functionalities work OOTB, not to > mention that I don't have a test environment to give it a try. Maybe the > MCF committers of these solr related updates could give it a look if I > commit a final version of the connector on a dedicated branch ? > > Julien > > On 29/11/2022 22:35, Karl Wright wrote: > > Hi Julien, > > > > Sorry for the delay; I've been under intense pressure at work of late and > > just saw this email now. > > > > Regarding library updates: we should generally go ahead and do those > > FIRST. There are custom fixes for httpclient checked into the ManifoldCF > > code base so we may need to work a little to get those to build properly. > > But I'm reasonably sure it can be done. Libraries are backwards > compatible > > at the minor version level so all is good there. When somebody wants to > go > > to HttpClient 5, though, we are in trouble. > > > > AFTER that is done we should evaluate whether the 9.x Solr library is > > backwards compatible enough with 8.x to work. We had to do very little > to > > go from 7.x to 8.x, so unless the Solr people suddenly changed their > > philosophy dramatically, it should be possible to do this too. But we > will > > see. > > > > Karl > > > > > > On Tue, Nov 29, 2022 at 9:59 AM Julien Massiera < > > julien.massi...@francelabs.com> wrote: > > > >> Hi Karl, > >> > >> the Solr output connector does not seem to work with Solr 9.x according > >> to our tests. We are going to either update or develop a new connector > >> but there is a problem concerning the libraries required. A solr 9.x > >> connector will of course involve a solrj 9.x lib but also the update of > >> the following libs in MCF: > >> > >> - zookeeper from 3.4.10 to >= 3.7.0 (current 3.8.0) > >> - httpcomponent.httpclient.version from 4.5.3 to 4.5.13 > >> - httpcomponent.httpcore.version from 4.4.6 to 4.4.15 > >> - httpcomponent.httpmime.version from 4.5.3 to 4.5.13 > >> > >> Those updates should not cause problems to other connectors in MCF, the > >> real problem here concerns the current Solr connector as I am not sure > >> that an updated version would be compatible with a Solr < 9.x. > >> There is also the modified solr clients using the custom multi-parts > >> http post methods that will cause some troubles to be ported on Solrj 9 > >> .x according to me. > >> > >> If I am not wrong, historically those custom clients were developed to > >> avoid errors with the embedded Tika of Solr for some documents. But > >> IMHO, it has become a challenge that is not worth the effort: the way to > >> go should be to have the documents processed by Tika BEFORE the Solr > >> indexation. Not to mention that the tika embedded in Solr is too old > >> (1.28.1) and will most certainly be removed someday (as stated in this > >> tickethttps://issues.apache.org/jira/browse/SOLR-13973). Thus, I think > >> it is not worth it to port the custom solr clients in the new connector. > >> This would ease the creation of the Solr 9 output connector. > >> > >> Whatever happens, if we want to maintain output connectors for different > >> versions of Solr, and IF the Solr 9 output connector is not compatible > >> with previous versions of Solr (still needs to be checked), we'll end up > >> with several versions of the libs in ManifoldCF. To be honest, I do not > >> see a proper way to deal with the libs conflicts between the two > >> connectors... > >> > >> What do you think ? > >> > >> Regards, > >> Julien > >> > -- > Julien MASSIERA > Directeur développement produit > France Labs – Les experts du Search > Datafari – Vainqueur du trophée Big Data 2018 au Digital Innovation Makers > Summit > www.francelabs.com > >
