Re: Need help on LTR
Does your feature definitions and the feature names used in the model match? On Tue, Mar 19, 2019 at 10:17 AM Amjad Khan wrote: > Yes, I did. > > I can see the feature that I created by this > schema/feature-store/exampleFeatureStore and it return me the features I > created. But issue is when I try to put store-model. > > > On Mar 19, 2019, at 12:18 AM, Mohomed Rimash > wrote: > > > > Hi Amjad, After adding the libraries into the path, Did you restart the > > SOLR ? > > > > On Tue, 19 Mar 2019 at 08:45, Amjad Khan wrote: > > > >> I followed the Solr LTR Documentation > >> > >> https://lucene.apache.org/solr/guide/7_4/learning-to-rank.html < > >> https://lucene.apache.org/solr/guide/7_4/learning-to-rank.html> > >> > >> 1. Added library into the solr-config > >> > >> >> regex=".*\.jar" /> > >> >> regex="solr-ltr-\d.*\.jar" /> > >> 2. Successfully added feature > >> 3. Get schema to see feature is available > >> 4. When I try to push model I see the error below, however I added the > lib > >> into solr-cofig > >> > >> Response > >> { > >> "responseHeader":{ > >>"status":400, > >>"QTime":1}, > >> "error":{ > >>"metadata":[ > >> "error-class","org.apache.solr.common.SolrException", > >> "root-error-class","java.lang.NullPointerException"], > >>"msg":"org.apache.solr.ltr.model.ModelException: Model type does not > >> exist org.apache.solr.ltr.model.LinearModel", > >>"code":400}} > >> > >> Thanks > >
Response time creep in Solr
Hi All, >From the past few months there has been a steady increase in the Solr response time in our application, yes there are enhancements and index size increase. How to approach this issue to find the root cause for this slow and constant increase? What parameters to check? Any pointers on this is appreciated! Thank you, Roopa
Re: Learning to Rank (LTR) with grouping
Hi Ilay, I am still on Solr 6.6.0 and did not patch the grouping fix. I implemented a temporary workaround solution to have 2 async request from the web application 1st with grouping 2nd without grouping and merged the results. This solution worked for my case as we were getting grouping results for specific tiles in the page. Roopa On Mon, Apr 2, 2018 at 2:57 AM, ilayaraja wrote: > Hi Roopa & Deigo, > > I am facing same issue with grouping. Currently, am on Solr 7.2.1 but > still > see that grouping with LTR is not working. Did you apply it as patch or the > latest solr version has the fix already? > > Ilay > > > > - > --Ilay > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: LTR not able to upload org.apache.solr.ltr.model.MultipleAdditiveTreesModel
Here is the stacktrace Caused by: org.apache.solr.common.SolrException: Failed to create new ManagedResource /schema/model-store of type org.apache.solr.ltr.store.rest.ManagedModelStore due to: org.apache.solr.common.SolrException: org.apache.solr.ltr.model.ModelException: Model type does not exist org.apache.solr.ltr.model.MultipleAdditiveTreesModel at org.apache.solr.core.SolrCore.(SolrCore.java:965) at org.apache.solr.core.SolrCore.reload(SolrCore.java:641) at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1107) ... 37 more Caused by: org.apache.solr.common.SolrException: Failed to create new ManagedResource /schema/model-store of type org.apache.solr.ltr.store.rest.ManagedModelStore due to: org.apache.solr.common.SolrException: org.apache.solr.ltr.model.ModelException: Model type does not exist org.apache.solr.ltr.model.MultipleAdditiveTreesModel at org.apache.solr.rest.RestManager.createManagedResource(RestManager.java:700) a at org.apache.solr.rest.RestManager.addRegisteredResource(RestManager.java:666) at org.apache.solr.rest.RestManager.access$300(RestManager.java:59) at org.apache.solr.rest.RestManager$Registry.registerManagedResource(RestManager.java:231) at org.apache.solr.ltr.store.rest.ManagedModelStore.registerManagedModelStore(ManagedModelStore.java:52) at org.apache.solr.ltr.search.LTRQParserPlugin.inform(LTRQParserPlugin.java:119) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:719) at org.apache.solr.core.SolrCore.(SolrCore.java:944) ... 39 more Caused by: org.apache.solr.common.SolrException: org.apache.solr.ltr.model.ModelException: Model type does not exist org.apache.solr.ltr.model.MultipleAdditiveTreesModel at org.apache.solr.ltr.store.rest.ManagedModelStore.addModelFromMap(ManagedModelStore.java:137) at org.apache.solr.ltr.store.rest.ManagedModelStore.loadStoredModels(ManagedModelStore.java:127) at org.apache.solr.ltr.search.LTRQParserPlugin.onManagedResourceInitialized(LTRQParserPlugin.java:133) at org.apache.solr.rest.ManagedResource.notifyObserversDuringInit(ManagedResource.java:115) at org.apache.solr.rest.ManagedResource.loadManagedDataAndNotify(ManagedResource.java:91) at org.apache.solr.rest.RestManager.createManagedResource(RestManager.java:694) ... 46 more Caused by: org.apache.solr.ltr.model.ModelException: Model type does not exist org.apache.solr.ltr.model.MultipleAdditiveTreesModel at org.apache.solr.ltr.model.LTRScoringModel.getInstance(LTRScoringModel.java:103) at org.apache.solr.ltr.store.rest.ManagedModelStore.fromLTRScoringModelMap(ManagedModelStore.java:235) at org.apache.solr.ltr.store.rest.ManagedModelStore.initWrapperModel(ManagedModelStore.java:254) at org.apache.solr.ltr.store.rest.ManagedModelStore.fromLTRScoringModelMap(ManagedModelStore.java:245) at org.apache.solr.ltr.store.rest.ManagedModelStore.addModelFromMap(ManagedModelStore.java:134) ... 51 more Caused by: org.apache.solr.common.SolrException: Error instantiating class: 'org.apache.solr.ltr.model.MultipleAdditiveTreesModel' at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:655) at org.apache.solr.ltr.model.LTRScoringModel.getInstance(LTRScoringModel.java:93) ... 55 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:637) ... 56 more Caused by: java.lang.NullPointerException at org.apache.solr.ltr.model.MultipleAdditiveTreesModel.(MultipleAdditiveTreesModel.java:308) ... 61 more On Wed, Mar 14, 2018 at 10:32 AM, Alessandro Benedetti wrote: > This is the piece of code involved : > > "try { > // create an instance of the model > model = solrResourceLoader.newInstance( > className, > LTRScoringModel.class, > new String[0], // no sub packages > new Class[] { String.class, List.class, List.class, String.class, > List.class, Map.class }, > new Object[] { name, features, norms, featureStoreName, > allFeatures, params }); > if (params != null) { > SolrPluginUtils.invokeSetters(model, params.entrySet()); > } > } catch (final Exception e) { > throw new ModelException("Model type does not exist " + className, > e); > }" > > I admit it is generic and contains even a catch "Exception" clause, but > wasn't it logging the stacktrace ? > Just out of curiosity, how was the entire stacktrace ? > > This may help to improve it. > > Regards > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-Use
Re: LTR not able to upload org.apache.solr.ltr.model.MultipleAdditiveTreesModel
Hi Alessandro, I figured the issue, the model was using a feature which was not in the features file. The error was very generic so was hard to find this. Thank you, Roopa On Wed, Mar 14, 2018 at 7:16 AM, Alessandro Benedetti wrote: > Hi Roopa, > that model changed name few times, which Apache Solr version are you using > ? > It is very likely you are using a class name not in sync with your Apache > Solr version. > > Regards > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: LTR Model size
What would be the best way to patch this to Solr 6.6 without having to do a full upgrade Thanks, Roopa On Fri, Mar 9, 2018 at 4:55 PM, Erick Erickson wrote: > Spoonerk: > > Please follow the instructions here: > http://lucene.apache.org/solr/community.html#mailing-lists-irc > > . You must use the _exact_ same e-mail as you used to subscribe. > > If the initial try doesn't work and following the suggestions at the > "problems" link doesn't work for you, let us know. But note you need > to show us the _entire_ return header to allow anyone to diagnose the > problem. > > > Best, > Erick > > On Fri, Mar 9, 2018 at 12:15 PM, spoonerk wrote: > > Please unsubscribe me. I have tried and tried but still get emails > > > > On Mar 9, 2018 10:19 AM, "Roopa Rao" wrote: > > > >> what is the way to configure the model size for LTR? We have about 3MB > >> model and Solr is not holding this model as a ManagedResource. > >> > >> How can this be configured? > >> > >> Thanks, > >> Roopa > >> >
Re: LTR Model size
Is there a way to patch the fix to use a external model to Solr 6.6 ? Thanks, Roopa On Fri, Mar 9, 2018 at 1:30 PM, tapan1707 wrote: > Could you elaborate a little bit more? Otherwise, I think that you might be > experiencing the same issue reported in > https://issues.apache.org/jira/browse/SOLR-11049 . > Default zookeeper znode limit is 1mb, so I think it might not be able to > handle the size of your model. > Correct me if I misunderstood anything. > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: LTR Model size
Thank you, this is exactly what I am facing. There is a mention of increasing jute.maxbuffer in the Jira I will try that option. Thanks, Roopa On Fri, Mar 9, 2018 at 1:30 PM, tapan1707 wrote: > Could you elaborate a little bit more? Otherwise, I think that you might be > experiencing the same issue reported in > https://issues.apache.org/jira/browse/SOLR-11049 . > Default zookeeper znode limit is 1mb, so I think it might not be able to > handle the size of your model. > Correct me if I misunderstood anything. > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
LTR Model size
what is the way to configure the model size for LTR? We have about 3MB model and Solr is not holding this model as a ManagedResource. How can this be configured? Thanks, Roopa
LTR not able to upload org.apache.solr.ltr.model.MultipleAdditiveTreesModel
Trying to upload a simple MultipleAdditiveTreesModel, however I am getting an error "msg":"org.apache.solr.ltr.model.ModelException: Model type does not exist org.apache.solr.ltr.model.MultipleAdditiveTreesModel" Root cause seems to be a syntax error in the model file? I did copy this from the example https://lucene.apache.org/solr/guide/6_6/learning-to-rank.html#LearningToRank-Examples Made sure all the values have surrounded by quotes Made sure features specified in the model are in the features file Did anyone else face this? what was the resolution? Thanks, Roopa -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
LTR not picking up modified features
Hi - There was an error in one of the feature definition in Solr LTR features.json file and I modified and uploaded it to Solr. I can see that the definition change is uploaded correctly using the feature store url such as http://servername/solr/techproducts/schema/feature-store/myFeatureStore I checked the _schema_feature-store.json file and I see that the change is present. However, during run time it is picking the old feature definition. Are the old features cached anywhere, any step I need to do to resolve this? Thanks! Roopa -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Turn on/off query based on a url parameter
Thanks, I got it working as below, features is true or false based on the efi parameter is_var. (Field and value should be substituted with correct names) { "store": "featurestore", "name": "isfeaturematch", "class": "org.apache.solr.ltr.feature.SolrFeature", "params": { "q": "{!func}and(${is_var:false}, query({!v='field:value'}))" } } On Thu, Feb 22, 2018 at 2:46 PM, Phil Scadden wrote: > I always filter solr request via a proxy (so solr itself is not exposed > directly to the web). In that proxy, the query parameters can be broken > down and filtered as desired (I examine authorities granted to a session to > control even which indexes are being searched) before passing the modified > url to solr. The coding of the proxy obviously depends on your application > environment. We use java and Spring. > > -Original Message- > From: Roopa Rao [mailto:roop...@gmail.com] > Sent: Friday, 23 February 2018 8:04 a.m. > To: solr-user@lucene.apache.org > Subject: Turn on/off query based on a url parameter > > Hi, > > I want to enable or disable a SolrFeature in LTR based on efi parameter. > > In simple the query should be executed only if a parameter is true. > > Any examples or suggestion on how to accomplish this? > > Functions queries examples are are using fields to give a value to. In my > case I want to execute the query only if a url parameter is true > > Thanks, > Roopa > Notice: This email and any attachments are confidential and may not be > used, published or redistributed without the prior written consent of the > Institute of Geological and Nuclear Sciences Limited (GNS Science). If > received in error please destroy and immediately notify GNS Science. Do not > copy or disclose the contents. >
Turn on/off query based on a url parameter
Hi, I want to enable or disable a SolrFeature in LTR based on efi parameter. In simple the query should be executed only if a parameter is true. Any examples or suggestion on how to accomplish this? Functions queries examples are are using fields to give a value to. In my case I want to execute the query only if a url parameter is true Thanks, Roopa
Using dynamic synonyms file
Hi, Is it possible to specify the synonyms file as a variable, set a default synonym file and passing the file name from the request? If so, is there an example of this? Such as, Thanks, Roopa
Re: Using Synonyms as a feature with LTR
I see okay, thank you. On Wed, Feb 14, 2018 at 10:34 AM, Alessandro Benedetti wrote: > I see, > According to what I know it is not possible to run for the same field > different query time analysis. > > Not sure if anyone was working on that. > > Regards > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: Using Synonyms as a feature with LTR
So, I would end up with ~6 copy fields with ~8 synonym files so that would be about 48 field/synonym combination. Would that be a significant in terms of index size. What would be the best way to measure this? Custom parser: This would take the file name, field to run the analysis on. This field need not be a copy field which holds data, since we can use this is only for getting the analysis. Get the synonyms for the user query as tokens. Create a edismax query based on the query tokens. Return the score This custom parser would be called in LTR as a scalar feature. I am at the stage I can get the synonyms from the analysis chain, however tokens are individual tokens and not phrases. So, I am stuck at how to construct a correct query based on the synonym tokens and positions. Thank you, Roopa On Wed, Feb 14, 2018 at 10:12 AM, Roopa Rao wrote: > So, I would end up with ~6 copy fields with ~8 synonym files so that would > be about 48 field/synonym combination. Would that be a significant in terms > of index size. I guess that depends on the thesaurus size, what would be > the best way to measure this? > > Custom parser: > This would take the file name, field to run the analysis on. This field > need not be a copy field which holds data, since we can use this is only > for getting the analysis. > Get the synonyms for the user query as tokens. > Create a edismax query based on the query tokens. > Return the score > > This custom parser would be called in LTR as a scalar feature. > > I am at the stage I can get the synonyms from the analysis chain, however > tokens are individual tokens and not phrases. So, I am stuck at how to > construct a correct query based on the synonym tokens and positions. > > Thank you, > Roopa > > > > On Wed, Feb 14, 2018 at 5:23 AM, Alessandro Benedetti < > a.benede...@sease.io> wrote: > >> "I can go with the "title" field and have that include the synonyms in >> analysis. Only problem is that the number of fields and number of synonyms >> files are quite a lot (~ 8 synonyms files) due to different weightage and >> type of expansion (exact vs partial) based on these. Hence going with this >> approach would mean creating more fields for all these synonyms >> (synonyms.txt) >> >> So, I am looking to build a custom parser for which I could supply the >> file >> and the field and that would expand the synonyms and return a score. " >> >> Having a binary or scalar feature is completely up to you and the way you >> configure the Solr feature. >> If you have 8 (copy?)fields with same content but different expansion, >> that >> is still ok. >> You can have 8 features, one per type of expansion. >> LTR will take care of the weight to be assigned to those features. >> >> "So, I am looking to build a custom parser for which I could supply the >> file >> and the field and that would expand the synonyms and return a score. "" >> I don't get this , can you elaborate ? >> >> Regards >> >> >> >> - >> --- >> Alessandro Benedetti >> Search Consultant, R&D Software Engineer, Director >> Sease Ltd. - www.sease.io >> -- >> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >> > >
Re: Using Synonyms as a feature with LTR
So, I would end up with ~6 copy fields with ~8 synonym files so that would be about 48 field/synonym combination. Would that be a significant in terms of index size. I guess that depends on the thesaurus size, what would be the best way to measure this? Custom parser: This would take the file name, field to run the analysis on. This field need not be a copy field which holds data, since we can use this is only for getting the analysis. Get the synonyms for the user query as tokens. Create a edismax query based on the query tokens. Return the score This custom parser would be called in LTR as a scalar feature. I am at the stage I can get the synonyms from the analysis chain, however tokens are individual tokens and not phrases. So, I am stuck at how to construct a correct query based on the synonym tokens and positions. Thank you, Roopa On Wed, Feb 14, 2018 at 5:23 AM, Alessandro Benedetti wrote: > "I can go with the "title" field and have that include the synonyms in > analysis. Only problem is that the number of fields and number of synonyms > files are quite a lot (~ 8 synonyms files) due to different weightage and > type of expansion (exact vs partial) based on these. Hence going with this > approach would mean creating more fields for all these synonyms > (synonyms.txt) > > So, I am looking to build a custom parser for which I could supply the file > and the field and that would expand the synonyms and return a score. " > > Having a binary or scalar feature is completely up to you and the way you > configure the Solr feature. > If you have 8 (copy?)fields with same content but different expansion, that > is still ok. > You can have 8 features, one per type of expansion. > LTR will take care of the weight to be assigned to those features. > > "So, I am looking to build a custom parser for which I could supply the > file > and the field and that would expand the synonyms and return a score. "" > I don't get this , can you elaborate ? > > Regards > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: Using Synonyms as a feature with LTR
Thank you, Alessandro, I was trying these options before replying. Yes, I am looking to generate a score for a query with synonym expansion (not binary feature) I can go with the "title" field and have that include the synonyms in analysis. Only problem is that the number of fields and number of synonyms files are quite a lot (~ 8 synonyms files) due to different weightage and type of expansion (exact vs partial) based on these. Hence going with this approach would mean creating more fields for all these synonyms (synonyms.txt) So, I am looking to build a custom parser for which I could supply the file and the field and that would expand the synonyms and return a score. Thanks, Roopa On Mon, Feb 12, 2018 at 6:23 AM, Alessandro Benedetti wrote: > In the end a feature will just be a numerical value. > How do you plan to use synonyms in a field to generate a numerical feature > ? > > Are you planning to define a binary feature for a field, in case there is a > match on the synonyms ? > Or a feature which contains a score for a query ( with synonyms expansion) > ? > > I would start from the SolrFeature, let's assume the "title" field has a > field type that includes synonyms ( query time) : > > { > "store" : "featureStore", > "name" : "hasTitleMatch", > "class" : "org.apache.solr.ltr.feature.SolrFeature", > "params" : { > "fq": [ "{!field f=title}${query}" ] > } > > Query time analysis will be applied and synonyms expanded. > So the feature will have a value , which is the score returned for the > query > and the document ( under scoring) . > You can play with that and design the feature that best fit your idea. > > Regards > > > > > > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Using Synonyms as a feature with LTR
I am trying to use synonyms expansion as a feature in LTR Any input on a feature using synonym expansion providing a field and the synonym file would be helpful. Thanks, Roopa -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Learning to Rank (LTR) with grouping
Hi Diego, I tried collapsing, unfortunately we are using a third party tool for Query processing, that does not support collapsing. Hence I am unable to go that route. Thank you, Roopa On Thu, Jan 11, 2018 at 2:59 PM, Diego Ceccarelli (BLOOMBERG/ LONDON) < dceccarel...@bloomberg.net> wrote: > Roopa, did you try collapsing instead of grouping? it would work with ltr.. > > From: solr-user@lucene.apache.org At: 01/11/18 16:48:26To: > solr-user@lucene.apache.org > Subject: Re: Learning to Rank (LTR) with grouping > > Solution that I implemented currently is: > Since we have a web application which takes the solr results and display in > the UI and I need LTR enabled for only one of the group, > I am executing two parallel queries to Solr from web app. > 1st query Get grouped results without LTR > 2nd query Get results with no grouping + LTR > Merge 1) & 2) in web app. > Doing a performance test now. Since there are only 10 results for these > queries and the queries are executed in parallel, I don't foresee any > problems. > > Thanks! > Roopa > > > On Thu, Jan 4, 2018 at 10:24 AM, Roopa Rao wrote: > > > Hi, > > > > Any guidance on this would be helpful. > > > > Thank you, > > Roopa > > > > On Tue, Dec 19, 2017 at 8:47 PM, Roopa Rao wrote: > > > >> Hi Diego, > >> > >> Thank you for looking into it further. > >> We recently ported over to 6.6 version solely to use LTR feature as it > is > >> critical for us. > >> > >> Since its not working with grouping in the base version, I am trying to > >> evaluate if there is any alternative way to make it work in 6.6 versus > >> upgrading to 7.0. > >> > >> Any guidance you could provide on what can be done to use 6.6 with > >> grouping + LTR or any alternatives would be helpful. Do I read your > >> response as needing to go to 7.0 when you say upstream? > >> > >> Thank you, > >> Roopa > >> > >> > >> On Tue, Dec 19, 2017 at 1:37 PM, Diego Ceccarelli < > >> diego.ceccare...@gmail.com> wrote: > >> > >>> Hi Roopa, unfortunately I can't port the patch to the branch_6_6, but > >>> soon I'll update to upstream. Sorry about that. > >>> > >>> On Mon, Dec 18, 2017 at 7:52 PM, Roopa Rao wrote: > >>> > Hi - > >>> > > >>> > I merged the code from the bloomberg master-solr-8776 branch to > >>> branch_6_6 > >>> > on Solr. > >>> > > >>> > When I tried to compile the solr source code, I am getting multiple > >>> > compilation errors (Attached), which seems to be due to the fact that > >>> the > >>> > branch master-solr-8776 may not be compatible with branch_6_6. > >>> > > >>> > Could you please provide your input if master-solr-8776 is compatible > >>> with > >>> > branch_6_6? > >>> > > >>> > If this is not the case then how to proceed with using fix in > >>> > master-solr-8776 with branch_6_6 can a new patch be created for this? > >>> > > >>> > Thank you, > >>> > Roopa > >>> > > >>> > On Mon, Dec 11, 2017 at 9:54 AM, Roopa Rao > wrote: > >>> >> > >>> >> Hi Diego, > >>> >> > >>> >> Thank you, > >>> >> > >>> >> I am interested in reranking the documents inside one of the groups. > >>> >> > >>> >> I will try the options you mentioned here. > >>> >> > >>> >> Thank you, > >>> >> Roopa > >>> >> > >>> >> On Mon, Dec 11, 2017 at 6:57 AM, Diego Ceccarelli (BLOOMBERG/ > LONDON) > >>> >> wrote: > >>> >>> > >>> >>> Hi Roopa, > >>> >>> > >>> >>> If you look at the diff: > >>> >>> > >>> >>> https://github.com/apache/lucene-solr/pull/162/files > >>> >>> > >>> >>> I didn't change much in SolrIndexSearcher, you can try to skip the > >>> file > >>> >>> when applying the patch and redo the changes after. > >>> >>> > >>> >>> Alternatively, the feature branch is available here: > >>> >>> > >>> >>> https://github.com/bloomberg/lucene-solr/commits/master-solr-
Re: Learning to Rank (LTR) with grouping
Solution that I implemented currently is: Since we have a web application which takes the solr results and display in the UI and I need LTR enabled for only one of the group, I am executing two parallel queries to Solr from web app. 1st query Get grouped results without LTR 2nd query Get results with no grouping + LTR Merge 1) & 2) in web app. Doing a performance test now. Since there are only 10 results for these queries and the queries are executed in parallel, I don't foresee any problems. Thanks! Roopa On Thu, Jan 4, 2018 at 10:24 AM, Roopa Rao wrote: > Hi, > > Any guidance on this would be helpful. > > Thank you, > Roopa > > On Tue, Dec 19, 2017 at 8:47 PM, Roopa Rao wrote: > >> Hi Diego, >> >> Thank you for looking into it further. >> We recently ported over to 6.6 version solely to use LTR feature as it is >> critical for us. >> >> Since its not working with grouping in the base version, I am trying to >> evaluate if there is any alternative way to make it work in 6.6 versus >> upgrading to 7.0. >> >> Any guidance you could provide on what can be done to use 6.6 with >> grouping + LTR or any alternatives would be helpful. Do I read your >> response as needing to go to 7.0 when you say upstream? >> >> Thank you, >> Roopa >> >> >> On Tue, Dec 19, 2017 at 1:37 PM, Diego Ceccarelli < >> diego.ceccare...@gmail.com> wrote: >> >>> Hi Roopa, unfortunately I can't port the patch to the branch_6_6, but >>> soon I'll update to upstream. Sorry about that. >>> >>> On Mon, Dec 18, 2017 at 7:52 PM, Roopa Rao wrote: >>> > Hi - >>> > >>> > I merged the code from the bloomberg master-solr-8776 branch to >>> branch_6_6 >>> > on Solr. >>> > >>> > When I tried to compile the solr source code, I am getting multiple >>> > compilation errors (Attached), which seems to be due to the fact that >>> the >>> > branch master-solr-8776 may not be compatible with branch_6_6. >>> > >>> > Could you please provide your input if master-solr-8776 is compatible >>> with >>> > branch_6_6? >>> > >>> > If this is not the case then how to proceed with using fix in >>> > master-solr-8776 with branch_6_6 can a new patch be created for this? >>> > >>> > Thank you, >>> > Roopa >>> > >>> > On Mon, Dec 11, 2017 at 9:54 AM, Roopa Rao wrote: >>> >> >>> >> Hi Diego, >>> >> >>> >> Thank you, >>> >> >>> >> I am interested in reranking the documents inside one of the groups. >>> >> >>> >> I will try the options you mentioned here. >>> >> >>> >> Thank you, >>> >> Roopa >>> >> >>> >> On Mon, Dec 11, 2017 at 6:57 AM, Diego Ceccarelli (BLOOMBERG/ LONDON) >>> >> wrote: >>> >>> >>> >>> Hi Roopa, >>> >>> >>> >>> If you look at the diff: >>> >>> >>> >>> https://github.com/apache/lucene-solr/pull/162/files >>> >>> >>> >>> I didn't change much in SolrIndexSearcher, you can try to skip the >>> file >>> >>> when applying the patch and redo the changes after. >>> >>> >>> >>> Alternatively, the feature branch is available here: >>> >>> >>> >>> https://github.com/bloomberg/lucene-solr/commits/master-solr-8776 >>> >>> >>> >>> you could try to merge with that or cheery-pick my changes. >>> >>> >>> >>> Are you interested in reranking the groups or also in reranking the >>> >>> documents inside each group? >>> >>> >>> >>> Cheers, >>> >>> Diego >>> >>> >>> >>> >>> >>> From: solr-user@lucene.apache.org At: 12/09/17 19:07:25To: >>> >>> solr-user@lucene.apache.org >>> >>> Subject: Re: Learning to Rank (LTR) with grouping >>> >>> >>> >>> Hi I tried to apply this JIRA SOLR-8776 as a patch as this feature is >>> >>> critical. >>> >>> >>> >>> Here are the steps I took on my mac: >>> >>> >>> >>> On branch branch_6_5 >>> >>> >>> >>> Your branch is up-to-date with
Re: Learning to Rank (LTR) with grouping
Hi, Any guidance on this would be helpful. Thank you, Roopa On Tue, Dec 19, 2017 at 8:47 PM, Roopa Rao wrote: > Hi Diego, > > Thank you for looking into it further. > We recently ported over to 6.6 version solely to use LTR feature as it is > critical for us. > > Since its not working with grouping in the base version, I am trying to > evaluate if there is any alternative way to make it work in 6.6 versus > upgrading to 7.0. > > Any guidance you could provide on what can be done to use 6.6 with > grouping + LTR or any alternatives would be helpful. Do I read your > response as needing to go to 7.0 when you say upstream? > > Thank you, > Roopa > > > On Tue, Dec 19, 2017 at 1:37 PM, Diego Ceccarelli < > diego.ceccare...@gmail.com> wrote: > >> Hi Roopa, unfortunately I can't port the patch to the branch_6_6, but >> soon I'll update to upstream. Sorry about that. >> >> On Mon, Dec 18, 2017 at 7:52 PM, Roopa Rao wrote: >> > Hi - >> > >> > I merged the code from the bloomberg master-solr-8776 branch to >> branch_6_6 >> > on Solr. >> > >> > When I tried to compile the solr source code, I am getting multiple >> > compilation errors (Attached), which seems to be due to the fact that >> the >> > branch master-solr-8776 may not be compatible with branch_6_6. >> > >> > Could you please provide your input if master-solr-8776 is compatible >> with >> > branch_6_6? >> > >> > If this is not the case then how to proceed with using fix in >> > master-solr-8776 with branch_6_6 can a new patch be created for this? >> > >> > Thank you, >> > Roopa >> > >> > On Mon, Dec 11, 2017 at 9:54 AM, Roopa Rao wrote: >> >> >> >> Hi Diego, >> >> >> >> Thank you, >> >> >> >> I am interested in reranking the documents inside one of the groups. >> >> >> >> I will try the options you mentioned here. >> >> >> >> Thank you, >> >> Roopa >> >> >> >> On Mon, Dec 11, 2017 at 6:57 AM, Diego Ceccarelli (BLOOMBERG/ LONDON) >> >> wrote: >> >>> >> >>> Hi Roopa, >> >>> >> >>> If you look at the diff: >> >>> >> >>> https://github.com/apache/lucene-solr/pull/162/files >> >>> >> >>> I didn't change much in SolrIndexSearcher, you can try to skip the >> file >> >>> when applying the patch and redo the changes after. >> >>> >> >>> Alternatively, the feature branch is available here: >> >>> >> >>> https://github.com/bloomberg/lucene-solr/commits/master-solr-8776 >> >>> >> >>> you could try to merge with that or cheery-pick my changes. >> >>> >> >>> Are you interested in reranking the groups or also in reranking the >> >>> documents inside each group? >> >>> >> >>> Cheers, >> >>> Diego >> >>> >> >>> >> >>> From: solr-user@lucene.apache.org At: 12/09/17 19:07:25To: >> >>> solr-user@lucene.apache.org >> >>> Subject: Re: Learning to Rank (LTR) with grouping >> >>> >> >>> Hi I tried to apply this JIRA SOLR-8776 as a patch as this feature is >> >>> critical. >> >>> >> >>> Here are the steps I took on my mac: >> >>> >> >>> On branch branch_6_5 >> >>> >> >>> Your branch is up-to-date with 'origin/branch_6_5' >> >>> >> >>> patch -p1 -i 162.patch --dry-run >> >>> >> >>> >> >>> I am getting Failures for certain Hunks >> >>> >> >>> Example: >> >>> >> >>> patching file >> >>> solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java >> >>> >> >>> Hunk #1 FAILED at 1471. >> >>> >> >>> >> >>> Could you please give your input on how to apply this ticket as a >> patch >> >>> for >> >>> branch_6_5 ? >> >>> >> >>> >> >>> Thank you, >> >>> >> >>> Roopa >> >>> >> >>> On Fri, Dec 8, 2017 at 6:52 PM, Roopa Rao wrote: >> >>> >> >>> > Hi Diego, >> >>> > >> >>> > Thank you, I will look into this and see how I could patch this. >> >>> > >> >>> > Thank you for your quick response, >> >>> > Roopa >> >>> > >> >>> > >> >>> > On Fri, Dec 8, 2017 at 5:44 PM, Diego Ceccarelli < >> >>> > diego.ceccare...@gmail.com> wrote: >> >>> > >> >>> >> Hi Roopa, >> >>> >> >> >>> >> LTR is implemented using RankQuery, and at the moment grouping >> doens't >> >>> >> support RankQuery. >> >>> >> I opened a jira item time ago >> >>> >> (https://issues.apache.org/jira/browse/SOLR-8776) and I would be >> happy >> >>> >> to receive feedback on that. You can find the code here >> >>> >> https://github.com/apache/lucene-solr/pull/162. >> >>> >> >> >>> >> Cheers, >> >>> >> diego >> >>> >> >> >>> >> On Fri, Dec 8, 2017 at 9:15 PM, Roopa Rao >> wrote: >> >>> >> > Hi, >> >>> >> > >> >>> >> > I am using grouping and LTR together and the results are not >> getting >> >>> >> > re-rank as it does without grouping. >> >>> >> > >> >>> >> > I am passing &rq parameter. >> >>> >> > >> >>> >> > Does LTR work with grouping on? >> >>> >> > Solr version 6.5 >> >>> >> > >> >>> >> > Thank you, >> >>> >> > Roopa >> >>> >> >> >>> > >> >>> > >> >>> >> >>> >> >> >> > >> > >
Re: Learning to Rank (LTR) with grouping
Hi Diego, Thank you for looking into it further. We recently ported over to 6.6 version solely to use LTR feature as it is critical for us. Since its not working with grouping in the base version, I am trying to evaluate if there is any alternative way to make it work in 6.6 versus upgrading to 7.0. Any guidance you could provide on what can be done to use 6.6 with grouping + LTR or any alternatives would be helpful. Do I read your response as needing to go to 7.0 when you say upstream? Thank you, Roopa On Tue, Dec 19, 2017 at 1:37 PM, Diego Ceccarelli < diego.ceccare...@gmail.com> wrote: > Hi Roopa, unfortunately I can't port the patch to the branch_6_6, but > soon I'll update to upstream. Sorry about that. > > On Mon, Dec 18, 2017 at 7:52 PM, Roopa Rao wrote: > > Hi - > > > > I merged the code from the bloomberg master-solr-8776 branch to > branch_6_6 > > on Solr. > > > > When I tried to compile the solr source code, I am getting multiple > > compilation errors (Attached), which seems to be due to the fact that the > > branch master-solr-8776 may not be compatible with branch_6_6. > > > > Could you please provide your input if master-solr-8776 is compatible > with > > branch_6_6? > > > > If this is not the case then how to proceed with using fix in > > master-solr-8776 with branch_6_6 can a new patch be created for this? > > > > Thank you, > > Roopa > > > > On Mon, Dec 11, 2017 at 9:54 AM, Roopa Rao wrote: > >> > >> Hi Diego, > >> > >> Thank you, > >> > >> I am interested in reranking the documents inside one of the groups. > >> > >> I will try the options you mentioned here. > >> > >> Thank you, > >> Roopa > >> > >> On Mon, Dec 11, 2017 at 6:57 AM, Diego Ceccarelli (BLOOMBERG/ LONDON) > >> wrote: > >>> > >>> Hi Roopa, > >>> > >>> If you look at the diff: > >>> > >>> https://github.com/apache/lucene-solr/pull/162/files > >>> > >>> I didn't change much in SolrIndexSearcher, you can try to skip the file > >>> when applying the patch and redo the changes after. > >>> > >>> Alternatively, the feature branch is available here: > >>> > >>> https://github.com/bloomberg/lucene-solr/commits/master-solr-8776 > >>> > >>> you could try to merge with that or cheery-pick my changes. > >>> > >>> Are you interested in reranking the groups or also in reranking the > >>> documents inside each group? > >>> > >>> Cheers, > >>> Diego > >>> > >>> > >>> From: solr-user@lucene.apache.org At: 12/09/17 19:07:25To: > >>> solr-user@lucene.apache.org > >>> Subject: Re: Learning to Rank (LTR) with grouping > >>> > >>> Hi I tried to apply this JIRA SOLR-8776 as a patch as this feature is > >>> critical. > >>> > >>> Here are the steps I took on my mac: > >>> > >>> On branch branch_6_5 > >>> > >>> Your branch is up-to-date with 'origin/branch_6_5' > >>> > >>> patch -p1 -i 162.patch --dry-run > >>> > >>> > >>> I am getting Failures for certain Hunks > >>> > >>> Example: > >>> > >>> patching file > >>> solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java > >>> > >>> Hunk #1 FAILED at 1471. > >>> > >>> > >>> Could you please give your input on how to apply this ticket as a patch > >>> for > >>> branch_6_5 ? > >>> > >>> > >>> Thank you, > >>> > >>> Roopa > >>> > >>> On Fri, Dec 8, 2017 at 6:52 PM, Roopa Rao wrote: > >>> > >>> > Hi Diego, > >>> > > >>> > Thank you, I will look into this and see how I could patch this. > >>> > > >>> > Thank you for your quick response, > >>> > Roopa > >>> > > >>> > > >>> > On Fri, Dec 8, 2017 at 5:44 PM, Diego Ceccarelli < > >>> > diego.ceccare...@gmail.com> wrote: > >>> > > >>> >> Hi Roopa, > >>> >> > >>> >> LTR is implemented using RankQuery, and at the moment grouping > doens't > >>> >> support RankQuery. > >>> >> I opened a jira item time ago > >>> >> (https://issues.apache.org/jira/browse/SOLR-8776) and I would be > happy > >>> >> to receive feedback on that. You can find the code here > >>> >> https://github.com/apache/lucene-solr/pull/162. > >>> >> > >>> >> Cheers, > >>> >> diego > >>> >> > >>> >> On Fri, Dec 8, 2017 at 9:15 PM, Roopa Rao > wrote: > >>> >> > Hi, > >>> >> > > >>> >> > I am using grouping and LTR together and the results are not > getting > >>> >> > re-rank as it does without grouping. > >>> >> > > >>> >> > I am passing &rq parameter. > >>> >> > > >>> >> > Does LTR work with grouping on? > >>> >> > Solr version 6.5 > >>> >> > > >>> >> > Thank you, > >>> >> > Roopa > >>> >> > >>> > > >>> > > >>> > >>> > >> > > >
Re: Learning to Rank (LTR) with grouping
Hi - I merged the code from the bloomberg master-solr-8776 branch to branch_6_6 on Solr. When I tried to compile the solr source code, I am getting multiple compilation errors (Attached), which seems to be due to the fact that the branch master-solr-8776 may not be compatible with branch_6_6. Could you please provide your input if master-solr-8776 is compatible with branch_6_6? If this is not the case then how to proceed with using fix in master-solr-8776 with branch_6_6 can a new patch be created for this? Thank you, Roopa On Mon, Dec 11, 2017 at 9:54 AM, Roopa Rao wrote: > Hi Diego, > > Thank you, > > I am interested in reranking the documents inside one of the groups. > > I will try the options you mentioned here. > > Thank you, > Roopa > > On Mon, Dec 11, 2017 at 6:57 AM, Diego Ceccarelli (BLOOMBERG/ LONDON) < > dceccarel...@bloomberg.net> wrote: > >> Hi Roopa, >> >> If you look at the diff: >> >> https://github.com/apache/lucene-solr/pull/162/files >> >> I didn't change much in SolrIndexSearcher, you can try to skip the file >> when applying the patch and redo the changes after. >> >> Alternatively, the feature branch is available here: >> >> https://github.com/bloomberg/lucene-solr/commits/master-solr-8776 >> >> you could try to merge with that or cheery-pick my changes. >> >> Are you interested in reranking the groups or also in reranking the >> documents inside each group? >> >> Cheers, >> Diego >> >> >> From: solr-user@lucene.apache.org At: 12/09/17 19:07:25To: >> solr-user@lucene.apache.org >> Subject: Re: Learning to Rank (LTR) with grouping >> >> Hi I tried to apply this JIRA SOLR-8776 as a patch as this feature is >> critical. >> >> Here are the steps I took on my mac: >> >> On branch branch_6_5 >> >> Your branch is up-to-date with 'origin/branch_6_5' >> >> patch -p1 -i 162.patch --dry-run >> >> >> I am getting Failures for certain Hunks >> >> Example: >> >> patching file >> solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java >> >> Hunk #1 FAILED at 1471. >> >> >> Could you please give your input on how to apply this ticket as a patch >> for >> branch_6_5 ? >> >> >> Thank you, >> >> Roopa >> >> On Fri, Dec 8, 2017 at 6:52 PM, Roopa Rao wrote: >> >> > Hi Diego, >> > >> > Thank you, I will look into this and see how I could patch this. >> > >> > Thank you for your quick response, >> > Roopa >> > >> > >> > On Fri, Dec 8, 2017 at 5:44 PM, Diego Ceccarelli < >> > diego.ceccare...@gmail.com> wrote: >> > >> >> Hi Roopa, >> >> >> >> LTR is implemented using RankQuery, and at the moment grouping doens't >> >> support RankQuery. >> >> I opened a jira item time ago >> >> (https://issues.apache.org/jira/browse/SOLR-8776) and I would be happy >> >> to receive feedback on that. You can find the code here >> >> https://github.com/apache/lucene-solr/pull/162. >> >> >> >> Cheers, >> >> diego >> >> >> >> On Fri, Dec 8, 2017 at 9:15 PM, Roopa Rao wrote: >> >> > Hi, >> >> > >> >> > I am using grouping and LTR together and the results are not getting >> >> > re-rank as it does without grouping. >> >> > >> >> > I am passing &rq parameter. >> >> > >> >> > Does LTR work with grouping on? >> >> > Solr version 6.5 >> >> > >> >> > Thank you, >> >> > Roopa >> >> >> > >> > >> >> >> > [javac] Compiling 1089 source files to /Users/rrao/git/lucene-solr-all/lucene-solr/solr/build/solr-core/classes/java [javac] /Users/rrao/git/lucene-solr-all/lucene-solr/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java:67: error: cannot find symbol [javac] import org.apache.solr.core.SolrInfoBean; [javac]^ [javac] symbol: class SolrInfoBean [javac] location: package org.apache.solr.core [javac] /Users/rrao/git/lucene-solr-all/lucene-solr/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java:90: error: cannot find symbol [javac] public class SolrIndexSearcher extends IndexSearcher implements Closeable, SolrInfoBean, SolrMetricProducer { [javac]
Re: Learning to Rank (LTR) with grouping
Hi Diego, Thank you, I am interested in reranking the documents inside one of the groups. I will try the options you mentioned here. Thank you, Roopa On Mon, Dec 11, 2017 at 6:57 AM, Diego Ceccarelli (BLOOMBERG/ LONDON) < dceccarel...@bloomberg.net> wrote: > Hi Roopa, > > If you look at the diff: > > https://github.com/apache/lucene-solr/pull/162/files > > I didn't change much in SolrIndexSearcher, you can try to skip the file > when applying the patch and redo the changes after. > > Alternatively, the feature branch is available here: > > https://github.com/bloomberg/lucene-solr/commits/master-solr-8776 > > you could try to merge with that or cheery-pick my changes. > > Are you interested in reranking the groups or also in reranking the > documents inside each group? > > Cheers, > Diego > > > From: solr-user@lucene.apache.org At: 12/09/17 19:07:25To: > solr-user@lucene.apache.org > Subject: Re: Learning to Rank (LTR) with grouping > > Hi I tried to apply this JIRA SOLR-8776 as a patch as this feature is > critical. > > Here are the steps I took on my mac: > > On branch branch_6_5 > > Your branch is up-to-date with 'origin/branch_6_5' > > patch -p1 -i 162.patch --dry-run > > > I am getting Failures for certain Hunks > > Example: > > patching file > solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java > > Hunk #1 FAILED at 1471. > > > Could you please give your input on how to apply this ticket as a patch for > branch_6_5 ? > > > Thank you, > > Roopa > > On Fri, Dec 8, 2017 at 6:52 PM, Roopa Rao wrote: > > > Hi Diego, > > > > Thank you, I will look into this and see how I could patch this. > > > > Thank you for your quick response, > > Roopa > > > > > > On Fri, Dec 8, 2017 at 5:44 PM, Diego Ceccarelli < > > diego.ceccare...@gmail.com> wrote: > > > >> Hi Roopa, > >> > >> LTR is implemented using RankQuery, and at the moment grouping doens't > >> support RankQuery. > >> I opened a jira item time ago > >> (https://issues.apache.org/jira/browse/SOLR-8776) and I would be happy > >> to receive feedback on that. You can find the code here > >> https://github.com/apache/lucene-solr/pull/162. > >> > >> Cheers, > >> diego > >> > >> On Fri, Dec 8, 2017 at 9:15 PM, Roopa Rao wrote: > >> > Hi, > >> > > >> > I am using grouping and LTR together and the results are not getting > >> > re-rank as it does without grouping. > >> > > >> > I am passing &rq parameter. > >> > > >> > Does LTR work with grouping on? > >> > Solr version 6.5 > >> > > >> > Thank you, > >> > Roopa > >> > > > > > > >
Re: Learning to Rank (LTR) with grouping
Hi I tried to apply this JIRA SOLR-8776 as a patch as this feature is critical. Here are the steps I took on my mac: On branch branch_6_5 Your branch is up-to-date with 'origin/branch_6_5' patch -p1 -i 162.patch --dry-run I am getting Failures for certain Hunks Example: patching file solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java Hunk #1 FAILED at 1471. Could you please give your input on how to apply this ticket as a patch for branch_6_5 ? Thank you, Roopa On Fri, Dec 8, 2017 at 6:52 PM, Roopa Rao wrote: > Hi Diego, > > Thank you, I will look into this and see how I could patch this. > > Thank you for your quick response, > Roopa > > > On Fri, Dec 8, 2017 at 5:44 PM, Diego Ceccarelli < > diego.ceccare...@gmail.com> wrote: > >> Hi Roopa, >> >> LTR is implemented using RankQuery, and at the moment grouping doens't >> support RankQuery. >> I opened a jira item time ago >> (https://issues.apache.org/jira/browse/SOLR-8776) and I would be happy >> to receive feedback on that. You can find the code here >> https://github.com/apache/lucene-solr/pull/162. >> >> Cheers, >> diego >> >> On Fri, Dec 8, 2017 at 9:15 PM, Roopa Rao wrote: >> > Hi, >> > >> > I am using grouping and LTR together and the results are not getting >> > re-rank as it does without grouping. >> > >> > I am passing &rq parameter. >> > >> > Does LTR work with grouping on? >> > Solr version 6.5 >> > >> > Thank you, >> > Roopa >> > >
Re: Learning to Rank (LTR) with grouping
Hi Diego, Thank you, I will look into this and see how I could patch this. Thank you for your quick response, Roopa On Fri, Dec 8, 2017 at 5:44 PM, Diego Ceccarelli wrote: > Hi Roopa, > > LTR is implemented using RankQuery, and at the moment grouping doens't > support RankQuery. > I opened a jira item time ago > (https://issues.apache.org/jira/browse/SOLR-8776) and I would be happy > to receive feedback on that. You can find the code here > https://github.com/apache/lucene-solr/pull/162. > > Cheers, > diego > > On Fri, Dec 8, 2017 at 9:15 PM, Roopa Rao wrote: > > Hi, > > > > I am using grouping and LTR together and the results are not getting > > re-rank as it does without grouping. > > > > I am passing &rq parameter. > > > > Does LTR work with grouping on? > > Solr version 6.5 > > > > Thank you, > > Roopa >
Learning to Rank (LTR) with grouping
Hi, I am using grouping and LTR together and the results are not getting re-rank as it does without grouping. I am passing &rq parameter. Does LTR work with grouping on? Solr version 6.5 Thank you, Roopa