Hi:
When I use LTR, there are 72 features, 500 models and 200 million data. During
use, LTR execution was found to be extremely slow, with each query above 5s.
In LTRScoringQuery. ModelWeight scorer method in class,
for (final Feature.FeatureWeight featureWeight : extractedFeatureWeights) {
final Feature.FeatureWeight.FeatureScorer scorer =
featureWeight.scorer(context);
if (scorer != null) {
featureScorers.add(scorer);
}
}
Modified to a thread pool call. Please help to see if this can lead to
performance optimization.Thank you very much
List<Future<Feature.FeatureWeight.FeatureScorer> > futures = new
ArrayList<>(extractedFeatureWeights.length);
for (final Feature.FeatureWeight featureWeight : extractedFeatureWeights) {
MyCreateFeatureWeightCallable callable = new
MyCreateFeatureWeightCallable(featureWeight, context);
RunnableFuture<Feature.FeatureWeight.FeatureScorer> runnableFuture = new
FutureTask<>(callable);
ltrThreadMgr.execute(runnableFuture);//releases semaphore when done
futures.add(runnableFuture);
}
for (final Future<Feature.FeatureWeight.FeatureScorer> future : futures) {
featureScorers.add(future.get()); // future.get() will block if the job is
still running
}
public class MyCreateFeatureWeightCallable implements
Callable<Feature.FeatureWeight.FeatureScorer> {
private Feature.FeatureWeight featureWeight;
private LeafReaderContext context;
public MyCreateFeatureWeightCallable(Feature.FeatureWeight featureWeight,
LeafReaderContext context) {
this.featureWeight = featureWeight;
this.context = context;
}
@Override
public Feature.FeatureWeight.FeatureScorer call() throws Exception {
return featureWeight.scorer(context);
}
}
在 2019年6月9日，上午11:14，李世明 <limingni...@live.com<mailto:limingni...@live.com>> 写道：
Hi：
When I use LTR, there are 72 features, 500 models and 200 million data. During
use, LTR execution was found to be extremely slow, with each query above 5s.
Through the analysis found in LTRScoringQuery ModelWeight scorer method of
class 72 when creating FeatureScorer feature value in the use of content of the
slowly, and here is the synchronization of execution.
I tried to change this to multithreaded, which had some effect. The average
query time was 600ms.
Is it too slow for payload or is there room for optimization here?
thank you
