AddictedCS opened a new issue, #933:
URL: https://github.com/apache/lucenenet/issues/933
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the bug
After upgrading to .NET 8 I've noticed a dramatic performance decrease. I've
localized the issue to `SearcherTaxonomyManager.maybeRefresh` method. I don't
know the internals of this method though, so any help will be of an immense
value.
Below if a Benchmark that reproduces the issue. The benchmark is simulating
document updates, using `maybeRefresh` to refresh the indexer. This code has
been running absolutely fine on net5, net6, net7, but is now completely
unusable on .NET 8.
```csharp
namespace Emy.Benchmark;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Documents;
using Lucene.Net.Facet;
using Lucene.Net.Facet.Taxonomy;
using Lucene.Net.Facet.Taxonomy.Directory;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Store;
using Lucene.Net.Util;
using Directory = System.IO.Directory;
[SimpleJob(RuntimeMoniker.Net70, baseline: true)]
[SimpleJob(RuntimeMoniker.Net80)]
[RPlotExporter]
[GcServer(true)]
public class LuceneBenchmark
{
private DirectoryTaxonomyWriter taxonomyWriter;
private IndexWriter indexWriter;
private Document[] documents;
private FacetsConfig facetsConfig;
private SearcherTaxonomyManager searcherManager;
[GlobalSetup]
public void Setup()
{
if (Directory.Exists("test_index"))
{
Directory.Delete("test_index", true);
}
if (Directory.Exists("test_facets"))
{
Directory.Delete("test_facets", true);
}
var analyzer = new StandardAnalyzer(LuceneVersion.LUCENE_48);
var luceneDirectory = new MMapDirectory("test_index");
indexWriter = new IndexWriter(luceneDirectory, new
IndexWriterConfig(LuceneVersion.LUCENE_48, analyzer));
taxonomyWriter = new DirectoryTaxonomyWriter(new
MMapDirectory("test_facets"));
facetsConfig = new FacetsConfig();
searcherManager = new SearcherTaxonomyManager(indexWriter, true, new
SearcherFactory(), taxonomyWriter);
facetsConfig.SetRequireDimCount("track_id", true);
documents = new Document[N];
for (int i = 0; i < N; i++)
{
var facet = GenerateRandomString(5);
documents[i] = new Document
{
new StringField("_id", i.ToString(), Field.Store.YES),
new TextField("content", GenerateRandomString(10),
Field.Store.YES),
new FacetField("track_id", facet)
};
}
}
[Params(100)]
public int N;
[Benchmark]
public int IndexDocumentsBenchmark() => IndexDocuments();
private int IndexDocuments()
{
for (int i = 0; i < documents.Length; ++i)
{
var taxonomyDocument = facetsConfig.Build(taxonomyWriter,
documents[i]);
indexWriter.UpdateDocument(new Term("_id", i.ToString()),
taxonomyDocument);
searcherManager.MaybeRefresh(); // maybe refresh causing
dramatic performance drop on .NET 8.0
}
return documents.Length;
}
private static string GenerateRandomString(int length)
{
// more spaces added on purpose
const string chars =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 ";
char[] stringChars = new char[length];
for (int i = 0; i < length; i++)
{
stringChars[i] = chars[Random.Shared.Next(chars.Length)];
}
return new string(stringChars);
}
}
```
The results are attached below.
Attaching project details, for those who want to reproduce it locally:
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net7.0;net8.0</TargetFrameworks>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.13.12" />
<PackageReference Include="Lucene.Net" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Facet"
Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.QueryParser"
Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Analysis.Common"
Version="4.8.0-beta00016" />
</ItemGroup>
</Project>
```
Any help on this will be greatly appreciated, as I'm right now blocked from
migrating toward .NET 8 because of this issue.
I've noticed somehow similar issue #929, and the fix (using ServerGc) didn't
help.
Thanks!
### Expected Behavior
Better performance.
### Steps To Reproduce
Described in the attached benchmark.
### Exceptions (if any)
No exceptions
### Lucene.NET Version
4.8.0-beta00016
### .NET Version
.NET8
### Operating System
MacOS ARM
### Anything else?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
