NightOwl888 commented on issue #1013:
URL: https://github.com/apache/lucenenet/issues/1013#issuecomment-2745028174
Given that the crash happens in PriorityQueue (could it be
ValuePriorityQueue?), it seems possible that this is due to running on a
machine with very little stack space, although I would expect there to be a
StackOverflowException not an AccessViolationException. The default value that
we provided in beta 17 is pretty high, 2048 bytes (or 1024 characters). This
may not be optimal for all environments.
This is just the maximum value that is allowed on the stack before falling
back to using array pool on the heap, so it can be lowered to a more acceptable
value if your system is running into problems without any ill effects (I
wouldn't recommend going below 32, though).
The value can be changed, since it is a system property named
`maxStackByteLimit`.
### Option 1 - Environment Variables
Configure the `maxStackByteLimit` variable, which must be prefixed with
`lucene:`.
Name: `lucene:maxStackByteLimit`
Value: `256`
### Option 2 - Self-Contained Custom Configuration Provider
Here is a minimal self-contained implementation, that requires references on:
```xml
<ItemGroup>
<PackageReference
Include="Microsoft.Extensions.Configuration.Abstractions" Version="9.0.3" />
<PackageReference
Include="Microsoft.Extensions.FileProviders.Abstractions" Version="9.0.3" />
</ItemGroup>
```
```c#
public sealed class SystemConfigurationFactory : IConfigurationFactory
{
private static readonly IConfiguration config = new
SystemConfiguration();
public IConfiguration GetConfiguration() => config;
}
public sealed class SystemConfiguration : IConfiguration
{
private readonly ConcurrentDictionary<string, string?>
systemProperties = new ConcurrentDictionary<string, string?>
{
["maxStackByteLimit"] = "256",
};
public string? this[string key]
{
get => systemProperties.TryGetValue(key, out var value) ? value
: null;
set => systemProperties[key] = value;
}
public IEnumerable<IConfigurationSection> GetChildren()
{
// Return only the first-level keys as sections
var sectionKeys = systemProperties.Keys
.Select(k => k.Split([':'], 2)[0])
.Distinct();
return sectionKeys.Select(k => new ConfigurationSection(this,
k));
}
public IChangeToken GetReloadToken() => NullChangeToken.Singleton;
public IConfigurationSection GetSection(string key)
{
return new ConfigurationSection(this, key);
}
private class ConfigurationSection : IConfigurationSection
{
private readonly SystemConfiguration parent;
public string Key { get; }
public string Path => Key;
public string? Value
{
get => parent[Key];
set => parent[Key] = value;
}
public ConfigurationSection(SystemConfiguration parent, string
key)
{
this.parent = parent;
Key = key;
}
public IEnumerable<IConfigurationSection> GetChildren()
{
var sectionPrefix = Key + ":";
var childKeys = parent.systemProperties.Keys
.Where(k => k.StartsWith(sectionPrefix))
.Select(k =>
k.Substring(sectionPrefix.Length).Split(':')[0])
.Distinct();
return childKeys.Select(k => new
ConfigurationSection(parent, sectionPrefix + k));
}
public IChangeToken GetReloadToken() =>
NullChangeToken.Singleton;
public IConfigurationSection GetSection(string key)
{
return new ConfigurationSection(parent, $"{Key}:{key}");
}
// Explicit implementation for IConfiguration members
string? IConfiguration.this[string key]
{
get => parent[$"{Key}:{key}"];
set => parent[$"{Key}:{key}"] = value;
}
IEnumerable<IConfigurationSection> IConfiguration.GetChildren()
=> GetChildren();
IChangeToken IConfiguration.GetReloadToken() => GetReloadToken();
IConfigurationSection IConfiguration.GetSection(string key) =>
GetSection(key);
}
}
```
And your application startup would look like:
```c#
ConfigurationSettings.SetConfigurationFactory(new
SystemConfigurationFactory());
```
Note that I haven't tested this, so there may be some kinks to work out.
Also, be aware that this will disable the built-in environment variable
provider so none of the `lucene:` prefixed environment variables will work.
### Option 3 - Standard (or Custom) Configuration Providers
Alternatively, you could use existing `Microsoft.Extension.Configuration`
providers by providing a pass-through implementation of
`IConfiguarationFactory`, as follows.
```xml
<ItemGroup>
<PackageReference
Include="Microsoft.Extensions.Configuration.Abstractions" Version="9.0.3" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json"
Version="9.0.3" />
<PackageReference
Include="Microsoft.Extensions.Configuration.EnvironmentVariables"
Version="9.0.3" />
</ItemGroup>
```
```c#
public sealed class ConfigurationFactory : IConfigurationFactory
{
private readonly IConfiguration configuration;
public ConfigurationFactory(IConfiguration configuration)
{
this.configuration = configuration ?? throw new
ArgumentNullException(nameof(configuration));
}
public IConfiguration GetConfiguration()
{
return configuration;
}
}
```
This can be used like so:
```c#
var configuration = new ConfigurationBuilder() // Note there should only be
1 instance of ConfigurationBuilder for your entire app
.AddEnvironmentVariables(prefix: "lucene:") // Use a custom prefix to
only load Lucene.NET settings
.AddJsonFile("appsettings.json", optional: true)
.Build();
ConfigurationSettings.SetConfigurationFactory(new
ConfigurationFactory(configuration));
```
Then you could put your application settings in the `appsettings.json` file,
like so:
```c#
{
"maxStackByteLimit": "256"
}
```
Do note that the above implementation won't detect any changes to the file
at runtime, though.
You could alternatively use one or more of the other providers, for example
the
[Microsoft.Extensions.Configuration](https://www.nuget.org/packages/Microsoft.Extensions.Configuration/latest)
contains an in-memory provider.
----------------------------
All of that said, changing the directory implementation could be a way to
quickly rule out `MMapDirectory` as the cause, although the other
implementations are not designed to scale to as many reads as `MMapDirectory`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
