AW: BUG #17448: In Windows 10, version 1703 and later, huge_pages doesn't work.

2022-04-28 Thread Wilm Hoyer


On Wed, Apr 27, 2022 at 03:04:23PM +, Wilm Hoyer wrote:
>>
>> I used the following hack to get the "real" Major and Minor Version of 
>> Windows - it's in C# (.Net) and needs to be adjusted (you can compile 
>> as x64 and use a long-long as return value ) to return the Service 
>> Number too and translated it into C.
>> I share it anyways, as it might help - please be kind, as it really is 
>> a little hack.
>>
>> Situation:
>> Main Application can't or is not willing to add a manifest file into 
>> its resources.
>>
>> Solution:
>> Start a small executable (which has a manifest file compiled into its 
>> resources), let it read the OS Version and code the Version into its 
>> return Code.

> Thanks for sharing.

You are welcome.

> Having to compile another tool just for that seems like a very high price to 
> pay, especially since we don't have any C# code in the tree.  I'm not even 
> sure that compiling this wouldn't need additional  requirements and/or if it 
> would work on our oldest supported Windows versions.

With "translate it into C" I meant "tread it as pseudo code, for a solution in 
plain C" (e.g. substitude Environment.OSVersion with IsWindowsVersionOrGreater 
or GetVersion )

On Wed, Apr 27, 2022 at 05:13:12PM +0900, Michael Paquier wrote:
> On Tue, Apr 26, 2022 at 12:54:35PM +0800, Julien Rouhaud wrote:
> > so I'm still on the opinion that we should unconditionally use the 
> > FILE_MAP_LARGE_PAGES flag if it's defined and call it a day.
> 
> Are we sure that this is not going to cause failures in environments 
> where the flag is not supported?

I'm not that familiar with the Microsoft OS or C (that's why I haven't migrated 
the c# code to C in the first place) to have a clear answer to that question.

If there is any risk and you want to avoid it, I can share a search result when 
I faced the same issue for our application. I declined this solution in favor 
of the previously shared one.
It's from NUnit (and needs migration to C as well - but since it just involves 
the Registry this should be pretty forward).
Just in case the Framework is not known: NUnit is the most popular .Net port of 
the Unit Testing Framework JUnit. There exits a C port too (CUnit) Maybe in 
there you can find an OS Version check too.

// Copyright (c) Charlie Poole, Rob Prouse and Contributors. MIT License - see 
LICENSE.txt
[...]
namespace NUnit.Framework.Internal
{
[SecuritySafeCritical]
public class OSPlatform
{
[...]
/// 
/// Gets the actual OS Version, not the incorrect value that might be
/// returned for Win 8.1 and Win 10
/// 
/// 
/// If an application is not manifested as Windows 8.1 or Windows 10,
/// the version returned from Environment.OSVersion will not be 6.3 and 
10.0
/// respectively, but will be 6.2 and 6.3. The correct value can be 
found in
/// the registry.
/// 
/// The original version
/// The correct OS version
private static Version GetWindows81PlusVersion(Version version)
{
try
{
using (var key = 
Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Windows 
NT\CurrentVersion"))
{
if (key != null)
{
var buildStr = key.GetValue("CurrentBuildNumber") as 
string;
int.TryParse(buildStr, out var build);

// These two keys are in Windows 10 only and are DWORDS
var major = key.GetValue("CurrentMajorVersionNumber") 
as int?;
var minor = key.GetValue("CurrentMinorVersionNumber") 
as int?;
if (major.HasValue && minor.HasValue)
{
return new Version(major.Value, minor.Value, build);
}

// If we get here, we are not Windows 10, so we are 
Windows 8
// or 8.1. 8.1 might report itself as 6.2, but will 
have 6.3
// in the registry. We can't do this earlier because 
for backwards
// compatibility, Windows 10 also has 6.3 for this key.
var currentVersion = key.GetValue("CurrentVersion") as 
string;
if(currentVersion == "6.3")
{
return new Version(6, 3, build);
}
}
}
}
catch (Exception)
{
}
return version;
}
[...]
   }
}

Finally, my reasoning to use the executable solution in favor of the NUnit one: 
I found no guarantee from Microsoft regarding the keys and values in the 
registry - hence a change with an update or in a newer Windows is not likely, 
but still possible. That's no problem for a heavily used and supported 
framework like NUnit - 

AW: BUG #17448: In Windows 10, version 1703 and later, huge_pages doesn't work.

2022-04-27 Thread Wilm Hoyer


On Tue, Apr 26, 2022 at 12:54:35PM +0800, Julien Rouhaud wrote:
>> I searched a bit and apparently some people are using this function 
>> directly opening some dll, which seems wrong.

> I was wondering about this whole business, and the manifest approach is a 
> *horrible* design for an API where the goal is to know if your run-time 
> environment is greater than a given threshold.

Agreed for the use case at hand, where you want to use one core API Function or 
another depending on the OS Version.

One Blog from Microsoft, I remember, told that one reason for the change were 
the increase of false installation error messages "Install Error - Your system 
does not meet the minimum supported operating system and service pack level."
where the software in question was written for Windows XP and the user tried to 
install it on, say, Windows 8.
That is just a Developer-Pilot error, where the Developers forgot to anticipate 
future OS Versions and instead of checking for Version at least, where checking 
for Version equality of all at design time known Windows Version.
Since you can develop only against OS APIs known at design time, and Microsoft 
claims to be pretty good at maintaining backward compatible facades for their 
APIs, there is some reason in that decision.
(To only see the Versions and APIs you told the OS with the manifest, you knew 
about at compile time).
The core Problem at hand is, that ms broke the promise of backward 
compatibility, since the function in question is working differently, depending 
on windows version, although with the above reasoning we should get the exact 
same behavior on windows 10 as on windows 8.1 (as PostgreSql, per default, only 
claims to know about Windows 8.1 features).

That said, I can understand the design decision. Personally, I still don't like 
it a bit, since developers should be allowed to make some stupid mistakes.

>>> Another Idea on windows machines would be to use the commandline to 
>>> execute ver in a separate Process and store the result in a file.
>> 
>> That also seems hackish, I don't think that we want to rely on 
>> something like that.

>Hmm.  That depends on the dependency set, I guess.  We do that on Linux at 
>some extent to for large pages in sysv_shmem.c.  Perhaps this could work for 
>Win10 if this avoids the extra loopholes with the >manifests.

I used the following hack to get the "real" Major and Minor Version of Windows 
- it's in C# (.Net) and needs to be adjusted (you can compile as x64 and use a 
long-long as return value ) to return the Service Number too and translated it 
into C.
I share it anyways, as it might help - please be kind, as it really is a little 
hack.

Situation: 
Main Application can't or is not willing to add a manifest file into its 
resources.

Solution:
Start a small executable (which has a manifest file compiled into its 
resources), let it read the OS Version and code the Version into its return 
Code.

CInt32 is basically an integer redefinition, where one can access the lower and 
higher Int16 separately.

The Main Programm eventually calls this (locate the executable, adjust the 
Process startup to be minimal, call the executable as separate process and 
interpret the return value as Version):
private static Version ReadModernOsVersionInternal()
{
String codeBase = Assembly.GetExecutingAssembly().CodeBase;
Uri uri = new Uri(codeBase);

String localPath = uri.LocalPath;
String pathDirectory = Path.GetDirectoryName(localPath);

if (pathDirectory != null)
{
String fullCombinePath = Path.Combine(pathDirectory, 
"Cf.Utilities.ReadModernOSVersion");

ProcessStartInfo processInfo = new ProcessStartInfo
{
FileName = fullCombinePath,
CreateNoWindow = true,
UseShellExecute = false
};

Process process = new Process
{
StartInfo = processInfo
};

process.Start();

if (process.WaitForExit(TimeSpan.FromSeconds(1).Milliseconds))
{
CInt32 versionInteger = process.ExitCode;
return new Version(versionInteger.HighValue, 
versionInteger.LowValue);
}
}

return new Version();
}


The small Version Check executable:

static Int32 Main(String[] args)
{
return OsVersionErmittler.ErmittleOsVersion();
}

and

static class OsVersionErmittler
{
/// 
/// Ermittelt die OsVersion und übergibt diese als High und LowWord.
/// 
/// 
public static CInt32 ErmittleOsVersion()
{
OperatingSystem version = Environment.OSVersion;
if (version.Platform == PlatformID.Win32NT && version.Version >= new 
Version(6, 3))
{
String versionString = version.VersionString;
return new CInt32((Int16) version.Version.Major, (Int16) 
version.Version.Minor);
}
return 0;
}
}

The shortened manifest of the small executable: