Re: [Scons-dev] MSVC 2017+ Toolset Support
> So to summarize the only (possible) difference between pro,ent,community > would be in whatever optional and/or updated components were installed for > each? > Yes. Target architecture differences would likely cause the most grief. So selecting one over the other would be useful just to use those > differences (optional/selected tools/components available via the visual > studio installer). > Yes. Removing all ambiguity in the presence of multiple products could be seen as a benefit as well. Does anyone have pro, enterprise, and community (or more than just one of > those versions installed) that could do a dump of vswhere output to answer > the question of ordering? > A change could be made to the existing vswhere query so that the order is deterministic regardless of the order of instances returned from vswhere. From: VCVER_TO_VSWHERE_VER = { '14.2': [ ["-version", "[16.0, 17.0)", ], # default: Enterprise, Professional, Community (order unpredictable?) ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools ], '14.1':[ ["-version", "[15.0, 16.0)", ], # default: Enterprise, Professional, Community (order unpredictable?) ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools ], '14.1Exp': [ ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.WDExpress"], # Express ],} To: _VCVER_TO_VSWHERE_VER = { '14.2': [ ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.Enterprise"], ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.Professional"], ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.Community"], ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], ], '14.1':[ ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.Enterprise"], ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.Professional"], ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.Community"], ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], ], '14.1Exp': [ ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.WDExpress"], ],} Note that vswhere's source code is on github... > I had in the original email: I am not motivated enough to read the C++ source code for vswhere :) Depence on the resolution order can be worked around with the solution above. Joe ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
So to summarize the only (possible) difference between pro,ent,community would be in whatever optional and/or updated components were installed for each? So selecting one over the other would be useful just to use those differences (optional/selected tools/components available via the visual studio installer). Does anyone have pro, enterprise, and community (or more than just one of those versions installed) that could do a dump of vswhere output to answer the question of ordering? Note that vswhere's source code is on github... On Fri, Aug 14, 2020 at 7:40 AM Joseph Brill wrote: > On Tue, Aug 11, 2020 at 10:52 PM Jason Kenny wrote: > >> Just to chime in.. my understanding of the vc compiler is that it is the >> same for all these cases. The community, profession and enterprise is about >> extra paid features of the IDE. While these are all side by side >> installable.. the code you build, as I understand, is the same. For me at >> least I don’t see what value exists yet in selecting license level values >> of the MSVS >> > > *All – please respond with any corrections and clarifications if anything > below is inaccurate or misleading. * > > > > In general, this assessment is accurate. With respect to the current > SCons implementation however, an important qualification is necessary and > is illustrated below. > > > > There is a potentially rare problem that only manifests itself when > multiple product instances (i.e., Enterprise, Professional, Community, and > Build Tools) of the same version are installed due to the current instance > selection implementation for 2017 and 2019 based on the vswhere tool. > > > > Given two or more installations of Enterprise, Professional, and > Community: the expected build *behavior is the same given that the > installed features and target toolsets are the same*. > > > > Given two or more installations of Enterprise, Professional, and > Community: the expected build behavior *can be different if the > installed features or target toolsets are different*. Herein lies the > rub. > > > > Currently, the selected MSVS instance is independent of the required > host/target architecture and toolset for the build. With multiple > instances of the same product installed, it is possible an installed > instance is selected that does not fulfill the required features/toolsets > despite the presence of installed instances that do fulfill the required > features/toolsets. > > > > Consider the following installations of MSVS 2019: > > - MSVS 2019 Enterprise: only toolsets targeting windows x64/x86 > > - MSVS 2019 Community: only toolsets targeting arm/arm64. > > > > In this case: > > - If 2019 Enterprise is always returned: arm/arm64 builds would fail > > - If 2019 Community is always returned: x64/x86 builds would fail > > > > The Visual Studio Installer offers a smorgasbord of independently selected > features (e.g., ATL, MFC), targets (e.g., arm, arm64), and toolsets. > Toolsets can be identified by the folder layout and existence of > tool binaries within an MSVS installation. Features are likely difficult > or impossible to identify. > > > > It could be possible that an end-user may not want to modify a > corporate/IT supplied Enterprise instance or may be restricted from making > changes. In this case, installing the Community edition makes sense. For > example, x86/x64 software for work and arm/arm64 software for pleasure. > > > > Similar examples with “features” could apply as well: one instance might > have ATL+MFC support installed while another instance does not. In this > case, there would be a problem if the instance without ATL+MFC were > selected. > > > > Given multiple instances installed with the same products and features, it > may be desirable to install updates (e.g., minor toolset updates) in one > instance and test with that particular instance prior to propagating the > same updates to the other instances. While extremely rare, code > generation bugs are possible in a minor toolset update. Reliance on > “undocumented” features also can also be problematic when switching > products and toolsets. With a corporate/it managed instance, updates may > be less frequent than a user-managed instance. > > > > Given two or more instances installed, it may be desirable to force a > build using the same product type as a customer and/or a production build > environment for purposes of verification and validation. > > > > If nothing else, it is useful for testing SCons J > > > > I am unaware of any documentation that describes the order in which > vswhere would return instances of Enterprise, Professional, and Community > if all three were present. Alphabetical? Last installed to first > installed? License level? I don't have access to licensed versions of the > Enterprise and Professional products. > > > With respect to the current SCons implementation, expansion of the > user-defined product specification would be useful even without specific > tools
Re: [Scons-dev] MSVC 2017+ Toolset Support
On Tue, Aug 11, 2020 at 10:52 PM Jason Kenny wrote: > Just to chime in.. my understanding of the vc compiler is that it is the > same for all these cases. The community, profession and enterprise is about > extra paid features of the IDE. While these are all side by side > installable.. the code you build, as I understand, is the same. For me at > least I don’t see what value exists yet in selecting license level values > of the MSVS > *All – please respond with any corrections and clarifications if anything below is inaccurate or misleading. * In general, this assessment is accurate. With respect to the current SCons implementation however, an important qualification is necessary and is illustrated below. There is a potentially rare problem that only manifests itself when multiple product instances (i.e., Enterprise, Professional, Community, and Build Tools) of the same version are installed due to the current instance selection implementation for 2017 and 2019 based on the vswhere tool. Given two or more installations of Enterprise, Professional, and Community: the expected build *behavior is the same given that the installed features and target toolsets are the same*. Given two or more installations of Enterprise, Professional, and Community: the expected build behavior *can be different if the installed features or target toolsets are different*. Herein lies the rub. Currently, the selected MSVS instance is independent of the required host/target architecture and toolset for the build. With multiple instances of the same product installed, it is possible an installed instance is selected that does not fulfill the required features/toolsets despite the presence of installed instances that do fulfill the required features/toolsets. Consider the following installations of MSVS 2019: - MSVS 2019 Enterprise: only toolsets targeting windows x64/x86 - MSVS 2019 Community: only toolsets targeting arm/arm64. In this case: - If 2019 Enterprise is always returned: arm/arm64 builds would fail - If 2019 Community is always returned: x64/x86 builds would fail The Visual Studio Installer offers a smorgasbord of independently selected features (e.g., ATL, MFC), targets (e.g., arm, arm64), and toolsets. Toolsets can be identified by the folder layout and existence of tool binaries within an MSVS installation. Features are likely difficult or impossible to identify. It could be possible that an end-user may not want to modify a corporate/IT supplied Enterprise instance or may be restricted from making changes. In this case, installing the Community edition makes sense. For example, x86/x64 software for work and arm/arm64 software for pleasure. Similar examples with “features” could apply as well: one instance might have ATL+MFC support installed while another instance does not. In this case, there would be a problem if the instance without ATL+MFC were selected. Given multiple instances installed with the same products and features, it may be desirable to install updates (e.g., minor toolset updates) in one instance and test with that particular instance prior to propagating the same updates to the other instances. While extremely rare, code generation bugs are possible in a minor toolset update. Reliance on “undocumented” features also can also be problematic when switching products and toolsets. With a corporate/it managed instance, updates may be less frequent than a user-managed instance. Given two or more instances installed, it may be desirable to force a build using the same product type as a customer and/or a production build environment for purposes of verification and validation. If nothing else, it is useful for testing SCons J I am unaware of any documentation that describes the order in which vswhere would return instances of Enterprise, Professional, and Community if all three were present. Alphabetical? Last installed to first installed? License level? I don't have access to licensed versions of the Enterprise and Professional products. With respect to the current SCons implementation, expansion of the user-defined product specification would be useful even without specific toolset version support for the situations outlined above. The mechanism necessary to be able to select the “Build Tools” instance when one or more Enterprise/Professional/Community instances is likely general enough to be extended to cover all instances at little to no extra cost. As for generated output, starting with 2015 all of the product instances (i.e., Enterprise, Professional, Community, Build Tools, and Express) are binary compatible both within a toolset (e.g, v141) and across toolsets (e.g, v140, v141, and v142) subject to limitations. See below. Regards, Joe https://docs.microsoft.com/en-us/cpp/porting/binary-compat-2015-2017?view=vs-2019#:~:text=The%20Microsoft%20C%2B%2B%20(MSVC)%20compiler,executables%20built%20by%20different%20versions.&text=
Re: [Scons-dev] MSVC 2017+ Toolset Support
Just to chime in.. my understanding of the vc compiler is that it is the same for all these cases. The community, profession and enterprise is about extra paid features of the IDE. While these are all side by side installable.. the code you build, as I understand, is the same. For me at least I don’t see what value exists yet in selecting license level values of the MSVS Jason Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Joseph Brill<mailto:joseph.c.br...@gmail.com> Sent: Tuesday, August 11, 2020 1:55 PM To: Bill Deegan<mailto:b...@baddogconsulting.com> Cc: SCons developer list<mailto:scons-dev@scons.org> Subject: Re: [Scons-dev] MSVC 2017+ Toolset Support The was a previous iteration that allowed MSVC_VERSION to contain a toolset and added a MSVC_PRODUCT variable that could contain a product specification and optional component type. This was described in: https://github.com/SCons/scons/issues/3664#issuecomment-643782430<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSCons%2Fscons%2Fissues%2F3664%23issuecomment-643782430&data=02%7C01%7C%7C0c91771b51e348310f4908d83e2826ba%7C84df9e7fe9f640afb435%7C1%7C0%7C637327689453041815&sdata=2ZkIEEHXd9IK%2FgJV0HUmYnzmciAjj5FxUvI5TDtvuF0%3D&reserved=0> Unfortunately, this predates the code being added to version control as it was a proof-of-concept that evolved fairly rapidly. On Tue, Aug 11, 2020 at 2:06 PM Bill Deegan mailto:b...@baddogconsulting.com>> wrote: To be clear, I'm not in favor of your proposed MSVC_VERSION regex interpretation, so let's table any discussion of that as it's not relevant to figuring out how to properly select a version of MSVC and toolset and product type. Am I grossly mistaken, or would separate variables as I've proposed above also allow specifying as explicitly as the single string which needs to be regex'd? On Tue, Aug 11, 2020 at 9:24 AM Joseph Brill mailto:joseph.c.br...@gmail.com>> wrote: On Tue, Aug 11, 2020 at 11:51 AM Mats Wichmann mailto:m...@wichmann.us>> wrote: To throw another spanner into the works, there are Visual Studio product codes too, which are 16.X for 2019, 15.x for 2017 and 14.0 for 2015... Nothing is straightforward with the microsoft version numbers. A mapping was necessary from the vswhere query results containing the MSVS version to the MSVC version number. The explicitly supported component ids are listed by product as there are a quite a few component ids in which we are not interested. And yes, one of the components that we are not-interested in caused a previous version to fail. And it looks like I have a typographical error to fix as well... # Vswhere query elements (reverse dictionaries computed during intialization) # MSVS_MAJOR: MSVS installationVersion.split('.')[0] <-> to _VCVER # Products: expected product components returned in vswhere query _MSVC_PRODUCTVERSION_COMPONENTIDS = { '14.2': { 'MSVS_MAJOR' : '16', 'Products' : ['Enterprise', 'Professional', 'Community', 'BuildTools'] }, '14.1': { 'MSVS_MAJOR' : '15', 'Products' : ['Enterprise', 'Professional', 'Community', 'BuildTools', _MSVC_COMPONENT_EXPRESS] }, } ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
The was a previous iteration that allowed MSVC_VERSION to contain a toolset and added a MSVC_PRODUCT variable that could contain a product specification and optional component type. This was described in: https://github.com/SCons/scons/issues/3664#issuecomment-643782430 Unfortunately, this predates the code being added to version control as it was a proof-of-concept that evolved fairly rapidly. On Tue, Aug 11, 2020 at 2:06 PM Bill Deegan wrote: > To be clear, I'm not in favor of your proposed MSVC_VERSION regex > interpretation, so let's table any discussion of that as it's not relevant > to figuring out how to properly select a version of MSVC and toolset and > product type. > > Am I grossly mistaken, or would separate variables as I've proposed above > also allow specifying as explicitly as the single string which needs to be > regex'd? > > > On Tue, Aug 11, 2020 at 9:24 AM Joseph Brill > wrote: > >> On Tue, Aug 11, 2020 at 11:51 AM Mats Wichmann wrote: >> >>> To throw another spanner into the works, there are Visual Studio product >>> codes too, which are 16.X for 2019, 15.x for 2017 and 14.0 for 2015... >>> >> >> Nothing is straightforward with the microsoft version numbers. >> >> A mapping was necessary from the vswhere query results containing the >> MSVS version to the MSVC version number. The explicitly supported >> component ids are listed by product as there are a quite a few component >> ids in which we are not interested. And yes, one of the components that we >> are not-interested in caused a previous version to fail. >> >> And it looks like I have a typographical error to fix as well... >> >> # Vswhere query elements (reverse dictionaries computed during >> intialization)# MSVS_MAJOR: MSVS installationVersion.split('.')[0] <-> >> to _VCVER# Products: expected product components returned in vswhere >> query >> _MSVC_PRODUCTVERSION_COMPONENTIDS = { >> '14.2': { >> 'MSVS_MAJOR' : '16', >> 'Products' : ['Enterprise', 'Professional', 'Community', >> 'BuildTools'] >> }, >> '14.1': { >> 'MSVS_MAJOR' : '15', >> 'Products' : ['Enterprise', 'Professional', 'Community', >> 'BuildTools', _MSVC_COMPONENT_EXPRESS] >> }, >> >> } >> >> >> > ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
On Tue, Aug 11, 2020 at 2:06 PM Bill Deegan wrote: > Am I grossly mistaken, or would separate variables as I've proposed above > also allow specifying as explicitly as the single string which needs to be > regex'd? > You are not mistaken. Multiple variables are certainly possible. However, they are more difficult to reconcile and validate for consistency and in general more verbose from an end-user perspective. That said, the current regex and validation/verification is simply an interface to map a user specification to an internal representation. There's no reason an alternate user-facing interface cannot be adopted (other then catching the author on a bad day). A combination of toolset/product/component (some or all of which are optional), while more difficult to validate and reconcile, could certainly be mapped into the same internal data structures. What may not be obvious from the regex is that it does not allow a product type (component) without being attached to a toolset and/or product. Not sure if this is desirable or not. If it is desireable, then the search algorithm would almost certainly have to be expanded likely with an additional table with a different ranking scheme. Admittedly, have not put much thought into that particular scenario. ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
To be clear, I'm not in favor of your proposed MSVC_VERSION regex interpretation, so let's table any discussion of that as it's not relevant to figuring out how to properly select a version of MSVC and toolset and product type. Am I grossly mistaken, or would separate variables as I've proposed above also allow specifying as explicitly as the single string which needs to be regex'd? On Tue, Aug 11, 2020 at 9:24 AM Joseph Brill wrote: > On Tue, Aug 11, 2020 at 11:51 AM Mats Wichmann wrote: > >> To throw another spanner into the works, there are Visual Studio product >> codes too, which are 16.X for 2019, 15.x for 2017 and 14.0 for 2015... >> > > Nothing is straightforward with the microsoft version numbers. > > A mapping was necessary from the vswhere query results containing the MSVS > version to the MSVC version number. The explicitly supported component ids > are listed by product as there are a quite a few component ids in which we > are not interested. And yes, one of the components that we are > not-interested in caused a previous version to fail. > > And it looks like I have a typographical error to fix as well... > > # Vswhere query elements (reverse dictionaries computed during > intialization)# MSVS_MAJOR: MSVS installationVersion.split('.')[0] <-> to > _VCVER# Products: expected product components returned in vswhere query > _MSVC_PRODUCTVERSION_COMPONENTIDS = { > '14.2': { > 'MSVS_MAJOR' : '16', > 'Products' : ['Enterprise', 'Professional', 'Community', > 'BuildTools'] > }, > '14.1': { > 'MSVS_MAJOR' : '15', > 'Products' : ['Enterprise', 'Professional', 'Community', > 'BuildTools', _MSVC_COMPONENT_EXPRESS] > }, > > } > > > ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
On Tue, Aug 11, 2020 at 11:51 AM Mats Wichmann wrote: > To throw another spanner into the works, there are Visual Studio product > codes too, which are 16.X for 2019, 15.x for 2017 and 14.0 for 2015... > Nothing is straightforward with the microsoft version numbers. A mapping was necessary from the vswhere query results containing the MSVS version to the MSVC version number. The explicitly supported component ids are listed by product as there are a quite a few component ids in which we are not interested. And yes, one of the components that we are not-interested in caused a previous version to fail. And it looks like I have a typographical error to fix as well... # Vswhere query elements (reverse dictionaries computed during intialization)# MSVS_MAJOR: MSVS installationVersion.split('.')[0] <-> to _VCVER# Products: expected product components returned in vswhere query _MSVC_PRODUCTVERSION_COMPONENTIDS = { '14.2': { 'MSVS_MAJOR' : '16', 'Products' : ['Enterprise', 'Professional', 'Community', 'BuildTools'] }, '14.1': { 'MSVS_MAJOR' : '15', 'Products' : ['Enterprise', 'Professional', 'Community', 'BuildTools', _MSVC_COMPONENT_EXPRESS] }, } ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
On 8/11/20 9:23 AM, Joseph Brill wrote: > I would like to take an opportunity to document some of the behavior of > the toolset version and some of the terminology used as a basis for > further discussion. Also, I believe that the consistency gap is for a > narrow subset of the possible extended specifications. > > > > Toolset: 14.2[X[.Y]], 14.1[X[.Y]], 14.0 > > Product: 14.2 (MSVC 2019), 14.1 (MSVC 2017), 14.0 (MSVC 2015) > > Component: Enterprise (Ent), Professional (Pro), Community (Com), > BuildTools (BT), WDExpress (Exp) To throw another spanner into the works, there are Visual Studio product codes too, which are 16.X for 2019, 15.x for 2017 and 14.0 for 2015... ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
I would like to take an opportunity to document some of the behavior of the toolset version and some of the terminology used as a basis for further discussion. Also, I believe that the consistency gap is for a narrow subset of the possible extended specifications. Toolset: 14.2[X[.Y]], 14.1[X[.Y]], 14.0 Product: 14.2 (MSVC 2019), 14.1 (MSVC 2017), 14.0 (MSVC 2015) Component: Enterprise (Ent), Professional (Pro), Community (Com), BuildTools (BT), WDExpress (Exp) The component ids used are from Microsoft’s documentation of the workloads and component ids for 2019 and 2017. The abbreviations are similar to that used for Express. The component ids are returned in vswhere queries. The component id tables are shown at the end of the document along with the reference links. Temporarily suspend the distaste of the extended syntax (there is a method to the madness). The extended syntax for the MSVC_VERSION is not as “free-wheeling” as it might appear and “binds” to specific products in a number of cases. This is by design. The point of departure is the semantics for specifications that do not bind to a specific product. This may be able to be resolved using a command-line option to SCons and/or another argument in the environment. I believe Bill may have suggested this earlier. A command-line argument would have significant utility as it would not require changes to build scripts. The regular expression that parses/validates the format is shown below. Currently, any specification that includes a component is automatically bound to a specific product. The following table includes examples from the original email and illustrates how the specification is resolved initially by default: User Internal (Resolved Once Upon Entry) MSVC_VERSION MSVC_VERSION Toolset Product Comp 14.1 14.1+ 14.1 * * 14.16 14.1+ 14.16 * * 14.16.27 14.1+ 14.16.27 * * 14.16.27023 14.1+ 14.16.27023 * * 14.26Ent 14.2 14.26 14.2 Ent 14.2Pro 14.2 14.2 14.2 Pro 14.26.27Com 14.2 14.26.27 14.2 Com 14.2BT 14.2 14.2 14.2 BT 14.1Exp 14.1 14.1 14.1 Exp 14.0 -> 14.2 14.2 14.0 14.2 * 14.1 -> 14.2BT 14.2 14.1 14.2 BT 14.16 -> 14.2Com 14.2 14.16 14.2 Com + Indicates that the preferred product version but may be “promoted” to a later version. This is the undesirable behavior. * Indicates that the product or component is not bound to a specific product and/or component. The toolset is a “proper” prefix (i.e., “startswith”). Again, I believe this may have been suggested earlier: the default behavior could be changed to “bind” to the native product for the toolset (i.e., the “Product” for the first four rows in the table above would be “14.1”). In this case, the only ambiguity in the selection would be which product “kind” (i.e., component id) is selected which is what happens in the current msvc detection. The alternative behavior, either via command-line option and/or additional variable, would be the toolset functionality as currently implemented. Issue #3664 succinctly illustrates the differences between the two behaviors. The two options are shown below in the context of the current extended toolset version specification: Assume that no 14.1 products are installed, and the 14.2 Build Tools edition is installed with the 14.1 toolset. No promotion Result MSVC_VERSION="14.1" Error – 14.1 Not Found MSVC_VERSION="14.1->14.2" 14.2 Build Tools MSVC_VERSION=”14.1->14.2BT” 14.2 Build Tools Promotion Result MSVC_VERSION="14.1" 14.2 Build Tools MSVC_VERSION="14.1->14.2" 14.2 Build Tools MSVC_VERSION=”14.1->14.2BT” 14.2 Build Tools In either case, the issue of using the 14.1 toolset with the 14.2 product can be resolved with the extended syntax and without product ambiguity (i.e., the last row). The second specification would leave open the kind (i.e., component) of product selected. >From an end-user perspective, adding “->14.2” is easier than going via the custom batch file route or even adding an additional variable. A command-line option to enable the extended behavior may be preferable especially if using a batch file to launch SCons. The point of departure is simply what should the default behavior be when a specification is not bound to a specific product and what should the mechanism be to enable the other behavior. The extended MSVC_VERSION format regular expression and Microsoft component ids for 2019 and 2017 are shown below. Regards, Joe *Extended MSVC_VERSION format:* *# Regular Expression to validate/parse MSVC_VERSION* *#* *#named groups:* *# version: _VCVER version or specific version* *# vcver:_VCVER version or None* *# specific: specific version or None* *# product: _VCVER version or None* *# suffix: product type suffix or None* _MSVC_EXTENDED_VERSION_RE = re.compile(""" *# anchor at start of string* ^ *# version nu
Re: [Scons-dev] MSVC 2017+ Toolset Support
Thanks for that. Actually, I do want to know. AFAIK, the handling of 14.0 is quite different as it is not installed side-by-side in 2017 and 2019. I'm not sure because I already had 2015 installed, but I believe the installation option in 2017 may be to install the 2015 tools "normally" (i.e., outside of the 2017 folder). For the 14.0 toolset with 2017 and 2019 there is "special case" handling and a specific batch file: vcvars140.bat. While you may not be able to install the 2015 tools with the 2019 installed, you can still pass "14.0" as the toolset. In 2019, vcvars140.bat is in the folder "Common7\Tools\vsdevcmd\ext\vcvars\vcvars140.bat". The vcvars140.bat file queries the registry to find the installation location of the 2015 tools (comments extracted from my working version): # vcvars140.bat compatible registry queries: # HKLM\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\SxS\VC7 /v "14.0" # HKLM\SOFTWARE\Microsoft\VisualStudio\SxS\VC7 /v "14.0" On Mon, Aug 10, 2020 at 8:36 PM Mats Wichmann wrote: > On 8/10/20 6:03 PM, Joseph Brill wrote: > > > Yes. I believe the number is 9 separate product installations for 2017 > > (build tools, express, community, professional, and enterprise) and 2109 > > (build tools, community, professional, and enterprise). I don't have > > access to professional or enterprise versions and a I believe the > > installer for 2017 build tools is rarer than hen's teeth (although it > > may be available via Chocolatey). As far as I know, there is not an > > express version of 2019. Locally, I have been using four installations > > for testing of the 14.1 (14.16) toolsets: 2017 Community, 2017 Express, > > 2019, Community, and 2019 Build Tools. > > Only minor correction: you can get eight via just the standard installer > - build tools 2017 is not rare at all; while express (yes, "expressly" > discontinued after 2017) is quite hard to find, and you can't get at it > through the standard VS installer - until you actually have it > installed, then the installed VS 2017 installer will let you manipulate > it, but it's invisible otherwise. > > > It is possible to specify the 2015 toolset (14.0) via MSVC 2017 and MSVC > > 2019 as well. > > the installation policy seems to be you get one toolset version from the > previous major release, only. I don't "know" this (as I've read no doc > or anything), deducing this from looking at my one machine with a bunch > of different VCs installed: > > the 2017 VS Installer offers me "2015.3 v14.00" v140 toolset, along with > v141 14.11 through 14.16. > > The 2019 VS installer offers me v141 14.16, along with v142 14.20 > through 14.27 - note no toolset for 2015 here. > > > all this is probably getting down to "more than anyone wants to know" :) > > ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
On 8/10/20 6:03 PM, Joseph Brill wrote: > Yes. I believe the number is 9 separate product installations for 2017 > (build tools, express, community, professional, and enterprise) and 2109 > (build tools, community, professional, and enterprise). I don't have > access to professional or enterprise versions and a I believe the > installer for 2017 build tools is rarer than hen's teeth (although it > may be available via Chocolatey). As far as I know, there is not an > express version of 2019. Locally, I have been using four installations > for testing of the 14.1 (14.16) toolsets: 2017 Community, 2017 Express, > 2019, Community, and 2019 Build Tools. Only minor correction: you can get eight via just the standard installer - build tools 2017 is not rare at all; while express (yes, "expressly" discontinued after 2017) is quite hard to find, and you can't get at it through the standard VS installer - until you actually have it installed, then the installed VS 2017 installer will let you manipulate it, but it's invisible otherwise. > It is possible to specify the 2015 toolset (14.0) via MSVC 2017 and MSVC > 2019 as well. the installation policy seems to be you get one toolset version from the previous major release, only. I don't "know" this (as I've read no doc or anything), deducing this from looking at my one machine with a bunch of different VCs installed: the 2017 VS Installer offers me "2015.3 v14.00" v140 toolset, along with v141 14.11 through 14.16. The 2019 VS installer offers me v141 14.16, along with v142 14.20 through 14.27 - note no toolset for 2015 here. all this is probably getting down to "more than anyone wants to know" :) ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MSVC 2017+ Toolset Support
> "Can I end up with 14.1 installed in both a MSVC2017 and MSVC2019 install (and of course also all the flavors of MSVC, build tools, express, community, enterprise), for a possible total of 8 separate installs? (or some smaller non singular number?)" Yes. I believe the number is 9 separate product installations for 2017 (build tools, express, community, professional, and enterprise) and 2109 (build tools, community, professional, and enterprise). I don't have access to professional or enterprise versions and a I believe the installer for 2017 build tools is rarer than hen's teeth (although it may be available via Chocolatey). As far as I know, there is not an express version of 2019. Locally, I have been using four installations for testing of the 14.1 (14.16) toolsets: 2017 Community, 2017 Express, 2019, Community, and 2019 Build Tools. It is possible to specify the 2015 toolset (14.0) via MSVC 2017 and MSVC 2019 as well. The argument to the vcvars batch file is "14.0". The batch file then effectively finds the 14.0 build tools. The working version uses the registry keys from the msvc batch files to determine if the 14.0 tools are installed. I believe the registry keys used by the msvc batch files.are slightly different that the registry keys that SCons uses for 14.0. I am time limited today and would like to spend some more time thinking about the rest of your email before responding. Implementation note: one advantage of the more complicated syntax is that it is validated with a single fairly straightforward regular expression. The components of interest are captured groups. The string match is done once upon entry. MSVC_VERSION is immediately set to the compatible "14.X" format for the existing code. Except for initial entry, MSVC_VERSION is the currently used format. One reason this was necessary was that the existing code was converting MSVC_VERSION to a floating point number. Even adding the second digit to the MSVC_VERSION (e.g., 14.16) would/could have "broken" the existing code. This was the key to not having to modify a sizable chunk of the existing code. The remaining fields are stored in an internal data structure due to the gap between determining if the tools are installed and actually determining the host/target batch file arguments. > (And yes. I know.. "Foolish consistency is the hobgoblin of little minds", but I think in this case it's not foolish) Emerson. Nice. As you are probably painfully aware by now: "I would have written a shorter letter, but I did not have the time". On Mon, Aug 10, 2020 at 7:23 PM Bill Deegan wrote: > Can I end up with 14.1 installed in both a MSVC2017 and MSVC2019 install > (and of course also all the flavors of MSVC, build tools, express, > community, enterprise), for a possible total of 8 separate installs? (or > some smaller non singular number?) > > I'm not a fan of the more complicated syntax for MSVC_VERSION and would > rather split out the selection to either separate variables for product, > toolset, and product version and/or a callable which handles selection. > MSVC_VERSION -> sounds like after MSVC2017 defaults to latest installed > toolset unless you set toolset version > MSVC_PRODUCT_VERSION -> Enterprise, Professional, Community, Express, or > not set > MSVC_TOOLSET_VERSION -> This may only apply for MSVC2017 or above (or > could be used if MSVC_VERSION is not set for MSVC < 2017, though that would > likely introduce confusion > > MSVC_SELECTOR -> Callable (too allow more flexibility than static versions > above) > > Otherwise you end up encoding the information in a string and then you > have to decode it. > Seems more complicated than necessary, and we don't have any other > syntaxes embedded in variables thus far. > > I realize you have lots of time invested in your MSVC_VERSION > implementation you've explained, but the goal here is to be consistent with > expected behavior in the face of changed MSVC. > > (And yes. I know.. "Foolish consistency is the hobgoblin of little minds", > but I think in this case it's not foolish) > > Welcome feedback on above. > > > > > On Sun, Aug 9, 2020 at 2:50 PM Joseph Brill > wrote: > >> This email is an attempt to focus the discussion concerning support for >> MSVC toolsets moving forward in a rational, dispassionate discourse. >> >> I am advocating a minor shift in orientation or "world view" for MSVC >> 2017 and later: the semantic transition from thinking of the msvc version >> as a product to thinking of the msvc version as a toolset. This actually >> makes the implementation of toolset support easier. >> >> There is a working version of the support for toolsets at varying levels >> of precision that can be enabled and disabled. The intent is that a >> command-line switch could be added to enable the toolset specific >> capabilities. By default, the toolset specific capabilities would be >> disabled in which case the code behaves exactly like the current >> master. This
Re: [Scons-dev] MSVC 2017+ Toolset Support
Can I end up with 14.1 installed in both a MSVC2017 and MSVC2019 install (and of course also all the flavors of MSVC, build tools, express, community, enterprise), for a possible total of 8 separate installs? (or some smaller non singular number?) I'm not a fan of the more complicated syntax for MSVC_VERSION and would rather split out the selection to either separate variables for product, toolset, and product version and/or a callable which handles selection. MSVC_VERSION -> sounds like after MSVC2017 defaults to latest installed toolset unless you set toolset version MSVC_PRODUCT_VERSION -> Enterprise, Professional, Community, Express, or not set MSVC_TOOLSET_VERSION -> This may only apply for MSVC2017 or above (or could be used if MSVC_VERSION is not set for MSVC < 2017, though that would likely introduce confusion MSVC_SELECTOR -> Callable (too allow more flexibility than static versions above) Otherwise you end up encoding the information in a string and then you have to decode it. Seems more complicated than necessary, and we don't have any other syntaxes embedded in variables thus far. I realize you have lots of time invested in your MSVC_VERSION implementation you've explained, but the goal here is to be consistent with expected behavior in the face of changed MSVC. (And yes. I know.. "Foolish consistency is the hobgoblin of little minds", but I think in this case it's not foolish) Welcome feedback on above. On Sun, Aug 9, 2020 at 2:50 PM Joseph Brill wrote: > This email is an attempt to focus the discussion concerning support for > MSVC toolsets moving forward in a rational, dispassionate discourse. > > I am advocating a minor shift in orientation or "world view" for MSVC 2017 > and later: the semantic transition from thinking of the msvc version as a > product to thinking of the msvc version as a toolset. This actually makes > the implementation of toolset support easier. > > There is a working version of the support for toolsets at varying levels > of precision that can be enabled and disabled. The intent is that a > command-line switch could be added to enable the toolset specific > capabilities. By default, the toolset specific capabilities would be > disabled in which case the code behaves exactly like the current master. > This would allow field testing and evaluation of the functionality in other > environments and to easily test the differences between the current master > and the toolset functionality. The functional aspects are feature > complete. The working version is a product of more than one man-month of > full-time development activity and the result of the third iteration of > implementation strategy. > > Existing PR #3717 was closed and the branch renamed due in part to make it > easier to identify and maintain for myself as I fear that I might be time > limited in the near term and due in part to the existing discussion > unraveling fairly quickly with dim prospects of inclusion moving forward. > Despite > being closed, I recommend viewing the "documentation" (the PR text and the > first comment) and the vc.py code difference in PR #3717. At any time the > PR request can be reopened. I would need a git guru to walk me through > associating the new branch name with the "old" PR. > > The implementation knocks an item or two off the internal vc.py TODO list > such as a single vswhere query and processing the json results for all > known installations. Ranked toolset lists are constructed by host/target. > The ranking takes into account the toolset version and product type. A > single toolset instance may be a member of multiple toolset lists. > Expensive lookups are "runtime memoized/cached" so that they are only > computed once. While there is a fair amount of initialization, the > processing burden is based on the number of products installed and the > number of toolsets for each product. Typical environments likely have a > single product installation with a spartan number of different toolsets > installed. There is nothing particularly "clever" done in the > implementation and should not be a maintenance burden. The bulk of the > toolset support implementation follows the code in the existing master > (i.e., the bottom of the file). The modifications to support toolsets > weighs in at approximately 1400 lines of code. > > Prior to MSVS 2017, there was a direct 1:1 mapping between the msvc > *product* and the msvc *toolset*. The toolset might be upgraded in an > update or service pack but there was no way to select a specific toolset as > side-by-side installations were not supported. That changed with the > release of MSVC 2017 which allows multiple side-by-side toolset > installations by target. MSVC 2017 and later, can be viewed as *toolset* > oriented > (i.e., a MSVC 2019 installation can be viewed as a container for 14.1X, > 14.2X, and transparently the 14.0 toolsets). Moving forward, I believe > that it will be advantageous to change from a
[Scons-dev] MSVC 2017+ Toolset Support
This email is an attempt to focus the discussion concerning support for MSVC toolsets moving forward in a rational, dispassionate discourse. I am advocating a minor shift in orientation or "world view" for MSVC 2017 and later: the semantic transition from thinking of the msvc version as a product to thinking of the msvc version as a toolset. This actually makes the implementation of toolset support easier. There is a working version of the support for toolsets at varying levels of precision that can be enabled and disabled. The intent is that a command-line switch could be added to enable the toolset specific capabilities. By default, the toolset specific capabilities would be disabled in which case the code behaves exactly like the current master. This would allow field testing and evaluation of the functionality in other environments and to easily test the differences between the current master and the toolset functionality. The functional aspects are feature complete. The working version is a product of more than one man-month of full-time development activity and the result of the third iteration of implementation strategy. Existing PR #3717 was closed and the branch renamed due in part to make it easier to identify and maintain for myself as I fear that I might be time limited in the near term and due in part to the existing discussion unraveling fairly quickly with dim prospects of inclusion moving forward. Despite being closed, I recommend viewing the "documentation" (the PR text and the first comment) and the vc.py code difference in PR #3717. At any time the PR request can be reopened. I would need a git guru to walk me through associating the new branch name with the "old" PR. The implementation knocks an item or two off the internal vc.py TODO list such as a single vswhere query and processing the json results for all known installations. Ranked toolset lists are constructed by host/target. The ranking takes into account the toolset version and product type. A single toolset instance may be a member of multiple toolset lists. Expensive lookups are "runtime memoized/cached" so that they are only computed once. While there is a fair amount of initialization, the processing burden is based on the number of products installed and the number of toolsets for each product. Typical environments likely have a single product installation with a spartan number of different toolsets installed. There is nothing particularly "clever" done in the implementation and should not be a maintenance burden. The bulk of the toolset support implementation follows the code in the existing master (i.e., the bottom of the file). The modifications to support toolsets weighs in at approximately 1400 lines of code. Prior to MSVS 2017, there was a direct 1:1 mapping between the msvc *product* and the msvc *toolset*. The toolset might be upgraded in an update or service pack but there was no way to select a specific toolset as side-by-side installations were not supported. That changed with the release of MSVC 2017 which allows multiple side-by-side toolset installations by target. MSVC 2017 and later, can be viewed as *toolset* oriented (i.e., a MSVC 2019 installation can be viewed as a container for 14.1X, 14.2X, and transparently the 14.0 toolsets). Moving forward, I believe that it will be advantageous to change from a *product* orientation to a *toolset* orientation. This change is only for the 2017 and later installations and currently really only affects 2017. With MSVC 2017 and MSVC 2019, an SCons version request of "14.1" or "14.2", respectively, *is a toolset request* that cannot be specified at a finer granularity. The current msvc behavior uses the *default toolset* which is generally the newest installed *toolset* version when the "--vcvars_ver" argument is not specified for the vcvars batch file. For example, a version request of "14.1" is likely to use the "14.16.27023" toolset when MSVC 2017 is installed. Effectively, the *product request* is a *toolset request* identical to "find the newest toolset version installed that starts with '14.1'". However, "14.2" (MSVC 2019) will return a *toolset* based on the latest update which likely varies across users and computers. Currently, a request for "14.1" will fail if MSVC 2019 is installed with the 14.1 toolset. With a toolset orientation, a 14.1 toolset request would be satisfied first by the native product (i.e., MSVC 2017) for the toolset and then by the latest product that contains the user specified toolset for a given host/target combination. This two-stage query, if necessary, is implemented in the working toolset oriented version. Currently, the actual toolset versions used for 2017 and 2019 are based on end-user's installation options and frequency of updates. While two different users may be using the same *product* (e.g., 2019) they may not be using the same *toolset*. Similarly, the installed toolset version is not necessarily