Re: [Scons-dev] This morning's WTF moment
On Tue, Sep 9, 2014 at 4:03 AM, Jean-Baptiste Lab jeanbaptiste@gmail.com wrote: Or wouldn't it be enough to simply mandate that exists() return something that can be tested against True/False? If that's the case, wouldn't a bit of wrapping around and implementing the __eq__/__neq__ descriptors (possibly __cmp__) be good enough so that we can get to the error description when needed (if False) without breaking existing usages? I did think about that. It's hard for me to imagine something that can test as false while still having a string value. Not impossible, but pretty weird and a bit un-pythonic. I prefer simplicity over cleverness. Still, if you have an idea, let me know. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
On Mon, Sep 8, 2014 at 10:24 PM, William Blevins wblevins...@gmail.com wrote: This might be obvious, but it the exception pattern not popular in python? Sure, but we don't want everyone testing for tool existence to have to wrap that in an exception handler. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
To test against True/False one should better implement __nonzero__, not __eq__ or something :-) That way one could then use the form if obj:, not an ugly form of if obj == True:. Thanks, Vasily 09 сент. 2014 г. 12:03 пользователь Jean-Baptiste Lab jeanbaptiste@gmail.com написал: Or wouldn't it be enough to simply mandate that exists() return something that can be tested against True/False? If that's the case, wouldn't a bit of wrapping around and implementing the __eq__/__neq__ descriptors (possibly __cmp__) be good enough so that we can get to the error description when needed (if False) without breaking existing usages? Or maybe I'm just missing the point, in which case I apologize ;) Cheers, JB On 9 September 2014 04:24, William Blevins wblevins...@gmail.com wrote: This might be obvious, but it the exception pattern not popular in python? On Sep 8, 2014 9:19 PM, Gary Oberbrunner ga...@oberbrunner.com wrote: On Mon, Sep 8, 2014 at 4:13 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: Ideally I always viewed this as a True False statement. I see you have it returning a tuple. I only worry that I have seen a lot push with certain python developers to say stuff like if not tool.exists(): # do something… This will not work as we will have a (True,””) or (False,””) return API. This seems to me to more complex to use and understand. At the very least east to trip up on. If we want an object returned. I think it will be better to define a error object that can be tested as True or False vs forcing tuple separation on returns values. Excellent point. The 'if not tool.exists()' pattern needs to work. I'll rethink that. Maybe something as simple as tool.exist_error() which can be called just after exists() returns False... -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
On 9 September 2014 11:56, Gary Oberbrunner ga...@oberbrunner.com wrote: On Tue, Sep 9, 2014 at 4:03 AM, Jean-Baptiste Lab jeanbaptiste@gmail.com wrote: Or wouldn't it be enough to simply mandate that exists() return something that can be tested against True/False? If that's the case, wouldn't a bit of wrapping around and implementing the __eq__/__neq__ descriptors (possibly __cmp__) be good enough so that we can get to the error description when needed (if False) without breaking existing usages? I did think about that. It's hard for me to imagine something that can test as false while still having a string value. Not impossible, but pretty weird and a bit un-pythonic. I prefer simplicity over cleverness. Still, if you have an idea, let me know. It might be a bit weird, granted, but I think it might be beneficial to sacrifice simplicity in that particular case so that the revamping of tools detection does not require a backward compatibility break of a possibly quite large public use-case. I'm currently at work and cannot focus on that much more, I'll try to think about it a bit more later on. Cheers, JB -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
What are your thoughts on infra to help provide a common mean to find tools for different platforms. I believe what I have in Parts for this work pretty well. It allow an extensible and generally easy way for one to define how to find a given tool version(s) for a some combination of host and target. Being able to update a file with information about a new version without having to modify existing code I have found to be a big win. Given cases the how SCons deal with the Intel Compler vs how Parts does. It has been very easy for me in Parts to support the intel compiler versions and different platforms such as x86, x86_64, phi (k1om),ia64 , and some other case I cannot talk about for different system ( as window, posix, mac, and some others…). Likewise I have little issue with msvc for x86,x86_64, arm as well as WDK cases. I believe we when we talk about the toolchains and tools we need to consider: 1) Host we are on 2) What target platform we are building for ( so we select the best tool) 3) What version ( normally use exactly this version, or best version of a certain version ie latest 4.x drop, or the latest). The point here is that a tool needs to have a version value ( it might be wild in certain cases, but the common case for most tools, is that you can have more than one version at a time installed) When we configure an environment we need to consider all a setting up the value via: 1) Some hand defaults 2) Processing some script 3) Allowing the user to saw just use the shell and trust me Make users like 3) the most as that is how make works, and so simple builds this is not so bad. But for cross building this is a mess, and for teams 1,2 become very useful as this allow for a duplicate-exact setup which means the builds are reproducible, cases like 3) require a copy-exact setup, and I generally see this get messed up to easily, causing wasted time on some strange issue, cause by some difference that was not duplicated correctly. I think what I have here in parts is generally useful. I also believe that for use to get tools and toolchain more correct, we need to deal with the platform better in SCons, I have a SystemPlatform object and API to allow adding new platform combination easily. Likewise I have a generally useful Version object. Jason From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Gary Oberbrunner Sent: Tuesday, September 9, 2014 9:38 AM To: SCons developer list Subject: Re: [Scons-dev] This morning's WTF moment This is python, so a get_last_error type API is really simple. It can just store the message in the tool class. Also, I expect exists() to be cached for speed (based on the tool's args) so when the tool doesn't exist it will need to store the message anyway, to return it to future callers. exists_error() just becomes an accessor for that member var. That basically means your double-negative version is no different than the regular version in terms of implementation, and I think the exists() + exists_error() API is nicer. I don't really like the error object, i.e. something that tests as False but has a string value. -- Gary On Tue, Sep 9, 2014 at 9:56 AM, Kenny, Jason L jason.l.ke...@intel.commailto:jason.l.ke...@intel.com wrote: We have two options it seems: 1) Add a exists_error_msg() like API. 2) Or return an error object. Thoughts on 1): The main value with 1) is the exists is very simple in that it returns True or False. The main issue with it is that it is a bit more complex to implement a get last error like API. There are a number possible Issues such as memory waste can easily become an problem, Not having a simple API to use this for a tool writer I would see as being needed to help keep this logic easy to implement from the tool writers point of view and not adding resource and or correctness issues to SCons. Or the implementation may just need to re-test. So maybe the way to view this API is to instead view it like the inverse ie.. not_exist() ie this would return a string ( ie the message) when it does not exists and None when it does exists. Then exist would always be a impl like: def exists(): return not_exists() When the user wants to see why it does not exist they would call the not_exists() api. Which would provide a message. If used directly the logic of If !not_exists(): will work correctly ( yes it a double neg here.. ewww) This might be a way to do this, without a special object while keeping it simpler to implement. Thoughts on 2) The value of an error object is that it can work be easily made to work in basic testing cases, while allowing the returning of extra information such as information about what is wrong. It could be mixed with use of True, as in this case we most likely will not have a message to share whereas false we will. The main con to an error object is that the user has to return this object, which
Re: [Scons-dev] This morning's WTF moment
On Tue, Sep 9, 2014 at 11:08 AM, Kenny, Jason L jason.l.ke...@intel.com wrote: What are your thoughts on infra to help provide a common mean to find tools for different platforms. I believe what I have in Parts for this work pretty well. It allow an extensible and generally easy way for one to define how to find a given tool version(s) for a some combination of host and target. Being able to update a file with information about a new version without having to modify existing code I have found to be a big win. Given cases the how SCons deal with the Intel Compler vs how Parts does. It has been very easy for me in Parts to support the intel compiler versions and different platforms such as x86, x86_64, phi (k1om),ia64 , and some other case I cannot talk about for different system ( as window, posix, mac, and some others…). Likewise I have little issue with msvc for x86,x86_64, arm as well as WDK cases. I believe we when we talk about the toolchains and tools we need to consider: 1) Host we are on 2) What target platform we are building for ( so we select the best tool) 3) What version ( normally use exactly this version, or best version of a certain version ie latest 4.x drop, or the latest). The point here is that a tool needs to have a version value ( it might be wild in certain cases, but the common case for most tools, is that you can have more than one version at a time installed) When we configure an environment we need to consider all a setting up the value via: 1) Some hand defaults 2) Processing some script 3) Allowing the user to saw just use the shell and trust me Make users like 3) the most as that is how make works, and so simple builds this is not so bad. But for cross building this is a mess, and for teams 1,2 become very useful as this allow for a duplicate-exact setup which means the builds are reproducible, cases like 3) require a copy-exact setup, and I generally see this get messed up to easily, causing wasted time on some strange issue, cause by some difference that was not duplicated correctly. I deliberately want to avoid the complexity of most of what you're suggesting, at least at the most basic level. Allow people to build fancy structures on top if they want. The current proposal involves a tool registry (not yet adequately documented in the wiki, sorry) which will help with enumerating available (and unavailable) tools. Also, tools can take args, so it'll be possible to say intelc = ToolIntelC(abi='x86', version='11.1') to get specific ABIs, path usage, or whatever. This will be left up to the tool, but I assume some common conventions will appear. Your ideas about paths, using scripts, etc. could be represented as tool args. As far as making it easier to support different Intel compiler versions, I don't see any way to make that easier. Different versions use different version-numbering conventions, different Windows registry layouts, different directory layouts... I don't see any way to write common code to support them all. But that is not a problem I'm trying to solve for the Toolchain revamp. If a particular tool is painful or complex inside, that's it's problem. As long as it can present a simple interface to the outside world, that's good. Tools in this proposal also have versions themselves, but that's more to enable a global tool repository, so (someday) people could auto-install tools, auto-update them, etc. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
I think you are missing the point or maybe I am. Given the tool revamp. How will we support: 1) Cross-builds. a. I want to build 32-bit and 64- at the same time b. I want to build for android arm and x86 2) Selecting different versions of gcc? 3) How do I iterate over the different versions of a tool that are installed 4) How do I know this this gcc tool will build x86 code or x86_64 code, will it be android, mac or linux, phi, etc? 5) I want use gcc not clang with icc or I want to use a given version of gcc (or cl) with icc. For me the issue is that SCons makes this HARDER than it needs to be. What I am suggesting is that tools have certain traits. Not a lot, just some basic stuff, I am suggesting that we need to define in SCons these objects to make easy building blocks: 1) Platform Object – defines a system os, arch ( maybe more as it can be clearly defined). Used to define a HOST and TARGET value in the environment ( like in Parts) 2) Tools Object - defines a tool builder, basic variable, tells us information ( such as it exists), populates the env[ENV] with needed values to run. 3) Toolchain allows us to define changes, much like you define 4) Configuration – to allow one to easily define common setting to apply to a configured tool 5) Toolsetting/info/finders – a set of basic objects to help find information. You seem against this, but I suggest this as building blocks to make it easy for a tool to setup and cache, etc a given tool. The fact is that most tools have the same pattern and can be configured by replacing some basic values. 6) A version object. I know you might find this complex, but more complex version handling this is really useful. And honestly is a common need when one is making larger system. You can live without it, but having it is really nice, and reduces common errors. I think the idea of a tool registry is a great idea, as long as it can support different tools impls of the same tool in some way. Jason fyi As far as configuring the Intel compiler, there is something coming that will should make this better. Honestly this is moving a mountain…. From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Gary Oberbrunner Sent: Tuesday, September 9, 2014 10:24 AM To: SCons developer list Subject: Re: [Scons-dev] This morning's WTF moment On Tue, Sep 9, 2014 at 11:08 AM, Kenny, Jason L jason.l.ke...@intel.commailto:jason.l.ke...@intel.com wrote: What are your thoughts on infra to help provide a common mean to find tools for different platforms. I believe what I have in Parts for this work pretty well. It allow an extensible and generally easy way for one to define how to find a given tool version(s) for a some combination of host and target. Being able to update a file with information about a new version without having to modify existing code I have found to be a big win. Given cases the how SCons deal with the Intel Compler vs how Parts does. It has been very easy for me in Parts to support the intel compiler versions and different platforms such as x86, x86_64, phi (k1om),ia64 , and some other case I cannot talk about for different system ( as window, posix, mac, and some others…). Likewise I have little issue with msvc for x86,x86_64, arm as well as WDK cases. I believe we when we talk about the toolchains and tools we need to consider: 1) Host we are on 2) What target platform we are building for ( so we select the best tool) 3) What version ( normally use exactly this version, or best version of a certain version ie latest 4.x drop, or the latest). The point here is that a tool needs to have a version value ( it might be wild in certain cases, but the common case for most tools, is that you can have more than one version at a time installed) When we configure an environment we need to consider all a setting up the value via: 1) Some hand defaults 2) Processing some script 3) Allowing the user to saw just use the shell and trust me Make users like 3) the most as that is how make works, and so simple builds this is not so bad. But for cross building this is a mess, and for teams 1,2 become very useful as this allow for a duplicate-exact setup which means the builds are reproducible, cases like 3) require a copy-exact setup, and I generally see this get messed up to easily, causing wasted time on some strange issue, cause by some difference that was not duplicated correctly. I deliberately want to avoid the complexity of most of what you're suggesting, at least at the most basic level. Allow people to build fancy structures on top if they want. The current proposal involves a tool registry (not yet adequately documented in the wiki, sorry) which will help with enumerating available (and unavailable) tools. Also, tools can take args, so it'll be possible to say intelc = ToolIntelC(abi='x86
Re: [Scons-dev] This morning's WTF moment
On Tue, Sep 9, 2014 at 12:03 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: I think you are missing the point or maybe I am. Given the tool revamp. How will we support: 1) Cross-builds. a. I want to build 32-bit and 64- at the same time b. I want to build for android arm and x86 I have in mind something like this: # define and register the tools, by name Tool(name='intelc_32_arm', class=Tool.IntelC, abi='ia32', sys='arm') Tool.IntelC(name='intelc_64', class=Tool.IntelC, , abi='x86_64', sys='intel') # use them env1=Environment(tools=['intelc_32_arm', ...]) env2 Environment(tools=['intelc_64', ...]) 2) Selecting different versions of gcc? Same method. Tool(name='ancient_gcc', class=Tool.GCC, version='3.4') Environment(tools=['ancient_gcc']) 3) How do I iterate over the different versions of a tool that are installed I haven't defined a tool enumeration API yet, but since there's a registry that stores everything, it shouldn't be hard. 4) How do I know this this gcc tool will build x86 code or x86_64 code, will it be android, mac or linux, phi, etc? If you use the default version (don't pass special args to the tool), it ought to auto-detect the current platform. 5) I want use gcc not clang with icc or I want to use a given version of gcc (or cl) with icc. Use a toolchain: my_tools = Toolchain(['my_icc', 'ancient_gcc']) For me the issue is that SCons makes this HARDER than it needs to be. What I am suggesting is that tools have certain traits. Not a lot, just some basic stuff, I am suggesting that we need to define in SCons these objects to make easy building blocks: 1) Platform Object – defines a system os, arch ( maybe more as it can be clearly defined). Used to define a HOST and TARGET value in the environment ( like in Parts) I want to avoid having to define and implement this right now -- I think it's a fine idea, it's just orthogonal to revamping the _basic infrastructure_ of tools and chains. If we define a Platform object and Tool authors take a Platform as one arg for their tool: my_platform = Platform(...) Tool(name='my_icc', class=Tool.IntelC, platform=my_platform) then that is great. But LaTex, m4, SQL, and a million other tools wouldn't find that Platform object useful or important. We can layer it in later. 2) Tools Object - defines a tool builder, basic variable, tells us information ( such as it exists), populates the env[ENV] with needed values to run. Yes. I hope I am achieving this. 3) Toolchain allows us to define changes, much like you define I don't see it as defining changes. I see it as enabling grouping of tools into larger clusters or configurations or whatever you call it. Both AND (all must exist) and OR (select an alternative) are important. 4) Configuration – to allow one to easily define common setting to apply to a configured tool Not sure what you're getting at here. Do tool args help? 5) Toolsetting/info/finders – a set of basic objects to help find information. You seem against this, but I suggest this as building blocks to make it easy for a tool to setup and cache, etc a given tool. The fact is that most tools have the same pattern and can be configured by replacing some basic values. Are you talking about adding some utility methods to the base Tool class, like looking up things in paths and registry? I'm fine with that. 6) A version object. I know you might find this complex, but more complex version handling this is really useful. And honestly is a common need when one is making larger system. You can live without it, but having it is really nice, and reduces common errors. I think this would be nice. Again, not necessarily as part of the toolchain revamp, but yes if we had a version object that allowed flexible comparisons that would be very useful. I think the idea of a tool registry is a great idea, as long as it can support different tools impls of the same tool in some way. See above. My current implementation memoizes based on the tool's class and all the args passed to it. This assumes that the construction of a tool isn't dependent on external state, only the class and args. I actually think that restriction is useful to clarify and enforce the underlying concept. (Right now I have it so if you try to re-register the tool with the same args, it just returns the original.) If your tool should behave differently (when constructed, not just when applied to an env) then it should take an arg to indicate that. abi is an obvious one for C-like tools. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
Thanks Gary for your thoughts! I have a few thought about the response. But I think I would start off with just one item. When you look at what you have suggested, we have a cross build you suggest something to what I would think.. env1=Environment(tools=['intelc_32_arm', ...]) What is wrong with this is that is that the user can say this: env1=Environment(tools=['intelc_32_arm', ‘mslink_64_arm’]); This allow for a bad and confusing environment. Tools as I would see them care about the platform or they don’t ( as you point out many tools may not care as they don’t output data that is independent. Ie work on ‘any’ platform, but even in all your cases they are often 32 and 64 bit version of them. which one is being used?) Having the ability to mix 32-bit and 64-bit tools seems dangerous. That is why I suggest having the environment have a built in notion of HOST_PLATFORM and TARGET_PLATFORM. I think that having a restriction that and environment is configured for some host-target combination and that the tools configure themselves based on that value. Many tools output, or view of Target is a general ‘any’/’noarch’ I don’t care. However many of these may still value this information to help configure which tool to select. It does seem to me that we already have a BKM to try to do this. I agree that we could not do this, but I feel that this would add a larger burden on the user to do what is right. Given the samples so far, there is a suggestion of lots of tools, with possible random names. This could get confusing quickly. I fear that the error handing will become hard, as giving a clear message to the user that something is wrong and way will be very difficult and will temp many people to start defining tools in a way in which they try to know about other tools in unhealthy ways. The user at the end of the day just wants to say build this stuff with this tool chain for some platform. Ie ‘any’ in some cases, or for android, window, posix, mac, etc… I think one of the values of SCons is to be easy to use. We tell it what we need, and it does it. It has domain knowledge of the “creation” chain, so the user can work on their problem. Jason From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Gary Oberbrunner Sent: Tuesday, September 9, 2014 12:22 PM To: SCons developer list Subject: Re: [Scons-dev] This morning's WTF moment On Tue, Sep 9, 2014 at 12:03 PM, Kenny, Jason L jason.l.ke...@intel.commailto:jason.l.ke...@intel.com wrote: I think you are missing the point or maybe I am. Given the tool revamp. How will we support: 1) Cross-builds. a. I want to build 32-bit and 64- at the same time b. I want to build for android arm and x86 I have in mind something like this: # define and register the tools, by name Tool(name='intelc_32_arm', class=Tool.IntelC, abi='ia32', sys='arm') Tool.IntelC(name='intelc_64', class=Tool.IntelC, , abi='x86_64', sys='intel') # use them env1=Environment(tools=['intelc_32_arm', ...]) env2 Environment(tools=['intelc_64', ...]) 2) Selecting different versions of gcc? Same method. Tool(name='ancient_gcc', class=Tool.GCC, version='3.4') Environment(tools=['ancient_gcc']) 3) How do I iterate over the different versions of a tool that are installed I haven't defined a tool enumeration API yet, but since there's a registry that stores everything, it shouldn't be hard. 4) How do I know this this gcc tool will build x86 code or x86_64 code, will it be android, mac or linux, phi, etc? If you use the default version (don't pass special args to the tool), it ought to auto-detect the current platform. 5) I want use gcc not clang with icc or I want to use a given version of gcc (or cl) with icc. Use a toolchain: my_tools = Toolchain(['my_icc', 'ancient_gcc']) For me the issue is that SCons makes this HARDER than it needs to be. What I am suggesting is that tools have certain traits. Not a lot, just some basic stuff, I am suggesting that we need to define in SCons these objects to make easy building blocks: 1) Platform Object – defines a system os, arch ( maybe more as it can be clearly defined). Used to define a HOST and TARGET value in the environment ( like in Parts) I want to avoid having to define and implement this right now -- I think it's a fine idea, it's just orthogonal to revamping the _basic infrastructure_ of tools and chains. If we define a Platform object and Tool authors take a Platform as one arg for their tool: my_platform = Platform(...) Tool(name='my_icc', class=Tool.IntelC, platform=my_platform) then that is great. But LaTex, m4, SQL, and a million other tools wouldn't find that Platform object useful or important. We can layer it in later. 2) Tools Object - defines a tool builder, basic variable, tells us information ( such as it exists), populates the env[ENV] with needed values to run. Yes. I hope I am achieving this. 3
Re: [Scons-dev] This morning's WTF moment
On Tue, Sep 9, 2014 at 2:07 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: Thanks Gary for your thoughts! I have a few thought about the response. But I think I would start off with just one item. When you look at what you have suggested, we have a cross build you suggest something to what I would think.. env1=Environment(tools=['intelc_32_arm', ...]) What is wrong with this is that is that the user can say this: env1=Environment(tools=['intelc_32_arm', ‘mslink_64_arm’]); This allow for a bad and confusing environment. Tools as I would see them care about the platform or they don’t ( as you point out many tools may not care as they don’t output data that is independent. Ie work on ‘any’ platform, but even in all your cases they are often 32 and 64 bit version of them. which one is being used?) This is what a toolchain is for, IMHO. Someone (SCons, tool author or user) would set up proper toolchains including compiler, linker etc. Users would use those as if they were tools: env1 = Environment(tools=['intel_32_arm_toolchain']). Again, at this most basic tool level, we need to define mechanism and not restrict policy. Having the ability to mix 32-bit and 64-bit tools seems dangerous. Sure, so is mixing wrong versions of NVIDIA CUDA and gcc. Or MSVC compiler and Gnu linker. Or MySQL with Postgres, R with SAS. And so on and so on. That is why I suggest having the environment have a built in notion of HOST_PLATFORM and TARGET_PLATFORM. I dislike global things like this in general. But I understand the idea behind it, so let me think about it for a while. It may be we can come up with a compromise, like adding a set of kwargs to a Toolchain that gets merged into all the Tools of that chain. The current syntax doesn't allow for that but maybe we can extend it. I think that having a restriction that and environment is configured for some host-target combination and that the tools configure themselves based on that value. Many tools output, or view of Target is a general ‘any’/’noarch’ I don’t care. Well, much more interestingly, many tools' idea of the Platform has nothing to do with what CPU it's running on. Might be GPU (NVIDIA/AMD), or network, or Amazon instance type, or SQL configuration... who knows. What you're proposing is too C-centric for a general build tool. Not saying it's not hugely useful in that C-centric context, just that it's a layer up from the base I'm trying to define now. However many of these may still value this information to help configure which tool to select. It does seem to me that we already have a BKM to try to do this. Not sure what a BKM is. So far I don't think we need (e.g.) a callback in an OR toolchain to decide which one to try first, given current machine state, but it's a possibility. Or maybe, even going beyond that, we could have a GeneratorToolchain that doesn't have a static list of tools, but runs a method to return its tool list. Hmm, that seems like it might cover a lot of use cases. I agree that we could not do this, but I feel that this would add a larger burden on the user to do what is right. Given the samples so far, there is a suggestion of lots of tools, with possible random names. This could get confusing quickly. I fear that the error handing will become hard, as giving a clear message to the user that something is wrong and way will be very difficult and will temp many people to start defining tools in a way in which they try to know about other tools in unhealthy ways. The user at the end of the day just wants to say build this stuff with this tool chain for some platform. Ie ‘any’ in some cases, or for android, window, posix, mac, etc… I think one of the values of SCons is to be easy to use. We tell it what we need, and it does it. It has domain knowledge of the “creation” chain, so the user can work on their problem. Yes. Proper definition of toolchains (including shipping plenty of useful ones) should help. Users who want to define their own chains will of course have to nail down what they want and be responsible for the results. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
I know that I am late in the talk here and some of this may have been answered already.. Windows has a sub-systems that run on the kernel. One of them is win32. ( there used to be a posix one FYI, and with win 8 there is a new winrt layer). So while it looks like a joke to on windows 64-bit to see win32 as a platform it is technically correct.. This is why I pushed a while ago to get (TARGET|HOST)_(OS|ARCH) added to SCons. I know that when this was proposed there had been different takes on what to do. .. ie do we have one object deal with this, or do we make different variable to deal with this. There are plus and minus to but cased for different use cases. This is why I added to Part the SystemPlatform object. To main goal is to enhance the target and host idea from the simple Scons PLATFORM concept. I believe this has been a great addition. And should be added in some way to SCon, it is a basic building block to make cross platform builds work better and to help make it easy to define more complex toolchain setups in an easy way. At anyrate SCons current imple of the tool should be fixed to correct the warning issue. The way I have some people fix this with Parts is to have them add a scons-site directory with an empty tools/defaults.py. Jason -Original Message- From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Russel Winder Sent: Saturday, September 6, 2014 4:43 AM To: SCons_Developers Subject: [Scons-dev] This morning's WTF moment OK so I am abandoning all my scruples and trying to get SCons running on my wife's Windows 7 machine so as to run some tests on Linux, OSX *and* Windows. I have now discovered that on a 64-bit laptop running Windows 7: Environment()['PLATFORM'] return win32. One assumes there is some existentialist humour present in this result? Also at every turn I am told: scons: warning: No version of Visual Studio compiler found – C/C++ compilers most likely not set correctly File … engine\SCons\Script\Main.py, line 602, in _scons_internal_warning Yes I know this, I haven't installed any C or C++ compilers on this machine, I don't need to be told at every turn. However not only does SCons insists on telling me this, it tells me twice, at Reading SConscript files phase *AND* Building targets phase. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
SO I am all for improving the Tools logic. This was a big part of the work I did in Parts. Given what I have, I know there are some more tweaks I would like to make. Is there a process in how to add proposal to this wiki page. I know I would like to propose a possible infra set of objects to make it easier to find and set up a working tools environment. ( ie what is need to run command correctly) Also a general statement. Do we want to say SCons errors or warns when a tool in a toolchain is not found. I have taken a view that it should error out with information. ( for example the user might have stated they want icc v12.1, parts might error out given that it is not installed tell the user that 13.1 was found not 12.1). I have found that warnings turn to noise more often than not and are ignored ( or missed as the text just scrolls to fast). When the “error” does happen later ( and it will) the user is annoyed that had time wasted. For me it seems to me that is a toolchain is not resolvable we need to error. I would also state that we want to allow define one toolchain per env. Some toolchains cannot be mixed. And having a different env just makes it work better. D and C++ seems to a common case here. However this is more complex, as different chains could be mixed as they are independent. Being able to define what toolchain to use up front, vs having a default chain ( which takes time and is a result of certain annoying warning on windows at time) seem to be a good solution, as we can provide chains, and allow then chain to complain is there are known incompatible issues. Jason From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Gary Oberbrunner Sent: Saturday, September 6, 2014 8:09 AM To: SCons developer list Subject: Re: [Scons-dev] This morning's WTF moment On Sat, Sep 6, 2014 at 8:41 AM, anatoly techtonik techto...@gmail.commailto:techto...@gmail.com wrote: I disagree. I am currently taking action on it. There is some documentation on the wiki describing my direction, and I'm writing some test code to explore further. I am afraid you're the only one who knows what you're doing. =) If you could paste a reference to this specific wiki location, I could change my mind, but so far I am definitely not in the list of people who are able to track this progress. http://www.scons.org/wiki/ToolchainRevamp (and related sub-pages). There was some mailing list discussion which I wanted to cut and paste into the discussion page but didn't get around to that yet. Admittedly this is not 100% up to date but it describes the general approach I'm investigating. I have a separate repo where I'm working on some test implementations, starting with a basic test framework for a new Tool base class and a ToolRegistry (my tasks for this weekend if I can get enough time). But it's nowhere near ready to share, which is why I just posted some pseudocode on that wiki page. I need to strike a balance between sharing the design and being transparent so people can give feedback, and trying things out. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
Conceptually, I like Jason's suggestions. I don't know how the 1 toolchain per env would work in practice. What about SWIG or JNI or other mixed builds? On Sep 8, 2014 12:19 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: SO I am all for improving the Tools logic. This was a big part of the work I did in Parts. Given what I have, I know there are some more tweaks I would like to make. Is there a process in how to add proposal to this wiki page. I know I would like to propose a possible infra set of objects to make it easier to find and set up a working tools environment. ( ie what is need to run command correctly) Also a general statement. Do we want to say SCons errors or warns when a tool in a toolchain is not found. I have taken a view that it should error out with information. ( for example the user might have stated they want icc v12.1, parts might error out given that it is not installed tell the user that 13.1 was found not 12.1). I have found that warnings turn to noise more often than not and are ignored ( or missed as the text just scrolls to fast). When the “error” does happen later ( and it will) the user is annoyed that had time wasted. For me it seems to me that is a toolchain is not resolvable we need to error. I would also state that we want to allow define one toolchain per env. Some toolchains cannot be mixed. And having a different env just makes it work better. D and C++ seems to a common case here. However this is more complex, as different chains could be mixed as they are independent. Being able to define what toolchain to use up front, vs having a default chain ( which takes time and is a result of certain annoying warning on windows at time) seem to be a good solution, as we can provide chains, and allow then chain to complain is there are known incompatible issues. Jason *From:* Scons-dev [mailto:scons-dev-boun...@scons.org] *On Behalf Of *Gary Oberbrunner *Sent:* Saturday, September 6, 2014 8:09 AM *To:* SCons developer list *Subject:* Re: [Scons-dev] This morning's WTF moment On Sat, Sep 6, 2014 at 8:41 AM, anatoly techtonik techto...@gmail.com wrote: I disagree. I am currently taking action on it. There is some documentation on the wiki describing my direction, and I'm writing some test code to explore further. I am afraid you're the only one who knows what you're doing. =) If you could paste a reference to this specific wiki location, I could change my mind, but so far I am definitely not in the list of people who are able to track this progress. http://www.scons.org/wiki/ToolchainRevamp (and related sub-pages). There was some mailing list discussion which I wanted to cut and paste into the discussion page but didn't get around to that yet. Admittedly this is not 100% up to date but it describes the general approach I'm investigating. I have a separate repo where I'm working on some test implementations, starting with a basic test framework for a new Tool base class and a ToolRegistry (my tasks for this weekend if I can get enough time). But it's nowhere near ready to share, which is why I just posted some pseudocode on that wiki page. I need to strike a balance between sharing the design and being transparent so people can give feedback, and trying things out. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
On Mon, Sep 8, 2014 at 12:19 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: SO I am all for improving the Tools logic. This was a big part of the work I did in Parts. Given what I have, I know there are some more tweaks I would like to make. Is there a process in how to add proposal to this wiki page. I know I would like to propose a possible infra set of objects to make it easier to find and set up a working tools environment. ( ie what is need to run command correctly) Also a general statement. Do we want to say SCons errors or warns when a tool in a toolchain is not found. I have taken a view that it should error out with information. ( for example the user might have stated they want icc v12.1, parts might error out given that it is not installed tell the user that 13.1 was found not 12.1). I have found that warnings turn to noise more often than not and are ignored ( or missed as the text just scrolls to fast). When the “error” does happen later ( and it will) the user is annoyed that had time wasted. For me it seems to me that is a toolchain is not resolvable we need to error. The current proposal is that a Tool's exists() should _return_ an error message but not throw or print anything. The toolchain logic above it can then test silently and decide what to do. I would also state that we want to allow define one toolchain per env. Some toolchains cannot be mixed. And having a different env just makes it work better. D and C++ seems to a common case here. However this is more complex, as different chains could be mixed as they are independent. Being able to define what toolchain to use up front, vs having a default chain ( which takes time and is a result of certain annoying warning on windows at time) seem to be a good solution, as we can provide chains, and allow then chain to complain is there are known incompatible issues. The current proposal is that a Toolchain is either AND (all must exist) or OR (first one wins). Toolchains can have other toolchains as members as well as Tools. Any element in a Toolchain can be marked as Optional (which means if it's in an AND toolchain it doesn't fail that toolchain if it doesn't exist). I have some simple test code for this working. I hope this architecture is flexible enough that we can have one master toolchain per system; that one would have sub-toolchains for C/C++ (which would consist of an OR toolchain for Intel, MSVC, gcc/mingw, gcc/cygwin and whatever else we want), SWIG, D, LaTeX, and whatever else we want. I'm also hoping (don't have any of this working or even really designed yet) to make it easy enough to replace or add to the default toolchain that we can make the default pretty minimal; users would add what they need. I also think it's flexible enough to give decent and appropriate error messages when the toolchain requirements aren't met. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
I should clarify that by toolchain I literally mean chain of tools that coexist.. What interesting about this is that a java toolchain or a c toolchain are a chain of tools by themselves, but can also be composed as one bigger chain. When I think of D and C/C++ ( or fortran and C/C++) I think chains that can coexist, but a tool in the chain may need to behave differently ( ie the linker), or we might want to have a special tool to replace the default link tool. There are other cases in which seem obvious to me at time, but I see confused a lot, is that we have a C tool chain for different compilers, ie the GCC toolchain has a compiler and linker in it, but the CC and LINK vars for GCC are very different than the ones used for MSVC or intelc or Clang. These have different default flags and configuration added as well completely different binaries and shell environments. So from a technical point of view I know in Parts I would want to look at tools chains as maybe having an notion of variations, ie CC.CL or CC.GCC or CC.INTEL. with certain properties to control how the variation is configured. ie I wanted to change the parts –toolchain=tool_ver logic to a –toolchain=[toolchain[@properity:value…],…] set up.. ie --tc=cc.cl@version:12mailto:--tc=cc.cl@version:12 or for intel compiler something like --tc=cc.icl@ver:13.*,mstools@ver:12mailto:--tc=cc.icl@ver:13.*,mstools@ver:12 which would setup latest version of 13 to work with msvc version 12 ( ie VS 2013) Just some thought I have had… Jason From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of William Blevins Sent: Monday, September 8, 2014 1:47 PM To: SCons developer list Subject: Re: [Scons-dev] This morning's WTF moment Conceptually, I like Jason's suggestions. I don't know how the 1 toolchain per env would work in practice. What about SWIG or JNI or other mixed builds? On Sep 8, 2014 12:19 PM, Kenny, Jason L jason.l.ke...@intel.commailto:jason.l.ke...@intel.com wrote: SO I am all for improving the Tools logic. This was a big part of the work I did in Parts. Given what I have, I know there are some more tweaks I would like to make. Is there a process in how to add proposal to this wiki page. I know I would like to propose a possible infra set of objects to make it easier to find and set up a working tools environment. ( ie what is need to run command correctly) Also a general statement. Do we want to say SCons errors or warns when a tool in a toolchain is not found. I have taken a view that it should error out with information. ( for example the user might have stated they want icc v12.1, parts might error out given that it is not installed tell the user that 13.1 was found not 12.1). I have found that warnings turn to noise more often than not and are ignored ( or missed as the text just scrolls to fast). When the “error” does happen later ( and it will) the user is annoyed that had time wasted. For me it seems to me that is a toolchain is not resolvable we need to error. I would also state that we want to allow define one toolchain per env. Some toolchains cannot be mixed. And having a different env just makes it work better. D and C++ seems to a common case here. However this is more complex, as different chains could be mixed as they are independent. Being able to define what toolchain to use up front, vs having a default chain ( which takes time and is a result of certain annoying warning on windows at time) seem to be a good solution, as we can provide chains, and allow then chain to complain is there are known incompatible issues. Jason From: Scons-dev [mailto:scons-dev-boun...@scons.orgmailto:scons-dev-boun...@scons.org] On Behalf Of Gary Oberbrunner Sent: Saturday, September 6, 2014 8:09 AM To: SCons developer list Subject: Re: [Scons-dev] This morning's WTF moment On Sat, Sep 6, 2014 at 8:41 AM, anatoly techtonik techto...@gmail.commailto:techto...@gmail.com wrote: I disagree. I am currently taking action on it. There is some documentation on the wiki describing my direction, and I'm writing some test code to explore further. I am afraid you're the only one who knows what you're doing. =) If you could paste a reference to this specific wiki location, I could change my mind, but so far I am definitely not in the list of people who are able to track this progress. http://www.scons.org/wiki/ToolchainRevamp (and related sub-pages). There was some mailing list discussion which I wanted to cut and paste into the discussion page but didn't get around to that yet. Admittedly this is not 100% up to date but it describes the general approach I'm investigating. I have a separate repo where I'm working on some test implementations, starting with a basic test framework for a new Tool base class and a ToolRegistry (my tasks for this weekend if I can get enough time). But it's nowhere near ready to share, which is why I just posted some pseudocode on that wiki page
Re: [Scons-dev] This morning's WTF moment
On Mon, Sep 8, 2014 at 4:13 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: Ideally I always viewed this as a True False statement. I see you have it returning a tuple. I only worry that I have seen a lot push with certain python developers to say stuff like if not tool.exists(): # do something… This will not work as we will have a (True,””) or (False,””) return API. This seems to me to more complex to use and understand. At the very least east to trip up on. If we want an object returned. I think it will be better to define a error object that can be tested as True or False vs forcing tuple separation on returns values. Excellent point. The 'if not tool.exists()' pattern needs to work. I'll rethink that. Maybe something as simple as tool.exist_error() which can be called just after exists() returns False... -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
This might be obvious, but it the exception pattern not popular in python? On Sep 8, 2014 9:19 PM, Gary Oberbrunner ga...@oberbrunner.com wrote: On Mon, Sep 8, 2014 at 4:13 PM, Kenny, Jason L jason.l.ke...@intel.com wrote: Ideally I always viewed this as a True False statement. I see you have it returning a tuple. I only worry that I have seen a lot push with certain python developers to say stuff like if not tool.exists(): # do something… This will not work as we will have a (True,””) or (False,””) return API. This seems to me to more complex to use and understand. At the very least east to trip up on. If we want an object returned. I think it will be better to define a error object that can be tested as True or False vs forcing tuple separation on returns values. Excellent point. The 'if not tool.exists()' pattern needs to work. I'll rethink that. Maybe something as simple as tool.exist_error() which can be called just after exists() returns False... -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
On Sat, Sep 6, 2014 at 5:42 AM, Russel Winder rus...@winder.org.uk wrote: OK so I am abandoning all my scruples and trying to get SCons running on my wife's Windows 7 machine so as to run some tests on Linux, OSX *and* Windows. I have now discovered that on a 64-bit laptop running Windows 7: Environment()['PLATFORM'] return win32. One assumes there is some existentialist humour present in this result? It's the Windows Way -- sys.platform() also returns win32 on 64-bit machines; system files are in /Windows/System32, and so on and so on. Also at every turn I am told: scons: warning: No version of Visual Studio compiler found – C/C++ compilers most likely not set correctly File … engine\SCons\Script\Main.py, line 602, in _scons_internal_warning Yes, known problem. The right solution is the toolchain revamp. A less invasive solution is surprisingly hard to find, though Anatoly has a possible idea. If you initialize your Environment with only the tools of interest, you won't see that warning. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] This morning's WTF moment
On Sat, Sep 6, 2014 at 3:06 PM, Dirk Bächle tshor...@gmx.de wrote: On 06.09.2014 13:59, Gary Oberbrunner wrote: This should be described before moving further and before I can support any talks about revamp. Sorry if you don't support it, but I'm making progress nonetheless. :-) +1 from me! =) ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
