Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Fri, 2014-08-08 at 19:07 +0200, Dirk Bächle wrote: Hi there, we have another user reporting trouble with SCons 2.3.2, in connection to the D tool...so it's time for us to act. I checked the sources, and the user's analysis appears to be partly correct: even when no D tool is present in the current system, the dmd tool gets loaded (meaning SCons.Tool.dmd.generate is executed). That should not be the case, and wasn't for the tests I made. However I only tested on Linux and OSX and this user is using Windows, a platform I cannot test on and no-one else bothered to. In general, the D tools use their own set of variables starting with D* (or _D*) so they shouldn't create much of a problem. But the D tools also set STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME to 0, independent of any setting that a tool like msvs.py made before. :( The idea was to keep the D tool names separate, except for where smart_link is involved. The issue here is that D stands in the same relationship to C and C++ and D. So what is good for Fortran is good for D and vice versa, there should be symmetry. I'm open for a discussion and would like to hear other ideas. My way to fix this would be: - D tools shouldn't be loaded by default, but get an optional tool ( - tools=['default', 'dmd'] is mandatory). It's not enough to say Let's try to detect them, and load them automatically., as long as the STATIC_SHARED flag gets set to a fixed value. An option would be to use env.Default() to set this flag, but I don't know if D can then cope with a possible setting of 1 (Russel?). Then the Fortran tools shouldn't either. Of course why should any tool be loaded by default? - Docs should get amended, to warn the user about the overwrite of the STATIC_AND_SHARED flag. Or better still find a way of the bug not happening. - Then push out a new patch release as soon as possible. Definitely. So what is needed is a volunteer who used C and C++ on Windows who has two machines one with D installed and one without, who can run the tests from a feature clone SCons repository. I am away from home for 2.5 weeks so have no access to OSX. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Sat, 2014-08-09 at 01:12 +0300, anatoly techtonik wrote: […] There is an explicit check on dmd binary. What is dmd binary then? https://bitbucket.org/scons/scons/commits/b757fe34f9fe#Lsrc/engine/SCons/Tool/dmd.pyF233T135 DMD is one of the three D compilers, the one that is actually the mainline one. GDC is D in the GCC system, LDC is D on LLVM. In general, the D tools use their own set of variables starting with D* (or _D*) so they shouldn't create much of a problem. But the D tools also set STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME to 0, independent of any setting that a tool like msvs.py made before. :( Why D uses global keyword that is reserved for C stuff? Either because it was necessary or because I made an error. If an error (likely I suspect) we should find the right solution and put it in place. I'm open for a discussion and would like to hear other ideas. My way to fix this would be: - D tools shouldn't be loaded by default, but get an optional tool ( - tools=['default', 'dmd'] is mandatory). It's not enough to say Let's try to detect them, and load them automatically., as long as the STATIC_SHARED flag gets set to a fixed value. An option would be to use env.Default() to set this flag, but I don't know if D can then cope with a possible setting of 1 (Russel?). +1 on disabling it by default for the performance reasons listed here https://bitbucket.org/scons/scons/commits/b757fe34f9fe#chg-src/engine/SCons/Tool/__init__.py Which is an argument for getting rid of Fortran, C++ and C, and also LaTeX, actually all the tools, from begin loaded by default. - Docs should get amended, to warn the user about the overwrite of the STATIC_AND_SHARED flag. - Then push out a new patch release as soon as possible. The problems are caused by this commit: https://bitbucket.org/scons/scons/commits/b757fe34f9fe which is huge (which is bad), which is also a source of this bug As we know Gary and I are Git people who like transient feature branches that can be packaged for merge and are not Mercurial experts. The D changes extended over a very long period in a Mercurial feature fork with me keeping the fork up to date with the mainline. On merge Mercurial was unable to cope with this, creating zillions of spurious crap. Really we do not have a good workflow and that is at the heart of the big commit. I just made a single commit of all the D changes from over a two year period because it was the only way to not have a huge internal Mercurial mess. If there is a workflow that works with Mercurial for SCons then we need education and a document explaining what to do. http://scons.tigris.org/issues/show_bug.cgi?id=2966 I think this and the current problem may be related since the word def appears in both problems. in which Gary accepts that the commit is large, but since he is the one who merged it, I guess it didn't pass proper review https://bitbucket.org/scons/scons/commits/40fa954282a3893b809eb4782efe231a95ded10f and the worse part that was already reverted once https://bitbucket.org/scons/scons/commits/8764000345e06e326ef68fd0acf9366c1f3eb885 which raises a question about our review process and debt of competence. I feel quite competent actually. Most of the blame for this can be laid squarely at the door of Mercurial not being a good fit for managing the SCons repository with the people and tools we have. If on the other hand people think Gary and I acted improperly, then there is always pistols at dawn? I made a quick review too, and noticed a new default tool, but the fact that D tool became new default tool that *changes default behavior* is something that I didn't think might happen. But what it is not reflected in CHANGES.txt I think is a major ommision. It doesn't change the default behaviour, or at least shouldn't. What is being changed that you notice? Now a big thing. CHANGES.txt explicitly says Tools for DMD, GDC and LDC provided and integrated with the C and C++ linking. That should say something for people who now C/C++ (i.e. not to me). Looking at pull request, I don't see tests that cover this integration on C and C++ side. The tests need to be updated in any case. This is a very sensible suggestion and one well worth following up. Feel free to create some and do pull requests or send me suggestions and I will do some. I will see what I can do, but it will be a couple of weeks tie yet. About solving regressions with documentation - I am not a C/C++ coder, but I am not a fan of dealing with things this way. Nor am I. This is not a regression, but the bugs need to be fixed. As noted elsewhere the bugs come from Windows people not Linux or OSX people highlighting that no-on bothered to test 2.3.2 on Windows before it was released. I cannot since I don't have Windows, something I repeatedly brought to people's notice about this long ago. -- Russel.
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
I'll be home in a few days. I have plenty of Windows machines and can do the required tests. As for not noticing that that change broke Windows def files, I should have checked that. (There was a new failure in the Windows buildbot but I didn't notice it.) As for the large change, Russel has it right. Git would have made that much simpler. And Anatoly, please be nice. We're all volunteers here, trying to do a good job with limited time available. -- Gary Oberbrunner (sent from my Android) On Aug 9, 2014 2:23 PM, Russel Winder rus...@winder.org.uk wrote: On Sat, 2014-08-09 at 01:12 +0300, anatoly techtonik wrote: […] There is an explicit check on dmd binary. What is dmd binary then? https://bitbucket.org/scons/scons/commits/b757fe34f9fe#Lsrc/engine/SCons/Tool/dmd.pyF233T135 DMD is one of the three D compilers, the one that is actually the mainline one. GDC is D in the GCC system, LDC is D on LLVM. In general, the D tools use their own set of variables starting with D* (or _D*) so they shouldn't create much of a problem. But the D tools also set STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME to 0, independent of any setting that a tool like msvs.py made before. :( Why D uses global keyword that is reserved for C stuff? Either because it was necessary or because I made an error. If an error (likely I suspect) we should find the right solution and put it in place. I'm open for a discussion and would like to hear other ideas. My way to fix this would be: - D tools shouldn't be loaded by default, but get an optional tool ( - tools=['default', 'dmd'] is mandatory). It's not enough to say Let's try to detect them, and load them automatically., as long as the STATIC_SHARED flag gets set to a fixed value. An option would be to use env.Default() to set this flag, but I don't know if D can then cope with a possible setting of 1 (Russel?). +1 on disabling it by default for the performance reasons listed here https://bitbucket.org/scons/scons/commits/b757fe34f9fe#chg-src/engine/SCons/Tool/__init__.py Which is an argument for getting rid of Fortran, C++ and C, and also LaTeX, actually all the tools, from begin loaded by default. - Docs should get amended, to warn the user about the overwrite of the STATIC_AND_SHARED flag. - Then push out a new patch release as soon as possible. The problems are caused by this commit: https://bitbucket.org/scons/scons/commits/b757fe34f9fe which is huge (which is bad), which is also a source of this bug As we know Gary and I are Git people who like transient feature branches that can be packaged for merge and are not Mercurial experts. The D changes extended over a very long period in a Mercurial feature fork with me keeping the fork up to date with the mainline. On merge Mercurial was unable to cope with this, creating zillions of spurious crap. Really we do not have a good workflow and that is at the heart of the big commit. I just made a single commit of all the D changes from over a two year period because it was the only way to not have a huge internal Mercurial mess. If there is a workflow that works with Mercurial for SCons then we need education and a document explaining what to do. http://scons.tigris.org/issues/show_bug.cgi?id=2966 I think this and the current problem may be related since the word def appears in both problems. in which Gary accepts that the commit is large, but since he is the one who merged it, I guess it didn't pass proper review https://bitbucket.org/scons/scons/commits/40fa954282a3893b809eb4782efe231a95ded10f and the worse part that was already reverted once https://bitbucket.org/scons/scons/commits/8764000345e06e326ef68fd0acf9366c1f3eb885 which raises a question about our review process and debt of competence. I feel quite competent actually. Most of the blame for this can be laid squarely at the door of Mercurial not being a good fit for managing the SCons repository with the people and tools we have. If on the other hand people think Gary and I acted improperly, then there is always pistols at dawn? I made a quick review too, and noticed a new default tool, but the fact that D tool became new default tool that *changes default behavior* is something that I didn't think might happen. But what it is not reflected in CHANGES.txt I think is a major ommision. It doesn't change the default behaviour, or at least shouldn't. What is being changed that you notice? Now a big thing. CHANGES.txt explicitly says Tools for DMD, GDC and LDC provided and integrated with the C and C++ linking. That should say something for people who now C/C++ (i.e. not to me). Looking at pull request, I don't see tests that cover this integration on C and C++ side. The tests need to be updated in any case. This is a very sensible suggestion and one well worth following up. Feel free to create some and do pull requests or send me
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
Hi Russel, thanks for the quick reply. On 09.08.2014 14:02, Russel Winder wrote: On Fri, 2014-08-08 at 19:07 +0200, Dirk Bächle wrote: Hi there, we have another user reporting trouble with SCons 2.3.2, in connection to the D tool...so it's time for us to act. I checked the sources, and the user's analysis appears to be partly correct: even when no D tool is present in the current system, the dmd tool gets loaded (meaning SCons.Tool.dmd.generate is executed). That should not be the case, and wasn't for the tests I made. However I only tested on Linux and OSX and this user is using Windows, a platform I cannot test on and no-one else bothered to. Can you do the following test please: - In the D tools of your local installation, change the names of the executables that get detected...such that they can't possibly be found in the system (in exists() rename dmd - foodmd, ...). - Then create a simple SConstruct: env = Environment() print env.Dump() - Run through SCons, and observe that the Environment has D variables (= dmd.py Tool got loaded anyway) This happens, because the line d_compiler = FindTool(d_compilers, env) or d_compilers[0] adds dmd to the tool list anyway. In the default Tool's generate() method, the returned list is simply loaded without checking the existence of tools again. It is okay, and wanted, for as many Tools to get loaded by default...as long as their variables don't clash. The STATIC_AND_SHARED variable is only set by: - C/C++ compilers (where only the first match gets selected, based on the current platform) and - masm (only under Windows, where it sets the same value as a Windows C/C++ compiler - no clash) Only the D tools are forcing it back to 0, and this is not compatible with the setting that the msvc and mingw Tool make. This is the main culprit, and we either have to remove the explicit setting of STATIC_AND_SHARED from the D tools (relying on the C/C++ tools to do the right thing, or, if they really need the setting under Windows, we have to remove the D tools from the default toolchain. So, is STATIC_AND_SHARED=0 *required* for the D tools to work properly under Windows? Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 15:18, Gary Oberbrunner wrote: I'll be home in a few days. I have plenty of Windows machines and can do the required tests. As for not noticing that that change broke Windows def files, I should have checked that. (There was a new failure in the Windows buildbot but I didn't notice it.) As for the large change, Russel has it right. Git would have made that much simpler. And Anatoly, please be nice. We're all volunteers here, trying to do a good job with limited time available. Yep, I second that. Things like these just happen sometimes, so we can only try to be professional about it and get the issue out of the way asap. I also screwed up badly several times when trying to revamp the doc toolchain, so, Russel, please don't feel offended. You did a great job on the D integration, it's just this tiny issue that's left...so let's work things out. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
As we know Gary and I are Git people who like transient feature branches that can be packaged for merge and are not Mercurial experts. The D changes extended over a very long period in a Mercurial feature fork with me keeping the fork up to date with the mainline. On merge Mercurial was unable to cope with this, creating zillions of spurious crap. Really we do not have a good workflow and that is at the heart of the big commit. I just made a single commit of all the D changes from over a two year period because it was the only way to not have a huge internal Mercurial mess. +1, I'm new, so my voice counts less, but my past experiences with revision management tools is that GIT Mercurial in every way; Mercurial is more like distributed CVS (and CVS was a lifetime/nightmare ago). I could make a long rant here, but my primary point is that maybe a motion should be made for moving SCons to GIT? We could at least take a vote and see if anyone is adamantly opposed. I'm against using a hammer to do the job of a screwdriver. The problems are caused by this commit: https://bitbucket.org/scons/scons/commits/b757fe34f9fe which is huge (which is bad), which is also a source of this bug http://scons.tigris.org/issues/show_bug.cgi?id=2966 in which Gary accepts that the commit is large, but since he is the one who merged it, I guess it didn't pass proper review https://bitbucket.org/scons/scons/commits/40fa954282a3893b809eb4782efe231a95ded10f and the worse part that was already reverted once https://bitbucket.org/scons/scons/commits/8764000345e06e326ef68fd0acf9366c1f3eb885 which raises a question about our review process and debt of competence. Not too long ago, we had a 30+ email chain for a small 20-line patch, which you basically through a tantrum over. Ironically, it was reviewed for nearly two weeks and you hadn't bothered to care until it was merged. You jumped to conclusions and that mail chain turned out to be a weeks worth of effort just explaining that you were confused about what the patch was doing! I tried to be professional about it even though you were being pretty venomous. If you want to help, then help, but your comments are unwarranted. -William On Sat, Aug 9, 2014 at 8:23 AM, Russel Winder rus...@winder.org.uk wrote: On Sat, 2014-08-09 at 01:12 +0300, anatoly techtonik wrote: […] There is an explicit check on dmd binary. What is dmd binary then? https://bitbucket.org/scons/scons/commits/b757fe34f9fe#Lsrc/engine/SCons/Tool/dmd.pyF233T135 DMD is one of the three D compilers, the one that is actually the mainline one. GDC is D in the GCC system, LDC is D on LLVM. In general, the D tools use their own set of variables starting with D* (or _D*) so they shouldn't create much of a problem. But the D tools also set STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME to 0, independent of any setting that a tool like msvs.py made before. :( Why D uses global keyword that is reserved for C stuff? Either because it was necessary or because I made an error. If an error (likely I suspect) we should find the right solution and put it in place. I'm open for a discussion and would like to hear other ideas. My way to fix this would be: - D tools shouldn't be loaded by default, but get an optional tool ( - tools=['default', 'dmd'] is mandatory). It's not enough to say Let's try to detect them, and load them automatically., as long as the STATIC_SHARED flag gets set to a fixed value. An option would be to use env.Default() to set this flag, but I don't know if D can then cope with a possible setting of 1 (Russel?). +1 on disabling it by default for the performance reasons listed here https://bitbucket.org/scons/scons/commits/b757fe34f9fe#chg-src/engine/SCons/Tool/__init__.py Which is an argument for getting rid of Fortran, C++ and C, and also LaTeX, actually all the tools, from begin loaded by default. - Docs should get amended, to warn the user about the overwrite of the STATIC_AND_SHARED flag. - Then push out a new patch release as soon as possible. The problems are caused by this commit: https://bitbucket.org/scons/scons/commits/b757fe34f9fe which is huge (which is bad), which is also a source of this bug As we know Gary and I are Git people who like transient feature branches that can be packaged for merge and are not Mercurial experts. The D changes extended over a very long period in a Mercurial feature fork with me keeping the fork up to date with the mainline. On merge Mercurial was unable to cope with this, creating zillions of spurious crap. Really we do not have a good workflow and that is at the heart of the big commit. I just made a single commit of all the D changes from over a two year period because it was the only way to not have a huge internal Mercurial mess. If there is a workflow that works with Mercurial for SCons then we need education and a document explaining what
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 18:01, William Blevins wrote: As we know Gary and I are Git people who like transient feature branches that can be packaged for merge and are not Mercurial experts. The D changes extended over a very long period in a Mercurial feature fork with me keeping the fork up to date with the mainline. On merge Mercurial was unable to cope with this, creating zillions of spurious crap. Really we do not have a good workflow and that is at the heart of the big commit. I just made a single commit of all the D changes from over a two year period because it was the only way to not have a huge internal Mercurial mess. +1, I'm new, so my voice counts less, but my past experiences with revision management tools is that GIT Mercurial in every way; Mercurial is more like distributed CVS (and CVS was a lifetime/nightmare ago). I don't see it that way. I could make a long rant here, but my primary point is that maybe a motion should be made for moving SCons to GIT? We could at least take a vote and see if anyone is adamantly opposed. I'm against using a hammer to do the job of a screwdriver. Have to disagree again. Feel free to raise a poll for Git (I think we did that some time ago, and hg won...so why don't we stick to that?), but I don't see how this can help us. Work doesn't get less with another transition, and we just moved from svn-hg. All the changes in the docs, the changes in the basic workflows (branching, merging, releasing) and whatever, now start to settle finally...such that one could start to really do some proper work. I just put quite some energy into updating the Wiki pages about how to branch/merge with Mercurial...and you want to move *again*? I've seen pull requests with several commits on a branch, where the changes were very easy to follow. That's when people say: Oh, why don't you squash stuff together, so the log graph looks nicer. On other occasions, reviewers complained because the single commit was too big and too difficult to review. Well, yeah, that happens...it's life. The merge that Russel mentioned was actually, as far as I remember, successful and left the repo in a consistent state. It was again the only problem that the log graph looked a bit odd, based on Mercurial's inner workings. I really don't want to transition to another VCS again, only for that... -1 from me at the moment. Let's not concentrate on having pretty revision graphs, but on providing and releasing a software that works for our users... ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Sat, 2014-08-09 at 15:28 +0200, Dirk Bächle wrote: […] Can you do the following test please: - In the D tools of your local installation, change the names of the executables that get detected...such that they can't possibly be found in the system (in exists() rename dmd - foodmd, ...). Given everything is Debian package installed this means removing the package, I do not tinker with files in /usr/bin! :-) I think I am going to have to find a way of using Docker or equivalent to create new Linux systems without having a full virtual machine. - Then create a simple SConstruct: env = Environment() print env.Dump() - Run through SCons, and observe that the Environment has D variables (= dmd.py Tool got loaded anyway) I'll take OPs and your word for it. Actually I just tried it on my server and yes this is true. But… all the Fortran symbols are brought in as well and I have no Fortran compiler on that machine. So everything being said about D is true for Fortran as well. This happens, because the line d_compiler = FindTool(d_compilers, env) or d_compilers[0] adds dmd to the tool list anyway. In the default Tool's generate() method, the returned list is simply loaded without checking the existence of tools again. Indeed as OP said. However all the Fortran tools are being brought in as well even though I have no Fortran compiler, why is this happening and should it be fixed? It is okay, and wanted, for as many Tools to get loaded by default...as long as their variables don't clash. The STATIC_AND_SHARED variable is only set by: - C/C++ compilers (where only the first match gets selected, based on the current platform) and - masm (only under Windows, where it sets the same value as a Windows C/C++ compiler - no clash) Only the D tools are forcing it back to 0, and this is not compatible with the setting that the msvc and mingw Tool make. This is the main culprit, and we either have to remove the explicit setting of STATIC_AND_SHARED from the D tools (relying on the C/C++ tools to do the right thing, or, if they really need the setting under Windows, we have to remove the D tools from the default toolchain. So, is STATIC_AND_SHARED=0 *required* for the D tools to work properly under Windows? OK, this seems to be the real point here, the rest is, forgive me, flummery. The D tool should behave in the same way the Fortran tool does regarding this sort of thing and it seems Fortran does not touch this. Given that SCons is a C/C++ build framework with other stuff bolted on, then clearly the C/C++ stuff is special and has priority. I can see it was an error of mine to force a change of this symbol in what appears to be a by default situation. So I suggest the way forward is to remove the setting of this from the D tools and forget it for now — the reason I original set this a couple of years ago was due to a (mis-)feature of D that is changing very rapidly just now. How this is treated for D needs reviewing in a few months time when the next GDC and LDC compilers get formally released. Then what we need is a test for this that shows a fail on Windows without a D compiler as well as with — and the usual battery of tests on Linux and OSX with and without. It appears only to be a Windows problem and I cannot do anything requiring Windows. I have made the remove the six lines change in my Mercurial repository and the D tests all run as expected (*), I am just running the full of tests set now. I can make a pull request but without running a test highlighting the problem (which we don't actually have just now) run on a Windows machine (which I don't have) this doesn't mean very much. :-( (*) It still worries me that the tool for checking the presence of an executable for use in a SCons test uses the user PATH but the execution of the test uses the default PATH only. Thus there are test fails which can never pass on my machine even though everything works fine. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 19:33, William Blevins wrote: Fair; I realize its non-trivial. My complaints with Mercurial aren't related to one particular event. It's more a personal preference in tool usability. If HG suites the needs of the team, I'll just have to muddle through it. Again, you are right. This issue has nothing to do with Mercurial. Sorry, William No need to excuse, and it's totally fine to be a little biased. It's just that exactly these kind of discussions keep the core developers from their actual work...which might be one of the reasons why so many bugs in our tracker are untouched. Sorry, I just couldn't resist to give this wink. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Sat, 2014-08-09 at 15:33 +0200, Dirk Bächle wrote: […] Yep, I second that. Things like these just happen sometimes, so we can only try to be professional about it and get the issue out of the way asap. I also screwed up badly several times when trying to revamp the doc toolchain, so, Russel, please don't feel offended. You did a great job on the D integration, it's just this tiny issue that's left...so let's work things out. Exactly. We have two genuine problems that need fixing, so let's do it. In amongst all the bluster and anger, I think we have iterated to a very simple solution to both bugs: remove the setting of the symbol in the D tools since it is problematic and the Fortran tools do not touch it (even if they ought to be :-). I will commit the change and push to my Mercurial repository on BitBucket, but won't ask for a pull request, unless that would be helpful to people with Windows kit trying this out. We do need a test for this, but I am not sure what form it should take so all suggestions welcome. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 19:35, Russel Winder wrote: [...] This happens, because the line d_compiler = FindTool(d_compilers, env) or d_compilers[0] adds dmd to the tool list anyway. In the default Tool's generate() method, the returned list is simply loaded without checking the existence of tools again. Indeed as OP said. However all the Fortran tools are being brought in as well even though I have no Fortran compiler, why is this happening and should it be fixed? I think the reasoning behind this is, that if a Fortran compiler isn't found during construction of the Environment (because PATH is not setup properly), it gives the user a chance to directly set env['FORTRAN'] = /path/to/fc and get things going for him. Whether this was a good idea is quite debatable, but that would be my explanation for why things are as they are. [...] So I suggest the way forward is to remove the setting of this from the D tools and forget it for now — the reason I original set this a couple of years ago was due to a (mis-)feature of D that is changing very rapidly just now. How this is treated for D needs reviewing in a few months time when the next GDC and LDC compilers get formally released. Another option would be to not completely remove it, but use env.Default() such that the variable is only set if it doesn't exist yet. That should do the trick too. Thanks a lot for looking at this immediately. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Sat, 2014-08-09 at 13:33 -0400, William Blevins wrote: Fair; I realize its non-trivial. My complaints with Mercurial aren't related to one particular event. It's more a personal preference in tool usability. If HG suites the needs of the team, I'll just have to muddle through it. Again, you are right. This issue has nothing to do with Mercurial. Sorry, Please don't apologise for raising a perfectly reasonable point, there is nothing wrong with that. The history of this is that Python went with Mercurial and Steven wanted to go with Mercurial, so we went with Mercurial. At the time it seemed fine and reasonable and I had no problem with it. And in many ways still don't even though I prefer Git for the remote tracking branches and transient internal feature branches. (Well actually I still prefer Bazaar, but that is now an ex-DVCS, given it's level of support since Canonical have dropped funded support.) The core issue here is that Mercurial enforces maintaining the names of branches in all repositories. Thus the technique of internal feature branches fails with Mercurial. This leads to having branches only for long lived things such as maintenance branches. Everything else needs to be on default. This means clones for feature branches or use of bookmarks (MQ seems to be deprecated in favour of rebase and graft and things I don't understand about Mercurial – which includes bookmarks). Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. I am getting to the stage of not doing anything because any process I use leads to problems. I am thinking that we should use clones for feature branches synchronizing with the mainline and even publish them but before making the final pull request perform a full rebase so as to create a pull request sitting on the current mainline default/tip. Anatoly had, I believe, an alternative suggestion, but I have forgotten what it was. As he is more of a Mercurial expert, his suggestion may well be far more sensible. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 19:42, Russel Winder wrote: On Sat, 2014-08-09 at 15:33 +0200, Dirk Bächle wrote: […] In amongst all the bluster and anger, I think we have iterated to a very simple solution to both bugs: remove the setting of the symbol in the D tools since it is problematic and the Fortran tools do not touch it (even if they ought to be :-). I will commit the change and push to my Mercurial repository on BitBucket, but won't ask for a pull request, unless that would be helpful to people with Windows kit trying this out. I think it is extremely helpful to create this PR right now. The sooner we can test this in the mainline (and against our Buildbots), and the sooner users can pull a fixed version straight from the repo for testing, the better it is for us. We have clearly identified the issue and have a fix for it. Let's not wait... We do need a test for this, but I am not sure what form it should take so all suggestions welcome. I don't think we need an extra test. There are lots of D tool tests already, and the existing C/C++ tests actually triggered errors on the Windows buildslave...but nobody had a closer look because the buildbot stuff was in a bad state recently. So the all red appeared to be normal. This is a fix for a previous commit, so there doesn't have to be a test for it, IMO. The actual testing would be to have one Buildslave under Windows with D tools installed, and another one without. So again, feel free to leap ahead and remove the offending line...no tests required from my side. And if things go havoc this time, you can put the blame on me afterwards. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Sat, 2014-08-09 at 19:54 +0200, Dirk Bächle wrote: […] I think the reasoning behind this is, that if a Fortran compiler isn't found during construction of the Environment (because PATH is not setup properly), it gives the user a chance to directly set env['FORTRAN'] = /path/to/fc and get things going for him. Whether this was a good idea is quite debatable, but that would be my explanation for why things are as they are. If it's good enough for Fortran it's good enough for D. :-) I suggest that the primary issue is the smart_link which if I remember correctly only works properly if the infrastructure for C, C++, Fortran and D is all present. […] Another option would be to not completely remove it, but use env.Default() such that the variable is only set if it doesn't exist yet. That should do the trick too. I have implemented the removal for now, but it needs testing by someone who might see the problem, I cannot. Though the full tests do highlight some test fails: test/Deprecated/SourceCode/SourceCode.py test/Docbook/basic/epub/epub.py test/Docbook/basic/html/html.py test/Docbook/basic/htmlchunked/htmlchunked.py test/Docbook/basic/htmlhelp/htmlhelp.py test/Docbook/basic/man/man.py test/Docbook/dependencies/xinclude/xinclude.py test/Interactive/shell.py Most of the 105 skipped tests I understand, some have me worried, especially the Fortran ones and some of the LaTeX ones. Thanks a lot for looking at this immediately. It acted as entertainment and distraction from getting Python courses ready for presentation next week and in three weeks time! -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
The core issue here is that Mercurial enforces maintaining the names of branches in all repositories. Thus the technique of internal feature branches fails with Mercurial. This leads to having branches only for long lived things such as maintenance branches. Everything else needs to be on default. This means clones for feature branches or use of bookmarks (MQ seems to be deprecated in favour of rebase and graft and things I don't understand about Mercurial – which includes bookmarks). Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. Bookmarks have been alright, but only for like 1 or 2 small commits and trying to work on several small independent articles is frustrating. To get around this I either clone a new repo for that item or I bootstrap a GIT repo and manage everything that way until I'm done. It may be my misunderstanding of the way Mercurial was meant to be used. Too many important features come as plug-ins rather than base-line features. I have co-workers who use HG for their projects and I was trying to see I was just being silly, but there response lead me to believe that they cloned a new repo per work item. -William On Sat, Aug 9, 2014 at 2:05 PM, Russel Winder rus...@winder.org.uk wrote: On Sat, 2014-08-09 at 13:33 -0400, William Blevins wrote: Fair; I realize its non-trivial. My complaints with Mercurial aren't related to one particular event. It's more a personal preference in tool usability. If HG suites the needs of the team, I'll just have to muddle through it. Again, you are right. This issue has nothing to do with Mercurial. Sorry, Please don't apologise for raising a perfectly reasonable point, there is nothing wrong with that. The history of this is that Python went with Mercurial and Steven wanted to go with Mercurial, so we went with Mercurial. At the time it seemed fine and reasonable and I had no problem with it. And in many ways still don't even though I prefer Git for the remote tracking branches and transient internal feature branches. (Well actually I still prefer Bazaar, but that is now an ex-DVCS, given it's level of support since Canonical have dropped funded support.) The core issue here is that Mercurial enforces maintaining the names of branches in all repositories. Thus the technique of internal feature branches fails with Mercurial. This leads to having branches only for long lived things such as maintenance branches. Everything else needs to be on default. This means clones for feature branches or use of bookmarks (MQ seems to be deprecated in favour of rebase and graft and things I don't understand about Mercurial – which includes bookmarks). Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. I am getting to the stage of not doing anything because any process I use leads to problems. I am thinking that we should use clones for feature branches synchronizing with the mainline and even publish them but before making the final pull request perform a full rebase so as to create a pull request sitting on the current mainline default/tip. Anatoly had, I believe, an alternative suggestion, but I have forgotten what it was. As he is more of a Mercurial expert, his suggestion may well be far more sensible. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 20:05, Russel Winder wrote: On Sat, 2014-08-09 at 13:33 -0400, William Blevins wrote: [...] Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. I still don't feel the logic behind this reasoning. For me, the fact that the end result looks messy doesn't have anything to do with the workflow itself. I am getting to the stage of not doing anything because any process I use leads to problems. Oh Russel, I didn't know you're *that* sensible... ;) Please don't give up messing with our repos and minds! Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Sat, 2014-08-09 at 20:09 +0200, Dirk Bächle wrote: […] I think it is extremely helpful to create this PR right now. The sooner we can test this in the mainline (and against our Buildbots), and the sooner users can pull a fixed version straight from the repo for testing, the better it is for us. We have clearly identified the issue and have a fix for it. Let's not wait... Duly done, #166. We do need a test for this, but I am not sure what form it should take so all suggestions welcome. I don't think we need an extra test. There are lots of D tool tests already, and the existing C/C++ tests actually triggered errors on the Windows buildslave...but nobody had a closer look because the buildbot stuff was in a bad state recently. So the all red appeared to be normal. This is a fix for a previous commit, so there doesn't have to be a test for it, IMO. The actual testing would be to have one Buildslave under Windows with D tools installed, and another one without. So again, feel free to leap ahead and remove the offending line...no tests required from my side. And if things go havoc this time, you can put the blame on me afterwards. ;) Anatoly is right to point out that there are some claimed situations not supported by tests that should be supported by tests. So there ought to be more tests. I am not sure though there is a specific test for this case anyway apart from the ones there. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
[Scons-dev] Version control (was Re: Scons 2.3.2 regression, D tool...)
On Saturday, August 09, 2014 02:18:49 PM William Blevins wrote: The core issue here is that Mercurial enforces maintaining the names of branches in all repositories. Thus the technique of internal feature branches fails with Mercurial. This leads to having branches only for long lived things such as maintenance branches. Everything else needs to be on default. This means clones for feature branches or use of bookmarks (MQ seems to be deprecated in favour of rebase and graft and things I don't understand about Mercurial – which includes bookmarks). Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. Bookmarks have been alright, but only for like 1 or 2 small commits and trying to work on several small independent articles is frustrating. To get around this I either clone a new repo for that item or I bootstrap a GIT repo and manage everything that way until I'm done. It may be my misunderstanding of the way Mercurial was meant to be used. Too many important features come as plug-ins rather than base-line features. I have co-workers who use HG for their projects and I was trying to see I was just being silly, but there response lead me to believe that they cloned a new repo per work item. I've been using DVCs since 2003 (Tom Lord's larch, followed by tla, hg, bzr, and recently git). I've even looked at fossil and darcs. Git is the strange one of the bunch. There are things in git that go by the name of X that have things in the other DVCSs that are also named X that are almost naming the same concept. It's the details that throw you. Such as git thinks that history is malleable by default while none of the other DVCSs do. (That's why you have to turn on the rebase extension to hg; it's part of the base code, but using it changes the basic model.) My colleagues at work are moving from ClearCase UCM to git and they eff things up every day. (So do I. The git user interface is absolutely horrible.) A different colleague of mine moved from ClearCase UCM to Mercurial and hasn't had a problem. (In fairness, he's working pretty much by himself so I don't expect that his merge problems are that complicated.) http://mercurial.selenic.com/wiki/BranchingExplained describes the differences rather well, I think, on the branching strategies of the two DVCSs. A changeset (aka commit object) in hg does not have to have a named reference to survive. http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/ also has good information. From what I've read on hg's bookmark implementation and what I've seen recently on git's branching, I think that http://mercurial.selenic.com/wiki/Bookmarks is a very close analog. Note that the bookmark namespace is global, not local like the git branch namespace. -- Mark A. Flacy ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On Saturday, August 09, 2014 08:22:40 PM Dirk Bächle wrote: On 09.08.2014 20:05, Russel Winder wrote: On Sat, 2014-08-09 at 13:33 -0400, William Blevins wrote: [...] Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. I still don't feel the logic behind this reasoning. For me, the fact that the end result looks messy doesn't have anything to do with the workflow itself. Well, the messy bit is information that nobody really cares about; so why keep it around? -- Mark A. Flacy ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 20:28, Russel Winder wrote: On Sat, 2014-08-09 at 20:09 +0200, Dirk Bächle wrote: […] I think it is extremely helpful to create this PR right now. The sooner we can test this in the mainline (and against our Buildbots), and the sooner users can pull a fixed version straight from the repo for testing, the better it is for us. We have clearly identified the issue and have a fix for it. Let's not wait... Duly done, #166. Merged, Buildbots are through...and the .def errors are gone. Thanks again for the fix! So we can now point users to the latest dev version, if this error shows up for them. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
