Re: Make [was Re: SCons and gdc]
On Tuesday, 30 October 2012 at 09:56:08 UTC, Russel Winder wrote: On Tue, 2012-10-30 at 10:08 +0100, Rob T wrote: […] Also scons has no built-in ability to scan subfolders for source files. You can manually specify sub-level source files, but that's unreasonable to do for large projects. You can write custom Python code to scan subfolders, but that is a lot more complicated than it should be. Do you know of a decent solution to scan sub-folders? Using os.walk is quite natural in SCons since SCons is just a Python internal DSL. SConstruct and SConscript are internal DSL files (not external DSL as Makefiles are), thus they are just Python programs. Scons does look like a very powerful tool, and I really do appreciate your input and the other posters as well. I am determined not to continue with Make, so maybe I will have to keep trying to get scons to do what I want. Mayhap then what you need to do is look at Parts. This is Jason Kenny's superstructure over SCons to deal with huge projects scattered across everywhere. This is the build framework Intel uses for their stuff. Intel have more and bigger C and C++ projects than I think anyone else around One more question: I am wondering how the scons D support locates dependencies from the imports specfied in the source files? So far I have not been able to get automatic dependency inclusions to work, so I end up manually specifying the import files. With C++ and Make, I could get gcc to scan source files for the #include files, and dump the list to a dependency file (.dep), and then I could include the .dep file(s) into the make process. Can scons with D support do something similar, or deal with it better? Now this is a whole different issue! I have no idea. If it is a problem it needs fixing. The general philosophy in SCons is to have scanners that find the dependencies. In C and C++ this is the awful #include. In D it's import. The code should create a graph of the dependencies that can be walked to create the list of inputs. There is a bit of code in the D tooling that is supposed to do this. If it doesn't then we need a bug report and preferably a small project exhibiting the problem that can be entered into the test suite — SCons development is obsessively TDD. I can only do stuff for Linux and OS X, I cannot do anything for Windows, and Windows is the "big problem" for the SCons D tooling at the moment which is why it has not got merged into the SCons mainline for release in 2.3.0. A plea for help: in order for SCons to have sane support for D, it needs people to step up and actively assist evolve the tools either by providing small test projects exhibiting fails, or running the tests on Windows and providing patches for fails. Currently I am not working in D, just in Python, Java, Groovy, Scala, Ceylon, Kotlin, so I have no D code activity I can use to evolve the D support in SCons. But if the D community can provide test cases, I can get SCons evolved to do the needful. But only on Linux and OS X, I definitely need a collaborator interested in working on Windows. I also belong to the JVM/.NET languages at work camp, but since I use both Windows and UNIX based systems, I could execute Windows tests if required. -- Paulo
Re: Make [was Re: SCons and gdc]
Rob T wrote:
> On Tuesday, 30 October 2012 at 09:42:50 UTC, Russel Winder wrote:
>> I would suggest you haven't given SCons long enough to get into the
>> SCons idioms, but you have to go with the route that is most comfortable
>> for you.
>
> You are right, I'll shut up and keep at it. I get frustrated sometimes,
> but that's no reason to vent in here. Sorry.
>
>> I do not see why you need to scan subfolders recursively. This is a
>> traditional Make approach. SCons does things differently even for highly
>> hierarchical systems. In particular use of SConscript files handles
>> everything. So I do not think this is a SCons problem.
>
> The only reason is to automate the construction of a list of source
> files for building. I'm used to using automated build scripts, which
> require only a minimal of manual input. For example, if I add a new
> source file to the mix, then I am expecting to not have to modify a
> SConstruct file to include it.
>
> I see mention of placing a SConsript file in each subdir, but that's a
> fair amount of manual overhead to bother with. so there must be another
> way?
>
The following SConstruct will scan all subfolders of the current
folder for d sources and compile them into a "foo" program.
==8<--
import os
sources = []
for dirpath, dirnames, filenames in os.walk ("."):
sources += [ os.path.join (dirpath, f)
for f in filenames
if f.endswith (".d") ]
Program ("foo", sources)
-->8==
Jerome
--
mailto:jeber...@free.fr
http://jeberger.free.fr
Jabber: jeber...@jabber.fr
signature.asc
Description: OpenPGP digital signature
Re: Make [was Re: SCons and gdc]
On Tuesday, 30 October 2012 at 09:56:08 UTC, Russel Winder wrote: Mayhap then what you need to do is look at Parts. This is Jason Kenny's superstructure over SCons to deal with huge projects scattered across everywhere. This is the build framework Intel uses for their stuff. Intel have more and bigger C and C++ projects than I think anyone else around For the record, my projects are not very small but also not very big (whatever that means), mid size I suppose? I'll have a look at Parts, but I have a hard case of Bleeding Edge Syndrome, and I may not survive. Now this is a whole different issue! I have no idea. If it is a problem it needs fixing. The general philosophy in SCons is to have scanners that find the dependencies. In C and C++ this is the awful #include. In D it's import. The code should create a graph of the dependencies that can be walked to create the list of inputs. There is a bit of code in the D tooling that is supposed to do this. If it doesn't then we need a bug report and preferably a small project exhibiting the problem that can be entered into the test suite — SCons development is obsessively TDD. So far I've had no indication that it's working. Perhaps it is only working if the import files are not located in subfolders? Will it account for something like this? import a.b; There must be something you did with the D support to give scons the ability to scan for imports. I know that #include can have sub-folders, which is common to see, such as #include So scons must be able to include dependencies located in sub-folders. I can only do stuff for Linux and OS X, I cannot do anything for Windows, and Windows is the "big problem" for the SCons D tooling at the moment which is why it has not got merged into the SCons mainline for release in 2.3.0. Currently I'm Linux only, but I have in the past built fairly complicated projects in Windows and I expect to be doing this again. This is another reason why I considered scons as a build solution because it is cross-platform. A plea for help: in order for SCons to have sane support for D, it needs people to step up and actively assist evolve the tools either by providing small test projects exhibiting fails, or running the tests on Windows and providing patches for fails. Once I get things working well enough on Linux, I'll give it a try. Right now though, I'm bogged down with Linux, I need to make some breathing space first. BTW, I use virtualbox as well as KVM extensively to run different OS's on same machine, so maybe you can try that route to get a Windows VM running for tests? --rt
Re: Make [was Re: SCons and gdc]
On Tuesday, 30 October 2012 at 09:42:50 UTC, Russel Winder wrote: I would suggest you haven't given SCons long enough to get into the SCons idioms, but you have to go with the route that is most comfortable for you. You are right, I'll shut up and keep at it. I get frustrated sometimes, but that's no reason to vent in here. Sorry. I do not see why you need to scan subfolders recursively. This is a traditional Make approach. SCons does things differently even for highly hierarchical systems. In particular use of SConscript files handles everything. So I do not think this is a SCons problem. The only reason is to automate the construction of a list of source files for building. I'm used to using automated build scripts, which require only a minimal of manual input. For example, if I add a new source file to the mix, then I am expecting to not have to modify a SConstruct file to include it. I see mention of placing a SConsript file in each subdir, but that's a fair amount of manual overhead to bother with. so there must be another way? Of course if you have to do things recursively then os.walk is the function you need. The solutions I looked at did not mention os.walk, so I'll have a look. Thanks for the tip. As noted in the exchanges to which I cc you in everything I sent, SCons does insist on having all directories in use under the directory with the SConstruct – unless you have with Default. On reflection I am now much less worried about this that I was at first. Out of source tree I find essential in any build, SCons does this well, Make less so. Out of project tree builds I am now not really worried about. Yes, I see I can build out of the source tree, that's half the battle solved, but it still insists on building in the project tree, which I was hoping to also do away with. There's a disadvantage for me doing it in this way, so it's something of a step backwards for me (in terms of organizing things), which I'd rather not have to do, hence the fustration I've expressed. There must be a way to solve it somehow. I disagree. I find Make totally rigid and unyielding. Not to mention rooted in 1977 assumptions of code. Yes I agree that Make sucks, and I hope I won't offend anyone by saying that. ;) You don't need to install SCons to use it, you can use it from a clone directly using the bootstrap system. I have an alias alias scons='python /home/users/russel/Repositories/Mercurial/Masters/SCons_D_Tooling/bootstrap.py' Sounds great, but my lack of Python expertise means that I do not fully understanding how this will work for me. I'll diginto it ... Thanks for the input. --rt
Re: Make [was Re: SCons and gdc]
On Tue, 2012-10-30 at 10:08 +0100, Rob T wrote: […] > Also scons has no built-in ability to scan subfolders for source > files. You can manually specify sub-level source files, but > that's unreasonable to do for large projects. You can write > custom Python code to scan subfolders, but that is a lot more > complicated than it should be. Do you know of a decent solution > to scan sub-folders? Using os.walk is quite natural in SCons since SCons is just a Python internal DSL. SConstruct and SConscript are internal DSL files (not external DSL as Makefiles are), thus they are just Python programs. > Scons does look like a very powerful tool, and I really do > appreciate your input and the other posters as well. I am > determined not to continue with Make, so maybe I will have to > keep trying to get scons to do what I want. Mayhap then what you need to do is look at Parts. This is Jason Kenny's superstructure over SCons to deal with huge projects scattered across everywhere. This is the build framework Intel uses for their stuff. Intel have more and bigger C and C++ projects than I think anyone else around > One more question: I am wondering how the scons D support locates > dependencies from the imports specfied in the source files? So > far I have not been able to get automatic dependency inclusions > to work, so I end up manually specifying the import files. With > C++ and Make, I could get gcc to scan source files for the > #include files, and dump the list to a dependency file (.dep), > and then I could include the .dep file(s) into the make process. > Can scons with D support do something similar, or deal with it > better? Now this is a whole different issue! I have no idea. If it is a problem it needs fixing. The general philosophy in SCons is to have scanners that find the dependencies. In C and C++ this is the awful #include. In D it's import. The code should create a graph of the dependencies that can be walked to create the list of inputs. There is a bit of code in the D tooling that is supposed to do this. If it doesn't then we need a bug report and preferably a small project exhibiting the problem that can be entered into the test suite — SCons development is obsessively TDD. I can only do stuff for Linux and OS X, I cannot do anything for Windows, and Windows is the "big problem" for the SCons D tooling at the moment which is why it has not got merged into the SCons mainline for release in 2.3.0. A plea for help: in order for SCons to have sane support for D, it needs people to step up and actively assist evolve the tools either by providing small test projects exhibiting fails, or running the tests on Windows and providing patches for fails. Currently I am not working in D, just in Python, Java, Groovy, Scala, Ceylon, Kotlin, so I have no D code activity I can use to evolve the D support in SCons. But if the D community can provide test cases, I can get SCons evolved to do the needful. But only on Linux and OS X, I definitely need a collaborator interested in working on Windows. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Make [was Re: SCons and gdc]
On Tue, 2012-10-30 at 00:19 +0100, Rob T wrote: […] > I definitely do not like Make. The scripts are made out of > garbage, and maintaining garbage just produces more waste. > Unfortunately for me, my attempts to make use of scons is not > encouraging. It may be better than Make, but not enough for me to > settle down with it. I would suggest you haven't given SCons long enough to get into the SCons idioms, but you have to go with the route that is most comfortable for you. > The two problems I mentioned were encountered almost immediately. > These are inability to scan subfolders recursively, and inability > to build to a level above the source folder. I don't think that > neither requirement has anything to do with thinking in terms of > Make. It could be that solving these two deficiencies may be > enough to keep me going with scons, I don't know. I do not see why you need to scan subfolders recursively. This is a traditional Make approach. SCons does things differently even for highly hierarchical systems. In particular use of SConscript files handles everything. So I do not think this is a SCons problem. Of course if you have to do things recursively then os.walk is the function you need. […] > I don't think it's a bug, because it's actually documented as a > feature. It may however be a "bug" in terms of the assumptions > about how applications should be built. As noted in the exchanges to which I cc you in everything I sent, SCons does insist on having all directories in use under the directory with the SConstruct – unless you have with Default. On reflection I am now much less worried about this that I was at first. Out of source tree I find essential in any build, SCons does this well, Make less so. Out of project tree builds I am now not really worried about. […] > I have only used Make, and as bad as it is, at least I can scan > subfolders with one built-in command. But why do you want to do this? Why doesn't os.walk achieve what you need with SCons? […] > Scons is far too rigid with the assumptions it makes, and IMO > some of the assumptions are plain wrong. I disagree. I find Make totally rigid and unyielding. Not to mention rooted in 1977 assumptions of code. > For example, building to a location out of the source tree has > the obvious advantage that your source tree remains a source > tree. I don't understand how anyone can consider this unusual or > not necessary. If a source tree is to be a tree containing source > code, then recursive scanning and building out of the tree is an > essential requirement. I always build out of source tree using SCons, to do otherwise is insanity, yes Autotools I am looking at you. However I have a source directory in my project directory and can then have many build directories in the project directory. Building a project for multiple platforms makes this essential. SCons supports this very well with the Variant system. […] > You are correct, only Python, which on a Linux system is normally > installed by default. I was refering to the need to manually > build scons from from a source repository in order to get latest > D support. I know I'm in the bleeding edge zone when it comes to > D, so a certain amount of hacking is needed, but I'd like to > minimize it as much as possible. You don't need to install SCons to use it, you can use it from a clone directly using the bootstrap system. I have an alias alias scons='python /home/users/russel/Repositories/Mercurial/Masters/SCons_D_Tooling/bootstrap.py' […] > > Or fix SCons? > > I thought of that, however in order to fix scons, I would have to > learn a lot about scons, and also learn Python. The flaws that I > see with scons are so basic, I probably would not fit in with the > scons culture, so I see nothing but pain in trying to fix scons. > I'm also learning D, and would rather spend more of my time > learning D than something else. My only interest with scons is > for using it, not fixing it, and I have no interest in learning > Python. Please stick with the "I don't want to learn Python" as your reason for not working with SCons. That is fine. I have no problem with that. I do have a problem with you saying "the flaws with SCons are so basic". This is just FUD from a person who hasn't really learnt the SCons way of doing things. So the resolution here is to stop mud-slinging at SCons and say "I am not going to use SCons because it involve working with Python and I don't want to do that." Then people can respect your position. […] > > (*) Think SCons → Python → Monty Python. > > That's how I view most of what is going on in programming land. :-) -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_win
Re: Make [was Re: SCons and gdc]
On Tuesday, 30 October 2012 at 01:22:17 UTC, H. S. Teoh wrote:
To each his own, but I honestly don't see what's so difficult
about
this:
# src/SConscript
Import('env')
env.Program('mydprogram', ['main.d', 'abc.d', 'def.cc'])
# lib1/SConscript
Import('env')
env.Library('mylib1', ['mod1.d', 'mod2.d'])
# lib2/SConscript
Import('env')
env.Library('mylib2', ['mod2.d', 'mod3.d'])
# SConstruct
objdir = 'build'
env = Environment()
Export(env)
env.SConscript('src/SConscript', build_dir=objdir)
env.SConscript('lib1/SConscript', build_dir=objdir)
env.SConscript('lib2/SConscript', build_dir=objdir)
Main program in src/, two libraries in lib1, lib2, and
everything builds
in build/ instead of the respective source trees. No problem. I
even
threw in a C++ file for kicks.
You are right, Make cannot do something like that in a reasonable
way, and it looks great.
You are describing one main project, with sub-level projects
inside, so the build dump is still going into the project tree.
This may work for some people, but it is frustrating for someone
who wishes to dump the build files outside of the project tree. I
guess there's just no solution to do different at this time. Not
what I want, but not a terminal problem either.
Also scons has no built-in ability to scan subfolders for source
files. You can manually specify sub-level source files, but
that's unreasonable to do for large projects. You can write
custom Python code to scan subfolders, but that is a lot more
complicated than it should be. Do you know of a decent solution
to scan sub-folders?
Scons does look like a very powerful tool, and I really do
appreciate your input and the other posters as well. I am
determined not to continue with Make, so maybe I will have to
keep trying to get scons to do what I want.
One more question: I am wondering how the scons D support locates
dependencies from the imports specfied in the source files? So
far I have not been able to get automatic dependency inclusions
to work, so I end up manually specifying the import files. With
C++ and Make, I could get gcc to scan source files for the
#include files, and dump the list to a dependency file (.dep),
and then I could include the .dep file(s) into the make process.
Can scons with D support do something similar, or deal with it
better?
--rt
Re: Make [was Re: SCons and gdc]
On Tue, Oct 30, 2012 at 12:19:46AM +0100, Rob T wrote:
[...]
> Scons is far too rigid with the assumptions it makes, and IMO some
> of the assumptions are plain wrong.
>
> For example, building to a location out of the source tree has the
> obvious advantage that your source tree remains a source tree. I
> don't understand how anyone can consider this unusual or not
> necessary. If a source tree is to be a tree containing source code,
> then recursive scanning and building out of the tree is an essential
> requirement.
>
> Scons however assumes that your source tree must be flat, and that
> your source tree must be polluted with build files.
I don't know where you got this assumption from, but it's plain wrong.
SCons supports out-of-source-tree builds. In fact, I have a project in
which I generate multiple builds from the same source tree (and I can
build *all* build variants in a single command, parallelized - something
that will cause make to keel over and die).
The only bug is that SCons assumes that the source tree and build
tree(s) must be under a common root, which may not be the case.
Nevertheless, even this is not a fatal problem, as you can just put your
SConstruct in the directory above the source tree, then you can build to
other subdirectories easily.
> >SCons depends only on Python. What are these other dependencies that
> >you speak of?
>
> You are correct, only Python, which on a Linux system is normally
> installed by default. I was refering to the need to manually build
> scons from from a source repository in order to get latest D support.
> I know I'm in the bleeding edge zone when it comes to D, so a certain
> amount of hacking is needed, but I'd like to minimize it as much as
> possible.
This is just what one puts up with when working with bleeding edge
technology. If there weren't kinks in the works, it'd be mainstream
already.
[...]
> >As far as I am aware there are no D coded build frameworks that can
> >handle C, C++, Fortran, D, LaTeX, Vala, Haskell, OCaml, Java, Scala.
>
> I'm currently only interested in building C++ and D, generalized
> tools that can manage multiple languages tend to be much more
> complex than I need.
To each his own, but I honestly don't see what's so difficult about
this:
# src/SConscript
Import('env')
env.Program('mydprogram', ['main.d', 'abc.d', 'def.cc'])
# lib1/SConscript
Import('env')
env.Library('mylib1', ['mod1.d', 'mod2.d'])
# lib2/SConscript
Import('env')
env.Library('mylib2', ['mod2.d', 'mod3.d'])
# SConstruct
objdir = 'build'
env = Environment()
Export(env)
env.SConscript('src/SConscript', build_dir=objdir)
env.SConscript('lib1/SConscript', build_dir=objdir)
env.SConscript('lib2/SConscript', build_dir=objdir)
Main program in src/, two libraries in lib1, lib2, and everything builds
in build/ instead of the respective source trees. No problem. I even
threw in a C++ file for kicks.
Now granted, SCons does have its own flaws, but railing about how
useless it is when one hasn't even bothered to learn what it can do
sounds rather unfair to me.
T
--
Ruby is essentially Perl minus Wall.
Re: Make [was Re: SCons and gdc]
On Saturday, 27 October 2012 at 18:11:30 UTC, Russel Winder wrote: Or it says you know Make but not SCons. All build frameworks have their computational models, idiosyncrasies , and points of pain. Make and SCons both have these. I definitely do not like Make. The scripts are made out of garbage, and maintaining garbage just produces more waste. Unfortunately for me, my attempts to make use of scons is not encouraging. It may be better than Make, but not enough for me to settle down with it. The problem most people have when moving from Make to SCons is that they think Make computational models and idioms. It takes a while to get over these and appreciate that SCons is very different from Make even though it is fundamentally the same. The two problems I mentioned were encountered almost immediately. These are inability to scan subfolders recursively, and inability to build to a level above the source folder. I don't think that neither requirement has anything to do with thinking in terms of Make. It could be that solving these two deficiencies may be enough to keep me going with scons, I don't know. Hummm... Whilst I am a fan of "out of source tree builds" I have always build within the project tree, so I have never noticed that trying to build in a directory that can only be reached from .. of the SConstruct appears to be impossible – without use of symbolic links. Will you put in the bug report or should I? I don't think it's a bug, because it's actually documented as a feature. It may however be a "bug" in terms of the assumptions about how applications should be built. Does Make? CMake, Autotools, Waf? I have only used Make, and as bad as it is, at least I can scan subfolders with one built-in command. Yes and no. Clearly there is a core of idiomatic things that every build framework should have. Then there is stuff that is unique. Scons is far too rigid with the assumptions it makes, and IMO some of the assumptions are plain wrong. For example, building to a location out of the source tree has the obvious advantage that your source tree remains a source tree. I don't understand how anyone can consider this unusual or not necessary. If a source tree is to be a tree containing source code, then recursive scanning and building out of the tree is an essential requirement. Scons however assumes that your source tree must be flat, and that your source tree must be polluted with build files. SCons depends only on Python. What are these other dependencies that you speak of? You are correct, only Python, which on a Linux system is normally installed by default. I was refering to the need to manually build scons from from a source repository in order to get latest D support. I know I'm in the bleeding edge zone when it comes to D, so a certain amount of hacking is needed, but I'd like to minimize it as much as possible. At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want. Or fix SCons? I thought of that, however in order to fix scons, I would have to learn a lot about scons, and also learn Python. The flaws that I see with scons are so basic, I probably would not fit in with the scons culture, so I see nothing but pain in trying to fix scons. I'm also learning D, and would rather spend more of my time learning D than something else. My only interest with scons is for using it, not fixing it, and I have no interest in learning Python. As far as I am aware there are no D coded build frameworks that can handle C, C++, Fortran, D, LaTeX, Vala, Haskell, OCaml, Java, Scala. I'm currently only interested in building C++ and D, generalized tools that can manage multiple languages tend to be much more complex than I need. (*) Think SCons → Python → Monty Python. That's how I view most of what is going on in programming land. --rt
Re: Make [was Re: SCons and gdc]
On Sat, 2012-10-27 at 00:15 +0200, Rob T wrote: > I'm trying to do a very simple build, but with scons I find > myself spending much more time with it (and getting nowhere) than > the actual coding, and that tells me that it's no better and may > be even worse than Make. Or it says you know Make but not SCons. All build frameworks have their computational models, idiosyncrasies , and points of pain. Make and SCons both have these. The problem most people have when moving from Make to SCons is that they think Make computational models and idioms. It takes a while to get over these and appreciate that SCons is very different from Make even though it is fundamentally the same. The short form is probably: Make is about a set of relationship rules and actions to achieve a target from a source. SCons is about specifying a toolchain and then construction a graph of the dependencies. > As an example of the sort of nonsense scons dishes out, I cannot > in a reasonable way, specify a build folder above my root source > folder, such a thing should be extremely easy to specify, but no > it isn't. Hummm... Whilst I am a fan of "out of source tree builds" I have always build within the project tree, so I have never noticed that trying to build in a directory that can only be reached from .. of the SConstruct appears to be impossible – without use of symbolic links. Will you put in the bug report or should I? > I also found that scons is lacking basic features that any build > tool should have. For example, when I tried to generate a list of > my source files, I discovered that scons it has no built in > ability to recursively scan a folder to include all subfolders. > WTF!? Does Make? CMake, Autotools, Waf? > I know I can write custom code in python to get around these > problems, but that will depeat the entire purpose of having a > build tool in the first place. Yes and no. Clearly there is a core of idiomatic things that every build framework should have. Then there is stuff that is unique. > scons also comes with a ton of dependency baggage that I simply > do not need, therefore I am giving up on it entirely. SCons depends only on Python. What are these other dependencies that you speak of? Giving up on SCons is fine, if it doesn't work for you. > At this point I'm considering looking at those old build tools > written in D, perhaps I can patch one of them up to get it to do > what I want. Or fix SCons? > If anyone has a suggestion as to which of the (I think) 2 or 3 > build tools coded in D looked the best, that would be appreciated. As far as I am aware there are no D coded build frameworks that can handle C, C++, Fortran, D, LaTeX, Vala, Haskell, OCaml, Java, Scala. Well SCons isn't very good at Java and Scala, but that is a whole different argument. Probably a full half hour and not 5 mins. (*) (*) Think SCons → Python → Monty Python. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Make [was Re: SCons and gdc]
On Saturday, 27 October 2012 at 03:33:37 UTC, H. S. Teoh wrote: On Sat, Oct 27, 2012 at 04:11:02AM +0200, RenatoUtsch wrote: On Friday, 26 October 2012 at 22:15:13 UTC, Rob T wrote: [...] >At this point I'm considering looking at those old build tools >written in D, perhaps I can patch one of them up to get it to >do >what I want. > >If anyone has a suggestion as to which of the (I think) 2 or 3 >build tools coded in D looked the best, that would be >appreciated. > >--rt I am currently learning more D to design a new one, that should really end the need for other ones. If you can wait 1 or 2 months for an alpha release... If anyone has any suggestion, I would be thankful. If you're going to write a new build tool, you should at least take a look at some of the ideas and concepts in tup: http://gittup.org/tup/ Do take some time to read the PDF under "additional info"; it is very insightful and contains some possibly revolutionary algorithms. Don't just reinvent the same old broken system that make was 30 years ago, just with different packaging and eye-candy. (Caveat: I've never used tup before. I just read up on it today, and was quite impressed by some of the innovations. Even if you decide to take another approach, you should at least be aware of what's cutting-edge these days so that you aren't just rehashing the same old stuff.) T Tup has some interesting concepts, maybe I can adapt them to my project. But I was thinking in making something more authomatized, like CMake or SCons but with tons of innovations.
Re: Make [was Re: SCons and gdc]
On Sat, Oct 27, 2012 at 04:11:02AM +0200, RenatoUtsch wrote: > On Friday, 26 October 2012 at 22:15:13 UTC, Rob T wrote: [...] > >At this point I'm considering looking at those old build tools > >written in D, perhaps I can patch one of them up to get it to do > >what I want. > > > >If anyone has a suggestion as to which of the (I think) 2 or 3 > >build tools coded in D looked the best, that would be appreciated. > > > >--rt > > I am currently learning more D to design a new one, that should > really end the need for other ones. If you can wait 1 or 2 months > for an alpha release... > > If anyone has any suggestion, I would be thankful. If you're going to write a new build tool, you should at least take a look at some of the ideas and concepts in tup: http://gittup.org/tup/ Do take some time to read the PDF under "additional info"; it is very insightful and contains some possibly revolutionary algorithms. Don't just reinvent the same old broken system that make was 30 years ago, just with different packaging and eye-candy. (Caveat: I've never used tup before. I just read up on it today, and was quite impressed by some of the innovations. Even if you decide to take another approach, you should at least be aware of what's cutting-edge these days so that you aren't just rehashing the same old stuff.) T -- Nothing in the world is more distasteful to a man than to take the path that leads to himself. -- Herman Hesse
Re: Make [was Re: SCons and gdc]
If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated. --rt Rob, I added D support to Premake (http://industriousone.com/premake) which is available here (https://bitbucket.org/goughy/premake-dev-d) which supports make & VStudio. Note that some parts are not feature complete yet (out of source builds) but _may_ work. Its a work in progress. Premake is written in Lua and packaged as a single executable so there are zero runtime dependencies. I'd appreciate feedback as to whether it works for your use cases. Goughy
Re: Make [was Re: SCons and gdc]
On Friday, 26 October 2012 at 22:15:13 UTC, Rob T wrote: I'm trying to do a very simple build, but with scons I find myself spending much more time with it (and getting nowhere) than the actual coding, and that tells me that it's no better and may be even worse than Make. As an example of the sort of nonsense scons dishes out, I cannot in a reasonable way, specify a build folder above my root source folder, such a thing should be extremely easy to specify, but no it isn't. I also found that scons is lacking basic features that any build tool should have. For example, when I tried to generate a list of my source files, I discovered that scons it has no built in ability to recursively scan a folder to include all subfolders. WTF!? I know I can write custom code in python to get around these problems, but that will depeat the entire purpose of having a build tool in the first place. scons also comes with a ton of dependency baggage that I simply do not need, therefore I am giving up on it entirely. At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want. If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated. --rt I am currently learning more D to design a new one, that should really end the need for other ones. If you can wait 1 or 2 months for an alpha release... If anyone has any suggestion, I would be thankful.
Re: Make [was Re: SCons and gdc]
On Wednesday, 24 October 2012 at 02:35:11 UTC, 1100110 wrote: I would say the above folder structure doesn't make much sense for a single program, unless those are third-party libs. Yes, that would be for storing separate libraries, each of which will be independent projects separate from the others. Of course other projects are expected to make use of the libraries, and some libraries may make use of other libraries, which means at least some of the projects will require access to the source interfaces of some of the others. I would expect that this kind of organization is standard practice, otherwise you'll have a lot of otherwise unnecessary difficulties keeping projects clean and identifiably separate. If anyone can suggest alternate ways of organizing projects, I'll be happy to read about it. --rt
Re: Make [was Re: SCons and gdc]
I'm trying to do a very simple build, but with scons I find myself spending much more time with it (and getting nowhere) than the actual coding, and that tells me that it's no better and may be even worse than Make. As an example of the sort of nonsense scons dishes out, I cannot in a reasonable way, specify a build folder above my root source folder, such a thing should be extremely easy to specify, but no it isn't. I also found that scons is lacking basic features that any build tool should have. For example, when I tried to generate a list of my source files, I discovered that scons it has no built in ability to recursively scan a folder to include all subfolders. WTF!? I know I can write custom code in python to get around these problems, but that will depeat the entire purpose of having a build tool in the first place. scons also comes with a ton of dependency baggage that I simply do not need, therefore I am giving up on it entirely. At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want. If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated. --rt
Re: Make [was Re: SCons and gdc]
On Tue, 23 Oct 2012 18:49:45 -0500, Rob T wrote: I may be getting messed up by the way modules map directly to folder structures, and how to separate an interface from the implementation. Eg. import A.m1; I understand that the above import requires that a folder "A" be found at the root level of the source folder (where ever it may be), and module m1.d must be located directly in A. So far so good, however A may be located in a folder outside of the current project. For example, the std lib imports are located outside you projects root folder (this is a goal to acheive what I want to reproduce with my own D prebuilt libraries). When I try to import from another project located in another set of folders, how do I tell the compiler to start looking at a certain point? For example /projects/ /p1 /src /A /B /C /p2 /src /D /E /F With the above folder structiure, how can I get this to work? import A.m1; import D.m2; I would say the above folder structure doesn't make much sense for a single program, unless those are third-party libs. The -I flag specifies where to look for imports, but go one folder above the A/ . So assuming we are in projects/: rdmd -Ip1/src -Ip2/src Notice there is no space between -I and the folder to look at. You could run the command from the bin/ dir if you have one, which will dump the obj in bin/ (rdmd -I../p1/src ...) or you can use the -of flag. It seems to be pretty similar to C, C++... I've used make on a project before, but make is a clusterfuck IMHO. I think I spent more time simply RTM for make than getting anything to work. I generally just create a file 'DBuild' with this: #!/bin/sh rdmd -IimportPath main.d And then modify it as necessary. I may be able to manually get something to work, but how can it be automated with scons? I know I'll have to manually supply the project search paths somehow, but will scons be able to figure out that "import D.m2;" means to look under /projects/p2/src/D/? I know nothing about scons. Also if I make an edit to D/m2.d will scons be able to figure out that D/m2.d needs to be rebuilt and/or that all files that import D/m2.d must be rebuilt? rdmd will, don't know anything about scons. In C/C++ full rebuilding is only required when header files (.h) are modified and included, not when the implementation is modified. How do we make the distinction between the interface and the implementation in D? Perhaps I should be building interface files (.di), how is that done and how do you refer to them after they are built? either -H flag or manually. -H pretty much just strips the comments since certain constructs need the full source. If you write a library and want to hide the source, then you can write .di files just like .h files, which include minimal source. But just like cpp macros, certain things (templates I believe are one) must be defined in the .di file. The functions can simply be declared. Check out the deimos project, which is a collection of bindings to C libraries. The .di files there should show you what is necessary. https://github.com/D-Programming-Deimos Nothing better than learn-by-example. Finally how do you specify an alternate folder for dumping the build stuff to separate it from being dumped into your source folders? Also how to you specify an installation folder, eg /usr/local/bin along with location of necessary import folders. I definitely do not want to install full source code so that imports will work, so I assume the .di files are installed instead. I just use shell scripts. or -of flag. Or running dmd from that directory, so it outputs there. There are many many ways to do this. Or use rdmd, which compiles and runs, but doesn't output obj files except in /tmp I know I'm asking a lot of basic questions which means I havn't much clue how to build D apps yet, so are there any good examples or documentation I can look at that will supply me with the answers? Phobos uses Make files I believe, vibe.d simply recompiles vibe.d plus your code usually. ps: I'm an experienced C++ programmer, so the tendancy is to replicate the same practice, however I'm definietly open to better ways that make the most out of D. Thanks for any help you gave give! --rt It's largely the same... There might be others who can give better advice, but it seems to be scons, shell files, dmd/rdmd flags, Make, CMake... not necessarily in order of preference. There's an old tool called dsss, but I've never had any luck with that one. I recommend running dmd --help and reading the output. It's pretty self explanatory. http://dlang.org/pretod.html#headerfiles That one might be useful. http://dlang.org/cpptod.html This one is specifically for C++ programmers. But neither o
Re: Make [was Re: SCons and gdc]
On Tue, 23 Oct 2012 18:49:45 -0500, Rob T wrote: I may be getting messed up by the way modules map directly to folder structures, and how to separate an interface from the implementation. Eg. import A.m1; I understand that the above import requires that a folder "A" be found at the root level of the source folder (where ever it may be), and module m1.d must be located directly in A. So far so good, however A may be located in a folder outside of the current project. For example, the std lib imports are located outside you projects root folder (this is a goal to acheive what I want to reproduce with my own D prebuilt libraries). When I try to import from another project located in another set of folders, how do I tell the compiler to start looking at a certain point? For example /projects/ /p1 /src /A /B /C /p2 /src /D /E /F With the above folder structiure, how can I get this to work? import A.m1; import D.m2; I may be able to manually get something to work, but how can it be automated with scons? I know I'll have to manually supply the project search paths somehow, but will scons be able to figure out that "import D.m2;" means to look under /projects/p2/src/D/? Also if I make an edit to D/m2.d will scons be able to figure out that D/m2.d needs to be rebuilt and/or that all files that import D/m2.d must be rebuilt? In C/C++ full rebuilding is only required when header files (.h) are modified and included, not when the implementation is modified. How do we make the distinction between the interface and the implementation in D? Perhaps I should be building interface files (.di), how is that done and how do you refer to them after they are built? Finally how do you specify an alternate folder for dumping the build stuff to separate it from being dumped into your source folders? Also how to you specify an installation folder, eg /usr/local/bin along with location of necessary import folders. I definitely do not want to install full source code so that imports will work, so I assume the .di files are installed instead. I know I'm asking a lot of basic questions which means I havn't much clue how to build D apps yet, so are there any good examples or documentation I can look at that will supply me with the answers? ps: I'm an experienced C++ programmer, so the tendancy is to replicate the same practice, however I'm definietly open to better ways that make the most out of D. Thanks for any help you gave give! --rt rdmd -Ip1/src -Ip2/src appThatImports.d Or dmd appThatImports.d p1/src/A/m1.d p2/src/B/m2.d. But you might still need the -I flag for the second, I don't remember... I prefer simply using rdmd and -I flags to figure out the import paths. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Make [was Re: SCons and gdc]
I may be getting messed up by the way modules map directly to folder structures, and how to separate an interface from the implementation. Eg. import A.m1; I understand that the above import requires that a folder "A" be found at the root level of the source folder (where ever it may be), and module m1.d must be located directly in A. So far so good, however A may be located in a folder outside of the current project. For example, the std lib imports are located outside you projects root folder (this is a goal to acheive what I want to reproduce with my own D prebuilt libraries). When I try to import from another project located in another set of folders, how do I tell the compiler to start looking at a certain point? For example /projects/ /p1 /src /A /B /C /p2 /src /D /E /F With the above folder structiure, how can I get this to work? import A.m1; import D.m2; I may be able to manually get something to work, but how can it be automated with scons? I know I'll have to manually supply the project search paths somehow, but will scons be able to figure out that "import D.m2;" means to look under /projects/p2/src/D/? Also if I make an edit to D/m2.d will scons be able to figure out that D/m2.d needs to be rebuilt and/or that all files that import D/m2.d must be rebuilt? In C/C++ full rebuilding is only required when header files (.h) are modified and included, not when the implementation is modified. How do we make the distinction between the interface and the implementation in D? Perhaps I should be building interface files (.di), how is that done and how do you refer to them after they are built? Finally how do you specify an alternate folder for dumping the build stuff to separate it from being dumped into your source folders? Also how to you specify an installation folder, eg /usr/local/bin along with location of necessary import folders. I definitely do not want to install full source code so that imports will work, so I assume the .di files are installed instead. I know I'm asking a lot of basic questions which means I havn't much clue how to build D apps yet, so are there any good examples or documentation I can look at that will supply me with the answers? ps: I'm an experienced C++ programmer, so the tendancy is to replicate the same practice, however I'm definietly open to better ways that make the most out of D. Thanks for any help you gave give! --rt
Re: Make [was Re: SCons and gdc]
On 23 October 2012 22:58, H. S. Teoh wrote: > On Tue, Oct 23, 2012 at 11:35:29PM +0200, Rob T wrote: > [...] >> Currently I am trying very hard to get rid of Make, really I don't >> have much choice because AFAIK there's no easy way to get a useful >> dependency list out of gdc or dmd that can be used with Make. There >> is an option to produce a dependency list, but the output seems to >> be useless because it does not include full path for some of the >> dependecies, and the format is wrong to boot (needs Make to run an >> editor to clean up), and cgd 4.7 has a bug with producing the output >> rendering it totally useless (I'll try and report this bug on the >> tracker, now that I have an account). > > SCons can figure out the dependencies without needing to be told > explicitly. That is, if it's working correctly. Currently I do have > multi-file D projects, but they haven't grown into multi-folder projects > yet, so admittedly I don't have too much experience in that area. > > >> Given the significant problems I'm experiencing, I really wonder how >> anyone is building anything of significance in D? Since it appears >> that significant apps are being built, I figure I'm trying to do >> things in a C/C++ way when I am expected to do things in a different >> "D way". > [...] > > Well, dmd tends to work best when given the full list of D files, as > opposed to the C/C++ custom of per-file compilation. (It's also faster > that way---significantly so.) The -op flag is your friend when it comes > to using dmd with multi-folder projects. > > And I just tried: gdc works with multiple files too. I'm not sure how > well it handles a full list of D files, though, if some of those files > may not necessarily be real dependencies. > > To compile multiple modules into one object file, you need to specify '-o', else it compiles one-at-a-time. Otherwise it should work pretty well... -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Re: Make [was Re: SCons and gdc]
On Tue, Oct 23, 2012 at 11:35:29PM +0200, Rob T wrote: [...] > Currently I am trying very hard to get rid of Make, really I don't > have much choice because AFAIK there's no easy way to get a useful > dependency list out of gdc or dmd that can be used with Make. There > is an option to produce a dependency list, but the output seems to > be useless because it does not include full path for some of the > dependecies, and the format is wrong to boot (needs Make to run an > editor to clean up), and cgd 4.7 has a bug with producing the output > rendering it totally useless (I'll try and report this bug on the > tracker, now that I have an account). SCons can figure out the dependencies without needing to be told explicitly. That is, if it's working correctly. Currently I do have multi-file D projects, but they haven't grown into multi-folder projects yet, so admittedly I don't have too much experience in that area. > Given the significant problems I'm experiencing, I really wonder how > anyone is building anything of significance in D? Since it appears > that significant apps are being built, I figure I'm trying to do > things in a C/C++ way when I am expected to do things in a different > "D way". [...] Well, dmd tends to work best when given the full list of D files, as opposed to the C/C++ custom of per-file compilation. (It's also faster that way---significantly so.) The -op flag is your friend when it comes to using dmd with multi-folder projects. And I just tried: gdc works with multiple files too. I'm not sure how well it handles a full list of D files, though, if some of those files may not necessarily be real dependencies. T -- If it breaks, you get to keep both pieces. -- Software disclaimer notice
Re: Make [was Re: SCons and gdc]
You know there's a really big problem to solve when getting Make to build properly can be more difficult than gettiing the program it is supposed to be building to work. My biggest problem with D right now, is that I cannot easily build anything of significance, even with Make which I know quite well, or scone, which I know next to nothing about at this point (just started using it). I've tried scons with the D support, but I have had the same issues as has been reported in here, so it's been a very rough ride. I may be having two problems, one with understanding how D apps are supposed to be built, and problems with the build tools that are still in development. Currently I am trying very hard to get rid of Make, really I don't have much choice because AFAIK there's no easy way to get a useful dependency list out of gdc or dmd that can be used with Make. There is an option to produce a dependency list, but the output seems to be useless because it does not include full path for some of the dependecies, and the format is wrong to boot (needs Make to run an editor to clean up), and cgd 4.7 has a bug with producing the output rendering it totally useless (I'll try and report this bug on the tracker, now that I have an account). Given the significant problems I'm experiencing, I really wonder how anyone is building anything of significance in D? Since it appears that significant apps are being built, I figure I'm trying to do things in a C/C++ way when I am expected to do things in a different "D way". Can anyone point me towards documentation that explains the best practices to build D apps? I'm not talking about simple one file programs, but multiple files that may have dependencies spanning across alternate project folders. --rt
Re: Make [was Re: SCons and gdc]
On Tue, 2012-10-23 at 14:17 +0200, Paulo Pinto wrote: […] > Even worse is having new generations of developers learning that > the 70's way of Make and Autotools is the way to go. I suppose one could attempt a (failed from the beginning) analogy with programming: 1950s approached gave way to 1970s approaches which gave way to 1990s approaches which gave way to 1930s approaches. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Make [was Re: SCons and gdc]
On 10/23/2012 11:06 AM, Russel Winder wrote: Waf and SCons work well across platforms for C, C++, D, Fortran, LaTeX, Vala, but not JVM-based languages. I wasn't familiar with SCons, though I'd heard you and others mention it before -- just took a look -- it looks like a promising tool. Thanks very much for the introduction!
Re: Make [was Re: SCons and gdc]
On Tuesday, 23 October 2012 at 09:07:34 UTC, Russel Winder wrote: On Mon, 2012-10-22 at 13:19 -0700, H. S. Teoh wrote: […] Well, this is just my biased arrogant opinion, but the root of the problem is that make is an antiquated overly-simplistic script that has long outlived its time, but due to historical reasons still survive festering under the layers of patches that it has acquired over the course of its sad life. Automake and its ilk are just yet another (system of) layer of patches upon the same broken system that doesn't address the fundamental design flaws in make. It's an edifice of cards that nobody dares touch because, well, it would take too much effort to reproduce all the tiny obscure cases it has been tweaked for over the years. But it's nonetheless a nigh unmaintainable fortress of cards that will collapse at the slightest provocation in the most unhelpful of ways. It's like implementing the whole of Windows 8 in K&R C. In this day and age, one would *think* we could do better, but no, this fossil from the 70's still shambles on, to the unnecessary suffering of countless generations of new programmers. Make was a revelation and a revolution in 1977. Surprisingly Make remains very useful for small tasks not requiring cross-platform portability. Autotools is very UNIX biased. CMake keeps Make going. Just. Waf and SCons work well across platforms for C, C++, D, Fortran, LaTeX, Vala, but not JVM-based languages. Gradle rules on the JVM, along with SBT and Leiningen. Gradle is also trying to invade the C++ space. Even worse is having new generations of developers learning that the 70's way of Make and Autotools is the way to go. -- Paulo
Re: Make [was Re: SCons and gdc]
On Mon, 2012-10-22 at 13:19 -0700, H. S. Teoh wrote: […] > > Well, this is just my biased arrogant opinion, but the root of the > problem is that make is an antiquated overly-simplistic script that has > long outlived its time, but due to historical reasons still survive > festering under the layers of patches that it has acquired over the > course of its sad life. Automake and its ilk are just yet another > (system of) layer of patches upon the same broken system that doesn't > address the fundamental design flaws in make. It's an edifice of cards > that nobody dares touch because, well, it would take too much effort to > reproduce all the tiny obscure cases it has been tweaked for over the > years. But it's nonetheless a nigh unmaintainable fortress of cards that > will collapse at the slightest provocation in the most unhelpful of > ways. It's like implementing the whole of Windows 8 in K&R C. In this > day and age, one would *think* we could do better, but no, this fossil > from the 70's still shambles on, to the unnecessary suffering of > countless generations of new programmers. > Make was a revelation and a revolution in 1977. Surprisingly Make remains very useful for small tasks not requiring cross-platform portability. Autotools is very UNIX biased. CMake keeps Make going. Just. Waf and SCons work well across platforms for C, C++, D, Fortran, LaTeX, Vala, but not JVM-based languages. Gradle rules on the JVM, along with SBT and Leiningen. Gradle is also trying to invade the C++ space. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
