Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 25, 2014 at 2:24 AM, anatoly techtonik techto...@gmail.comwrote: I am actually thinking about stripping Docbook toolchain altogether (better sooner than later), and move its maintenance into separate repo, because its addition tripled repository size for all subsequent versions. That seems a bit extreme to me, but perhaps I don't understand what you're proposing. I guess you're proposing to remove src/engine/SCons/Tool/docbook (i.e. most of commit 5ba470ff00b2) on the principle that it is just a copy of something that is source-controlled elsewhere? I could see that I guess. But it won't actually make a fresh hg clone much smaller since it'll still be in the history (unless we strip it, which I don't think is a good idea at all). Just removing that dir takes a clean checked out repo from 67MB to 48MB for me, which is not a huge savings. I think the whole repo is still very reasonable in size. -- Gary ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 25, 2014 at 5:17 PM, Gary Oberbrunner ga...@oberbrunner.com wrote: On Tue, Feb 25, 2014 at 2:24 AM, anatoly techtonik techto...@gmail.com wrote: I am actually thinking about stripping Docbook toolchain altogether (better sooner than later), and move its maintenance into separate repo, because its addition tripled repository size for all subsequent versions. That seems a bit extreme to me, but perhaps I don't understand what you're proposing. I guess you're proposing to remove src/engine/SCons/Tool/docbook (i.e. most of commit 5ba470ff00b2) on the principle that it is just a copy of something that is source-controlled elsewhere? I could see that I guess. Yes. Not only source controlled - it just doesn't belong. I'd like to see SCons lean and small handy utility - not an enterprisey XML driven monster. Having a clean history will help me to market for it properly and get additional proof that the project is well maintained. But it won't actually make a fresh hg clone much smaller since it'll still be in the history (unless we strip it, which I don't think is a good idea at all). Strip, right. Storing editor configuration in repository is a bad move, and it is also time that it takes to clone this stuff that is bad. It is hard to explain and you may think that I am a nitpicker, but I stripped branches during migration also because of these reasons and seeing 3x increase in size is like wasting some part of this work. Just removing that dir takes a clean checked out repo from 67MB to 48MB for me, which is not a huge savings. I think the whole repo is still very reasonable in size. I am not sure you're operating with a clean clone - here is the graph that I've got while measuring repository growth - see 3rd tab - http://goo.gl/ZOEc8e My Mercurial strip doesn't remove commit from the repository. It just merely hides it. I didn't try stripping it yet. Maybe you also have this behavior. -- anatoly t. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Wed, Feb 19, 2014 at 10:49 PM, Dirk Bächle tshor...@gmx.de wrote: On 19.02.2014 18:07, Bill Deegan wrote: Might I suggest we stop discussing it and just propose pull requests. If you have a specific change in mind, then make it and send a pull request. Yup, I'm all for it. @Anatoly: the commits that introduced the new doc toolchain are 8ca01af:0c9c8af (pull request #77, 2013-05-04). I am actually thinking about stripping Docbook toolchain altogether (better sooner than later), and move its maintenance into separate repo, because its addition tripled repository size for all subsequent versions. https://www.google.com/fusiontables/DataSource?docid=17oZw7PWFFDtmYpnj4JSJDSMZfrZM5z8hkfLpa8w I also find it a disadvantage that irrelevant lines from Docbook sources mess with grep results. This will desync all current pull requests, but will save a few minutes of developer time for every clone (change) that people make. Of course, ideally it will be good to merge these PR if not only me thinks that such cleanup is needed. -- anatoly t. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On 19.02.2014 06:15, Bill Deegan wrote: Anatoly, bootstrap.py is not meant to be run by users, only developers. -Bill I'd even go one step further and say: it's primarily meant to be run by release managers. It's okay if you take on this role for yourself as a developer while you're hacking away with things, but as far as I know there is nowhere documented that this is actually required from you. Nobody forces you now or has forced you in the past, to run this additional step, right? Or is it your understanding that every developer is required to run the full build scenario? I can understand that you are a little confused, and maybe even frustrated, because suddenly things that seemed to work for you show a different behaviour. But that's what happens. Time goes by and things change. And we want some changes for the SCons project, to make it better, right? And that's what we did, we made SCons better such that you don't have to write MAN pages by hand anymore for example. As a consequence of this, you simply don't get away anymore with what you did in the past: running only half of the packaging test without the documentation. But this is also a change to the better side and not meant to be against you personally. It reduces the work load for the actual release managers because errors in the documentation syntax are revealed much earlier in the development process. And you can still get back to your old routine and workflow and help the project even more and better than before, if you decide to take that little step and install the libxml2 or lxml Python bindings. And if you decide to not install it, and simply skip the full packaging build, that'll be fine with everyone too...and you can save even more of your time and invest it in development itself. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Wed, Feb 19, 2014 at 8:15 AM, Bill Deegan b...@baddogconsulting.com wrote: Anatoly, bootstrap.py is not meant to be run by users, only developers. I believe there is a terminology confusion. User is anybody who uses SCons. Developer is anybody with commit right to the SCons repository. When a person makes checkout of SCons to debug, check fixes or try new features, this person is still a user and this person needs a quick way to know that the SCons is not broken. Running tests takes a lt of time, so most users won't survive long enough to become developers. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Wed, 2014-02-19 at 16:44 +, Managan, Rob wrote: […] You can execute the local SCons directly from the src/ subdirectory by first setting the SCONS_LIB_DIR environment variable to the local src/engine subdirectory, and then executing the local src/script/scons.py script to populate the build/scons/ subdirectory. You would do this as follows on a Linux or UNIX system (using sh or a derivative like bash or kWh): $ setenv MYSCONS=`pwd`/src $ setenv SCONS_LIB_DIR=$MYSCONS/engine $ python $MYSCONS/script/scons.py [arguments] […] I guess the reason I haven't tried this is that it populates the source tree with .pyc files whereas using the bootstrap.py it leaves the source tree untouched. Maybe I should just get used to pollution :-) -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On 19.02.2014 18:07, Bill Deegan wrote: Might I suggest we stop discussing it and just propose pull requests. If you have a specific change in mind, then make it and send a pull request. Yup, I'm all for it. @Anatoly: the commits that introduced the new doc toolchain are 8ca01af:0c9c8af (pull request #77, 2013-05-04). Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
Hi Anatoly, On 18.02.2014 05:46, anatoly techtonik wrote: Why SCons bootstrap became dependent on external libraries? I find it a major usability regression. Can this be fixed? it didn't suddenly become dependent, it always was. We're now using DocBook, so we need to process and transform XML...that's the single dependency to either libxml2 or lxml. And it's not a dependency for the regular user, but the role of the developer... Before that we required the full list of required tools as given at the bottom of http://www.scons.org/wiki/DeveloperGuide/Documentation . If you'd switch to another tool like e.g. Sphinx, as you suggested under http://www.scons.org/wiki/DeveloperGuide/Documentation/Discussion, then that would be the new dependency. It's just switching names, so I don't see how this is any better... So, - the external dependencies have actually been reduced, and - no, I don't think it can't be fixed. ;) Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 18, 2014 at 11:31 AM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 05:46, anatoly techtonik wrote: Why SCons bootstrap became dependent on external libraries? I find it a major usability regression. Can this be fixed? it didn't suddenly become dependent, it always was. There is a mistake. The bootstrap process never require documentation tools to be present. hg up 2.3.0 bootstrap.py C:\Python27\python.exe R:\scons\bootstrap\src\script\scons.py scons: Reading SConscript files ... doc: jw not found, skipping building User Guide. doc: WARNING: no groff, skipping text and PostScript versions of man pages ... The bootstrap.py is a minimal build of SCons to bootstrap the full build of all the packages, as specified in our local SConstruct file. We're now using DocBook, so we need to process and transform XML...that's the single dependency to either libxml2 or lxml. And it's not a dependency for the regular user, but the role of the developer... Although that's not relevant. I think that we need to summarize Sphinx vs Docbook vs Semantic MediaWiki page at some place as well as previous workflow. We've already got a lot of info, but it is not compressed for public consumption. I feel that Sphinx is not ideal for our workflow, but I also feel that it is easier to augment Sphinx with our specific markup than to maintain Docbook toolchain. This needs to be more than just a feeling. Before that we required the full list of required tools as given at the bottom of http://www.scons.org/wiki/DeveloperGuide/Documentation . If you'd switch to another tool like e.g. Sphinx, as you suggested under http://www.scons.org/wiki/DeveloperGuide/Documentation/Discussion, then that would be the new dependency. It's just switching names, so I don't see how this is any better... So, - the external dependencies have actually been reduced, and - no, I don't think it can't be fixed. ;) Sphinx is pure Python dependency, but the point is that documentation dependencies for the bootstrap script should remain optional as it already was before, so that SCons was self-sufficient to build itself and run from checkout. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On 18.02.2014 12:12, anatoly techtonik wrote: [...] There is a mistake. The bootstrap process never require documentation tools to be present. Correct, the bootstrap process doesn't require doc tools...but the SConstruct at the top-level does. So unless you call bootstrap.py from the top-level source dir everything should be fine, do you agree? For the other case see my comments below please. hg up 2.3.0 bootstrap.py C:\Python27\python.exe R:\scons\bootstrap\src\script\scons.py scons: Reading SConscript files ... doc: jw not found, skipping building User Guide. doc: WARNING: no groff, skipping text and PostScript versions of man pages ... The bootstrap.py is a minimal build of SCons to bootstrap the full build of all the packages, as specified in our local SConstruct file. Doesn't the full build of all the packages,... include the full documentation? If yes, then it requires one of the libxml2/lxml bindings installed...and I don't see a problem with that. If not, what command do I have to call (assuming the role of a release manager) to get a full release build, creating all packages such that they're ready for shipping with the guarantee that no files are missing? Dirk P.S.: So, - the external dependencies have actually been reduced, and - no, I don't think it can't be fixed. ;) The last item was supposed to read: I don't think it can be fixed. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 18, 2014 at 8:57 PM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 12:12, anatoly techtonik wrote: [...] There is a mistake. The bootstrap process never require documentation tools to be present. Correct, the bootstrap process doesn't require doc tools...but the SConstruct at the top-level does. Right. SConstruct now requires doc tools, which wasn't the case for 2.3.0, was it? So unless you call bootstrap.py from the top-level source dir everything should be fine, do you agree? No. =) The previous behavior allowed me to run bootstrap, which together with SConstruct is a quick integration test that SCons works ok. It is not a replacement for runtest.py, but really saves a lot of time. hg up 2.3.0 bootstrap.py C:\Python27\python.exe R:\scons\bootstrap\src\script\scons.py scons: Reading SConscript files ... doc: jw not found, skipping building User Guide. doc: WARNING: no groff, skipping text and PostScript versions of man pages ... The bootstrap.py is a minimal build of SCons to bootstrap the full build of all the packages, as specified in our local SConstruct file. Doesn't the full build of all the packages,... include the full documentation? Just update to 2.3.0 and check. I believe that the answer is no. Also, because role of bootstrap.py script is to bootstrap the build. Having to fetch dependencies manually diminishes its purpose. In any case - make it optional - is a good principle. I need documentation in my browser, and Google is a better helper that files on disk, so I really want to leave documentation toolchain optional as it was before. If yes, then it requires one of the libxml2/lxml bindings installed...and I don't see a problem with that. If not, what command do I have to call (assuming the role of a release manager) to get a full release build, creating all packages such that they're ready for shipping with the guarantee that no files are missing? You need to ensure that there are no warnings during the build process and the warning about missing documentation build is among those that you especially should not ignore as a release manager. ;) ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On 18.02.2014 19:59, anatoly techtonik wrote: [...] You need to ensure that there are no warnings during the build process and the warning about missing documentation build is among those that you especially should not ignore as a release manager. ;) I could live with both variants for the top-level build (packaging n' stuff): a) The default is to build and package SCons as far as the tools and requirements for the single steps are met. Documentation may get built and archives might get packaged and tested or not, depending on which tools/modules you have installed in your system. The focus is on trying out integration with a minimum of effort, especially regarding running time of the build. b) The default is to guarantee a correct build, which means that all packages get created such that they contain all required files and documents. All these packages pass their tests, especially the ones for self-containment, and are ready to get shipped. If one or more of these goals are not met, the build breaks. So I'd like to let the actual release managers decide. Just chime in guys... Personally, I'd always do the full build anyway. Sorry, but I simply don't believe in getting half-pregnant. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 18, 2014 at 10:54 PM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 19:59, anatoly techtonik wrote: [...] You need to ensure that there are no warnings during the build process and the warning about missing documentation build is among those that you especially should not ignore as a release manager. ;) I could live with both variants for the top-level build (packaging n' stuff): a) The default is to build and package SCons as far as the tools and requirements for the single steps are met. Documentation may get built and archives might get packaged and tested or not, depending on which tools/modules you have installed in your system. The focus is on trying out integration with a minimum of effort, especially regarding running time of the build. b) The default is to guarantee a correct build, which means that all packages get created such that they contain all required files and documents. All these packages pass their tests, especially the ones for self-containment, and are ready to get shipped. If one or more of these goals are not met, the build breaks. So I'd like to let the actual release managers decide. Just chime in guys... Personally, I'd always do the full build anyway. Sorry, but I simply don't believe in getting half-pregnant. ;) I don't understand what do you mean by being half-pregnant. Can you expand on this? -- anatoly t. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 18, 2014 at 11:12 PM, anatoly techtonik techto...@gmail.com wrote: On Tue, Feb 18, 2014 at 10:54 PM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 19:59, anatoly techtonik wrote: [...] You need to ensure that there are no warnings during the build process and the warning about missing documentation build is among those that you especially should not ignore as a release manager. ;) I could live with both variants for the top-level build (packaging n' stuff): a) The default is to build and package SCons as far as the tools and requirements for the single steps are met. Documentation may get built and archives might get packaged and tested or not, depending on which tools/modules you have installed in your system. The focus is on trying out integration with a minimum of effort, especially regarding running time of the build. b) The default is to guarantee a correct build, which means that all packages get created such that they contain all required files and documents. All these packages pass their tests, especially the ones for self-containment, and are ready to get shipped. If one or more of these goals are not met, the build breaks. So I'd like to let the actual release managers decide. Just chime in guys... Personally, I'd always do the full build anyway. Sorry, but I simply don't believe in getting half-pregnant. ;) I don't understand what do you mean by being half-pregnant. Can you expand on this? Ok. I'll put it the other way. Between automating the job of release manager, which is done once in few months and automating the job of developer, which is more than once in a while, you choose the former. Why? My opinion is that by adding additional dependencies to run the SCons without errors from a fresh checkout we are significantly increasing contribution barrier and discouraging people from participating. People need to checkout and run to see the power of SCons. Not read, checkout, install, setup, run cycle. Something like this. -- anatoly t. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Tue, Feb 18, 2014 at 11:50 PM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 21:19, anatoly techtonik wrote: [...] Ok. I'll put it the other way. Between automating the job of release manager, which is done once in few months and automating the job of developer, which is more than once in a while, you choose the former. Why? Because my workflow seems to differ from yours. Do you use bootstrap.py to call a development version of SCons, or how do you do it? I use bootstrap.py for a quick test that development version is not completely broken. -- anatoly t. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On 19.02.2014 00:00, anatoly techtonik wrote: On Tue, Feb 18, 2014 at 11:50 PM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 21:19, anatoly techtonik wrote: [...] Ok. I'll put it the other way. Between automating the job of release manager, which is done once in few months and automating the job of developer, which is more than once in a while, you choose the former. Why? Because my workflow seems to differ from yours. Do you use bootstrap.py to call a development version of SCons, or how do you do it? I use bootstrap.py for a quick test that development version is not completely broken. Okay, and when you have a simple SConstruct in a folder like /tmp/sconstest, change into this folder via cd /tmp/sconstest and then call python /full/path/to/scons/repo/bootstrap.py , does that work in 2.3.0 without having libxml2/lxml installed or do you see an error? ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Wed, Feb 19, 2014 at 2:08 AM, Dirk Bächle tshor...@gmx.de wrote: On 19.02.2014 00:00, anatoly techtonik wrote: On Tue, Feb 18, 2014 at 11:50 PM, Dirk Bächle tshor...@gmx.de wrote: On 18.02.2014 21:19, anatoly techtonik wrote: [...] Ok. I'll put it the other way. Between automating the job of release manager, which is done once in few months and automating the job of developer, which is more than once in a while, you choose the former. Why? Because my workflow seems to differ from yours. Do you use bootstrap.py to call a development version of SCons, or how do you do it? I use bootstrap.py for a quick test that development version is not completely broken. Okay, and when you have a simple SConstruct in a folder like /tmp/sconstest, change into this folder via cd /tmp/sconstest and then call python /full/path/to/scons/repo/bootstrap.py , does that work in 2.3.0 without having libxml2/lxml installed or do you see an error? There is no error and should not be. -- anatoly t. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On 19.02.2014 00:14, anatoly techtonik wrote: On Wed, Feb 19, 2014 at 2:08 AM, Dirk Bächle tshor...@gmx.de wrote: [...] Okay, and when you have a simple SConstruct in a folder like /tmp/sconstest, change into this folder via cd /tmp/sconstest and then call python /full/path/to/scons/repo/bootstrap.py , does that work in 2.3.0 without having libxml2/lxml installed or do you see an error? There is no error and should not be. Good, so you are able to develop SCons and run a checked-out, or even modified, version of SCons against a build project, right? Because in your earlier mail you said: My opinion is that by adding additional dependencies to run the SCons without errors from a fresh checkout we are significantly increasing contribution barrier and discouraging people from participating. People need to checkout and run to see the power of SCons. Not read, checkout, install, setup, run cycle. Something like this. But this is obviously not the case. When following the first instructions in the top-level README.rst, people are able to call SCons without installing it and without having to resolve any further dependencies. So there is actually no reason to fear that users or first-time developers get a bad first impression of SCons, when they try to use the latest development version. Can you see that too, and agree with me that we don't have a real problem in this very specific use case (cloning the repo, and calling SCons directly)? ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons doesn't bootstrap without libxml2
On Wed, Feb 19, 2014 at 2:41 AM, Dirk Bächle tshor...@gmx.de wrote: On 19.02.2014 00:14, anatoly techtonik wrote: On Wed, Feb 19, 2014 at 2:08 AM, Dirk Bächle tshor...@gmx.de wrote: [...] Okay, and when you have a simple SConstruct in a folder like /tmp/sconstest, change into this folder via cd /tmp/sconstest and then call python /full/path/to/scons/repo/bootstrap.py , does that work in 2.3.0 without having libxml2/lxml installed or do you see an error? There is no error and should not be. Good, so you are able to develop SCons and run a checked-out, or even modified, version of SCons against a build project, right? No. The user experience is that the run failed while previously the same user scenario worked without problem. Because in your earlier mail you said: My opinion is that by adding additional dependencies to run the SCons without errors from a fresh checkout we are significantly increasing contribution barrier and discouraging people from participating. People need to checkout and run to see the power of SCons. Not read, checkout, install, setup, run cycle. Something like this. But this is obviously not the case. The two things do not contradict. When following the first instructions in the top-level README.rst, people are able to call SCons without installing it and without having to resolve any further dependencies. Ok. I'll correct myself. For users: - read, checkout, read, run + checkout, run For me: - edit, runtests.py -a + edit, bootstrap.py So there is actually no reason to fear that users or first-time developers get a bad first impression of SCons, when they try to use the latest development version. Just make a corridor testing. Mine failed. Can you see that too, and agree with me that we don't have a real problem in this very specific use case (cloning the repo, and calling SCons directly)? It depends on how seriously you take the user experience discipline, but let's just say that I am a stubborn conservative freak and want the previous behavior back. =) ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
