Re: [Scons-dev] Testing external tools
On 05.05.2014 18:21, Russel Winder wrote: I think I have forgotten how to use the runtest.py script for executing tests of tools that are not in the SCons heirarchy and the help isn't helping. | python /home/users/russel/Repositories/Mercurial/Masters/SCons/runtest.py test 1/1 (100.00%) /usr/bin/python -tt test/Issue_1/sconstest-A.py Traceback (most recent call last): File test/Issue_1/sconstest-A.py, line 34, in module test = TestSCons.TestSCons() File /home/users/russel/Repositories/Mercurial/Masters/SCons/QMTest/TestSCons.py, line 232, in __init__ os.chdir(script_dir) OSError: [Errno 2] No such file or directory: '/home/users/russel/Repositories/Mercurial/Masters/SCons_Protobuf/src/script' Exception AttributeError: 'TestSCons' object has no attribute '_dirlist' in bound method TestSCons.__del__ of 7ff37614b9d0 ignored Try -e, like for external tool. ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] MANIFEST.in file
Hi Rob, On 06.05.2014 17:37, Managan, Rob wrote: I probably have not read any of the workflow docs lately but I don't remember having to update the MANIFEST.in file when a new file is added. Of course, I may never have added a file before. Is it the responsibility of a developer that adds a file to the source to add it to the MANIFEST.in file so bootstrap.py works? Until I tried using bootstrap.py I had no clue that something may be broken since the tests all passed, etc. you're right, this isn't really obvious at the moment. We should add this info to the README.rst, and also to the introduction of the Developer Guide in the Wiki. Can you possibly suggest a good place within those documents, like where would you search first? In addition to that, it might be a good idea to try and add some kind of automatic check. A simple script or test, that gets started with every runtest.py -a for example. I'm not sure about how to properly detect files that are missing in the MANIFEST.in, such that the script can emit a warning. If this were an easy task, we could probably throw out the MANIFEST.in files in general and use the script instead. ;) But maybe someone has a clever idea. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] WTF NB?
On 06.05.2014 20:51, anatoly techtonik wrote: PTAL. https://bitbucket.org/scons/scons/pull-request/135/ What does PTAL mean? ;) Sorry, but I couldn't resist. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Bugtracker and stuff...
Hi all, On 05.05.2014 02:45, Dirk Bächle wrote: On 05.05.2014 02:01, William Deegan wrote: Do we know the specs of the server pair is providing us with? roundup suggests mysql or postgressql for database. Need be we can put roundup on the same server as buildbot. I’ve installed and customized bugzilla for many many clients, so if you like I can take a whack at this. Cool, just go ahead...I won't mind. In the meantime I'll try to figure out how to push attachments via xmlrpc. I now have a first version of a Tigris-roundup importer ready, please find the Python script attached. It's not perfect, but I could successfully import all current 2947 issues into a roundup demo instance running locally on my machine. The things that get properly imported are: - users, issues, attached files, single messages, keywords . Most attributes like priority, status and assigned to get set, so I think it's at least good enough to setup a demo server for testing out roundup with our tracker database. Some things are missing (can't get copied): - The last activity time for an issue, as well as the upload time for files are protected items in the database. So, the timeline of events gets skewed in some places. It might be possible to correct this by exporting the database again in CSV/JSON (roundup itself offers this for migration to newer versions), manipulating the data, and then importing again. I haven't looked further into this...anyone cares to give it a try? - I'm not sure what to do with depends on and duplicate of information. Roundup has a superseder list field, but the meaning of it isn't clear (has no intentional meaning). So we may have to come up with something ourselves... - Currently no passwords are set for added users, so they can't login. We could set the username as password too? For my test I used the following steps: 1.) Download the Tigris bugs in XML format with python import_tigris.py files scons import 2.) Start a local roundup demo (in virtualenv) with env/bin/python demo.py nuke 3.) Push data to roundup via xmlrpc python import_tigris.py push http://admin:admin@localhost:8917/demo/xmlrpc import There is also a short usage description at the start of the script. For testing purposes you might want to move all except the first XML file (1.xml) from the import folder to a safe place. Pushing all the issues took about 3 hours on my side. ;) Best regards, Dirk roundup_tigris_import.tgz Description: application/compressed-tar ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Status of Issue Tracker at scons.tigris.org
Hi Michael, On 13.05.2014 13:50, Michael Haubenwallner wrote: Hi, I'm wondering if the Issue Tracker still is used by the SCons developers, it's still the only official tracker that we have...and use. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Emulate SVR4/Linux like SONAME for AIX
Hi Michael, On 16.05.2014 18:42, Michael Haubenwallner wrote: Hi, based on the pull requests #138 #139 #140, fixing various issues, now I've been able to get the aix-soname thingy into SCons. However, as a SCons newbie, I'm unsure whether the way I've implemented this into SCons is based on correct assumptions on how SCons should work. Actually, I've been able to reduce the number of failing tests from 90 to 65 on the AIX box I'm working on. https://bitbucket.org/haubi/scons/commits/76108c06ac81150a4d79a52583d3cf7a5e6fa1cd Thoughts? I had a quick glance at your pull requests and the commit mentioned above. To me it looks like you're doing solid work, please keep it up. Someone else (Gary?) should comment on your introduction of CCVENDOR, I'm a bit unsure whether this is the way to go while having the new toolchain design in mind. Please try to use bookmarks instead of named branches for organizing your local work and commits. See also: http://www.scons.org/wiki/SconsMercurialWorkflows Other than that, good job and thanks for all the effort you have put into this! Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
[Scons-dev] SCons issues - An overview...
Hi there, I've done some cleanup work in our Tigris bug tracker (marking duplicates and such) and noticed a few issues that might get closed right away. When I list items in the following, complete with ID and assignee, this is not meant as an order to take immediate action. Please don't take offense when your name pops up, these are friendly reminders...if at all. ;) It would be cool, if these issues could get revised in the near future: - #2490,#1912,#1917: is this development of a C# tool superseded by Russel's repo? - #2538, SK: create IRC freenode.net channel with a group as owner, is this done already? - #2533, Gary: installer issue under Vista, does this still happen? - #2703, Bill: ditto for Windows 7, still an issue? - #2669, Gary: fixed completely, or only for Linux? (commit 64f297, 2010-09-16) - #2451, Gary: infrastructure for extensions (=ToolsIndex), close? - #2403, Bill: Problem when not specifying buildtarget with MSVS Projects, problem or not? - #2124, Azverkan: Install() race condition, might be gone in newer Python versions? - #1900, Gary: intel detection tool is hard coded for lookups, superseded by current code? - #1915, Bill: VPATH Search bug - #2114, SK: .F files no longer recognized on some platforms - #2544, Anatoly: Remove links for archived documentation - #2564, Gary: Wiki upgrade + OpenID - #2697, #2675, #2667, #2663, #2661, #2612, #2570, #2148, #2212, #1753, #2885, #2820, #2622, #2694: lots of Windows/MSVC related bugs that might be resolved after the big toolchain revamp Then, I also found several issues that have some importance for development, in my opinion. They probably need to get discussed in the group, because often a direction has to be taken (either, or). I'm not sure how we would go about this, but here are the IDs: - #608: Unices multi platform support - #2439: Expanding # for RPATH - #2167: subprocess for win32.py - #2581: Alias looking like a file gets ignored when chaining aliases - #2648: Dir causes TypeError in build phase - #2550: Combining Java, Install and Jar builders can cause AttributeError - #2594: source=None disables dependency on action - #2517: Java buidler does nothing when duplicate=1 - #1708: sources scanned even with --implicit-cache - #1406: Spaces in path incorrectly handled by Qt tool - #2455: confdefs.h is never created by configure context - #2498: Delayed creation of default Environment overwrites scanner assignments - #2608: Add a progress indicator (# of # targets) - #1945: scons should not cache implicit dependnecy found from a directory scanner - #2460: Alias with depends instead of sources is never rebuilt - #824: Remove os.chdir() from SCons itself - #1328: Program() has Value node handling bug - #2831: AddOption and help display with -h - #2611: -h flag does not print local option help - #2081: Better messaging need for *** Two environments with different actions were specified for the same target error Best regards, Dirk Finally, here are lists of low hanging fruit: - #1991, Bill: document CheckTypeSize() - #1985, Bill: document Platform() - #2688, SK: add scons to PATH after installation under Windows - #2652, Gary: document CopyTo()/CopyAs() - #2477, Bill: amend MAN page entry for DefaultEnvironment - #798, Bill: Export() in subsconscript can override values (Export is global!) - #2633, Anatoly: update Getting Started - #2444, Bill: Unclear where env.WhereIs searches for executables - #780, Gary: Document setting BUILDERS value to a callable wrapper - #2361, SK: class Tool: self.options is not a callable entity - #1874, Gary: scons fails to append .so if the library name os a period in it - #2427, Bill: STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME not in man page - #2600, Gary: Command not executed when args 8191 (document MAXLINELENGTH) - #2565, SK: link SHCXXCOMSTR to CXXCOMSTR in docs and patches that don't seem to require much work: - #2553, Gary: Allow specification of non-standard package type in Package() - #2696, SK: Improve performance with --duplicate=copy - #2630, SK: Problem using tempfile class with batch building - #2569, SK: Regexp for RCScan - #2575, Jason: Add chdir functionallity to zip-builder - #1920, Gary: Support parameter replacement in files - #1910, Gary: Recursive flag ignored in dictionary based scanners - #2450, Gary: Improvement for Variables updating - #2431, Bill: Allow customisation of tempfile extension - #806, Gary: Builder for mozilla xpidl files. - #, Ray: Created Builder for Motif Uil files - #1615, David: Fortran scanner not picking up #include statements - #1761, Gary: Support for precompiled headers in intelc - #2799, Issues: Mingw tool does not respect SHLINKCOMSTR and LDMODULECOMSTR ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
[Scons-dev] Your advice wanted on issue #1742: TryLink() is too successful sometimes...
Hi there, I'd like to get a second opinion (and third, and...) on the following problem: I'm currently debugging issue #1742 and found out why the TryLink() sometimes returns successful, although this shouldn't happen. In the given example env = Environment(tools = ['g++', 'gnulink']) printCC(env) config = env.Configure() print 'C++ toolchain', config.TryLink('int foo() {}', '.c') config.Finish() the variable $CC is empty, since the gcc tool doesn't get loaded. But a *.c file gets compiled, so the smart_link() function returns $CC for $SMARTLINK, which is also the definition of $LINK. Finally, the command $LINK -o $TARGET $LINKFLAGS $__RPATH $SOURCES $_LIBDIRFLAGS $_LIBFLAGS evaluates to -o , and the leading - suppresses the actual return value of the spawn(). So we know what happens, but should we do something about it? Does this need to get documented somewhere, or would it be good to catch it somehow? I don't have a clear idea for the latter option...your comments are welcome. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Your advice wanted on issue #1742: TryLink() is too successful sometimes...
On 20.05.2014 23:58, Dirk Bächle wrote: [...] I'm currently debugging issue #1742 and found out why the TryLink() sometimes returns successful, although this shouldn't happen. In Sorry, just noticed this right now (better late than never ;) )...it should be either ...sometimes returns success or ...sometimes returns successfully or ...sometimes is successful. Correct? Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Your advice wanted on issue #1742: TryLink() is too successful sometimes...
On 21.05.2014 00:53, Gary Oberbrunner wrote: That's an insidious combination of problems there. I don't think much is expected to work if only g++ is defined and not gcc, but it would be nice if it didn't fail this badly. Of course the mythical toolchain revamp will prevent this kind of thing or at least make it harder. But I don't know how much can be done for this particular case. I'm more worried about the general case...this can happen in other contexts too. The basic problem is that we use - to signal a command to be ignored, and it's the option char at the same time. This makes the underlying grammar for the current command string ambiguous. One option would be to do the check on leading - and @ on the unexpanded string only (before the subst). But then a user couldn't switch on/off these via a leading variable anymore...like in ($IGNORE$LINK ...). It would always have to get hardcoded into the Action command, or you'd have to use a generator function. Maybe that's a way out, without sacrificing too much flexibility. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons issues - An overview...
On 24.05.2014 20:34, Gary Oberbrunner wrote: I've just reviewed and closed many of the issues in your first set. More to do of course but it's a start. Cool, that's a whole bunch off issues fixed/resolved. Nice, and thanks for having a look! Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Toolchain requirements
Hi Gary, here are my comments: On 25.05.2014 19:14, Gary Oberbrunner wrote: I'd like to kick off a round of discussion about toolchains, so can make some progress toward a design. I have some preliminary thoughts. Please comment. Apologies for the HTML mail, let me know if this isn't readable for you. * Allow installing external tools (pip install or ...) * o scons --version (or similar) should list installed tools and toolchains o missing external tools should give sensible errors I'm not sure I understand what you mean by missing here. An external Tool that isn't able to find the executables or packages it depends on in the current system? Like an overview, where you get an Available/NotFound for each Tool? What I wouldn't like is, if we'd define a set of external Tools that the user would *have* to install. We can provide a package of external tools in a bundled fashion, something like our best of. But each user should be free to install it, or only the single Tools he needs, without getting confusing messages about a Tool not being found, just because we think he needs to have it. * Tool setup must happen before reading SConstruct somehow I don't know if this is really required, see my comment below about lazy initialization of Tools. It would be good to have a pool of Platform-related info (which OS/Python/Kernel are we running on?) at this early stage, that all environments/tools can access directly. * o DefaultEnvironment and all new Environments should know about all tools o alternative: lazy-construct DefaultEnvironment o user-specified tools and toolchains need to be specifiable at beginning of build * User should be able to set default tools and toolchains ...and the order in which they are probed, if possible this specification should be per OS. * o unused tools shouldn't take any startup time * Lazy init of tools and chains * o This is faster because unused tools don't matter o It allows missing unused tools to not give errors, but missing used tools can (and should) o But it makes configuring environments much harder for users, because they can't override or append to tool-provided variables until those exist. This would break a lot of existing SConstructs. o We need to find some kind of compromise here: o + Explicitly list tools required by build (where?): this should work well because only the needed tools will be initialized + if nothing explicitly specified, fall back to current method We should also take the case into account where a user might set up a dummy environment. Depending on which Toolchain seems to be available, he then wants to setup his actual environments, or install additional packages first, or... I, personally, think that we don't need lazy initialization here. If we give a user more option to control which Tools/Toolchains are probed in which order, he can size down the effort for detecting stuff nicely. At least, the latter (=toolchain control) should have higher priority over the certainly nice and cool lazy init feature. Doing things lazy, also makes things like debugging, writing test cases or creating issue reports a little harder... * Within a tool: * o specify dependencies on other tools What kind of dependencies are you thinking about? Module-wise, method-wise or both? Is it more about data dependencies, or order dependencies for the probing phase? * o detect existence on system reliably, and without modifying env o + need better error messaging: ability to probe silently, but also give sensible errors when needed o constructor needs to allow args: version, path, ABI, etc. (this is important) o allow for common setup (all C compilers, etc.) as now o tools should be versioned so user can check if up to date, etc. * Tool chains: * o either-or o and o collections * Platform * o How much do we need to know about the platform, for tools to initialize themselves? o Cross-compilation comes into this, but may be too much to include as a general part of this project. It's really hard to draw the line here. For my personal taste, we should have a PlatformInfo with only the basic data about OS/CPU/RAM/Kernel and the Python version we're running under. Everything else, like latest Java version or available cross-compilers, should be handled by the rest of the toolchain/tools stuff somehow. In general, I just don't want to have things getting all muddy and confused to fast. This will happen anyway down the road of implementation, faster than we can think. ;) So let's try to be as clear and concise as possible right now at the beginning, and establish clear visions and borders about what we want to do, and what
Re: [Scons-dev] SCons and Java
Hi William, please find a few comments below. On 30.05.2014 02:42, William Blevins wrote: Team, I have a couple points I'd like to discuss, but for the sake of organization, I intend to split them into separate emails. Java Part #2 There has been some discussion on making Scons more ANT-like, http://www.scons.org/wiki/JavaSupport, and that might solve some issues like identifying Java targets. There is merit here assuming that Java support stays SCons-like at the same time; otherwise, why not just use ant or maven? From this I get that you'd like to have some form of Java support in SCons directly, and don't want to delegate things to an external program, correct? At least it's my assumption for the following remarks... Problem(s) this will solve: 1. Determining the source - target(s) from *.java - *.class is painful, possibly java version dependent, and performance intensive (possibly); thus, we should optimize this out. 2. Java developers are plagued requiring manual cleaning of a jar far too often with ant-like system(s) and the current scon(s) methodology. 1. Clean could be required when deleting or moving a file since the last build output isn't removed otherwise; avoidable in current SCons depending on how sources are specified (not obvious and/or intuitive coming from ant-like system(s)). 3. Simplify the SCons Java API to allow building jar file(s) without having to interact with *.class file(s) at all. 4. Simplify the SCons Java support for resource file(s); currently, you have to do copying by hand (if it's supported I know not where or how). 5. Reduce the amount of code required to build a jar (by a lot). This sounds like an even more simplified Java Builder than what we have now. If you could come up with a SoSimpleJava() Builder, we'd probably add it to the core and let people have their own choice. How I imagine this should work: 1. Treat class file(s) like side-effect(s) and do not include them (at all) in the dependency tree. 1. This means we don't have to scan the Java file(s) and determine *.class targets. I don't think that you can have both. If you treat files as SideEffects (in the SCons sense), they have to be part of the dependency DAG. See next comment... 2. Remove the Java(...) function from SCons; thus, do not interact with *.class file(s) directly ever. There isn't a *good* reason to do this. Here is just one scenario, where dealing with *.class files actually makes sense: If you're using anonymous classes in a Java source, a bunch of additional class files get generated. How do you remove these files on a clean, if they are located in a source folder (which people might prefer for some reason)? More experienced Java developers can probably come up with a dozen more useful cases...I'm not sure how often people would want to integrate pre-built *.class files (coming from Scala perhaps?) into their builds, for example. You may be able to work around the clean problem, by creating all *.class files in a separate folder structure (pretty much like a VariantDir). But what you'll end up with is a somewhat crippled DAG, forcing you to remove this output dir completely and rebuild everything from scratch...even when only a comment in a single Java source is changed. Only to be sure, just like in make or Ant...there is not much of a win here for the user, I'm afraid. Users might also accidentally delete *.class files in your output dir, between two builds. And they expect SCons to rebuild those intermediate targets on its next invocation. How do you want to handle this? Finally, you should take into account that users might want to create different versions of the same source code. Just as an example, let's assume they want to create an original and obfuscated target(s), or a simple and extended release where additional packages get included based on environment variables. And for maximum speedup they do this in parallel, by using the -j option...so a single dedicated output dir won't do probably. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] SCons and Java
William, thanks for your quick reply. On 31.05.2014 01:39, William Blevins wrote: Dirk, Team, I have a couple points I'd like to discuss, but for the sake of organization, I intend to split them into separate emails. Java Part #2 There has been some discussion on making Scons more ANT-like, http://www.scons.org/wiki/JavaSupport, and that might solve some issues like identifying Java targets. There is merit here assuming that Java support stays SCons-like at the same time; otherwise, why not just use ant or maven? From this I get that you'd like to have some form of Java support in SCons directly, and don't want to delegate things to an external program, correct? At least it's my assumption for the following remarks... SCons already has Java support (right?). I have been a SCons user for a few years and am interested in trying to make the Java toolkit better because I dislike Ant-like build system(s), but I feel that SCons still has room for improvement. I am braining storming ways to improve the setup. I have a bunch of idea(s) that I would like to discuss and this just happens to be one of them (#2 in fact). I know that the SCons teams has talked about refactoring the SCons Java toolkit, but outside the notes from the link above, I haven't seen any further discussions. I am the new kid on the scons-dev mailing list block, so feel free to give me a tour. I would be willing to work on SCons with some direction. That's good to hear, any helping hand is welcome. I haven't been involved in the latest Java development, so the link list I gave you in the mailing thread http://four.pairlist.net/pipermail/scons-users/2014-February/002225.html is all I got. Problem(s) this will solve: 1. Determining the source - target(s) from *.java - *.class is painful, possibly java version dependent, and performance intensive (possibly); thus, we should optimize this out. 2. Java developers are plagued requiring manual cleaning of a jar far too often with ant-like system(s) and the current scon(s) methodology. 1. Clean could be required when deleting or moving a file since the last build output isn't removed otherwise; avoidable in current SCons depending on how sources are specified (not obvious and/or intuitive coming from ant-like system(s)). 3. Simplify the SCons Java API to allow building jar file(s) without having to interact with *.class file(s) at all. 4. Simplify the SCons Java support for resource file(s); currently, you have to do copying by hand (if it's supported I know not where or how). 5. Reduce the amount of code required to build a jar (by a lot). This sounds like an even more simplified Java Builder than what we have now. If you could come up with a SoSimpleJava() Builder, we'd probably add it to the core and let people have their own choice. That's fair, but since I am talking about not tracking *.class file(s) at all. The old function(s) couldn't exist. How I imagine this should work: 1. Treat class file(s) like side-effect(s) and do not include them (at all) in the dependency tree. 1. This means we don't have to scan the Java file(s) and determine *.class targets. I don't think that you can have both. If you treat files as SideEffects (in the SCons sense), they have to be part of the dependency DAG. See next comment... I meant this in the truer form of side-effect and not the SCons.SideEffects sense. Getting from A - C by passing through B, but B is just along the way. The only way the *.class file(s) could be part of the DAG is if they were identified after the Java compiler had already created the *.class file(s) by basically Glob(*.class). I would really like to get away from SCons being required to scan Java source file(s) to determine the targets due to the 1 - N nature of *.java to *.class. This would resolve several issues with the SCons Java toolkit including incorrect dependency tree(s) with derived (generated) Java source file(s) and some build issue(s) that require manual clean operation(s) on Java code (which may be alleviated depending on how you setup your SConscript, but may not be obvious coming from an ant-like system). Testing out this idea looks like a straightforward task to me. I'd really like to encourage you to just go ahead and implement a new Tool/Builder in parallel to the existing ones. Then it's easier to discuss dis-/advantages and point out the problems of each approach. However, there certainly will be more obstacles ahead that I can't think of right now. It's a perfectly understandable reaction to restrict functionality, such that certain issues don't even emerge in first place. But one has to be careful about not driving this too far, because then it's actually more problem avoidance than
Re: [Scons-dev] Toolchain requirements
On 25.05.2014 19:14, Gary Oberbrunner wrote: I'd like to kick off a round of discussion about toolchains, so can make some progress toward a design. I have some preliminary thoughts. Please comment. Apologies for the HTML mail, let me know if this isn't readable for you. I just stumbled over this link https://github.com/pradal/openalea/tree/master/sconsx , maybe this implementation of an SCons extension gives us a few additional ideas... Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Buildbot
Hi Bill, On 10.06.2014 20:03, William Deegan wrote: Dirk, I’d have to disagree. That's okay. ;) I don’t think new toolchain logic is imminent. Having a clean build on buildbot across all the available platforms is a simple way to verify thinks are “OK”. Right, and not all dots are on the i currently. The LaTeX support has a problem with biblatex vs. biber detection under Fedora (talked to Rob about this some time ago, but I don't know what the current status is), and the Gnu Java installation under SuSE doesn't understand the -sourcepath option. I can still install Sun Java instead, but the last time I suggested this you rather wanted to have the tests marked broken until someone fixed them. So what has higher priority: getting the slaves green or fixing things properly? (Yes, I know that this is over-simplified... ;) ) Probably can ignore the 2.6 failures though.. Perhaps I should retire that buildbot slave? I wouldn't mind, or would upgrading to 2.7 be an option? Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] OP me, plz.
Hi Anatoly, On 05.07.2014 09:20, anatoly techtonik wrote: Hi William, Because nobody from developers objected, I already sent request to botbot.me, even though I didn't tick the [ ] op mark. Another reason for me to pushing on op is to increase the bus factor on the channel. I understand your intentions behind this request, and value all your efforts to drive this item forward. But I, personally, don't like the way you're hammering this through. As Bill hinted at, this is a somewhat sensitive topic because recording of what other people say and write actually touches part of their private sphere. So it would be a good idea to ask the developer community *and* give it some time to answer first, before leaping ahead and just doing it. You probably missed the point with the private sphere a little, which is okay because it can happen to anyone of us. But this also makes clear how important it is to ask and get other people's opinions first, instead of saying I'll do this now. Stop me if you can.. In the first case you're a valid part of the community, in the latter you're doing things alone. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] OP me, plz.
On 05.07.2014 10:30, Dirk Bächle wrote: Hi Anatoly, On 05.07.2014 09:20, anatoly techtonik wrote: Hi William, Because nobody from developers objected, I already sent request to botbot.me, even though I didn't tick the [ ] op mark. Another reason for me to pushing on op is to increase the bus factor on the channel. I understand your intentions behind this request, and value all your efforts to drive this item forward. But I, personally, don't like the way you're hammering this through. As Bill hinted at, this is a somewhat sensitive topic because recording of what other people say and write actually touches part of their private sphere. So it would be a good idea to ask the developer community *and* give it some time to answer first, before leaping ahead and just doing it. You probably missed the point with the private sphere a little, which is okay because it can happen to anyone of us. But this also Sorry, I just looked this up...there is no such word as private sphere. I meant privacy... Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Is Tigris issue tracking actively used?
Andrew, On 06.07.2014 22:25, Andrew Featherstone wrote: Hi Dirk, Ok it's good to know where to be looking. For me the number of open issues gives a (false) negative impression that the project's development is stale. For me, P1 bugs are triaged as is an issue which causes detrimental behaviour (e.g. deletes source code, compiles source code with different flags, misses changes in source code), and must be fixed in the next release. Only two of the P1 issues have been commented on in the past two years, and some have sat still since 2009! As I said, this is confusing at best to someone who wants to get a feel for the current status of an open source project. I can only agree. ;) That's why I started to do something about it...would you like to help? Moving forwards it'd be nice to know that tackling issues in the issue tracker is worthwhile, that comments don't go unread, etc. Who marks what issues must be fixed for the next version? Is there any plan for existing issues to be updated? Do the developers communicate through the issue tracker or some other method e.g. IRC? Regarding issues there is no real planning or update process in place. We used to have a triaging process (BugParty) via IRC, on a bi-weekly basis...but with the dwindling number of core developers it petered out. At the moment, the development version is quite stable. I don't know of any showstopper bugs that would have to get fixed immediately (no P1 issues). The few really serious issues get discussed and assigned here on the dev ML. We also have a small roadmap at http://scons.org/wiki/Roadmap ...aaand that's it, I guess. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Is Tigris issue tracking actively used?
Andrew, On 07.07.2014 19:47, Andrew Featherstone wrote: I'm likewise, William, but I think that all users can add some value by wandering through the open issue list and seeing if we can improve things by clearing issues that are known to be resolved, testing old issues against current versions of SCons, etc. Dirk, I've started traipsing through the open bugs, looking for things I think already are fixed, or verifying if issues are still present, and commenting accordingly. At 4 issues a day it'd be 3 months before the open issues list has boiled down to nothing, but hopefully just keeping a downward trend on this for a period will feel worthwhile. thanks so much for all the work you've done so far! The roadmap is a high-level thing, and doesn't explain for example what triggered the 2.3.2 release, which I'm interested in understanding. Cheers, Andrew P.S. on the homepage the links to the individual mailing lists point to the old Tigris ones. Can they point at the Mailman-based ones instead? That's #2947 already. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Is Tigris issue tracking actively used?
On 07.07.2014 22:01, anatoly techtonik wrote: On Sun, Jul 6, 2014 at 3:25 PM, Andrew Featherstone andrew.featherst...@cantab.net wrote: Hi All, I was wondering what the current state of issue tracking is for the SCons project. There was some talk a little while back about moving away from Tigris to a different issue tracker, has that happened? No. Sorry. Dirk made script to export tigris.org issues, which I saved here: https://bitbucket.org/techtonik/dataliberation/src/tip/issues/?at=default but Google deprecated their Issue API, probably due a lot of spam: https://code.google.com/p/support/wiki/IssueTrackerAPIPython Bitbucket supports import/export, but was considered awful by me: https://confluence.atlassian.com/display/BITBUCKET/Export+or+import+issue+data and then I stuck with reversing Roundup tracker model. Moving to Roundup as bug tracker got discussed recently on the dev ML: http://two.pairlist.net/pipermail/scons-dev/2014-May/001369.html I have patched the export scripts from the dataliberation repo above, and am able to export most of the current Tigris tracker's info to a Roundup instance. I also have two final pull requests pending at OpenHatch: https://github.com/openhatch/oh-bugimporters/pull/57 https://github.com/openhatch/oh-mainline/pull/293 which will make our database available (read-only) via their Roundup tracker. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Issues that can potentially be closed
William, On 11.07.2014 05:03, William Blevins wrote: I'm not sure how to progress with this? We discussed cleaning up the open issues in another thread, but I don't know enough about how issues are managed to be making changes without direction and/or get permission. I'll go through the Issue ML at Tigris over the weekend, and close the bugs that have been identified as resolved or duplicates by you and Andrew. Thanks again, you both did a great job! Also, there are lots of comments about bug triage, and I have no idea what that means. Don't care about them so much. We're in another bug triaging process right now, so only the issues we declare as important today really are... ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Issues that can potentially be closed
On 10.07.2014 02:37, William Blevins wrote: *More duplicates (just looking at java issues atm):* 2432 -- Duplicates 1772 I'm not sure about this one. Gary, can you have a short look please? 1849 -- Duplicates 1594 2547 -- Duplicates 1594 2548 -- Duplicates 1594 2046 -- Duplicates 1594 2733 -- Duplicates 1594 Declared #2733 as the new parent bug. 2359 -- Duplicates 1766 No, one is about emitting proper class filenames for anonymous classes, the other about displaying the intermediate class names in the dependency tree output. 2931 -- Duplicates 1771 Okay, please attach your current modified version of the Protoc Builder to bug 1771 for later reference. 2483 -- Duplicates 2089 Good catch! Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Getting started with the SCons code base~
On 12.07.2014 17:26, William Blevins wrote: Is there also a guide for running the tests? Seems there are 3 sets. Yes, there is...check QMTest/test-framework.rst, also available at http://scons.org/wiki/DeveloperGuide/TestingMethodology . ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Is Tigris issue tracking actively used?
Hi there, On 06.07.2014 14:25, Andrew Featherstone wrote: Hi All, I was wondering what the current state of issue tracking is for the SCons project. There was some talk a little while back about moving away from Tigris to a different issue tracker, has that happened? I thought I'd have a quick look on Tigris, but clicking All Open Issues returns a warning that there are too many issues to be displayed! Viewing open bugs and sorting to priority gives a large chunk of bugs that haven't be touched in years, and some of which look like they are resolved (e.g. issue 1985 http://scons.tigris.org/issues/show_bug.cgi?id=1985 looks like it's implemented here http://www.scons.org/doc/2.3.1/HTML/scons-user.html#f-Platform). It's confusing that there are open P1 bugs that don't make it into subsequent releases, or at least have a comment that explains why they're being pushed to release n+1. after another bug scraping (thanks to everybody who volunteered!), I created two new HTML lists of the current state in our bugtracker. Please find those files attached for your convenience and general overview... Best regards, Dirk issues_overview.tgz Description: application/compressed-tar ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Getting started with the SCons code base~
On 12.07.2014 23:43, William Blevins wrote: Back to 2395: My python expertise isn't 5 star, and I'm confused on how to propagate a symlinks argument down. I'm a little confused, too... What I imagine my SConscript should do: env.InstallAs( a, b, symlinks=False ) because that's not what the issue description tells me. It's about Copy(), not Install(). Where the scons copy command exists: SCons.Tools.install.copyFunc There are so many wrapper layers I feel like I'm spinning in circles. I was able to follow it up until src/engine/SCons/Action.py:1065 and then I get an argument count mismatch for installFunc. I'm worried about changing the Action.execute interface, so I'm not sure how to proceed. Any thoughts? Please, concentrate on Copy() and its copy_function as defined in Defaults.py. Leave the Install()/CopyAs/CopyTo Tools as they are...there shouldn't be a need to change any of Action()'s, Builder()'s or Executor()'s interface. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Issues that can potentially be closed
On 12.07.2014 16:32, William Blevins wrote: For 1771, my protoc builder changes aren't important. This is not so much about importance, but traceability and being able to reproduce your findings later. Take a look at issue #1438 in the bug tracker. It's now suggested by Andrew to get closed, mainly because the error can't be reproduced...which later might happen to your issue too. I'm just sayin', it's still your decision. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
[Scons-dev] Slight confusion about the current Python floor version...
Hi there, a bug report about the allegedly broken Split() method, my comments and the submitted patch to reach backward compatibility for Python 2.6 have caused some confusion. So, I'd like to ask again: What is our current Python floor version for development (default branch)? My understanding was that we had agreed on already allowing Python 2.7 only changes some time ago. But SCons 2.3.2 and the website claim that we're still compatible with 2.4 and up. We either have to agree on supporting 2.6+2.7 for the current release 2.3.2, or the webpage has to get amended. And for the upcoming release we have to properly announce and make the clear cut to 2.7 only. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Pull requests
On 20.07.2014 17:13, anatoly techtonik wrote: On Sun, Jul 20, 2014 at 12:48 PM, Dirk Bächle tshor...@gmx.de wrote: [...] I do, because I'm subscribed to the Tigris Issue mailing list. Every other subscriber should see your comments too. Additionally, each issue has a CC list, where the creator and current assignee are added automatically. Like for the Issue ML, it's up to your own whether you want to watch a special issue or not. So unless you have concrete questions about the source code, it's probably enough to just add your comments to the issue itself. Please don't use the developer ML to bump your issues. What does bump mean? http://www.urbandictionary.com/define.php?term=bump ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] New debian maintainer
Hi Jörg, On 26.07.2014 22:16, Jörg Frings-Fürst wrote: Hi, my name is Jörg Frings-Fürst and I live in a small village near by Trier, Germany. and welcome to the scons-dev mailing list. Good to have you on board. I'm the new debian maintainer for your program scons and scons-doc. Now I'm working for a update the packages to version 2.3.2. I hope for a good collaboration. I think I'm speaking for all other developers here: If you encounter any problems or have questions about our packaging process, we'll be glad to help you out. And if there are points we could improve to make your life as package maintainer a little easier, just let us know please. Best regards, Dirk (near FaM) ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Pleased to announce we've added Dirk Bächle as a comitter to the core scons repo
On 29.07.2014 03:30, Manuel wrote: Cool that's great news!! El jul 28, 2014 9:46 PM, Kenny, Jason L jason.l.ke...@intel.com mailto:jason.l.ke...@intel.com escribió: Second that! *From:*Scons-dev [mailto:scons-dev-boun...@scons.org mailto:scons-dev-boun...@scons.org] *On Behalf Of *William Blevins *Sent:* Monday, July 28, 2014 7:34 PM *To:* SCons developer list *Subject:* Re: [Scons-dev] Pleased to announce we've added Dirk Bächle as a comitter to the core scons repo Yeah Dirk! Thanks guys! :) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
Hi, On 30.07.2014 19:05, William Deegan wrote: Dirk, What OS is your fedora7 buildbot slave running. Is it still Fedora 7? it's running Fedora17, as the info page correctly states: http://buildbot.scons.org/buildslaves/fedora-17 . It's just the slave's name that somehow (*cough*) got wrong. ;) Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
On 30.07.2014 19:59, Bill Deegan wrote: Dirk, I can change the name to 17 and restart if you can update the slave on your side? -Bill Okay, let's do that. Do I have to change some config first, or do I restart the slave right away? Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
Hi Russel, On 30.07.2014 22:18, Russel Winder wrote: On Wed, 2014-07-30 at 19:57 +0200, Dirk Bächle wrote: it's running Fedora17, as the info page correctly states: Isn't Fedora 17 just a teensy weensy bit out of date? (given Fedora 20 is current and 21 pending?) yes, but my understanding is that it's not required to always have the latest version of every OS running. I even regard it to be a little counter-productive for a system like Buildbot, which helps us to keep SCons working on a variety (!) of platforms. As long as Python 2.7 runs on it, the OS should be good enough for testing and actually working with it... Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Java Development
Hi Mark, On 02.08.2014 01:49, Mark A. Flacy wrote: On Friday, August 01, 2014 05:22:39 PM William Blevins wrote: SCons Java doesn't need to be that fancy, but I think the root problems can be solved. The SCons java tool simply doesn't get the love that some of the other tools get. One common use case is to generate java source from the xjc tool (which converts XML schema to java classes) for use by other java classes. The emitter in that case would need to parse a bindings file as well as the XML schema(s) that the binding file uses. However, the resulting java files should have no dependencies on any external libraries, so they could be compiled as a unit independently from anything using them. And there are Annotations, which can be used to (and *are* used to) generate source. See http://deors.wordpress.com/2011/09/26/annotation-types/ for a discussion of how to do that. it's okay to be concerned about all the other possible Java use cases and I understand that it would be a long road to fully support the toolchains you mentioned. But this is not the scope of William's current investigations. He's interested in improving the support for Java in SCons for the very basic usage scenarios. And he has my full support, which doesn't mean a lot because I don't have a large Java background but only some basic knowledge. In the background, I'm working on a recursive Glob() which fully supports Ant-like notation. Having this in place, we'd be able to link generated Java sources (like in your examples above) into a SCons build with commands like env.Jar('build', Glob('*/**/*.java')). So one could at least schedule the generating commands for the *.java files to run always, and then let SCons handle the rest. This would still be far from perfect, but it's better than what we have right now. And I, personally, am more interested in the latter. ;) Just my 2 cents. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
Hi guys, On 30.07.2014 16:11, Managan, Rob wrote: Hi Bill, For test/TEX/biber_biblatex.py we get: biber returned an error, check the blg file I suspect that the installed version of biber is mismstched with the version of biblatex and errors out. I guess I need to find a way to figure out what version of biber is installed and what version we require. At http://ctan.mirrorcatalogs.com/macros/latex/contrib/biblatex/doc/biblatex.pdf on page 6 I found the chart for what versions of biblatex and biber are compatible. I will email back later when I figure out how to get the version of biblatex. Probably a kpsewhich type call and then look at the top of the biblatex.sty file. I hope biber as an executable will tell you its version with a command line option. More when I get my ancient linux laptop up and running. Same fortest/TEX/biber_biblatex2.py, test/TEX/biblatex.py, and test/TEX/biblatex_plain.py I think I fixed the above errors for the Fedora box. Based on the description in http://tex.stackexchange.com/questions/140814/biblatex-biber-fails-with-a-strange-error-about-missing-recode-data-xml-file I removed the /tmp/par-* folders, and biber seems to work again (versions of biber and biblatex match locally). In the next days, I'll let another pull request follow to get my Buildslaves green again Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
On 07.08.2014 01:39, Dirk Bächle wrote: Hi guys, [...] I removed the /tmp/par-* folders, and biber seems to work again (versions of biber and biblatex match locally). In the next days, I'll let another pull request follow to get my Buildslaves green again Okay, I created pull request #162. Bill, can you review this one? Feel free to merge/accept it right away, it should be a fast-forward one... @Rob: In the pull request I'm removing the '.bcf' extension from the test biblatex_plain.py. The TeX file in question doesn't even contain a \bibliography entry, so it's probably too much to expect that a BCF is created. I hope this is okay, please object otherwise... Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
Rob, On 08.08.2014 00:04, Managan, Rob wrote: Dirk, I just realized that my desktop got updated to TeXLive 2013 recently and I now have biblatex 2.8 and biber 1.8 available (previously I was using versions 2.2 and 1.2 respectively). With those versions I still get the .bcf file generated. What versions did your system end up with? I am guessing it is an earlier system. I downgraded the SuSE 12.3 box to TeXLive 2012 again, which means biblatex.sty v1.7 and biber v0.9.9. I had the 2013 version first (installed from the internet directly), but then decided to switch to the packages as provided by SuSE...just to get all problems with biblatex/biber out of the way. But maybe the test is now still selecting the wrong bibtex backend? Thanks for your immediate feedback. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
[Scons-dev] Scons 2.3.2 regression, D tool...
Hi there, we have another user reporting trouble with SCons 2.3.2, in connection to the D tool...so it's time for us to act. I checked the sources, and the user's analysis appears to be partly correct: even when no D tool is present in the current system, the dmd tool gets loaded (meaning SCons.Tool.dmd.generate is executed). In general, the D tools use their own set of variables starting with D* (or _D*) so they shouldn't create much of a problem. But the D tools also set STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME to 0, independent of any setting that a tool like msvs.py made before. :( I'm open for a discussion and would like to hear other ideas. My way to fix this would be: - D tools shouldn't be loaded by default, but get an optional tool ( - tools=['default', 'dmd'] is mandatory). It's not enough to say Let's try to detect them, and load them automatically., as long as the STATIC_SHARED flag gets set to a fixed value. An option would be to use env.Default() to set this flag, but I don't know if D can then cope with a possible setting of 1 (Russel?). - Docs should get amended, to warn the user about the overwrite of the STATIC_AND_SHARED flag. - Then push out a new patch release as soon as possible. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Test failures with biblattex
On 08.08.2014 03:37, Bill Deegan wrote: Dirk, Merged. Let's see how that works! Looks good already...one failing test left, for which I opened PR #163. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Error in documentation
Hi Mark, On 08.08.2014 17:05, Mark A. Flacy wrote: Greetings, While answering a question on the users list, I referenced http://www.scons.org/doc/production/HTML/scons-user.html#idp24303632 I went back and read it more closely and noticed that the output in the final examples don't match the output when I run it. this typo is already filed as issue #2788. Can you attach your changes as patch there, please? Thanks a lot in advance. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
Hi Russel, thanks for the quick reply. On 09.08.2014 14:02, Russel Winder wrote: On Fri, 2014-08-08 at 19:07 +0200, Dirk Bächle wrote: Hi there, we have another user reporting trouble with SCons 2.3.2, in connection to the D tool...so it's time for us to act. I checked the sources, and the user's analysis appears to be partly correct: even when no D tool is present in the current system, the dmd tool gets loaded (meaning SCons.Tool.dmd.generate is executed). That should not be the case, and wasn't for the tests I made. However I only tested on Linux and OSX and this user is using Windows, a platform I cannot test on and no-one else bothered to. Can you do the following test please: - In the D tools of your local installation, change the names of the executables that get detected...such that they can't possibly be found in the system (in exists() rename dmd - foodmd, ...). - Then create a simple SConstruct: env = Environment() print env.Dump() - Run through SCons, and observe that the Environment has D variables (= dmd.py Tool got loaded anyway) This happens, because the line d_compiler = FindTool(d_compilers, env) or d_compilers[0] adds dmd to the tool list anyway. In the default Tool's generate() method, the returned list is simply loaded without checking the existence of tools again. It is okay, and wanted, for as many Tools to get loaded by default...as long as their variables don't clash. The STATIC_AND_SHARED variable is only set by: - C/C++ compilers (where only the first match gets selected, based on the current platform) and - masm (only under Windows, where it sets the same value as a Windows C/C++ compiler - no clash) Only the D tools are forcing it back to 0, and this is not compatible with the setting that the msvc and mingw Tool make. This is the main culprit, and we either have to remove the explicit setting of STATIC_AND_SHARED from the D tools (relying on the C/C++ tools to do the right thing, or, if they really need the setting under Windows, we have to remove the D tools from the default toolchain. So, is STATIC_AND_SHARED=0 *required* for the D tools to work properly under Windows? Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 15:18, Gary Oberbrunner wrote: I'll be home in a few days. I have plenty of Windows machines and can do the required tests. As for not noticing that that change broke Windows def files, I should have checked that. (There was a new failure in the Windows buildbot but I didn't notice it.) As for the large change, Russel has it right. Git would have made that much simpler. And Anatoly, please be nice. We're all volunteers here, trying to do a good job with limited time available. Yep, I second that. Things like these just happen sometimes, so we can only try to be professional about it and get the issue out of the way asap. I also screwed up badly several times when trying to revamp the doc toolchain, so, Russel, please don't feel offended. You did a great job on the D integration, it's just this tiny issue that's left...so let's work things out. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 18:01, William Blevins wrote: As we know Gary and I are Git people who like transient feature branches that can be packaged for merge and are not Mercurial experts. The D changes extended over a very long period in a Mercurial feature fork with me keeping the fork up to date with the mainline. On merge Mercurial was unable to cope with this, creating zillions of spurious crap. Really we do not have a good workflow and that is at the heart of the big commit. I just made a single commit of all the D changes from over a two year period because it was the only way to not have a huge internal Mercurial mess. +1, I'm new, so my voice counts less, but my past experiences with revision management tools is that GIT Mercurial in every way; Mercurial is more like distributed CVS (and CVS was a lifetime/nightmare ago). I don't see it that way. I could make a long rant here, but my primary point is that maybe a motion should be made for moving SCons to GIT? We could at least take a vote and see if anyone is adamantly opposed. I'm against using a hammer to do the job of a screwdriver. Have to disagree again. Feel free to raise a poll for Git (I think we did that some time ago, and hg won...so why don't we stick to that?), but I don't see how this can help us. Work doesn't get less with another transition, and we just moved from svn-hg. All the changes in the docs, the changes in the basic workflows (branching, merging, releasing) and whatever, now start to settle finally...such that one could start to really do some proper work. I just put quite some energy into updating the Wiki pages about how to branch/merge with Mercurial...and you want to move *again*? I've seen pull requests with several commits on a branch, where the changes were very easy to follow. That's when people say: Oh, why don't you squash stuff together, so the log graph looks nicer. On other occasions, reviewers complained because the single commit was too big and too difficult to review. Well, yeah, that happens...it's life. The merge that Russel mentioned was actually, as far as I remember, successful and left the repo in a consistent state. It was again the only problem that the log graph looked a bit odd, based on Mercurial's inner workings. I really don't want to transition to another VCS again, only for that... -1 from me at the moment. Let's not concentrate on having pretty revision graphs, but on providing and releasing a software that works for our users... ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 19:33, William Blevins wrote: Fair; I realize its non-trivial. My complaints with Mercurial aren't related to one particular event. It's more a personal preference in tool usability. If HG suites the needs of the team, I'll just have to muddle through it. Again, you are right. This issue has nothing to do with Mercurial. Sorry, William No need to excuse, and it's totally fine to be a little biased. It's just that exactly these kind of discussions keep the core developers from their actual work...which might be one of the reasons why so many bugs in our tracker are untouched. Sorry, I just couldn't resist to give this wink. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 19:35, Russel Winder wrote: [...] This happens, because the line d_compiler = FindTool(d_compilers, env) or d_compilers[0] adds dmd to the tool list anyway. In the default Tool's generate() method, the returned list is simply loaded without checking the existence of tools again. Indeed as OP said. However all the Fortran tools are being brought in as well even though I have no Fortran compiler, why is this happening and should it be fixed? I think the reasoning behind this is, that if a Fortran compiler isn't found during construction of the Environment (because PATH is not setup properly), it gives the user a chance to directly set env['FORTRAN'] = /path/to/fc and get things going for him. Whether this was a good idea is quite debatable, but that would be my explanation for why things are as they are. [...] So I suggest the way forward is to remove the setting of this from the D tools and forget it for now — the reason I original set this a couple of years ago was due to a (mis-)feature of D that is changing very rapidly just now. How this is treated for D needs reviewing in a few months time when the next GDC and LDC compilers get formally released. Another option would be to not completely remove it, but use env.Default() such that the variable is only set if it doesn't exist yet. That should do the trick too. Thanks a lot for looking at this immediately. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 19:42, Russel Winder wrote: On Sat, 2014-08-09 at 15:33 +0200, Dirk Bächle wrote: […] In amongst all the bluster and anger, I think we have iterated to a very simple solution to both bugs: remove the setting of the symbol in the D tools since it is problematic and the Fortran tools do not touch it (even if they ought to be :-). I will commit the change and push to my Mercurial repository on BitBucket, but won't ask for a pull request, unless that would be helpful to people with Windows kit trying this out. I think it is extremely helpful to create this PR right now. The sooner we can test this in the mainline (and against our Buildbots), and the sooner users can pull a fixed version straight from the repo for testing, the better it is for us. We have clearly identified the issue and have a fix for it. Let's not wait... We do need a test for this, but I am not sure what form it should take so all suggestions welcome. I don't think we need an extra test. There are lots of D tool tests already, and the existing C/C++ tests actually triggered errors on the Windows buildslave...but nobody had a closer look because the buildbot stuff was in a bad state recently. So the all red appeared to be normal. This is a fix for a previous commit, so there doesn't have to be a test for it, IMO. The actual testing would be to have one Buildslave under Windows with D tools installed, and another one without. So again, feel free to leap ahead and remove the offending line...no tests required from my side. And if things go havoc this time, you can put the blame on me afterwards. ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 20:05, Russel Winder wrote: On Sat, 2014-08-09 at 13:33 -0400, William Blevins wrote: [...] Our problem is that it seems that maintaining a long running synchronized clone of a default branch leads Mercurial to having problems on a final merge. Advice from a Mercurial expert was that actually the result was fine, just ugly. Gary felt it was an ugly too far and that we should not use that way of merging. Nothing wrong there per se. It just brings into stark relief that we do not have a reasonable workflow just now. I still don't feel the logic behind this reasoning. For me, the fact that the end result looks messy doesn't have anything to do with the workflow itself. I am getting to the stage of not doing anything because any process I use leads to problems. Oh Russel, I didn't know you're *that* sensible... ;) Please don't give up messing with our repos and minds! Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 09.08.2014 20:28, Russel Winder wrote: On Sat, 2014-08-09 at 20:09 +0200, Dirk Bächle wrote: […] I think it is extremely helpful to create this PR right now. The sooner we can test this in the mainline (and against our Buildbots), and the sooner users can pull a fixed version straight from the repo for testing, the better it is for us. We have clearly identified the issue and have a fix for it. Let's not wait... Duly done, #166. Merged, Buildbots are through...and the .def errors are gone. Thanks again for the fix! So we can now point users to the latest dev version, if this error shows up for them. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Scons 2.3.2 regression, D tool...
On 10.08.2014 08:43, Mark A. Flacy wrote: On Saturday, August 09, 2014 10:41:10 PM Dirk Bächle wrote: On 09.08.2014 22:02, Mark A. Flacy wrote: Well, the messy bit is information that nobody really cares about; so why keep it around? I may have read Russel's comment wrong, and understood it like The result is messy, so we don't have a proper workflow and need to switch the VCS.. This would be wrong in my opinion..but this meaning probably wasn't intended anyway. About information: I'm a fan of keeping history as it happens...including errors, and their immediate fixes. Today, I don't know what information I might need tomorrow, or in 2 weeks/months. The seemingly unusable commit (or even only its checkin comment) that I rebase/squash away today, might serve as a starting point for a fresh approach at a later time. Highly speculative, I know...but that's how I see it. Well, the attached graphic is a zoomed-out-to-the-max screenshot of the history of ONE file in our ClearCase repository. There are more branches to the right and bottom of the picture. *Many* more branches. Doesn't really shock me, I see these at work every day. ;) Well, maybe not *that* big, but large enough to make your head spin when you look at the screen for too long... Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] on demand Program tool initialization
Hi Anatoly, On 10.08.2014 10:16, anatoly techtonik wrote: Hi, I am looking at D SConstruct example. http://stackoverflow.com/a/9917185/239247 How does Program determine that it needs to call D tool? in this case the D Builders are registered with the Program() method as source builders. This means, if Program() encounters a non-object extension (like the suffix .d) it will look up its internal list of source builders, and try to find one for converting .d-*.o. So the decision is based on the suffix of the source file. What is the minimum information that tool should tell about itself so that Program can determine its source files and not confuse them with anything else? Because this way you don't need to search for tool or initialize it until there is a need to build source file for that tool. There might be need to adjust default parameters for that tool, but that's another problem. So *on demand* Program tool initialization means that tool is fully initialized only when build demands building targets that are associated with some tool. Is it possible to add this part already? We have started to discuss these kind of things on the ToolchainRevamp pages in the Wiki: http://www.scons.org/wiki/ToolchainRevamp and http://scons.org/wiki/RevampToolsSubsystem . Feel free to add your points and comments there. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Issue 2964
On 10.08.2014 10:59, Russel Winder wrote: Can Issue 2964 be marked as Fixed and Closed? http://scons.tigris.org/issues/show_bug.cgi?id=2964 Done, thanks for noting. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] on demand Program tool initialization
On 10.08.2014 19:24, anatoly techtonik wrote: Right now the scenario plot is this. [ ] I feel like SCons is slow. . it probably because SCons initializes unneeded tools, which take time to discover their binaries [ ] I don't want SCons to initialize unneeded tools or search for binaries if that slows me down Another scenario: [ ] I want to know what tools my script uses without reading docs (because they often outdated), compiling anything or installing all these tools A development: [ ] I want scons to tell me what tools do I needed to compile specific script or target That's very good, just continue like this...and feel free to use a tool of your own choice to select and maintain further paragraphs (=use cases). We can always reference them per URL from the Wiki pages, I guess. Thanks a lot for contributing to the new Toolchain design. Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Wow!
On 12.08.2014 02:50, Gary Oberbrunner wrote: I go on vacation for a couple of weeks and all this stuff happens while I'm gone! It'll take me a little while to catch up. Hi Gary, maybe you should do that more often then... ;) Just take your time to check what's new and important. I think we're quite okay in general at the moment. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Questions about the SCons release process.
William, On 12.08.2014 01:46, William Blevins wrote: Will the next SCons update (SCons 2.3.3) be released from default based? Based on the current commit workflow this seems to be the case. Is this a good model for project releases? In this case, the reason for releasing an update is to fix a VERY specific issue. Wouldn't it be better to release with a patch fix on top of branch rel_2.3.2 as 2.3.2-2 rather than release everything current in default which may include changes intended for SCons 3.X? I understand your concerns, but I don't see any changes in the diff 2.3.2-current that would break Python2.x compatibility. So I'd like to continue with release numbering as normal, everything else might confuse the user. I am concerned that the leading contributor to SCons 2.X instability, including breaking pre-2.7 compatibility, is the lack of separation between 2.X baselines and 3.X. Maybe there needs to be a workflow developed for this? I still think that we really have enough workflows, now we actually have to do wome work. Boring, I know...it's so much more fun to fiddle with tech stuff and new interfaces while migrating from one technology to the other. ;) BTW, the OpenHatch project just migrated its issues from Roundup to Github, because with having the tracker close to the sources they would save so much work. They wrote scripts and did all their stuff, and now the first users complain because Github doesn't have a notion of depends on, for example. Lots of hours down the drain basically, and let me just add: I warned them in advance... I don't want the same thing happening to us, our resources are still to scarce. We might consider handling version releases more like the python release model (or some other industry-standard-like release model). Releasing/branching off from default is an industry standard. Maybe I am misunderstanding. We are releasing SCons 2.3.3 from default, please consider merging pull request #164 to improve the documentation on http://scons.tigris.org/issues/show_bug.cgi?id=2395 I saw your documentation PRs, and I'll make sure that they go in before the release. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] catching up
On 17.08.2014 20:54, Gary Oberbrunner wrote: There's probably not much point in my trying to respond to many of the threads that have been running here in the last couple of weeks; if there are things that should be addressed please bring them up. As for dev priorities, I think we have two big important projects (not counting releasing 2.3.3 which I guess we should do after some further testing?): Python 3 port, and Toolchain. [...] (I know there are lots of other things going on -- I didn't intend the above to be exhaustive.) Making this a little more complete: - patch/bug release 2.3.3? (fix for D tools) - Node class patch (switch to __slots__) - Integration of stubprocess.py - release 2.4? - then, Python3 and Toolchain? Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] catching up
Hi Anatoly, On 18.08.2014 18:52, anatoly techtonik wrote: On Mon, Aug 18, 2014 at 12:36 AM, Dirk Bächle tshor...@gmx.de wrote: Making this a little more complete: - patch/bug release 2.3.3? (fix for D tools) - Node class patch (switch to __slots__) Will that be really useful? http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/ yes, it will be useful. It helps us, and our users, to save memory...right now, with the current Python versions. Not with some imaginary ones... Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] catching up
Hi there, On 24.08.2014 21:02, anatoly techtonik wrote: On Sun, Aug 24, 2014 at 9:09 PM, Gary Oberbrunner ga...@oberbrunner.com wrote: [...] Then I'd like to revisit Mercurial workflow, because we need to clarify how to rebase pull requests. I would really like to understand why we need a rebase for pull requests in the first place. I see these two features - stubprocess.py and __slots__ as branches (ideally all feature should be optional, so that you can turn off them, for example for porting code to Lua). Lua, what? Where does that suddenly come from? I don't see any porting activities to other languages on the roadmap, and I don't remember any discussions about it either. So why should we give our current development a direction based on imaginary features? Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] catching up
On 25.08.2014 20:54, Bill Deegan wrote: Looks like some of the memory clean up may have broken interactive mode. Take a look at: http://scons.tigris.org/issues/show_bug.cgi?id=2971 Yeah, I noticed. I'll contact the user and try to find out what goes wrong, hopefully resulting in a MWE and additional testcase(s). Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] catching up
Anatoly, please find a few comments below. On 25.08.2014 10:51, anatoly techtonik wrote: On Mon, Aug 25, 2014 at 1:52 AM, Dirk Bächle tshor...@gmx.de wrote: On 24.08.2014 21:02, anatoly techtonik wrote: On Sun, Aug 24, 2014 at 9:09 PM, Gary Oberbrunner ga...@oberbrunner.com wrote: Then I'd like to revisit Mercurial workflow, because we need to clarify how to rebase pull requests. I would really like to understand why we need a rebase for pull requests in the first place. 1. To get clean linear history, which humans can browse without advanced graphical tools. Well, unfortunately history isn't always linear when you're working in parallel with several people. I don't mind if a developer locally squashes his commits or uses the MQ extension to reorganize his current bug branch/bookmark for better readability. But once his commits are pushed to his fork at bitbucket (not even the master), history should be regarded frozen. So I wouldn't vote for making rebases mandatory for any pull request...or even go as far as refusing non-fast-forward pushes on the server (current master). I just don't think this is the way to go, especially if we're after getting more contributors into the project. We have to be open towards newcomers, for which we should offer a very basic workflow (like we have described now in the Wiki). And experienced developers can feel free to fiddle with their commits while preparing their next pull request. All together, we're already in a very good state...and for those long-termed dev branches it's just up to a few people to finally detect and embrace the MQ and rebase extensions of Mercurial. 2. To resolve conflicts. Just for example https://bitbucket.org/scons/scons/pull-request/155/adds-a-test-case-demonstrating-that-issue/diff#Ltest/SWIG/include-directive.pyT59 A conflict has to get resolved, for rebase as well as for a simple merge. I don't see how a rebase is more helpful in this situation. 2.1 To update CHANGES.txt after other PR did this 3. Addressing review comments Once a change was committed, I can only add another commit to amend things. I don't see how a (local) rebase can help here...as written above, I exclude remote rebasing from my considerations. 4. Testing PR on top of other fixes 5. Squashing commits 6. Moving stuff to a different branch There are other options available, like MQ again, to accomplish these. I still don't see the urgent need to have rebasing...sorry. I guess for every reference you show me, I can come up with a page against git's rebase, like the following: http://paul.stadig.name/2010/12/thou-shalt-not-lie-git-rebase-ammend.html http://rlc.vlinder.ca/blog/2013/03/flawed-ways-of-working-git-rebase/ http://jhw.dreamwidth.org/1868.html , but an URL fight is definitely not what I'm after. For me, it somehow shows that this is a highly opinionated discussion...further meaning that there are no clear facts to decide for one side or the other. So we probably can't lose if we switch (except the efforts for another migration), but we also don't lose a lot if the merge workflows stay as they are now. 7. Finally a reason to know Mercurial better As I said, a highly opinionated discussion... ;) I see these two features - stubprocess.py and __slots__ as branches (ideally all feature should be optional, so that you can turn off them, for example for porting code to Lua). Lua, what? Where does that suddenly come from? I don't see any porting activities to other languages on the roadmap, and I don't remember any discussions about it either. So why should we give our current development a direction based on imaginary features? Sorry, it is just a bit of forward thinking. The same stuff that made me mad when I saw the Docbook toolchain committed. Last time I tried to clone SCons over SSH it took 20 minutes and I knew it will happen. As for Lua. Right now there are better systems than SCons in some aspects, for example http://industriousone.com/premake in Lua which is absent from http://scons.org/wiki/SconsVsOtherBuildTools and http://martine.github.io/ninja/ used by Chromium guys, who I believe ditched SCons at some point even though they've built a Hammer harness on top of it. The reason why such tools appear is that the old code base becomes too overcomplicated for new features to add, and I am afraid that people primarily abandon projects for this reason. Okay, but that's your own perception. Do you know of any projects (names, please) that have ditched SCons because the code base was too complex? When I google for scons slow, I seem to get more project-related hits than for scons complicated or scons complex. And for projects like Ardour, Dolfin or Chromium, speed was the main reason to switch. I'm not trying to toss your concerns aside here, but I would like to take care of the highest prio items first...and highest prio gets somewhat dictated by what most users want, right? I want to be able
Re: [Scons-dev] User Guide generation questions
On 30.08.2014 00:12, William Blevins wrote: I am trying to make some misc updates to the User Guide, and I am having some issues. I thought it might be a problem with a document do I tried docs-validate and there were two errors: [...] 81.11% (176/217) doc/user/scons_ex.xml ERROR: doc/user/scons_ex.xml fails to validate: Element 'programlisting': This element is not expected. Expected is one of ( {http://www.scons.org/dbxsd/v1.0}refentry.class http://www.scons.org/dbxsd/v1.0%7Drefentry.class, {http://www.scons.org/dbxsd/v1.0}refentry http://www.scons.org/dbxsd/v1.0%7Drefentry, {http://www.scons.org/dbxsd/v1.0}section.class http://www.scons.org/dbxsd/v1.0%7Dsection.class, {http://www.scons.org/dbxsd/v1.0}section http://www.scons.org/dbxsd/v1.0%7Dsection, {http://www.scons.org/dbxsd/v1.0}toc http://www.scons.org/dbxsd/v1.0%7Dtoc, {http://www.scons.org/dbxsd/v1.0}lot http://www.scons.org/dbxsd/v1.0%7Dlot, {http://www.scons.org/dbxsd/v1.0}index http://www.scons.org/dbxsd/v1.0%7Dindex, {http://www.scons.org/dbxsd/v1.0}glossary http://www.scons.org/dbxsd/v1.0%7Dglossary, {http://www.scons.org/dbxsd/v1.0}bibliography http://www.scons.org/dbxsd/v1.0%7Dbibliography, {http://www.scons.org/dbxsd/v1.0}simplesect http://www.scons.org/dbxsd/v1.0%7Dsimplesect )., line 4116 81.57% (177/217) doc/user/sconf.xml [...] 92.63% (201/217) doc/user/scons_exi.xml ERROR: doc/user/scons_exi.xml fails to validate: Element 'programlisting': This element is not expected. Expected is one of ( {http://www.scons.org/dbxsd/v1.0}refentry.class http://www.scons.org/dbxsd/v1.0%7Drefentry.class, {http://www.scons.org/dbxsd/v1.0}refentry http://www.scons.org/dbxsd/v1.0%7Drefentry, {http://www.scons.org/dbxsd/v1.0}section.class http://www.scons.org/dbxsd/v1.0%7Dsection.class, {http://www.scons.org/dbxsd/v1.0}section http://www.scons.org/dbxsd/v1.0%7Dsection, {http://www.scons.org/dbxsd/v1.0}toc http://www.scons.org/dbxsd/v1.0%7Dtoc, {http://www.scons.org/dbxsd/v1.0}lot http://www.scons.org/dbxsd/v1.0%7Dlot, {http://www.scons.org/dbxsd/v1.0}index http://www.scons.org/dbxsd/v1.0%7Dindex, {http://www.scons.org/dbxsd/v1.0}glossary http://www.scons.org/dbxsd/v1.0%7Dglossary, {http://www.scons.org/dbxsd/v1.0}bibliography http://www.scons.org/dbxsd/v1.0%7Dbibliography, {http://www.scons.org/dbxsd/v1.0}simplesect http://www.scons.org/dbxsd/v1.0%7Dsimplesect )., line 4367 Validation failed! Please correct the errors above and try again. Can I remove those sections? I'm not that familiar with the schema. The failing files scons_ex.xml and scons_exi.xml are generated during the actual build process. Remove them by hand, or do a scons -c before trying to validate... Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] stubprocess and slots
Hi Gary, On 30.08.2014 14:36, Gary Oberbrunner wrote: I'll look at integrating stubprocess.py today. Jason, I presume you get credit for that in the changelog? I'll also see if I can track down any more failing tests, and look over the open pull requests. sounds very good. Dirk, I presume __slots__ is in your court? Sure, I have a full patch against the current trunk ready for this. Would you rather like to see this as simple PR, or should I open a named branch for it? I can try to split the numerous changes into several commits, if this somehow helps for the review...but I can't ensure that the sources work for any intermediate versions, only in the final state all tests pass. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] User Guide generation questions
William, On 31.08.2014 19:50, William Blevins wrote: This is happening from a clean scons checkout with no local changes. I am currently at revision 3187:605ae0a39155. I assume this is also an issue in default latest. Using python 2.6 on a RHEL6 or equivalent linux distro. I'm currently at revision 3189:98dea26fbc96 and I can build the docs just fine. My working directory is #doc/user and I call python ../../src/script/scons.py . Please ensure that you have the following Python bindings on your system libxml2+libxslt or lxml. Regards, Dirk V/R, William On Sun, Aug 31, 2014 at 12:56 PM, Mark Flacy mfl...@verizon.net mailto:mfl...@verizon.net wrote: Do you have a branch/repo I can pull from/clone to see? I use docbook a lot. On August 31, 2014 10:51:14 AM CDT, William Blevins wblevins...@gmail.com mailto:wblevins...@gmail.com wrote: That may be a valid problem though because when I run *scons* from #*doc/user* the index.html generated from the build is completely empty. V/R, William ___ Scons-dev mailing list Scons-dev@scons.org http://two.pairlist.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Buildbot, Java and D...
Hi William, On 13.09.2014 00:59, William Blevins wrote: # No result if tools not available test.no_result( condition=(test.where_is( 'javac' ) is None) ) test.no_result( condition=(test.where_is( 'jar' ) is None) ) # This test is known to fail as of July 2014; see Tigris issue 1771 and issue 2931. # Once the underlying issue is corrected, this test should be re-enabled. test.skip_test('Skipping derived-source test until issue 1771 is fixed.\n') Based on the above code snippet, I don't understand how that is possible. If Java is not installed, then the test should be a no result. and here is what the documentation of TestCmd.no_result() says about this: def no_result(self = None, condition = 1, function = None, skip = 0): Causes a test to exit with no valid result. By default, the no_result() method reports NO RESULT for the test and exits with a status of 2. If a condition argument is supplied, the test fails only if the condition is true. The point here is that the no_result() method stops the current test immediately with a sys.exit(2)...so in the case where one of javac/jar can't be found, the skip_test() isn't reached. That's where the exit code of 2 comes from, I guess. For an immediate fix, I'd like to suggest simply moving the skip_test() line above the no_result()s. Does that sound reasonable? Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Buildbot, Java and D...
On 13.09.2014 17:44, William Blevins wrote: Dirke, 'tis a week t'early ferrr that. ;) The top two no_result calls need to work when we add the test for real, so we might as well fix them. I think you answered your own question, looks like the skip option needs to be enabled, so the test is skipped rather than fails; however, the real question here is, why are we running tests on a buildbot without Java installed? Is ignoring a whole language toolchain reasonable? I would say so, yes. The testsuite can be run by any user who wants to check whether his patch introduces a regression. For this, he wants a clean run of all tests (passed or skipped) before, and after applying his changes. And we don't want to force him having to install Java, SWIG, Qt, D,..., and whatnot. The tests have to cope with uninstalled applications/tools, and this gets tested as well...by not installing anything on every Buildbot. As long as we have *some* machines with Java in the mix, we should be doing fine. Which leads to the more interesting question: How do we guarantee full test coverage, i.e. ensure that each test is properly run at least once on a Buildslave? Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] runtests.py and multithreading
On 13.09.2014 22:06, William Blevins wrote: Team, I was wondering if I could runtests.py multithreaded. It appears that someone started adding functionality for it, but an argument was never added for public use, so jobs is always hard-coded to 1. The runtest.py supports the standard -j / --jobs option for this, if you have the threading module installed. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Buildbot, Java and D...
Hey guys, On 14.09.2014 11:56, Russel Winder wrote: On Sat, 2014-09-13 at 14:46 -0400, William Blevins wrote: Just to clarify. I am assuming that the python3 baseline requires 2.7+, and the current 2.X baseline is still supposed to support 2.6+. If this is wrong, then I need some clarification because someone informed me incorrectly or my memory sucks or both. The decision was taken to make 2.7 the baseline for all of SCons. This would be helpful in going forward with a Python 2 and Python 3 compatible codebase since lots of Python 3 is being backported to 2.7 anyway. let's not fight too much over this, okay? It's clear that we're in a transition phase from 2.6 to 2.7. Python 2.7 really *is* the new floor for the Python3 branch, so once this gets merged back to default it's settled and done. For the current development on the default branch we don't deliberately try to break 2.6 compatibility, as long as possible. This is mainly for the convenience of users that try the latest commits of the dev version, e.g. if bugs of the released 2.3.3 (which had 2.6 as floor) are fixed. So, if you can develop or fix something by using Python 2.6 lingo *and* it also works in Python 2.7+3.x, that's the preferred method at the moment. If you simply can't do without a Python 2.7 feature, then it's okay to introduce the break. But we're not talking about format specifiers here, but concepts in the range of slots, metaclasses, functools,..well, I hope you get the idea. People then started the issue of 2.6 compliance and a few changes have been made to the default/tip codebase to support that, but currently officially as far as I know the base is 2.7 and all the 2.6 fixes are just making the code ugly. Just let the code be ugly. If most things still work for a user under 2.6, he certainly won't mind. ;) Confusion is rife on this one now. :-( I hope my comments above made things a little clearer. To put it shortly, the current floors are: default branch = 2.6 (as long as possible, before merging in python3 support) python3 branch = 2.7 Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Buildbot, Java and D...
On 14.09.2014 12:07, Russel Winder wrote: On Sat, 2014-09-13 at 00:06 +0200, Dirk Bächle wrote: Hi there, I just checked the latest Buildbot runs for my slaves and see test/D/SharedObjects/sconstest-ldc.py failing under Fedora20. More importantly, the other slaves can't get Hummm… how was LDC installed on this machine? Actually I am guessing using yum which means it is the 2.057 system, which is very old and doesn't support dynamic libraries. I am using a locally compiled system which is 2.065 which is the first to have reasonable dynamic library support. I installed ldc from the Package thingy (GUI), which means having used yum in the background, I guess. The exact version currently used is ldc-2-57.20140305git6e908ff.fc20 (32-bit) Thus the test should be switched off for CI usage. The question is what is the right way of ignoring the test unless the version of LDC is the right one: I guess run a trial LDC compile and discover the version number? I think the question is more like: How should the actual ldc Tool behave? What should happen when a user has a version of ldc installed in his system, but tries to build a project with dynamic libs involved? Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] JetBrains
On 16.09.2014 09:39, Arve Knudsen wrote: CLion is supposed to support more build tools in the future, I've been told. Maybe put some pressure on them to support SCons (via their forums f.ex.)? Building up pressure is the wrong approach, we could simply offer them our help...if they should be interested to add some basic SCons support to their tools. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Builbot test fails
Hi Russel, On 18.09.2014 07:50, Russel Winder wrote: Looking at recent Buildbot results, the D tools are causing test fails on the Fedora20 and Debian76 builders. I am not sure why the Fedora20 builder test/D/SharedObjects/sconstest-ldc.py is failing as the recent pull request was explicitly to fix that issue. I will investigate. I just checked on the machine directly: the phobos lib under Fedora20 has the name phobos-ldc, but your ldc Tool assumes phobos2 to be the default. On the Debian76 builder there appear to be three sorts of error: 1. [...] 2. ncurses is not installed in the debian76 builder and one of the D tests is about linking to it. Does the SCons test framework allow for detection of libraries as well as detection of executables? I'd rather rewrite the testcase to include local headers only, such that the build is self-contained. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] 2.3.3 Release issue ?
On 27.09.2014 16:41, Gary Oberbrunner wrote: On Sat, Sep 27, 2014 at 10:36 AM, Dirk Bächle tshor...@gmx.de wrote: ... Is it too late to stop you with that? I'm currently working on the fix for #2971, and would like to get that into 2.3.4 as well if possible. Sorry, I didn't know that a v2.3.4 was planned/pending, else I would've spoken up earlier. :P No problem, I can squeeze it in. Let me know when you're ready. Thanks for waiting, created PR #188. Latch on. :) Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] [scons-dev] Fwd: December SourceForge Project of the Month Ballot
Hi, I put my vote a few minutes ago. We should really take this serious and post a call to vote on the user ML as well. The two last months had only 13 and 34 votes in total each. So we have a good chance to win. ;) Best regards, Dirk On 17.10.2014 22:52, Steven Knight wrote: Not sure if this just went to me, or to any of you. Hope everyone's well... --SK -- Forwarded message -- From: *Elizabeth Daniels* edani...@slashdotmedia.com mailto:edani...@slashdotmedia.com Date: Fri, Oct 17, 2014 at 12:55 PM Subject: December SourceForge Project of the Month Ballot To: sfcommunity communityt...@sourceforge.net mailto:communityt...@sourceforge.net Dear SourceForge Project Admin, Your project has been selected for the ballot for December Community Choice Project of the Month. The vote is conducted on SourceForge and runs through Nov 15th. The blog post linking to this vote is @ _https://sourceforge.net/blog/community-choice-project-of-the-month-vote-december___, and the vote itself is @ _https://sourceforge.net/p/potm/discussion/vote/thread/7144005c/___ Anyone with a SourceForge account is eligible to vote. We require a SourceForge account because we have had problems with users trying to abuse the vote without some sort of authentication. The Project of the Month gets listed on top of the SourceForge.net website's front page, which is also linked to an interview about your project. The interview remains on our site in perpetuity so you can use the link to talk about your project, your community, and so on. Interviews are conducted by email. And you'll also be listed on http://sourceforge.net/potm http://sourceforge.net/potm. Thanks so much for being part of the SourceForge community and may the best project win! Let us know how we can help out with this process by getting in touch with us @ communityt...@sourceforge.net mailto:communityt...@sourceforge.net. Best wishes, -- Elizabeth Daniels Senior Content Editor SourceForge.net http://sourceforge.net | 415.713.0229 tel:415.713.0229 ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] License SConsBuildBricks_path.svg
Just a quick addendum: I checked the SVG in the design archive at https://bitbucket.org/dirkbaechle/scons_design and it's properly tagged as cc-by-nc-sa there...looks as if I simply forgot to replace the version in the actual SCons doc dirs with it. So, this can be fixed quite easily...but still leaves the question about which license forms would be okay for Debian. Dirk On 24.10.2014 07:39, Jörg Frings-Fürst wrote: Hello, I'm the new maintainer of the packages scons and scons-doc on Debian. There is a problem with the license of SConsBuildBricks_path.svg. The license cc-by-nc-sa is tagged as non free and therefore the package scons-doc must be removed. Output of lintian: E: scons-doc source: license-problem-cc-by-nc-sa doc/design/titlepage/SConsBuildBricks_path.svg N: N:The given source file is copyrighted under the non free license creative N:commons non commercial share alike. N: N:Severity: serious, Certainty: possible N: N:Check: cruft, Type: source Any hints? CU Jörg ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
[Scons-dev] Bonjour de Lyon...
Hi there, I just wanted to send you all a quick Hello from the PyConFR 2014 in Lyon, France. I gave my talk yesterday, so my adrenaline level has dropped to something close to normal again by now. ;) Ill continue enjoying the weather and the belgian beer, have some fun too! Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Bonjour de Lyon...
Hi Bill, well, the talk was okay, I hope. I had prepared too many slides, so I had to skip a few things. Not sure if all of the listeners could follow the red thread...but the slides are online (Ill post the link to the video, once its published) so people can look up things later. Anyway, one could see the SCons logo at the start, and lots of links to our project at the end...that was the most important part to me. ;) While talking to several people, they were quite interested and didnt seem to know that SCons really exists. This made me think about whether we should consider to translate (at least parts of) our basic documentation, or the basic webpages to some other languages like French, Spanish, Portuguese...but thats a discussion for another day. I also met David Cournapeau (some of you guys will remember him), and he told me to say hello. He was very pleased to hear that were moving on with the project, and dont stick to Python 1.5x anymore. ;) Ill hit the street now, and roam around Lyon a bit... Regards, Dirk Gesendet:Montag, 27. Oktober 2014 um 03:06 Uhr Von:Bill Deegan b...@baddogconsulting.com An:SCons developer list scons-dev@scons.org Betreff:Re: [Scons-dev] Bonjour de Lyon... How did the talk go? -Bill On Sun, Oct 26, 2014 at 7:02 PM, Gary Oberbrunner ga...@oberbrunner.com wrote: Bonsoir! Hope youre enjoying Lyon and the conference; wish I could be there! On Sun, Oct 26, 2014 at 4:45 AM, Dirk Bchle tshor...@gmx.de wrote: Hi there, I just wanted to send you all a quick Hello from the PyConFR 2014 in Lyon, France. I gave my talk yesterday, so my adrenaline level has dropped to something close to normal again by now. ;) Ill continue enjoying the weather and the belgian beer, have some fun too! Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Hi there, On 30.10.2014 16:01, Ben Golding wrote: Thanks Gary. http://scons.tigris.org/issues/show_bug.cgi?id=2982 I started to add some additional trace to Taskmaster.py, and compare the logs from sequential vs. parallel builds. The bug seems to relate to how the node states are used for the side-effect nodes: - In executed_with_callbacks(), the side-effect state is set to NODE_NO_STATE. - In postprocess(), the logic to decrement the parent ref count is only active for a side-effect node with state NODE_EXECUTING. Although this helped me to understand the bug a bit better, I'm not sure what is the correct fix. I don't think there is anything to fix here...and it's no bug for me either. Please read the man page for the definition of a SideEffect, and when it should be used. My understanding is, that in your case the conditions do not apply...so you should rather define an Emitter (see http://www.scons.org/wiki/ToolsForFools for example) that adds sf0 and sf1 to the list of targets. Then the Install() should work as expected. Just my 2 cents. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Gary, On 01.11.2014 12:03, Gary Oberbrunner wrote: SideEffect may be the root cause of the problem, but Install should not behave this way, IMHO. If you pass some nodes to Install, it should either install them or fail the build with an error (don't know how to build... or similar). It shouldn't ever _not_ install them silently. I gave this some more thought, and when we talk about proper targets, or source files that always exist, I agree with you. Install should just work... However, a file marked as SideEffect is a different story... And that's because it has a fixed name, but can have different contents... depending on whether you're currently building a.foo or b.foo, as introduced in my last example. This means that if we supported it as full target, we would soon run into trouble with things like: scons a.foo b.foo install vs. scons b.foo a.foo install . We're primarily a build system, not an install system, and care about secure and stable builds...which also should include convergence at the top of our priority list. With this I mean, that a user expects that the build converges to a settled state (I do, at least), where all targets are properly updated...independent of the order in which the single build steps for the targets are executed. And it's obvious to me that we'd start violating this principle, when treating SideEffects as fully equivalent targets. Arguing against myself :-), we could change the doc for SideEffect to indicate that nodes that are SideEffects are not guaranteed to be created, and may or may not be ignored by targets that use them as sources. But, this kind of indeterminacy doesn't seem like a good user experience to me. The problem I described above, doesn't yield a good user experience either... ;) In my opinion, this is the point where we have to educate (horrible word, but I can't think of a better term right now) the user to not use SideEffect as a workaround for properly defining a Builder/Emitter instead. SideEffect should be reserved for files that only the compiler (or any other build step) cares about. If the *user* cares about a target file, and wants to work with it in the DAG---by using it as input file to another build step for example---he has to define it as a proper target (=Emitter). On Fri, Oct 31, 2014 at 7:42 PM, William Blevins wblevins...@gmail.com wrote: Team, Not to be contrary here, but I think personal opinions should be postponed until we determine if the definition of SideEffect per the SCons User Guide matches the actual behavior. http://www.scons.org/doc/production/HTML/scons-user.html [...] Reading the description, I think that the SideEffect behavior doesn't cover the Depends behaviors which Andrew desires. I agree, in the sense that the usage of SideEffect is simply the wrong approach for this use case. From my angle, the documentation matches the current behaviour just fine... However, it's probably a good idea to make it more clear that defining a SideEffect file means that the user doesn't care about what happens with the file. So, he can't rely on the file to be properly updated, or have a determined content at a given time. Therefore, he also can't Install/Copy/OtherBuilder() the file...because its contents are actually unknown. So much for my latest thoughts about this, I hope they make some sense. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
Hi Ben, On 03.11.2014 11:48, Ben Golding wrote: It seems one possible fix for this bug could be a warning/error if the user tries to Install() a side-effect file, and expanding/clarifying the docs for SideEffect(). Also, if Install() is permitted with a warning, the current behaviour (silently ignoring the file) must be fixed. A few other comments/questions come to mind: 1) If the contents of a side-effect file are indeterminate, how does telling SCons about this file help to have a deterministic (or convergent) build? What purpose does SideEffect() have at all? What would be the remaining valid use-cases, if Install() is not permitted? I tried to explain this in my last mail. SideEffect() kind of marks a file that isn't supposed to be used as a target, but exists intermediary during the build process/step. The name of this intermediary file is always the same (it's hard-wired into the compiler/builder), so when you would run SCons in parallel mode (-j n) you could get in trouble: one build thread could get started, while another one is running...so the intermediary file (the SideEffect) could get mixed up. As far as I know, that's the scenario the SideEffect() functionality was designed for, and is actually the only situation where the method should be used. As soon as the name of the intermediary file isn't hard-wired for example, but built from the file stem of its target/source files, the usage of SideEffect() gets superfluous: each target would create a uniquely named intermediary file along with it, so there's no danger of mixing up things. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... 2) The example given in docs of .pdb file is an interesting one. It looks like SideEffect() is not actually used in msvc.py. It's very well possible that the docs aren't up-to-date, so let's do something about it! :) A better example would probably be the docbook Tool when it builds an EPUB file... In fact, this indeterminate behaviour depends on whether the option -Fd [1] is used, what filename is specified with -Fd, and the debug information format option [2]. It is both possible and desirable, particularly in a parallel build, to either use -Z7 (foo.cc = foo.obj; no .pdb), or to use -Zi / -ZI with -Fd and a suitably unique filename (foo.cc = foo.obj, foo.pdb). However, using -Zi without explicit -Fd gives something like (foo.cc = foo.obj, vc100.pdb). As I wrote above, if the filenames are guaranteed to be unique, you don't need SideEffect at all... Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Likely bug - installing side effect files
On 03.11.2014 14:44, Ben Golding wrote: Hi Dirk, Thanks for the clarification. Nevertheless, what effect does this marking of the side effect file actually have during the build? What can I usefully do with the object returned by SideEffect()? Not very much, I'm afraid...and why would you want to anyway? I'm not a native speaker, so take it with a grain of salt please that it appears obvious to me, that declaring a file as SideEffect means that I don't really care about it. Does it have an effect during the parallel build? (like a mutex, restricting that only one builder instance can run concurrently if they share the same hard-coded filename) Yes, it does exactly that. Another thing that SideEffect() does, is to mark the file for getting removed on a scons -c. In the past, this has misled many writers of builders/tools, to use SideEffect() in favour of the more correct Clean()... If you are correct, this behaviour is certainly in contradiction with the docs: http://scons.org/doc/latest/HTML/scons-user/apd.html#f-sideeffect In the passage above I was referring to the might be removed as part of cleaning the directory in which it lives part...sorry, if this got you confused. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] script/scons
Hi Jörg, On 08.11.2014 11:42, Jörg Frings-Fürst wrote: Hello, from Helmut Grohne hel...@subdivi.de I have just get: 2. I am very uneasy about the following hunk to script/scons: | +# - running from source takes priority (since 2.3.2), excluding SCONS_LIB_DIR settings | +script_path = os.path.abspath(os.path.dirname(__file__)) | +source_path = os.path.join(script_path, '..', 'engine') | +libs.append(source_path) Importing random python modules from .. is a route to security issues. Even if upstream is keen on keeping this hack to make scons work better when used from source, the Debian package almost certainly should revert it. Any hints about this? I fail to see how this affects the integrity and security of a Debian installation/distribution. When Helmut Grohne says that the Debian package almost certainly should revert it. is this based on anything more than his very personal opinion, and a good portion of FUD? Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] script/scons
On 08.11.2014 12:41, Jörg Frings-Fürst wrote: Hallo Dirk, Am Samstag, den 08.11.2014, 11:58 +0100 schrieb Dirk Bächle: Hi Jörg, On 08.11.2014 11:42, Jörg Frings-Fürst wrote: Hello, from Helmut Grohne hel...@subdivi.de I have just get: [...] Any hints about this? I fail to see how this affects the integrity and security of a Debian installation/distribution. When Helmut Grohne says that the Debian package almost certainly should revert it. is this based on anything more than his very personal opinion, and a good portion of FUD? Form irc: [08:00:45] helmut is having . in the library path for a python application generally considered a vulnerability? [08:45:03] womble helmut: It certainly isn't a *good* thing. If it runs with any sort of elevated privileges, it's *definitely* exploitable. [09:56:04] carnil helmut, womble: reminds me as example to perl e.g. there is #588017, one puppet CVE in similar regard was http://puppetlabs.com/security/cve/cve-2014-3248, or #591676 [09:56:14] [zwiebelbot] Debian#588017: perl: current directory in @INC potentially harmful - https://bugs.debian.org/588017 In these first two references (I followed the given links), they talk about adding . (the current working directory) to the python path. We don't do that, we add scriptdir + .. + engine...which is actually a fully qualified path. It's just not normalized in the sense that it has a .. in it. Other than that, it's not different to any other absolute path like, let's say, /usr/lib/python2.7/site-packages [09:56:15] [zwiebelbot] Debian#591676: pylint: please either disable or document dynamic checks - https://bugs.debian.org/591676 In this last link, there is no adding of . to the python path mentioned...and adding .. is neither. So I don't regard it as being relevant to the current discussion. Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Fwd: PlatformIO and SCons
Hi Ivan, thanks a lot for notifying us about your project. I just had a short look at the website, and it looks really slick. My congratulations! On 15.11.2014 23:44, Ivan Kravets wrote: Hi, Thanks a lot for great SCons project! I'm an author of PlatformIO open-source project http://platformio.ikravets.com PlatformIO Source Code builder is based on SCons. I promote SCons for embedded developers :) See the section Smart Code Builder. Fast and Reliable. on the home page of project. Could I ask you to put an information about PlatformIO to this page http://www.scons.org/refer.php ? @Gary, @Bill: Can one of you comment on this? I'm not a 100% sure where our web sources are, was it already migrated to hg, or is it still in SVN (what's the repo link)? By the way, we also have a list of projects in our Wiki at http://scons.org/wiki/SconsProjects , so I'll add you there as well, okay? P.S: The SCons-end developers have problem with PIP SCons (they can't include SCons to dependency list and to install Scons via PIP without egg). I've been monitoring this pull-request https://bitbucket.org/scons/scons/pull-request/113/fix-for-bug-2769-which-should-allow-scons/diff for several months. Do you have any plans to fix this issue? Yes, it's still on our Todo list...and work on it will continue soon. We're sorry for the inconvenience, but it's a tricky issue (as everything related to packaging in Python is) and we don't want to break anything for other users. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] [Scons-users] Man page problems with installing latest scons from Bitbucket
Hi Gary, On 16.11.2014 02:23, Gary Oberbrunner wrote: Just one more thing on this, redirecting to scons-dev. I just updated my SCons build VM to Trusty (14.04) from Precise (12.04), and I get the same error Michael was seeing when trying to do a full packaging build of SCons: error: [Errno 2] No such file or directory: '/usr/lib/python2.7/distutils/command/wininst-6.0.exe', /usr/lib/python2.7/distutils/command/wininst-6.0.exe not included in the Debian packages. It seems the wininst-*.exe files have been removed from the python2.7-dev package in 14.04, although they were there in 12.04. I worked around it by copying them from a Windows machine with python distutils. Grr. thanks for figuring this out! I updated the page http://scons.org/wiki/SconsBuildRequirements accordingly, and also provided a TGZ archive containing the missing EXEs. Additionally, I moved the link to the requirements directly into the Contributing section of the developer guide ( http://scons.org/wiki/DeveloperGuide ). I hope that makes sense... Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] [Scons-users] Man page problems with installing latest scons from Bitbucket
Hi Bill, On 16.11.2014 21:01, Bill Deegan wrote: Dirk, Are they available from another apt-repo? Maybe one of the unsafe ones? (Where they stick blobs) I'm not sure (didn't check)...but they're definitely not contained in the python source package. I added the source repos to my UpdateManager, and then did an apt-get source python. Nada... Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] [Scons-users] Man page problems with installing latest scons from Bitbucket
On 17.11.2014 00:38, Gary Oberbrunner wrote: On Sun, Nov 16, 2014 at 3:01 PM, Bill Deegan b...@baddogconsulting.com wrote: Dirk, Are they available from another apt-repo? Maybe one of the unsafe ones? (Where they stick blobs) I googled around for quite a while; I didn't find where or why they were removed but also didn't find any alternative (other than the Windows python setuptools). This is the link I found being closest to the problem at hand: https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1081155 Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
[Scons-dev] PyCon 2015, Montréal...
Hi there, the PyCon 2015 in Montréal ( https://us.pycon.org/2015/ ) still accepts proposals for the poster session (talks/sprints are through). So if anybody would like to go and represent SCons, I would be willing to help with preparing text/graphics...but I can't be on site at the 12th of April. ;) Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] PyCon 2015, Montréal...
On 17.11.2014 18:31, Dirk Bächle wrote: Hi there, the PyCon 2015 in Montréal ( https://us.pycon.org/2015/ ) still accepts proposals for the poster session (talks/sprints are through). So if anybody would like to go and represent SCons, I would be willing to help with preparing text/graphics...but I can't be on site at the 12th of April. ;) Upps, I'm sorry for the noise. I just realized that we already *have* November, so it's too late anyway. ;) As an excuse, today's my first working day after a full month vacation...so my internal clock is still somewhat askew. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
[Scons-dev] Question: VersionedLibrary, Name/Path in Node class...
Hi fellow devs, I'm currently working on my patch for switching the Node class to __slots__, and there is one final test in the suite which doesn't pass and keeps bugging me: test/LINK/VersionedLib.py. The basic problem seems to be that the shlib_emitter in the link.py tool rewrites the name of the target node: === def shlib_emitter(target, source, env): Verbose = True platform = env.subst('$PLATFORM') for tgt in target: tgt.attributes.shared = 1 try: # target[0] comes in as libtest.so. Add the version extensions version = env.subst('$SHLIBVERSION') if version: version_names = shlib_emitter_names(target, source, env) # change the name of the target to include the version number target[0].name = version_names[0] for name in version_names: env.SideEffect(name, target[0]) env.Clean(target[0], name) if Verbose: print shlib_emitter: add side effect - ,name except KeyError: version = None return (target, source) === but it doesn't amend the .path at the same time, so both entries are actually out of synch in the following: name = libtest.so.2.5.4 path = libtest.so for the test example. My question: Do we allow the name of a Node to be anything, or (as I assumed until now) is the name always the trailing part of the path? Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Question: VersionedLibrary, Name/Path in Node class...
On 25.11.2014 22:52, Gary Oberbrunner wrote: I'd certainly think those two should stay in sync (name and path). This would then mean that either VersionedSharedLibrary() in engine/SCons/Tool/__init__.py, ll. 336 : if version: # here we need the full pathname so the links end up in the right directory if Verbose: print VerShLib: target lib is = , map(str, target) print VerShLib: name is = , target[0].name print VerShLib: dir is = , target[0].dir.path libname = target[0].path linknames = VersionShLibLinkNames(version, libname, env) or the test in test/LINK/VersionedLib.py has to get fixed. At the moment, the test only passes because target[0].path doesn't contain the version number at its end. Dirk On Tue, Nov 25, 2014 at 3:37 PM, Dirk Bächle tshor...@gmx.de mailto:tshor...@gmx.de wrote: Hi fellow devs, I'm currently working on my patch for switching the Node class to __slots__, and there is one final test in the suite which doesn't pass and keeps bugging me: test/LINK/VersionedLib.py. The basic problem seems to be that the shlib_emitter in the link.py tool rewrites the name of the target node: === def shlib_emitter(target, source, env): Verbose = True platform = env.subst('$PLATFORM') for tgt in target: tgt.attributes.shared = 1 try: # target[0] comes in as libtest.so. Add the version extensions version = env.subst('$SHLIBVERSION') if version: version_names = shlib_emitter_names(target, source, env) # change the name of the target to include the version number target[0].name = version_names[0] for name in version_names: env.SideEffect(name, target[0]) env.Clean(target[0], name) if Verbose: print shlib_emitter: add side effect - ,name except KeyError: version = None return (target, source) === but it doesn't amend the .path at the same time, so both entries are actually out of synch in the following: name = libtest.so.2.5.4 path = libtest.so for the test example. My question: Do we allow the name of a Node to be anything, or (as I assumed until now) is the name always the trailing part of the path? Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org mailto:Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev -- Gary ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
Bill, On 12.12.2014 00:52, Bill Deegan wrote: I'm thinking a hg repo and some sort of ci to process it when new changes come in. Or use readthedocs.org http://readthedocs.org (It has integration with bitbucket for such already) That way users and do pull requests and also the webserver only has to handle serving static pages. Is it reasonable to expect that users who wanted to contribute to wiki-like content could handle mercurial? it's probably not so much about hg (or any other tool, for that matter), but about the workflow that's required for getting one's changes in. To me it feels like we're moving away from a more scratchpad-like medium, to static pages. That would be okay if we had several authors and technical writers that could create lots of pages with content. But this would mean that we provide all the information, and we are responsible for keeping things alive. If that's what we want, fine. It more or less boils down to the question: Do we want a Wiki (static pages) for us, or for our users? Just some late night thoughts, to fuel the discussion... Regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
On 12.12.2014 01:47, Bill Deegan wrote: Dirk, Trying to figure out a way we can avoid the spam problem we've had so far. Yes, I know and I'm not against it. Many thanks to you and Gary for caring to find an alternative. No matter what the final technical solution will look like, I'll definitely support it. I just wanted to point out that we're raising the barriers for the occasional newcomer and commenters. We do have a lot of stuff in the Wiki that's well worked out and mature enough to get served statically, complementing the UserGuide for example. But it would be good to still have a place for collecting ideas, snippets, organizing things...where people can go nuts if they have to. And that's what fits perfectly to a Wiki, in my opinion. We want new people and their ideas and writings coming in, not having to push them out ourselves...so we need to have low barriers for that. What about having a two stage thing, with a Wiki for collecting ideas 'n' stuff, and then statically served pages, protected by a VCS as well? A little bit like core tools vs. external ones? If we like a contribution, we'll pull it in? Finally, if we could find a Wiki, with something like the ApprovalQueue we had before, on a server that's less vulnerable to DOS attacks...that would be my favorite. Being robust against DOS is more important to me than the spam filtering/preventing issue. I don't care about the Wiki dialect so much, and I also think that the default rendering of Bitbucket looks quite okay. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
Gary, On 12.12.2014 15:00, Gary Oberbrunner wrote: I would like to find a system that has some kind of online editor/previewer, rather than a pure clone/edit/push/pull-request system (whether it's git or hg), because sometimes you want to see how your markup will actually look on the site before pushing it. (Dirk, I think that would also help the occasional contributor overhead you're concerned about.) I do think something hg or git-based would be preferable. sounds acceptable. What I'm mainly after is that a user can queue his changes...and then forget about them. So he doesn't have to cycle through all kinds of review steps, or answer further inquiries. That might put people off... It should be edit, save/commit...done. And yes, we have to move it off our current hosting provider because DoS attacks on our MoinMoin wiki bring the shared server to its knees on a somewhat regular basis. And I'm not ready to be sysadmin of another amazon micro instance, so we need a hosted solution. Do we have admin access to the host where our website is running on? What is it exactly, apached? I just stumbled over this module http://www.techrepublic.com/blog/smb-technologist/secure-your-apache-server-from-ddos-slowloris-and-dns-injection-attacks/ , not sure if it could really help. Anyway, I have the existing wiki mostly converted from moin to markdown (github flavor, but I could redo it as a different flavor depending on what we choose) with only a few hand edits necessary. Some things like the Bug() macro won't translate, but I think we can live without that. I didn't find any decent tools to convert from moin to rst, so I think going with a markdown solution would be easier than rst at this point, though I think pandoc can convert between markdown and rst, so if needed we can do that -- not sure how lossy that conversion would be. There is also Creole ( http://en.wikipedia.org/wiki/Creole_(markup) ), which is supported by Bitbucket. I have no personal experience with it, but it seems to be designed for exactly this purpose of migrating from one Wiki to another. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
On 12.12.2014 19:32, Gary Oberbrunner wrote: On Fri, Dec 12, 2014 at 1:20 PM, Dirk Bächle tshor...@gmx.de mailto:tshor...@gmx.de wrote: On 12.12.2014 18:43, Gary Oberbrunner wrote: [...] It would be a little odd to have our code at bitbucket and our wiki at github, but if github's wiki engine/editor is way better, I'd consider it. After all our wiki has been a separate thing for years already. Ahh, I can already see where you're going with this ;) You're thinking I want to move _everything_ to github I bet. Actually, no. I do like git better than mercurial, it's true; but bitbucket seems to have fine git support these days so I'm agnostic on that. I was just teasing there... :) I really just want to get the wiki back up soon, and not have to think about it anymore for a while :-). If someone proposes some other code-oriented wiki site I'd be just as happy to use that for the wiki. I really don't want to be the pessimist in this round, but I'd like to point out what's on the other side of the equation when we switch to a VCS-based editing of the Wiki: Someone has to accept (and review?) all the incoming pull requests. And you can't simply mark persons trustworthy for the future, as with the ApprovalQueue. So you have to accept/merge each time...just sayin'. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
Gary, On 13.12.2014 15:06, Gary Oberbrunner wrote: Here's a wiki progress report. * I've re-enabled the regular wiki, in read-only mode. That way at least people can see it. Pair may take it down again but it's better than nothing. * I put up test versions of the wiki, converted to markdown, on both bitbucket and github. - bitbucket: https://bitbucket.org/scons/scons/wiki/FrontPage (this is backed by an hg repo) - github: https://github.com/garyo/scons-wiki/wiki/FrontPage (this is backed by a git repo) IMHO, the bitbucket version looks better. They both are functional. They both have online editors, and both allow specifying a list of approved contributors who can edit directly, without making pull requests. They are both DVCS-backed and can accept pull requests (I think, haven't tried that.) I don't know much yet about searchability or TOCs. Feel free to play with them (but don't expect your changes to persist forever, these are just tests). Please let me know what you find. this is awesome! For me, both of the options look good enough to give them a go. The bitbucket version is my winner, because it uses the full page width for content. I think this is important, e.g. for places where we quote source passages with longer lines... One other thing I noticed is, that on the SconsIsNotSlow page the attached files were successfully converted...but the images seem to be missing. Maybe there is a convert option for that? Otherwise, as I said above, it's perfectly good to start from here and manually improve pages where required. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
On 13.12.2014 18:00, alexandre.feb...@gmail.com wrote: Hi, Just a side remark, as a former user of that wiki: The issue is that most of it is outdated: refers to obsolete methods, uses Python methods whereas there is now a SCons method, etc... And even when it's not, readers don't trust fully what they see, because, well, how can they know if this is still up to date or not? And this is caused by the very use of a wiki to hold/update the data. Whereas if all of this could be integrated in the **official** doc, and as such, be maintained/updated to follow SCons evolution, it would bring much more value to it. Maintained and updated by *whom*? Do you volunteer? ;) The intention of our Wiki is not to provide official documentation, but to collect additional material (sources/docs)...that might once get integrated into the core. But it's mainly there for the community (all users), and it's in the state the community left it in. At least that's the way I see it. Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
On 13.12.2014 17:53, Bill Deegan wrote: Can find search on bitbucket version, but it's there on github. Good point, that would speak for github then..since bitbucket isn't too interested in searching and hierarchies: https://groups.google.com/forum/#!topic/bitbucket-users/R0ZJrWhhMTo http://stackoverflow.com/questions/3050545/bitbucket-wiki-create-a-heirarchy-structure But even github doesn't do a full text search, it can only find page titles. :( Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] What to replace the wiki with?
On 13.12.2014 21:54, Bill Deegan wrote: All, I think two items for wiki are must have: 1) full text searchable from the wiki 2) index able by google and others. I'm pretty sure neither bitbucket nor github has both. (Though I suppose 1 would come with 2) As for giving specific contributors unfettered write access. I think we could just give them write access to a given repo. (which is where the wiki is). No such access control exists for a project's wiki under bitbucket. Assuming I have a host to put moin on, does anyone know how to configure it to minimize the load it causes? Can you put a caching proxy in front of it? I'm not an expert (I know jag about web stuff), so don't take my words for granted. But I read somewhere that one can roughly distinguish between servers that handle requests thread-based (apache), and those that work event-based (nginx). The former are more vulnerable to D(D)OS attacks, so if we setup all the stuff on our own, using nginx might be worth a try? I remember that there was a recent talk about Bitbucket and its internal architecture. If this sound interesting, I can try to dig up the link, and you can have a look at which components/configurations it uses. Why not learn from the best? Or let's say the successful at least... ;) Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Bug in env.Glob() - how do I report?
Hi Vasily, first of all Thanks a lot for asking first, before simply going ahead and submitting an issue. Asking on the User mailing list is the preferred method for finding out whether an odd behaviour is really a bug or not. On 15.12.2014 14:12, Vasily wrote: Hi everybody, I've stubmled upon the bug in env.Glob() that loses FS.Entry actual class when working with several VariantDir's coming from one source. Small reproducer attached. I tried to run your SConstruct on my machine, but I get an dependency cycle error. Can you provide a little bit more description about what output you would expect, and what you see instead? That'll help me (and other users) understand better what your actual problem is. I'm currently not able to reproduce your example on my side, so I can't debug it either. Please also note that you're using disambiguate on a Node, that might not yet exist physically. It gets created later, when the Command() builder is run during the actual build phase Where should I submit the issue? scons.tigris.org http://scons.tigris.org looks a bit outdated, should I submit a bug on bitbucket.org http://bitbucket.org? The Tigris bug tracker is still the correct place to enter issues, but first let's see if we can identify this as bug at all. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Bug in env.Glob() - how do I report?
Vasily, On 15.12.2014 20:35, Vasily wrote: Hi Dirk, Thanks for the reply! I get this running the SConstruct I attached: $ scons . scons: Reading SConscript files ... TypeError: Tried to lookup Dir 'some_entry' as a File.: File C:\Vass\Work\Temp\scons_glob_bug\SConstruct, line 22: d2[0].srcnode() File C:\Python27\scons-2.1.0\SCons\Node\FS.py, line 743: srcnode.must_be_same(self.__class__) File C:\Python27\scons-2.1.0\SCons\Node\FS.py, line 626: (self.__class__.__name__, self.path, klass.__name__)) Note: I've checked it agains SCons 2.1.0 (pretty old, but that's what we use internally right now, maybe it's already fixed?..) I had another look and I tested your example with the latest SCons 2.3.4 and 2.1.0 as well. Both times I get the cyclic dependency error... This seems to be logical to me, because of the following line: d1 = variant_exec(env1, 'var1', env1.Command, 'some_entry', 'some_entry', Mkdir($TARGET)) , where you specify some_entry as target *and* source for the Mkdir(). You don't need to do this, setting the source to [] (empty list) should work fine. After this change I was finally able to reproduce your error output: the problem is not so much in the Glob itself, but in the fact that you're trying to handle directories but aren't telling SCons so. As a default, SCons assumes that simple strings (like the ones returned from the Glob call) refer to file type entries. So, if a folder some_entry exists, but the Glob() returns a some_entry these two collide. For resolving this you have to explicitly convert the list of globbed strings to Dir nodes...find an example SConstruct attached. I hope this explains things and helps you further. Regards, Dirk def variant_exec(env, variant_dir, func, *args, **kw): oldwd = env.fs.getcwd() env.VariantDir(variant_dir, '.', duplicate=0) env.fs.chdir(env1.Dir(variant_dir)) try: return func(*args, **kw) finally: env.fs.chdir(oldwd) env1 = DefaultEnvironment() env2 = env1.Clone() d1 = variant_exec(env1, 'var1', env1.Command, env1.Dir('some_entry'), [], Mkdir($TARGET)) d1[0].srcnode() def DirGlob(env, pattern): return env.Dir(env.Glob(pattern)) d2 = variant_exec(env2, 'var2', DirGlob, env2, '*_entry') d1[0].disambiguate() d1[0].srcnode() d2[0].disambiguate() d2[0].srcnode() ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
[Scons-dev] Upcoming switch of the Node class to slots...
Hi there, here's the next little step towards the version 2.4 of SCons, which is planned to contain the switch to using __slots__ for reducing the overall memory consumption. I've rebased my changes against the current trunk (default) of the main repo, and pushed the resulting changes to my experimental SCons repo. Note, that this branch is not planned to get directly merged into the main repo...it's just a preview/rebase for developers (I'm thinking mainly at Jason for Parts, and Ivan for PlatformIO) that might want to check for collisions with their tools/extensions. So, if you fall in this category too, or simply want to test the changes against your current project, you can download the branch like this: hg clone http://bitbucket.org/dirkbaechle/scons_experimental -r switch_node_to_slots . Make sure that you execute this in a clean directory, such that your basic SCons repo doesn't get mixed up! If you are interested in the exact diff for this patch, the single commit above might be too big to chew on for you. Then, looking at my original development branch hg clone http://bitbucket.org/dirkbaechle/scons_experimental -r reduce_memory_footprint , containing all single debugging/fixing/amending steps I did along the way, might give you further insight. If everything else fails, just ask. ;) Please also take into account the mail https://pairlist4.pair.net/pipermail/scons-users/2014-July/002734.html , where I described the planned changes on the Node API level. That's it for now, I guess. Happy testing! Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev