Re: [Scons-dev] Py 3.5 support.. How important is it?

[Jonathon Reinhart]

No, only Python 3.5 is in the standard repo. As far as I know, the only way
to install something newer is to compile Python from source (an alternate
install).

On Tue, Jan 12, 2021, 00:05 Bill Deegan  wrote:

> Jonathon,
>
> Are there py 3.6 or newer python packages for Debian 9 in the standard
> repo?
> Or is it a special add on.
>
> -Bill
>
> On Mon, Jan 11, 2021 at 8:32 PM Jonathon Reinhart <
> jonathon.reinh...@gmail.com> wrote:
>
>> Debian 9 is still supported, and runs Python 3.5:
>> https://wiki.debian.org/Python
>>
>> But, that can be worked around by either adding a newer version of
>> Python, or holding back the SCons version.
>>
>> On Mon, Jan 11, 2021, 23:06 Bill Deegan 
>> wrote:
>>
>>> Greetings,
>>>
>>> Since Py 3.5 is EOL'd we're considering moving the floor version for
>>> SCons to 3.6.
>>>
>>> Please comment if 3.5 is still important to you and why
>>>
>>> Thanks,
>>> Bill
>>> SCons Project Co-manager
>>> ___
>>> Scons-dev mailing list
>>> Scons-dev@scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>
>> ___
>> Scons-dev mailing list
>> Scons-dev@scons.org
>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Py 3.5 support.. How important is it?

[Jonathon Reinhart]

Debian 9 is still supported, and runs Python 3.5:
https://wiki.debian.org/Python

But, that can be worked around by either adding a newer version of Python,
or holding back the SCons version.

On Mon, Jan 11, 2021, 23:06 Bill Deegan  wrote:

> Greetings,
>
> Since Py 3.5 is EOL'd we're considering moving the floor version for SCons
> to 3.6.
>
> Please comment if 3.5 is still important to you and why
>
> Thanks,
> Bill
> SCons Project Co-manager
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] bug prune

[Jonathon Reinhart]

That makes sense, Mats. Thanks for the additional insight.

On Mon, Sep 2, 2019, 10:30 Mats Wichmann  wrote:

> On 9/2/19 8:03 AM, Jonathon Reinhart wrote:
> > Note that you can subscribe to a GitHub issue without commenting. This
> > is preferred as it avoids spamming all currently-subscribed users.
> >
> > Also, I think mass/automated bug closing needs to be done very
> > conservatively. Closing an issue doesn't make the bug go away. There are
> > projects that have bots which close issues after 30 days of inactivity,
> > and I find it infuriating.
>
> I'm not a fan of the rapid/aggressive closing either, wasn't suggesting
> that. The idea of a bot is to keep there from being so much manual work
> to get the notifications sent, and the closing done. If the team prefers
> no not keep it after the prune, it can just be turned off.
>
> There are a decent number of bugs that were created over 10 years ago,
> and many in the 3-10 year range, and which, due to the migration from
> tigris, aren't really associated with people who may have reported them,
> or commented on them.
>
> The idea of commenting to keep a bug alive is to defeat the bot's idea
> of what is active (and to confirm "yes, this is still a problem").  I
> don't think subscribing to it does that.
>
>
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] bug prune

[Jonathon Reinhart]

Note that you can subscribe to a GitHub issue without commenting. This is
preferred as it avoids spamming all currently-subscribed users.

Also, I think mass/automated bug closing needs to be done very
conservatively. Closing an issue doesn't make the bug go away. There are
projects that have bots which close issues after 30 days of inactivity, and
I find it infuriating.





On Tue, Aug 27, 2019, 08:50 Mats Wichmann  wrote:

>
> Just to pull some thoughts together:
>
> there are currently 679 open scons issues on github.
>
> That number drops to 92 if you select only ones which have had a
> modification since the big migration from tigris. Try this query:
>
> is:issue is:open updated:>=2018-02-10
>
> or as a link:
>
>
> https://github.com/SCons/scons/issues?utf8=%E2%9C%93=is%3Aissue+is%3Aopen+updated%3A%3E%3D2018-02-10+
>
> I'm a relative newcomer around here, but I don't see the value of
> showing a ton of historical bugs that aren't being worked on; the newly
> filed ones don't even get a lot of attention - there just isn't a big
> scons team at this point and numerically most current contributors have
> a specific motivation ("itch to scratch" as it were) rather than the
> ability to just generally work on bugs.  To provide more visible focus
> there's already been some discussion of a bug prune.
>
> My suggestion is this:
>
> (a) close all open tigris bugs with a message that includes these items:
>
> * bug is now tracked on github [link]
> * bugs which have not had activity in 18 months are going to be closed
> (it doesn't have to be 18, but that was the cutover time)
> * we understand readers of this issue might not see messages from
> github, so if you want to keep this issue alive, make a comment - any
> comment - on the corresponding github issue.
>
> (b) fire up a bot to mark inactive github issues with a tag, and
> configure suitably.  Looks like there's an app in the github marketplace
> that is free so setup is just a YAML file. Example setup here:
>
>
> https://github.com/timgrossmann/InstaPy/commit/afd968dfa1ce1141456a207484d35f2766d5916b
>
> the app:
>
> https://github.com/marketplace/stale
>
> (c) someone scan through the first-time closure proposal list and
> manually update any which seem deserving of continued life.
>
>
> Closed-as-stale issues don't vanish, they are still there to be browsed
> as needed...
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Should we remove python 3.5 from our CI tests

[Jonathon Reinhart]

> True but who uses Debian Stable for anything relating to software
> development, it is a server distribution.

I'm not sure where you got the idea that Debian is a "server
distribution". Lots of people use Debian as a desktop OS, and my team
uses Debian for software development.

I think SCons would be making a serious mistake if it dropped support
for Python 3.5. Just because the distro is using an older version of
SCons, doesn't mean that SCons shouldn't support the latest version of
Python available on the system.

This is particularly troublesome for me as I consider putting together
a new build image, based on Debian 9, but with latest SCons installed
from source.

I'm all for cleaning up old cruft, but it seems like removing support
for a version of Python that's less than 3 years old seems a bit
aggressive.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Should we remove python 3.5 from our CI tests

[Jonathon Reinhart]

No way. Distros (e.g. Debian 9) package Python 3.5.

Are you going to drop Python 2 support too?

It's really not that big of a deal to test each minor version. Plenty
of open source software far less important than SCons does it.

On Fri, May 25, 2018 at 5:10 PM, Mats Wichmann  wrote:
> On 05/25/2018 02:59 PM, Daniel Moody wrote:
>> Opening discussion to remove 3.5 from the CI tests.
>>
>> Is there any reason we need 3.5 specifically?
>> 3.6 has been out for a while and is pretty stable.
>>
>> It's automated ci so there isn't much effort in keeping it part of the
>> tests, but we should only do it if there is reason.
>>
>> Downside is longer ci iterations and using more resources from the
>> generously free ci platforms.
>
> I'd vote - once 3.7 is out (that likely next month, no?), 3.5 should be
> dropped. Can't test every version.
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Enabling Travis CI service for SCons on github

[Jonathon Reinhart]

Bill,

I'm not sure if you were asking Daniel to enable those versions, or simply
if it were possible.

The answer to the latter is yes, easily:
https://github.com/JonathonReinhart/scuba/blob/master/.travis.yml

Jonathon Reinhart

On Mon, Dec 18, 2017 at 5:51 PM, Bill Deegan <b...@baddogconsulting.com>
wrote:

> Daniel,
>
> Can we get travis to test with py2.7, 3.5, and 3.6 ?
>
> -Bill
>
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Enabling Travis CI service for SCons on github

[Jonathon Reinhart]

Yes, it should automatically do that.

See this (merged) PR from one of my projects:
https://github.com/JonathonReinhart/scuba/pull/98

Towards the bottom you'll see a "View Details" button.
Clicking that will expand a box showing the results of all the "checks"
that ran.



On Tue, Dec 5, 2017 at 11:13 PM, Bill Deegan 
wrote:

> Is there a way to get travis to post the results back into the pull
> request?
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Enabling Travis CI service for SCons on github

[Jonathon Reinhart]

Is @SConsProject the Twitter account you're referring to?

Does SCons really want to post Tweets when a build fails? I've never seen
this before. To me, Twitter is used by open-source projects for things like
announcing new releases, bugfixes, special events, etc. and not posting to
the world when some Joe's feature branch failed to build.

Cheers,
Jonathon

On Tue, Dec 5, 2017 at 10:34 PM, Daniel Moody  wrote:

> Writing tweets to twitter will require private API keys, which implies a
> private server is needed.
>
> Is there any private server scons has that could run a small web server to
> receive the webhook notification from travis and write the tweet to twitter?
>
> For the web server, I was thinking of using a python web server to receive
> the notification and twython package to write the tweet.
>
>
>
>
>
> On Tue, Dec 5, 2017 at 1:22 PM, Bill Deegan 
> wrote:
>
>> Great!
>> That's all I can think of at the moment.
>>
>> Thanks,
>> -Bill
>>
>> On Tue, Dec 5, 2017 at 10:19 AM, Daniel Moody 
>> wrote:
>>
>>> https://docs.travis-ci.com/user/notifications/
>>>
>>> IRC is covered. Webhook seems open ended to setup for any site so seems
>>> possible for Twitter messages with some setup.
>>>
>>> Any other notification types we are interested in?
>>>
>>> I'll take a look and submit a PR.
>>>
>>> On Dec 5, 2017 12:55 PM, "Bill Deegan" 
>>> wrote:
>>>
 Should be enabled now.
 Just merged one of your pull requests..
 I'll keep an eye on it.
 Can we get the results to post on twitter? and/or IRC?

 On Sun, Dec 3, 2017 at 9:08 PM, Bill Deegan 
 wrote:

> Yes. I'll get to it in the next week or so.
> We do have buildbot doing similar at : buildbot.scons.org
>
> -Bill
>
> On Sun, Dec 3, 2017 at 2:02 PM, Daniel Moody 
> wrote:
>
>> In this pull request:
>> https://github.com/SCons/scons/pull/17
>>
>> SCons got a Travis CI script added to the repo. Travis CI is a free
>> service for running each new commit to github against a testing script
>> (.travis.yml). This takes place on Travis's servers and builds are 
>> recorded
>> and accessible via links. Below is an example:
>>
>> https://travis-ci.org/dmoody256/scons/builds/311007377
>>
>> The also add a nifty little check next to the commit if the commit
>> passed the test:
>> [image: Inline image 2]
>>
>> I was wondering if someone who has access could enable the service
>> for the SCons project?
>>
>> Below are some instructions on how to do it from the Github web
>> interface
>>
>> In order to utilize the script you need to add Travis CI as a
>> service, below are some basic instructions from Github:
>>
>>1. Navigate to the repository you want to connect to Travis.
>>2. In the right sidebar, click Settings.
>>3. In the left sidebar, click Webhooks & Services.
>>4. In the Services box, click Add service.
>>5. Select "Travis CI".
>>6. For public projects leave the configuration fields empty
>>7. Click Add service.
>>8. Go to https://travis-ci.org/profile
>>9. Turn Automatic CI on for the project you want to run builds for
>>
>> Best regards,
>>
>>
>>
>> ___
>> Scons-dev mailing list
>> Scons-dev@scons.org
>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>
>>
>

 ___
 Scons-dev mailing list
 Scons-dev@scons.org
 https://pairlist2.pair.net/mailman/listinfo/scons-dev


>>> ___
>>> Scons-dev mailing list
>>> Scons-dev@scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>
>>>
>>
>> ___
>> Scons-dev mailing list
>> Scons-dev@scons.org
>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>
>>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Hashes

[Jonathon Reinhart]

I believe you will never encounter an accidental MD5 collision in the way
that SCons uses it. [1] All of the MD5 collisions being publicized are
intentional; leveraging a chosen-prefix attack. Does SCons really care to
address the case where someone is intentionally generating collisions? I
imagine not.

MD5 is still the fastest general-purpose hashing algorithm [2]. So I so
reason for SCons to worry about changing hash algorithms.

Jonathon Reinhart

[1]: https://stackoverflow.com/a/937798/119527
[2]: https://stackoverflow.com/a/2723941/119527

On Thu, Oct 26, 2017 at 7:58 AM, Russel Winder <rus...@winder.org.uk> wrote:

> I may just be out of date: is SCons using MD5 for hashing?
>
> Clearly SCons is not that interested in security or true persistence
> level hashing, but given the issue of clashing might MD5 now not be
> useful?
>
> --
> Russel.
> 
> =
> Dr Russel Winder  t: +44 20 7585 2200   voip:
> sip:russel.win...@ekiga.net
> 41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] SCons moves to GitHub! https://github.com/SConsProject/scons

[Jonathon Reinhart]

I am confident that this was the right move. I see this really lowering the
barrier to entry for people wishing to contribute.

On Mon, Sep 18, 2017 at 5:12 PM, Bill Deegan 
wrote:

> Greetings,
>
> Well the day has finally come.
>
> SCons is moving to Github and git.
>
> Outstanding pull requests on bitbucket.org will need to be migrated to
> the new repo.
>
> The bugtracker is still at scons.tigris.org, but will be moved (real soon
> now) to github.
> Dirk has a start on a script to migrate from scons.tigris.org, but we
> need some help completing the logic to upload the bugs (and their
> attachments) to github.
>
> The wiki is still at https://bitbucket.org/scons/scons/wiki, but will
> also be migrated real soon now.
>
> The Project repo is now at:
>
> https://github.com/SConsProject/scons
>
> There is another test repo we will use to work on migrating the issues and
> wiki. Please do not use that repo. ( https://github.com/
> SConsProject/scons-gh-convert-git )
>
>
> -Bill
> SCons Project Co-Manager
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Sorry

[Jonathon Reinhart]

+1 for Travis. It works well for some of my weird Python projects:

https://github.com/JonathonReinhart/scuba/blob/master/.travis.yml

https://github.com/JonathonReinhart/staticx/blob/master/.travis.yml


On Fri, Sep 15, 2017 at 9:54 AM, Russel Winder  wrote:

> On Fri, 2017-09-15 at 09:44 -0400, BDC wrote:
> > Likely engage with any useful free third party services
>
> Whilst Travis-CI uses really ancient Ubuntu (they are just moving from
> Precise
> to Trusty and haven't even thought about Xenial, let alone use an actually
> useful one such as Zesty), having it as a barrier to merging a pull
> request is
> really good, and the integration with GitHub is excellent.
>
> You will already know this I have no doubt, I am just putting forward that
> I
> am for putting it in place and should shortly be able to put time in to
> making
> a matrix work.
>
> --
> Russel.
> 
> =
> Dr Russel Winder  t: +44 20 7585 2200   voip:
> sip:russel.win...@ekiga.net
> 41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] SCons performance investigations

[Jonathon Reinhart]

I just wanted to add some quick anecdotes.  In some of our largest, most
complicated builds, we have observed a lot of the same things as you all
have.

One time we did some quick profiling, and saw that much CPU time during a
null build was spent in the variable substitution.

Additionally, we also have a habit of cloning the environment before
passing it to a SConscript. This is for safety - to ensure that a child
SConscript can't mess up the environment for its siblings.


Jonathon Reinhart

On Sat, Jul 22, 2017 at 5:23 PM, Bill Deegan <b...@baddogconsulting.com>
wrote:

> Jason,
>
> Any chance you could add these comments to the wiki page?
> https://bitbucket.org/scons/scons/wiki/NeedForSpeed
>
> -Bill
>
> On Sat, Jul 22, 2017 at 10:09 AM, Jason Kenny <dragon...@live.com> wrote:
>
>> Some additional thoughts
>>
>>
>>
>> Serial DAG traversal:
>>
>>- On the issue here as well is that the Dag for doing builds is based
>>on nodes. There is a bit of logic to deal with handing side effects and
>>build actions that have multiple outputs. Greg Noel had made a push for
>>something called TNG taskmaster. I understand now the main fix he was 
>> going
>>for is to tweak SCons to navigate a builder Dag instead of Node DAG, the
>>node Dag is great to get the main organization but after that it is
>>generally trivial to make a DAG based on builder at the same time,
>>Traversing this is much faster, we require less “special” logic and will 
>> be
>>easier to parallelize.
>>   - On big improvement this provides is that we only need to test if
>>   the sources or targets are out of date if the dependent builders are 
>> all up
>>   to date. If one of the is out of date, we just build, This vs we check 
>> each
>>   node and see if the build action has been done which requires extra 
>> scans
>>   and work in the current logic.
>>   - Given a builder is out of data you just mark all parents out of
>>   date. We only care about builders in a set that we don’t know are out 
>> of
>>   date yet. Simple tweaks on how we go through the tree can mean we only 
>> need
>>   to touch a few nodes.
>>
>> Start up time:
>>
>>- Zero build time is going to be the worse case for a build up to
>>date, as we have to make sure all items are in a good state. Time to start
>>building on diff should be a lot faster. Scons spends a lot of time having
>>to read everything on second passes. We can use our cache much better to
>>store states on what builds what, etc to avoid even having to read a file.
>>If the file did not change we already know the node/builder tree it will
>>provide. We already know the actions. We can start building items as soon
>>as a md5/time stamp check fails most of the time. Globs can store
>>information about what it read and processed and only need to go off when
>>we notice a directory timestamp. Avoiding processing build files and
>>loading known state is much faster than processing the python code. My 
>> work
>>in Parts has shown this. The trick is knowing when you might have to load 
>> a
>>file again to make sure custom logic get processed correctly.
>>- In the case of Parts it would be great to load file concurrently
>>and in parallel. I think I have a way to go this concurrently which I have
>>not done yet. The main issue is the node FS object tree is a sync point 
>> for
>>being parallel.
>>
>> CacheDir:
>>
>> 100% agree..
>>
>> SConsign generation:
>>
>>- I think this is a bigger deal for larger builds. I have found in
>>Parts, as I store more data I would try to break up the items into
>>different files. This helps, but in the end, at some point a pickle or 
>> JSON
>>dump takes times. It also takes time to load them as in cases for builds I
>>have had, loading 700mb files takes even the best systems a moment to do.
>>This is a big waste when I only need to get a little bit of data. 
>> Likewise,
>>the storing of the data could and should be happening as we build items. 
>> As
>>noted we don’t have a good way to store a single item without storing all
>>the file. If the file is large 100MB to GBs this can take time, as in many
>>seconds, which in the end annoy users. I would say with what I do have
>>working well in Parts that the data storage, retrieval is the big time
>>suck. Addressing this would have the larg

Re: [Scons-dev] [Scons-users] Can we drop windows native installers if pip install works?

[Jonathon Reinhart]

On Mon, Apr 10, 2017 at 3:36 AM, Alexandre Feblot <
alexandre.feb...@gmail.com> wrote:

> And in some controlled environments, there might be no access to internet.
> A self contained installer is still the best solution in this situation.


That's not an issue. Pip will happily install from a zip file, tarball,
cloned git repo, etc. There is no internet access requirement to use Pip.

A lot of Windows users will, out of habit, come looking for "setup.exe",
but considering Pip comes pre-installed with Python 2.7 and up these days,
it'd be trivial to 'pip install scons'.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Why are Builders excluded from env.Clone()?

[Jonathon Reinhart]

Hi Bill,

On Fri, Dec 9, 2016 at 12:46 PM, Bill Deegan  wrote:
> Is your real issue (which caused you to dig into this)
> "The big problem here is that c_file, cxx_file are *not* unique to the
> passed-in
> environment."
>
> Perhaps a solution to just that piece of the puzzle would be simpler?

Mostly so, yes. In my experience, this applies to both of these functions:

   SCons.Tool.createCFileBuilders(env)
   SCons.Tool.createObjBuilders(env)

..but it could easily apply to anything from Tool/__init__.py, where
builders are added to an environment if they don't already exist.

A naive solution to this particular piece of the puzzle might be to
*always* create a new builder, apply it to the environment, and return
it. But this doesn't work, because many tools simply want to add
another way (suffix) to build the same type of thing (Builder). If the
tool always caused a new builder to be generated, all of the suffixes
for other tools would be lost.

As I reason through this, the fact that builders aren't cloned really
seems to be the central problem.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Why are Builders excluded from env.Clone()?

[Jonathon Reinhart]

Hi Dirk,

Thanks for pointing out that issue to me; I hadn't found it in my searches.

On Fri, Dec 9, 2016 at 12:31 PM, Dirk Bächle  wrote:
>> BUILDERS is explicitly excluded in the semi_deepcopy_dict() call.
>>
>> My questions:
>>  - Why are Builders explicitly excluded from the env.Clone()?
>
>
> Because it's possible that during the deep-copy, the original is altered too
> (see comment in BuilderDict::__semi_deepcopy__).

After reading the comment:

def __semi_deepcopy__(self):
# These cannot be copied since they would both modify the same
# builder object, and indeed
# just copying would modify the original builder
raise TypeError( 'cannot semi_deepcopy a BuilderDict' )

...and reading this on the original issue (#2821):

> The problem is that the BUIDLERS are handled via a special BuilderDict which
> cannot be properly cloned.  It has a deepcopy method, but the inherently 
> alters
> the original environment to which it is attached.

> I tried a few different ways to correct this. The first was to workaround the
> issue during cloning, but it felt like it was mainly a hack. So I took the
> approach that the BuilderDict should not be copied directly, and made deepcopy
> throw an exception. The Clone function then manages a proper cloning of the
> BuilderDict.

...and the patch
(http://scons.tigris.org/nonav/issues/showattachment.cgi/890/patch.diff)

...I still don't completely understand the original problem. Why
exactly can't the BuilderDict be deep-copied? How does it alter the
attached environment?

Conceptually, why can't BuilderDict.__semi_deepcopy__ (or a regular
__deepcopy__ for that matter) know to actually create copies of the
Builders referenced by the BuilderDict as well?
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

[Scons-dev] Why are Builders excluded from env.Clone()?

[Jonathon Reinhart]

Hello everyone,

Let me start out with a quick experiment:

env = Environment()
env2 = env.Clone()
print env['BUILDERS']['StaticObject'] is env2['BUILDERS']['StaticObject']

This surprisingly prints "True". The takeaway here is that builders are not
copied when environments are Clone()-ed. This seems to disagree with the
docstring for Environment.Clone() [1] which says:

"""Return a copy of a construction Environment.  The
copy is like a Python "deep copy"--that is, independent
copies are made recursively of each objects--except that
a reference is copied when an object is not deep-copyable
(like a function).  There are no references to any mutable
objects in the original Environment.
"""

This behavior does appear to be intentional, however:

builders = self._dict.get('BUILDERS', {})

clone = copy.copy(self)
# BUILDERS is not safe to do a simple copy
clone._dict = semi_deepcopy_dict(self._dict, ['BUILDERS'])
clone._dict['BUILDERS'] = BuilderDict(builders, clone)

BUILDERS is explicitly excluded in the semi_deepcopy_dict() call.

My questions:
 - Why are Builders explicitly excluded from the env.Clone()?
 - Would it be reasonable to add an optional argument to Clone()
   (e.g. really_deep) which causes Builders to not be excluded?


Some background:

Several times I've noticed that changes made by Tools can "leak" out into other
environments (than the one upon which the tool was called). For example,
consider the Cython tool [2] which does the following:

def generate(env):
env["CYTHON"] = "cython"
env["CYTHONCOM"] = "$CYTHON $CYTHONFLAGS -o $TARGET $SOURCE"
env["CYTHONCFILESUFFIX"] = ".c"

c_file, cxx_file = SCons.Tool.createCFileBuilders(env)

c_file.suffix['.pyx'] = cython_suffix_emitter
c_file.add_action('.pyx', cythonAction)

c_file.suffix['.py'] = cython_suffix_emitter
c_file.add_action('.py', cythonAction)

create_builder(env)

This code is consistent with the Tools included with SCons (e.g. gcc).

The big problem here is that c_file, cxx_file are *not* unique to the passed-in
environment. As my experiment above showed, builders are common to all Clone()s
of that environment.

This causes issues (that are incredibly difficult to track down!)
where Tools can
interact poorly, even when applied to different environments. Here's a scenario
using a hypothetical Zython tool that converts .py files to .c files:

base_env = Environment(
tool_path = ['somewhere'],
)

cython_env = base_env.Clone()
cython_env.Tool('cython')

zython_env = base_env.Clone()
zython_env.Tool('zython')

Because add_action() is a misnomer and should be called set_action(), this
would result in the zython tool being *the* .py -> .c builder for all
environments cloned from base_env.

Of course a workaround for this is to create a brand new Environment(), and the
builders will be created new as well. This is inconvenient though, and
according to the documentation, shouldn't be necessary.

What can we do about this?


Best regards,

Jonathon Reinhart



[1]: Environment.Clone()
 
https://bitbucket.org/scons/scons/src/3763c12a/src/engine/SCons/Environment.py#Environment.py-1377

[2]: cython Tool
 
https://github.com/cython/cython/blob/master/Tools/site_scons/site_tools/cython.py
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Hg vs Git

[Jonathon Reinhart]

On Mon, Nov 28, 2016 at 4:42 AM, Russel Winder <rus...@winder.org.uk> wrote:
> On Sun, 2016-11-27 at 14:11 -0500, Jonathon Reinhart wrote:
>> > The point here is that someone can mutate a branch locally and then
>>
>> force it to the mainline.
>>
>> No, that is specifically what protected branches prevent. If "master"
>> is
>> protected, then no one, not even an admin, can re-write history and
>> force
>> push to it.
>
> So if BitBucket supports this, the admins for the mainline SCons and
> SCons-Contrib repositories should mark all the branches as protected?

Not necessarily "all" of the branches, but definitely the ones that
are final/deliverable (e.g. "master" in git parlance). I also
recommend protecting long-lived feature branches, e.g. the Python 3
effort, where multiple people may be working on it for quite a while.

>> Personally, I find the rewriting extremely powerful for my local
>> development - I can re-arrange, split, and join commits in my feature
>> branch before it is merged into master. Very few people are
>> interested in
>> rewriting history of a published tree.
>
> I have never been a user of history rewriting as I tend to publish all
> my repositories all the time. Maybe my workfow and approach is wrong,
> and that I should keep all work private and so rebasable and squashable
> in both Hg and Git until the point of publishing for the pull request?

IMO, it's all about a published branch policy for the project. For example:
- master - Stable at all times; protected; periodically tagged for
major releases
- devel - Bleeding edge; mostly stable; protected; periodically merged
into master at stable points
- (everything else) - In-progress feature/issue branch; **not
protected** and may be rebased/rewritten at any time

This allows you to say "hey guys, check out my WIP branch", but with a
disclaimer that it may be rewritten, giving you the power to go back
and change things based on code review.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Hg vs Git

[Jonathon Reinhart]

> The point here is that someone can mutate a branch locally and then
force it to the mainline.

No, that is specifically what protected branches prevent. If "master" is
protected, then no one, not even an admin, can re-write history and force
push to it.

Personally, I find the rewriting extremely powerful for my local
development - I can re-arrange, split, and join commits in my feature
branch before it is merged into master. Very few people are interested in
rewriting history of a published tree.

On Nov 27, 2016 10:10 AM, "Russel Winder"  wrote:

> On Sun, 2016-11-27 at 12:39 +0100, rupert THURNER wrote:
>
> Absolutely no reason to apologise for contributing.
>
> > sorry for posting here, i usually just lurk on this list because i am
> > interested in build tools. i doubt that mercurial will die out -
> > their
> > mailing list seems more busy than ever. history rewrite can be done
> > with
> > mercurial nowadays with extensions, and will come even more, just
> > note
> > facebooks "hg absorb" extension. a nice write up about future plans
> > from
> > the mozilla dev list:
> > https://groups.google.com/d/topic/mozilla.dev.version-control/nh4fITF
> > lEMk/discussion
>
> In a sense introducing history rewriting is undermining the whole point
> of Mercurial in a world dominated by Git.
>
> The really important point in the email, at least for me, is that all
> that hoo-ha a couple of years ago that Mercurial would remain a Python
> 2 application as it was simply to hard to switch from ASCII string to
> Unicode strings has gone away and Mercurial will work on Python 3.
>
> Of course if a Rust version really does come out, it may sweep away the
> Python version!
>
> Interesting that groups within Google, Facebook, Mozilla, and Unity are
> Mercurial hold-outs in the tide of Git, and even fighting back with
> movers from Git to Mercurial.
>
> The write up makes it sound a bit like Google, Facebook, Mozilla, and
> Unity are single entities where in fact that a many hundreds of groups
> all acting independently within the organizations.
>
> That Mercurial will run on Python 3 re-energises my willingness to do
> things with Mercurial.
>
> --
> Russel.
> 
> =
> Dr Russel Winder  t: +44 20 7585 2200   voip:
> sip:russel.win...@ekiga.net
> 41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Hg vs Git

[Jonathon Reinhart]

Dirk Bächle  writes:

> I *don't* want the history in my repos to be mutable...

All of the major Git hosting providers have *protected branches* which are
immutable.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Windows executables

[Jonathon Reinhart]

On Tue, Sep 20, 2016 at 9:22 PM, William Blevins 
wrote:

> Can you not run an executable on Windows that doesn't have the extension?


You're not a Windows user, are you? :-)

The general answer is no: It must have a .exe extension to run via
double-clicking in Explorer, or by providing the path in a Command prompt.

Exceptions include .bat and .com files, but these don't really count for
what you're asking.

Regards,
Jonathon
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] D, SCons and Meson

[Jonathon Reinhart]

Here are some initial thoughts, based on my five-minute read of Russel's post
and mesonbuild.com. I'm not a D user, but perhaps my user story can provide
some perspective.

We switched from Make to SCons because of the complexity of our primarily
GCC-based builds. There are a lot of "layers", and getting the dependencies
correct was impossible - we would have been better off using a shell script, as
at least everything would have gotten re-built. Now we strongly leverage SCons'
ability to define custom builders which not only makes the build system cleaner
(putting the messiness in just one location) but also to ensure that
dependencies aren't missed. The fact that SConscript files are Python is a
definite plus for us. We write implement custom builders in Python all the
time.

Going in, the projects were very monolithic; (external) "dependencies" were not
used at all. As we converted things to SCons, we also took the time to pull out
pieces of code into libraries. There were also places where snapshots of
open-source libraries had been "dropped-in" and modified. Now those
modifications are being managed as branches on local forks of the open-source
Git repositories.

Managing those external dependencies has been tricky.  When I first saw the
word "repository" and env.SourceCode() show up in SCons' documentation, I was
very excited; "perhaps this will allow us to manage external code dependencies"
I thought. Then I realized that only archaic systems like CVS and BitKeeper
were supported, and the SourceCode() feature had been deprecated (emphasis
mine):

This function and its associate factory functions are deprecated. There is
no replacement. The intended use was to keep a local tree in sync with an
archive, but in actuality the function only causes the archive to be fetched
on the first run. Synchronizing with the archive is best done **external to
SCons**.

We're currently using Git submodules but some developers struggle with them.
Of course, we like to be able to develop/build/test all of these libraries
independently. Some libraries have dependencies themselves, but we didn't want
to git-submodule them into the higher-level library, because those dependencies
could then get duplicated many times in the tree of a large project that is
initialized with 'git submodule update --init --recursive'. So instead, we took
the approach that libraries expect consuming projects to provide their
dependencies; and for standalone development, libraries have a clone_deps.py
script. It's not the most elegant - there is duplication between .gitmodules
and clone_deps.py but it's generally manageable.

With that said, the (external) dependency-handling mechanism in Meson looks
intriguing. I'm not sure how well it would work in our use-case, but it
definitely highlights an area that SCons has deliberately chosen to stay out
of. Meson seems to consider this "subproject" concept as a first-class citizen
of their build system (https://github.com/mesonbuild/meson/wiki/Subprojects).
It would be great for us if SCons could support this concept a bit more.

The other thing that Meson has going for it (IMHO) is that it is managed in
Git, on GitHub, utilizing their standard issue tracker and pull requests, and
building/testing on Travis and AppVeyor. (See previous discussions on that
topic.)

On Mon, Aug 29, 2016 at 4:10 AM, Russel Winder  wrote:
> In case you hadn't heard, Meson is the "new kid in town" on the build
> front. It is a Python front end to a Ninja back end. It is clearly in
> the SCons and Waf tradition (with some CMake) but very much a Ninja
> backend replacement to Autotools in it's entirety. Meson will make Waf
> (and CMake?) irrelevant not for any technical reasons, but because the
> GNOME/GUADEC type folks are seeing Meson as the replacement for their
> Autotools build – for those that do not think Autotools is all they
> will ever need. i.e. it is getting rapid traction.
>
> Debian maintainers are already seeing it as a main build player where,
> sadly, SCons never really made it. Waf was rejected since it maintained
> the position of being in the project not in the distribution. This is a
> really interesting tension: Gradle has driven the "in the project" idea
> to being the norm in the JDK world (and also some of the C++ world),
> but this is not acceptable with many contexts including Debian and
> Fedora.
>
> Why am I emailing this at all? Well it relates to D-related stuff. I
> had been trying to make SCons the D build of choice for those not using
> Dub. However Meson has come charging in and has taken pole position in
> the Debian and likely Fedora communities as the build of choice for D-
> related packaging. Yes SCons could still be used, but I suspect Meson
> is going to win this game.
>
> I am still intending to create a dependency handling system in SCons
> for D to communicate with the Dub repository for downloads to get and
> build (using the Dub default 

Re: [Scons-dev] Rumour…

[Jonathon Reinhart]

Honestly, I would more more inclined to contribute to SCons if it were
developed using Git, and fully hosted on GitHub.

I think the GitHub model just works extremely well, and that's why so many
projects are moving to it. Having an integrated issue tracker, releases
tied to tags, and although I haven't used it, GitHub Pages is a perfect set
of features for most Open Source projects.  The integration with CI
services (e.g. Travis) and other services (like CodeCov) is a cherry on top.

There have been small bugs that I've gone to fix in SCons, but didn't feel
like dealing with mercurial / BitBucket; not because they're bad, but
because they're not what I'm familiar with.

I suspect there are many others like me.

Just my (unsolicited) $0.02.


On Tue, Jan 5, 2016 at 12:25 PM, Bill Deegan 
wrote:

> Not a rumor. ;)
> http://lwn.net/Articles/669924/
>
> http://thread.gmane.org/gmane.comp.python.devel/150459/focus=150960
>
>
> On Tue, Jan 5, 2016 at 9:13 AM, William Blevins 
> wrote:
>
>> Don't spread rumors ;)
>>
>> On Tue, Jan 5, 2016 at 4:58 PM, Russel Winder 
>> wrote:
>>
>>>
>>> I hear a rumour that Python is now switching to GitHub. Despite
>>> Mercurial having many great plusses over Git, it seems no big project
>>> is staying with Mercurial.
>>>
>>> --
>>> Russel.
>>>
>>> =
>>> Dr Russel Winder  t: +44 20 7585 2200   voip:
>>> sip:russel.win...@ekiga.net
>>> 41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
>>> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>>>
>>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Proper way to get File path (undocumented rfile)

[Jonathon Reinhart]

I, too have come across scenarios where rfile() was the only way to
accomplish exactly what I needed. I believe the scenario was this:

The arguments / command-line parameters to my external utility were so
complex, SCons couldn't handle it himself. So I wrote my own
variable-function-thing (like ${_concat}) in which I needed to expand the
Nodes to their paths, just like SCons would when expanding $TARGETS to
their paths. This is how I ended up finding rfile().

It seems like it should be documented.





On Fri, Oct 2, 2015 at 4:15 AM, anatoly techtonik 
wrote:

> Hi,
>
> Currently the way to get filename from File node is to str() that File.
> That's quite shady API, especially if used in function like:
>
> def convert(node):
> return str(node).replace('\\', '/')
>
> I mean you have no idea what types of node are expected and why
> there is slash escaping. The str(node) can return anything and
> works on any types of nodes.
>
> So, there is undocumented method File.rfile() with the path.
> http://www.scons.org/doc/HTML/scons-api/SCons.Node.FS.File-class.html#rfile
> Which contains os specific path and it is used for example in
> https://github.com/wesnoth/wesnoth/pull/481/files
>
> The questions.
> 1. Why it is called rfile?
> 2. Should it be documented?
> 3. What should be the proper API to get path info for File node?
> 4. What is the proper API to convert paths to system-specific and to
> canonical (forward slash) form?
> --
> anatoly t.
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>


-- 
Computers are incredibly fast, accurate and stupid. Human beings are
incredibly slow, inaccurate and brilliant. Together they are powerful
beyond imagination.
A. Einstein
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] Breaking changes in v2.4

[Jonathon Reinhart]

On Sat, Aug 1, 2015 at 9:37 AM, Alexandre Feblot alexandre.feb...@gmail.com
 wrote:

 Old attributes will still be supported, so that the upgrade should be
 transparent.


Okay, good to hear. Thank you. I will try this out soon.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

[Scons-dev] Breaking changes in v2.4

[Jonathon Reinhart]

Hello all,

I'm looking at the upcoming Node API changes (for __slots__) mentioned here:

https://pairlist4.pair.net/pipermail/scons-users/2014-July/002734.html

Is it correct that we will be unable to use t.abspath in v2.4, and will
instead need to use a new method, t.get_abspath()? Is there any reason the
previous attribute name couldn't instead be implemented with @property, so
it doesn't break any existing code?

There are a few places where our builds use .abspath because there seemed
to be no better way. I have no problem porting the code to work with 2.4
but I'm wondering how many others there are.

Jonathon
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev