Re: [Scons-dev] SCons speedup and profiling results...

[Dirk Bächle]

Hi Gary,

On 26.09.2013 02:08, Gary Oberbrunner wrote:



[...]

I think this is excellent work!  Solid analysis.  I know there's been 
some thought given to caching subst() before; it's trickier than one 
might think but in many cases it should work, and it definitely speeds 
things up.  I'm also impressed by a 30% memory reduction -- interested 
to hear how that comes out.




it will definitely take some more time. Not only subst() is quite 
tricky, but getting cyclic dependencies out of the way for 
garbage-collecting Nodes means a lot of fiddling. But there should be 
some light at the end of the tunnel soon


By the way: Does anybody know of a use case for having more than one FS 
during a build or interactive mode? I've wondered many times now, why on 
earth we keep a reference to the current DefaultFS in each Node? After 
all I don't see any place in the source code where I could initialize a 
second one, or one that's different from the Default FS...


Have you seen the speed and memory tests the buildbots used to run? 
 Not sure what their status is these days but perhaps your tests could 
be integrated into them at some point.


Yes, I considered using them. But as far as I know, they rely on 
everything that's being run to be a packaged archive and you always have 
to write some sort of config file. That's what put me off a little bit, 
and so I stuck to my own scripts/packages and developed them further...


Regards,

Dirk

___
Scons-dev mailing list
Scons-dev@scons.org
http://two.pairlist.net/mailman/listinfo/scons-dev

Re: [Scons-dev] SCons speedup and profiling results...

[Gary Oberbrunner]

On Wed, Sep 25, 2013 at 5:51 PM, Dirk Bächle  wrote:

> Hi there,
>
> a few minutes ago I added a new page to our Wiki. It's called
>
>   
> http://scons.org/wiki/**WhySconsIsNotSlow
>
> and shows a few results of the speedup and profiling experiments that I
> did recently. As mentioned
> in the "Repositories" section, you can also download the full set of
> results and the little test framework that
> I used from the Bitbucket repos:
>
>   
> http://www.bitbucket.org/**dirkbaechle/scons_testresults
>   
> http://www.bitbucket.org/**dirkbaechle/scons_testsuite
>
> So just dive in if you're interested, and let me hear what you think...
>
> By the way, I'm sorry for certain parts of the text being quite terse. The
> language could also be improved a little, I guess...but right now I'm
> lacking the time and patience for it (I just wanted this project off my
> desk ;) ). So if any native speakers out there should feel the itch to
> correct and extend my writings, just go ahead please. I'd highly appreciate
> it...and it's a Wiki after all.
>

I think this is excellent work!  Solid analysis.  I know there's been some
thought given to caching subst() before; it's trickier than one might think
but in many cases it should work, and it definitely speeds things up.  I'm
also impressed by a 30% memory reduction -- interested to hear how that
comes out.

Have you seen the speed and memory tests the buildbots used to run?  Not
sure what their status is these days but perhaps your tests could be
integrated into them at some point.

-- 
Gary
___
Scons-dev mailing list
Scons-dev@scons.org
http://two.pairlist.net/mailman/listinfo/scons-dev

Re: [Scons-dev] scons running on py2.7 and py3.3

[Gary Oberbrunner]

On Tue, Sep 24, 2013 at 7:15 AM, Neal Becker  wrote:

> Gary Oberbrunner wrote:
>
> > OK, I ported all of Neal's work onto a python3-port branch in the main
> > SCons repo.  I made an initial commit with just the 2to3, then added the
> > rest of his initial manual changes, and then cherry-picked all the rest
> of
> > his commits onto that branch.
> >
> > This is now the branch where all python3 work should be done.  I think
> it's
> > a good start.  Many tests currently don't pass in my Python 2.7.2 (on
> > Ubuntu 11.04); mostly str/unicode type errors so far.  Neal, did you run
> > the tests on python2?  Did things work for you?
> >
> >
>
> Tests are largely broken.  As you said, mostly str/unicode type errors.  I
> don't
> plan to work on fixing it in the near future.


I have a few fixes queued up.  But there's plenty of work ahead.  First
thing will be a sys.stdout.write wrapper, there's tons of those that fail
now.

-- 
Gary
___
Scons-dev mailing list
Scons-dev@scons.org
http://two.pairlist.net/mailman/listinfo/scons-dev

Re: [Scons-dev] scons running on py2.7 and py3.3

[Gary Oberbrunner]

On Tue, Sep 24, 2013 at 6:22 AM, Russel Winder  wrote:

> On Sun, 2013-09-22 at 13:33 -0400, Gary Oberbrunner wrote:
> > OK, I ported all of Neal's work onto a python3-port branch in the main
> > SCons repo.  I made an initial commit with just the 2to3, then added the
> > rest of his initial manual changes, and then cherry-picked all the rest
> of
> > his commits onto that branch.
> >
> > This is now the branch where all python3 work should be done.  I think
> it's
> > a good start.  Many tests currently don't pass in my Python 2.7.2 (on
> > Ubuntu 11.04); mostly str/unicode type errors so far.  Neal, did you run
> > the tests on python2?  Did things work for you?
>
> To date I have kept the D tooling variant of SCons as a separate
> repository using the default branch. This turns out to work fairly well,
> pending someone telling me when I can propose a merge into the mainline,
> which should have happened months ago ;-)
>
> Now with multiple branches how is one supposed to manage the differences
> so that the D Tooling appears in the Python 3 branch as well as the
> default branch?
>

I wondered about that as I was doing it, Russel.  I recognize that in hg
there's a contingent that says use separate repos for branches, and avoid
named branches.  But since we already have some named branches, and I'm
really at heart more of a git guy :-) I figured I'd do it that way -- but
I'm not wedded to it; I assume we can back it out and make a separate repo.
 But how do projects that work that way keep track of what's what?  Seems
like you need a meta-repo layer to communicate what's branched from where.

If we did it as separate repos per branch, would that address your
concerns?  You could of course merge your D stuff into each branch for
testing, or when it's merged to the mainline if we haven't merged the
python3 branch it should merge on top of your stuff (which may still need
some 2to3, as will any such branch getting merged from here on out).

-- 
Gary
___
Scons-dev mailing list
Scons-dev@scons.org
http://two.pairlist.net/mailman/listinfo/scons-dev

[Scons-dev] SCons speedup and profiling results...

[Dirk Bächle]

Hi there,

a few minutes ago I added a new page to our Wiki. It's called

  http://scons.org/wiki/WhySconsIsNotSlow

and shows a few results of the speedup and profiling experiments that I 
did recently. As mentioned
in the "Repositories" section, you can also download the full set of 
results and the little test framework that

I used from the Bitbucket repos:

  http://www.bitbucket.org/dirkbaechle/scons_testresults
  http://www.bitbucket.org/dirkbaechle/scons_testsuite

So just dive in if you're interested, and let me hear what you think...

By the way, I'm sorry for certain parts of the text being quite terse. 
The language could also be improved a little, I guess...but right now 
I'm lacking the time and patience for it (I just wanted this project off 
my desk ;) ). So if any native speakers out there should feel the itch 
to correct and extend my writings, just go ahead please. I'd highly 
appreciate it...and it's a Wiki after all.


Best regards,

Dirk

___
Scons-dev mailing list
Scons-dev@scons.org
http://two.pairlist.net/mailman/listinfo/scons-dev