[Scons-dev] buildbot.scons.org may be down for about an hour due to DNS issues..
It'll probably point to my baddogconsulting.com website until the dns update propagates. ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Hi everyone,
Why cannot .path, .abspath, etc. become properties (with getters set to
.get_path, .get_abspath, etc)? What's the need to break existing API? I
think there're TONS of files which are using those functions... and
thinking about patching all those makes me really sad :(
Thanks,
Vasily
2015-03-02 12:14 GMT+03:00 :
> Hum,
>
> in fact, 1 and 3 and also the idea to checkin Scons 2.3 in our current
> code (as well as all maintenance branches) are equivalent, and painful: We
> have many maintenance branches, and hundreds of tags that potentially may
> have to be checked out again and recompiled.
>
> 2 seems to be the only actual solution in order to keep all existing code
> working.
>
> However, what would you think about such implementation which keeps
> compatible with current code:
>
> - rename private attributes. example: abspath -> private_abspath
> - for new code, use new interface. example: get_abspath() { return
> self.private_abspath }
> - for old code, allow using the old private attribute thanks to
> __getattr__():
>
>
> class Node:
>
> __slots__ = ('private_abspath')
>
> def __init__(self):
> self.private_abspath = None
>
> def get_abspath(self):
> self.private_abspath = "/a/path" # Lazy initialization
> return self.private_abspath
>
> def __getattr__(self, attr):
> if attr == "abspath":
> print "warning: abspath supported only for API consistency
> with peformance loss. Use get_abspath()" # in real class,this warning
> should only be printed once.
> return self.get_abspath()
>
>
> n = Node()
> print n.get_abspath()
> print n.abspath
>
> result:
>
> /a/path
> warning: abspath supported only for API consistency with peformance loss.
> Use get_abspath()
> /a/path
>
> Note: I always considered .path and .abspath as public API as they are
> documented and used in many examples in the documentation. Was that a wrong
> assumption?
>
>
>
>
> Le 2 mars 2015 à 09:30, Dirk Bächle a écrit :
>
> Alexandre,
>
> On 02.03.2015 09:06, alexandre.feb...@gmail.com wrote:
>
> Hi Dirk,
>
> About those new interfaces for accessing node attributes that may get
> initialized lazily, do I understand here the consequences properly?
>
> - Using directly a_node.path or a_node.abspath *must* be replaced in all
> SConstructs/SConscripts by the new interfaces if we wan’t them to keep
> working properly
> - This lets us with 2 solutions:
>1- Do these changes in all our product maintenance branches, and even
> each time we need to checkout again an old tag we want to compile again.
>2- Keep an old Scons 2.3 to build our old code, and change our build
> infrastructure to be able to have different SCons paths, so we can chose
> the proper one.
>
>
> sorry, I forgot:
>
> 3- Factor out all your direct accesses to Node internals into a
> custom module/Builder, and then wrap it with a check for the SCons version.
> For 2.3.x and smaller, you access Node attributes as before, and for the
> newer versions you then branch to using the getter methods.
>
> This is pretty much what the compat layer would do, which I mentioned in
> my last reply.
>
> Best regards,
>
> Dirk
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Can we use property decorators to resolve this for standard Node properties?
-Bill
On Mon, Mar 2, 2015 at 8:40 AM, Kenny, Jason L
wrote:
> Hi dirk,
>
> I need to give this a run. As I am one of the people that would be in a
> dangerous area :-)
>
> My first thought however would be this:
>
> 1) did we add a __dict__ to the __slots__ ? if not can we, this would
> probally fix 99% of the issues I would have in Parts
> 2) stuff like t.abspath -> t.get_abspath(), can the t.abspath be a
> property that calls t.get_abspath()?
>
> This seems like this would deal with most of the concerns people might
> have. I understand that there might be good reason not to do this. I
> haven't looked in detail yet myself.
>
> Just some thoughts.
>
> Jason
>
>
> -Original Message-
> From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Dirk
> Bächle
> Sent: Saturday, February 28, 2015 6:03 AM
> To: SCons users mailing list
> Cc: SCons developer list
> Subject: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching
> the Node class to using slots...
>
> Hello there,
>
> as has been pre-announced at
>
>https://pairlist4.pair.net/pipermail/scons-users/2014-July/002734.html
>
> and
>
>
> https://pairlist2.pair.net/pipermail/scons-dev/2014-December/002107.html
>
> we're still planning to switch the Node class to using "slots" in the core
> sources, mainly for reducing the overall memory consumption by up to 35% in
> large build projects.
>
> We have now pushed a new branch named "switch_to_slots" to the main SCons
> repo, and ran it against our test suite in Buildbot (
> http://buildbot.scons.org/waterfall ). So far, the implementation looks
> stable and for the normal user there shouldn't be any danger of
> scripts/builds breaking after an update to the new release 2.4 (coming
> probably in a month or so).
>
> However, if you write and maintain your own extensions, where you
> regularly reach into the bowels of SCons to extract internal data or to
> override methods (monkey-patching), there is a chance that you'll get
> bitten by the changes as described in the appendix below.
>
> We invite any interested user/developer to try out the current "slots"
> branch, by cloning
>
>hg clone http://bitbucket.org/scons/scons -r switch_to_slots
>
> to a local folder (not your standard SCons repo!), and starting it via the
> "bootstrap.py" script:
>
>python ~/try_out_folder/scons/bootstrap.py
>
> Please report back any errors or inconveniences that you find, such that
> we can work on a solution together.
> If you have further questions, better ask them now. We won't look back
> after merging this. ;)
>
>
> Best regards,
>
> Dirk
>
>
> Appendix: Short list of changes to the Node* classes
>
>
> - Introduction of decorators for memoizer debugging/counting (we can't use
> metaclasses anymore).
> - Adding interface for accessing node attributes that may get initialized
> lazily:
> t.abspath -> t.get_abspath()
> t.labspath -> t.get_labspath()
> t.tpath -> t.get_tpath()
> t.path_elements -> t.get_path_elements()
> t.path -> t.get_internal_path()
>
> - Methods like t.exists(), t.rexists() and t.get_contents() can't be
> re-assigned directly anymore. They'll be realized via function maps
> internally ( {int : func} ).
>
> - NodeInfo and BuildInfo will also get switched to __slots__, and get a
> new internal version number (1.0 -> 2.0). This means that after the switch
> *all* targets are regarded to be out-of-date on the first build.
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Hi dirk,
I need to give this a run. As I am one of the people that would be in a
dangerous area :-)
My first thought however would be this:
1) did we add a __dict__ to the __slots__ ? if not can we, this would probally
fix 99% of the issues I would have in Parts
2) stuff like t.abspath -> t.get_abspath(), can the t.abspath be a property
that calls t.get_abspath()?
This seems like this would deal with most of the concerns people might have. I
understand that there might be good reason not to do this. I haven't looked in
detail yet myself.
Just some thoughts.
Jason
-Original Message-
From: Scons-dev [mailto:scons-dev-boun...@scons.org] On Behalf Of Dirk Bächle
Sent: Saturday, February 28, 2015 6:03 AM
To: SCons users mailing list
Cc: SCons developer list
Subject: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the
Node class to using slots...
Hello there,
as has been pre-announced at
https://pairlist4.pair.net/pipermail/scons-users/2014-July/002734.html
and
https://pairlist2.pair.net/pipermail/scons-dev/2014-December/002107.html
we're still planning to switch the Node class to using "slots" in the core
sources, mainly for reducing the overall memory consumption by up to 35% in
large build projects.
We have now pushed a new branch named "switch_to_slots" to the main SCons repo,
and ran it against our test suite in Buildbot (
http://buildbot.scons.org/waterfall ). So far, the implementation looks stable
and for the normal user there shouldn't be any danger of scripts/builds
breaking after an update to the new release 2.4 (coming probably in a month or
so).
However, if you write and maintain your own extensions, where you regularly
reach into the bowels of SCons to extract internal data or to override methods
(monkey-patching), there is a chance that you'll get bitten by the changes as
described in the appendix below.
We invite any interested user/developer to try out the current "slots" branch,
by cloning
hg clone http://bitbucket.org/scons/scons -r switch_to_slots
to a local folder (not your standard SCons repo!), and starting it via the
"bootstrap.py" script:
python ~/try_out_folder/scons/bootstrap.py
Please report back any errors or inconveniences that you find, such that we can
work on a solution together.
If you have further questions, better ask them now. We won't look back after
merging this. ;)
Best regards,
Dirk
Appendix: Short list of changes to the Node* classes
- Introduction of decorators for memoizer debugging/counting (we can't use
metaclasses anymore).
- Adding interface for accessing node attributes that may get initialized
lazily:
t.abspath -> t.get_abspath()
t.labspath -> t.get_labspath()
t.tpath -> t.get_tpath()
t.path_elements -> t.get_path_elements()
t.path -> t.get_internal_path()
- Methods like t.exists(), t.rexists() and t.get_contents() can't be
re-assigned directly anymore. They'll be realized via function maps internally
( {int : func} ).
- NodeInfo and BuildInfo will also get switched to __slots__, and get a new
internal version number (1.0 -> 2.0). This means that after the switch *all*
targets are regarded to be out-of-date on the first build.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Hum,
in fact, 1 and 3 and also the idea to checkin Scons 2.3 in our current code (as
well as all maintenance branches) are equivalent, and painful: We have many
maintenance branches, and hundreds of tags that potentially may have to be
checked out again and recompiled.
2 seems to be the only actual solution in order to keep all existing code
working.
However, what would you think about such implementation which keeps compatible
with current code:
- rename private attributes. example: abspath -> private_abspath
- for new code, use new interface. example: get_abspath() { return
self.private_abspath }
- for old code, allow using the old private attribute thanks to __getattr__():
class Node:
__slots__ = ('private_abspath')
def __init__(self):
self.private_abspath = None
def get_abspath(self):
self.private_abspath = "/a/path" # Lazy initialization
return self.private_abspath
def __getattr__(self, attr):
if attr == "abspath":
print "warning: abspath supported only for API consistency with
peformance loss. Use get_abspath()" # in real class,this warning should only be
printed once.
return self.get_abspath()
n = Node()
print n.get_abspath()
print n.abspath
result:
/a/path
warning: abspath supported only for API consistency with peformance loss. Use
get_abspath()
/a/path
Note: I always considered .path and .abspath as public API as they are
documented and used in many examples in the documentation. Was that a wrong
assumption?
> Le 2 mars 2015 à 09:30, Dirk Bächle a écrit :
>
> Alexandre,
>
> On 02.03.2015 09:06, alexandre.feb...@gmail.com wrote:
>> Hi Dirk,
>>
>> About those new interfaces for accessing node attributes that may get
>> initialized lazily, do I understand here the consequences properly?
>>
>> - Using directly a_node.path or a_node.abspath *must* be replaced in all
>> SConstructs/SConscripts by the new interfaces if we wan’t them to keep
>> working properly
>> - This lets us with 2 solutions:
>>1- Do these changes in all our product maintenance branches, and even
>> each time we need to checkout again an old tag we want to compile again.
>>2- Keep an old Scons 2.3 to build our old code, and change our build
>> infrastructure to be able to have different SCons paths, so we can chose the
>> proper one.
>>
>
> sorry, I forgot:
>
> 3- Factor out all your direct accesses to Node internals into a custom
> module/Builder, and then wrap it with a check for the SCons version. For
> 2.3.x and smaller, you access Node attributes as before, and for the newer
> versions you then branch to using the getter methods.
>
> This is pretty much what the compat layer would do, which I mentioned in my
> last reply.
>
> Best regards,
>
> Dirk
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Alexandre, On 02.03.2015 09:06, alexandre.feb...@gmail.com wrote: Hi Dirk, About those new interfaces for accessing node attributes that may get initialized lazily, do I understand here the consequences properly? - Using directly a_node.path or a_node.abspath *must* be replaced in all SConstructs/SConscripts by the new interfaces if we wan’t them to keep working properly - This lets us with 2 solutions: 1- Do these changes in all our product maintenance branches, and even each time we need to checkout again an old tag we want to compile again. 2- Keep an old Scons 2.3 to build our old code, and change our build infrastructure to be able to have different SCons paths, so we can chose the proper one. sorry, I forgot: 3- Factor out all your direct accesses to Node internals into a custom module/Builder, and then wrap it with a check for the SCons version. For 2.3.x and smaller, you access Node attributes as before, and for the newer versions you then branch to using the getter methods. This is pretty much what the compat layer would do, which I mentioned in my last reply. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Hi Alexandre, thanks for showing your interest in this thread/announcement. On 02.03.2015 09:06, alexandre.feb...@gmail.com wrote: Hi Dirk, About those new interfaces for accessing node attributes that may get initialized lazily, do I understand here the consequences properly? - Using directly a_node.path or a_node.abspath *must* be replaced in all SConstructs/SConscripts by the new interfaces if we wan’t them to keep working properly Yes, that is correct. - This lets us with 2 solutions: 1- Do these changes in all our product maintenance branches, and even each time we need to checkout again an old tag we want to compile again. 2- Keep an old Scons 2.3 to build our old code, and change our build infrastructure to be able to have different SCons paths, so we can chose the proper one. Depending on how exactly your build structure looks like, these are your main options, I guess. You might want to consider using an scons-local package for now, that you check in with your current source code (on a branch?). Like this, you just have to checkout the branch and can build your projects right away. For the "default" branch (trunk) you would then switch the interfaces to the SCons 2.4+, which you install on all machines per default as before. If this gives you headaches, let's talk about it some more. I had some ideas about a shim compatibility layer in mind...but in general it's better for you to get away from directly accessing SCons internals. Best regards, Dirk ___ Scons-dev mailing list Scons-dev@scons.org https://pairlist2.pair.net/mailman/listinfo/scons-dev
Re: [Scons-dev] Announcement: Core changes for v2.4 ahead, Switching the Node class to using slots...
Hi Dirk,
About those new interfaces for accessing node attributes that may get
initialized lazily, do I understand here the consequences properly?
- Using directly a_node.path or a_node.abspath *must* be replaced in all
SConstructs/SConscripts by the new interfaces if we wan’t them to keep working
properly
- This lets us with 2 solutions:
1- Do these changes in all our product maintenance branches, and even each
time we need to checkout again an old tag we want to compile again.
2- Keep an old Scons 2.3 to build our old code, and change our build
infrastructure to be able to have different SCons paths, so we can chose the
proper one.
Kind Regards
> Le 28 févr. 2015 à 13:03, Dirk Bächle a écrit :
>
> Appendix: Short list of changes to the Node* classes
>
>
> - Introduction of decorators for memoizer debugging/counting (we can't
> use metaclasses anymore).
> - Adding interface for accessing node attributes that may get
> initialized lazily:
> t.abspath -> t.get_abspath()
> t.labspath -> t.get_labspath()
> t.tpath -> t.get_tpath()
> t.path_elements -> t.get_path_elements()
> t.path -> t.get_internal_path()
>
> - Methods like t.exists(), t.rexists() and t.get_contents() can't be
> re-assigned directly anymore. They'll be realized via function maps
> internally ( {int : func} ).
>
> - NodeInfo and BuildInfo will also get switched to __slots__, and get a
> new internal version number (1.0 -> 2.0). This means that after the
> switch *all* targets are regarded to be out-of-date on the first build.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
