[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: I've updated and synced the What's New files across master, 3.7, 3.6, 3.5 (PR pending RM merge), and 2.7. So I think we are finally done here. -- priority: deferred blocker -> normal resolution: -> fixed stage: patch review -> resolved status: open ->

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: New changeset 4fde701133643d09354ec4bda6bdf78d85aed797 by Ned Deily in branch '2.7': bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7624) https://github.com/python/cpython/commit/4fde701133643d09354ec4bda6bdf78d85aed797 --

[issue23404] 'make touch' does not work with git clones of the source repository

Change by Ned Deily : -- pull_requests: +7243 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue23404] 'make touch' does not work with git clones of the source repository

Change by Ned Deily : -- pull_requests: +7242 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: New changeset 1b5731e2761a9a1b7394b3a81ed267f94fde42ca by Ned Deily in branch '3.6': bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7622) https://github.com/python/cpython/commit/1b5731e2761a9a1b7394b3a81ed267f94fde42ca --

[issue23404] 'make touch' does not work with git clones of the source repository

Change by Ned Deily : -- pull_requests: +7241 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: New changeset 144493dd7f2640f7e9091862ece3e0a6aca07884 by Ned Deily (Miss Islington (bot)) in branch '3.7': bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7620) (GH-7621)

[issue23404] 'make touch' does not work with git clones of the source repository

Change by miss-islington : -- pull_requests: +7240 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: New changeset 9d6171ded5c56679bc295bacffc718472bcb706b by Ned Deily in branch 'master': bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7620) https://github.com/python/cpython/commit/9d6171ded5c56679bc295bacffc718472bcb706b --

[issue23404] 'make touch' does not work with git clones of the source repository

Change by Ned Deily : -- pull_requests: +7239 stage: backport needed -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: No, I'm not currently working on this (to be completely honest, I entirely forgot we hadn't backported the What's New updates to the various branches). They'll need to be independent updates, since they affect a different file for each

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: Nick, were you planning to do the work for the other (non-3.6) branches? If not, can someone else pick this up please? -- ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: There still might be time to get this into 3.5.4 final, if Larry agrees. -- ___ Python tracker ___

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: Moving to deferred blocker and taking 3.6 off the affected versions list, since the 3.6 What's New has now been updated appropriately. Items still to be done: - make a similar change to the 3.5 What's New on the 3.5 branch - update the maintenance release

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: New changeset e1b690370fd8f93bef1e69eeea2695f95a7cfff5 by Nick Coghlan in branch '3.6': bpo-23404: `make regen-all` What's New entry (#2128) https://github.com/python/cpython/commit/e1b690370fd8f93bef1e69eeea2695f95a7cfff5 --

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: Thanks for that Victor. I started to review the PR, and then realised my suggestions were going to be extensive enough that it made more sense to post an alternate PR for you to review: https://github.com/python/cpython/pull/2128 --

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by Nick Coghlan : -- pull_requests: +2182 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: I wrote https://github.com/python/cpython/pull/2127 to document "make regen-all". I'm not sure that it's the correct way to document such build change in minor 3.6 releases. Can someone please take a look? -- ___

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by STINNER Victor : -- pull_requests: +2181 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: As Nick noted, this change needs to be properly documented for 3.6.2 and other maintenance releases. -- nosy: +benjamin.peterson, larry priority: critical -> release blocker ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: As noted on python-dev, it would be helpful if this change was mentioned in the 2.7, 3.5, and 3.6 What's New documents under a "Notable changes in maintenance releases" section. For 2.7, that would be a retitling of the existing section "New features added to

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: I modified Python 2.7, 3.5, 3.6 and master (3.7). I prefer to keep the issue open at least one week in a regression is found, like issue #30273 (this one should already be fixed). Maybe the devguide should be updated too? --

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: New changeset 8a19eb24c97ef43e9fc7d45af180334ac8093545 by Victor Stinner in branch '2.7': bpo-23404: make touch becomes make regen-all (#1466) https://github.com/python/cpython/commit/8a19eb24c97ef43e9fc7d45af180334ac8093545 --

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by STINNER Victor : -- pull_requests: +1565 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: New changeset ab6b962ef241be97536573d7490ce1cfc74fde18 by Victor Stinner in branch '3.5': bpo-23404: make touch becomes make regen-all (#1405) (#1461) (#1465) https://github.com/python/cpython/commit/ab6b962ef241be97536573d7490ce1cfc74fde18 --

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by STINNER Victor : -- pull_requests: +1563 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: New changeset 9d02f562961efd12d3c8317a10916db7f77330cc by Victor Stinner in branch '3.6': [3.6] bpo-23404: make touch becomes make regen-all (#1405) (#1461) https://github.com/python/cpython/commit/9d02f562961efd12d3c8317a10916db7f77330cc --

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: New changeset b109a1d3360fc4bb87b9887264e3634632d392ca by Victor Stinner in branch 'master': bpo-30273: Update sysconfig (#1464) https://github.com/python/cpython/commit/b109a1d3360fc4bb87b9887264e3634632d392ca --

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by STINNER Victor : -- pull_requests: +1562 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: I started a thread on python-dev to ask if it's ok to backport the change to 2.7, 3.5 and 3.6. I started to backport the change to 3.6: https://github.com/python/cpython/pull/1461 -- ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by STINNER Victor : -- pull_requests: +1557 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: I just created the issue #30259: Test somehow that generated files are up to date: run make regen-all. -- ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: New changeset a5c62a8e9f0de6c4133825a5710984a3cd5e102b by Victor Stinner in branch 'master': bpo-23404: make touch becomes make regen-all (#1405) https://github.com/python/cpython/commit/a5c62a8e9f0de6c4133825a5710984a3cd5e102b --

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: > If we go in this direction, maybe we can go further and also run autoconf && > autoheader? Oh, I just noticed that there is already a "make autoconf" target, I didn't know. -- ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: Nick Coghlan: "Could we use "regen-*" as the pattern for the file regeneration targets?" Sure, done. I checked the CPython documentation, build scripts, etc. I confirm that "rebuild" usually means "recompile Python from scratch", it's even an explicit

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: Could we use "regen-*" as the pattern for the file regeneration targets? I'd expect "make rebuild-all" to be akin to "make clean && make", since in the context of a C project, "build" maps to "generate the output binaries" for me. --

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: https://github.com/python/cpython/pull/1405#pullrequestreview-35923931 Zachary Ware: "LGTM, but I haven't fully parsed how the old mess worked. We should also have CI (either Travis or buildbot) confirming that make rebuild-all doesn't leave any checked-in

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: The issue #23404 has been marked as a duplicate of this bug: "AMD64 FreeBSD 9.x 3.x" tries to rebuild Include/opcode.h, timestamp issue. -- ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: I rewrote Python (UNIX/BSD) build system to not rebuild generated files based on file modification time: the action is now explicit. This change should not only fix buildbots, but also ease cross-compilation, and more generally make Python build less painful!

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by STINNER Victor : -- pull_requests: +1513 ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

Martin Panter added the comment: A while ago I wrote a patch targetting Issue 22359 that may be a starting point for “make regen”: . It pulled out three recipes into separate “phony” targets: “make graminit importlib importlib_external”.

[issue23404] 'make touch' does not work with git clones of the source repository

Nick Coghlan added the comment: As Martin notes, my concern is that I can go for years without having to touch the low level files (since syntax changes are rare), so it's important to me to make it easy to remember and/or relearn how to regenerate them. However, suitable make commands (e.g.

[issue23404] 'make touch' does not work with git clones of the source repository

Brett Cannon added the comment: Yep, that's exactly what I was suggesting, Martin (and sorry if I missed you saying that earlier; on vacation so trying to keep email/open source time to minimum :) . -- ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

Martin Panter added the comment: Do you mean a separate makefile rule to rebuild the generated files, rather than rebuilding them in a normal “make all” build? I would support this; this is what I meant with my “make regenerate” proposal in response to Nick linked above. --

[issue23404] 'make touch' does not work with git clones of the source repository

Brett Cannon added the comment: Would it make sense to have a `make rebuild` or something which explicitly ignores prebuilt files? That could then also serve as non-obvious documentation on how to build those files in the first place. -- nosy: +ncoghlan

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by Kubilay Kocak : -- nosy: +koobs ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

Martin Panter added the comment: Last time I proposed removing the automatic rebuilding of checked-in generated files, it seemed getting a consensus would not be trivial. Nick seemed strongly against changing the status quo:

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: > Anyway my preference was to stop automatically regenerating files, and to > keep the build system as simple as practical without hacks like the BOOT="#" > flag. In the past, this issue bite me so many times on Solaris, FreeBSD, etc. One because there was

[issue23404] 'make touch' does not work with git clones of the source repository

STINNER Victor added the comment: FYI 3 FreeBSD buildbots are currently broken because of this issue. -- nosy: +haypo ___ Python tracker ___

[issue23404] 'make touch' does not work with git clones of the source repository

Martin Panter added the comment: I had mainly been using my boot-flag.patch with BSD Make (bmake) rather than Gnu Make. It seems I was relying on a bug in BSD Make that merges escaped newlines in command lines, despite Posix and Gnu Make. Anyway, Chi’s fix seems appropriate. I am posting the

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by Martin Panter : Added file: http://bugs.python.org/file46773/boot-flag.py2.patch ___ Python tracker ___

[issue23404] 'make touch' does not work with git clones of the source repository

Zachary Ware added the comment: I've disabled the Touch step on the buildbots until this is fixed. -- keywords: +buildbot nosy: +zach.ware priority: normal -> critical versions: +Python 2.7, Python 3.7 ___ Python tracker

[issue23404] 'make touch' does not work with git clones of the source repository

Chi Hsuan Yen added the comment: Just tried boot-flag.patch with https://github.com/python/cpython/commit/16e07881bd3867d471dd0a25c5117672b65c90ee. Doesn't work out-of-box: #./Programs/_freeze_importlib \ ./Lib/importlib/_bootstrap.py Python/importlib.h /bin/sh: line 1:

[issue23404] 'make touch' does not work with git clones of the source repository

Martin Panter added the comment: Okay so my “make -t” trick has various flaws. You still have to know the filenames to specify, it requires Makefile to be generated by configure in the source tree, and it creates empty files if you use it in a separate build directory. Another idea: instead

[issue23404] 'make touch' does not work with git clones of the source repository

Martin Panter added the comment: Perhaps you can use “make -t ” instead of “make touch”. If you know the filename you want to avoid regenerating, that would not need any extra tools, no Mercurial or Git or any other version control. E.g.: make -t Include/Python-ast.h -- nosy:

[issue23404] 'make touch' does not work with git clones of the source repository

Changes by Ezio Melotti : -- nosy: +brett.cannon ___ Python tracker ___ ___

[issue23404] 'make touch' does not work with git clones of the source repository

Berker Peksag added the comment: Patch LGTM. Thanks Carol. I will left this issue open in case someone wants to work on a git equivalent of make touch as described by Ned in msg235514. -- components: -Devguide nosy: +berker.peksag stage: patch review - needs patch type: -

[issue23404] 'make touch' does not work with git clones of the source repository

Carol Willing added the comment: Using Ned Deily's feedback, I have added a devguide patch to clarify that 'make touch' only works with a Mercurial clone and was not designed for git clones. This patch, iss23404devguide.patch, resolves the devguide portion of this issue unless anyone has

[issue23404] 'make touch' does not work with git clones of the source repository

Vinson Lee added the comment: With the supplied patch on CentOS 6 , make passes and make test passes with the exception of test_readline that is Issue19884. 372 tests OK. 1 test failed: test_readline 7 tests altered the execution environment: test_calendar test_distutils test_float

[issue23404] 'make touch' does not work with git clones of the source repository

Vinson Lee added the comment: I tried a hg copy of the repository. make touch worked for me. ./configure make touch make -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23404 ___

[issue23404] 'make touch' does not work with git clones of the source repository

Ned Deily added the comment: Unfortunately, those syntax differences aren't the only problems you could run into so making these syntax changes isn't really a general solution, IMO. 'make touch' is the documented and supported way to ensure that the unnecessary build steps are not run. But,