[issue43567] regen.vcxproj cannot regenerate some necessary files

[Steve Dower]

Steve Dower  added the comment:

We're updating all files now, and doing it automatically on build (or manually 
with --regen).

--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Steve Dower]

Steve Dower  added the comment:


New changeset 748283819043c60b1cb272c2cc9ab5b457afb03a by Steve Dower in branch 
'master':
bpo-43567: Improved generated code refresh on Windows (GH-25120)
https://github.com/python/cpython/commit/748283819043c60b1cb272c2cc9ab5b457afb03a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

Jiaxin: "Resolution: works for me" is meant for the triager to indicate that 
there is no bug, or at least that the repro given in the bug report doesn't 
trigger the bug in the triager's environment. It does *not* mean that a patch 
works for the submitter of the issue. But thanks for letting us know that 
Steve's patch works for you! :-)

--
resolution: works for me -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Jiaxin Peng]

Jiaxin Peng  added the comment:

Steve's PR zooba:bpo-43567 works as expected. Really clean implementation!
Thanks!

--
resolution:  -> works for me

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[anthony shaw]

anthony shaw  added the comment:

Guido, regen.vcxproj targets 142 as the SDK version, which is most likely a 
mistake.
The other projects are part of the main PCBuild.sln solution, which has a 
variable for the base SDK version.

If you need to change it quickly, you can either open it in VS and right click 
on the project and change the properties to an older SDK, or edit the vcxproj 
file itself.

I can make this change in a patch, but because its a standalone project, it 
doesn't share the base SDK version with the others.

I've reviewed Steve's patch and it would fix this because it changes regen into 
a build target instead of a build project, so it doesn't specify the SDK 
version at all.
Steve's implementation is also much, much cleaner!

--
nosy: +anthony shaw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

@Jiaxin Please test steve's PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Steve Dower]

Steve Dower  added the comment:

I've attached my PR that streamlines this, it should be good (as in, 
reliable/fast) to do the regen on every build, but the "build.bat --regen" 
command is retained to force it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +23866
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25120

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

Anthony, can you help us out here? Is there a problem with regen.vcsproj, 
either by depending on VS 2019 (which we don't officially support yet) or by 
missing the changes in the names of some generated files?

--
nosy: +anthonypjshaw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[STINNER Victor]

STINNER Victor  added the comment:

> not having Include/Python-ast.h and Python/Python-ast.c

FYI Include/Python-ast.h was renamed to Include/internal/pycore_ast.h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +pablogsal, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Jiaxin Peng]

Jiaxin Peng  added the comment:

The regen.vcxproj was modified but not test well. In the first version of 
regen.vcxproj there were such build targets:

_RegenGrammar(BeforeTargets="Build") -> _RegenAST_H -> _RegenAST_C -> 
_RegenOpcodes -> _RegenTokens -> _RegenKeywords -> _RegenSymbols

In commit 1ed83ad: bpo-40939: Remove the old parser (GH-20768),
"_RegenGrammar" was removed, but there's still "_RegenAST_H" as AfterTargets of 
"_RegenGrammar".

@gvanrossum sorry about removing nosy folks, This website displays in my 
language so I did not realize what is "nosy list".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

So is there a bug in regen.vcxproj, or in 
https://devguide.python.org/setup/#windows-compiling ? The latter says VS 2017.

@pjx206: please stop clearing out the nosy list. You've done that twice now. If 
you do it again, I'll close this issue.

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Jiaxin Peng]

Jiaxin Peng  added the comment:

`build.bat --regen` produces such error because it uses build tool from visual 
studio to build regen.vcxproj, and you do not have v142 toolset. I'm using 
vs2019 and have v142 toolset installed and regen.vcxproj workd, just cannot 
actually regenerate files needed completely.

--
nosy: +pjx206

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Jiaxin Peng]

Change by Jiaxin Peng :


--
nosy:  -anthonypjshaw, lys.nikolaou, pablogsal, paul.moore, pjx206, 
steve.dower, tim.golden, vstinner, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

FWIW, could this issue recently merged by Victor be related?
bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

@Steve, Anthony: I am now in a state where I have *no* changes anywhere, 
build.bat completes without errors, but build.bat --regen produces this error 
(after first doing a regular build):

C:\Program Files (x86)\Microsoft Visual 
Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Platform.targets(67,5):
 error MSB8020: The build to
ols for v142 (Platform Toolset = 'v142') cannot be found. To build using the 
v142 build tools, please install v142 build tools.  Alternatively, you may u 
pgrade to the current Visual Studio tools by selecting the Project menu or 
right-click the solution, and then selecting "Retarget solution". [C:\Users\gv 
anrossum\cpython\PCbuild\regen.vcxproj]

Help!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

Can you past the full output from your command?

Also, try making a dummy change (e.g. add a blank line) to each of the input 
files -- this will trigger the rebuild.

PS. Please don't remove nosy folks unless they request so.

--
nosy: +anthonypjshaw, lys.nikolaou, pablogsal, paul.moore, steve.dower, 
tim.golden, vstinner, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Jiaxin Peng]

Jiaxin Peng  added the comment:

I did:
- modified Grammar/python.gram
- added new operator to compare_op_bitwise_or_pair
- added corresponding token to Grammer/Tokens
- build.bar --regen
- not having Include/Python-ast.h and Python/Python-ast.c

I agree that a regen.bat that using a system python is better solution. 
regen.vcxproj is confusing.

--
nosy:  -anthonypjshaw, lys.nikolaou, pablogsal, paul.moore, steve.dower, 
tim.golden, vstinner, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

Workaround:

For opcode.py changes, I can probably use the following strategy:

- build.bat
- Add new opcode to opcode.py
- build.bat --regen
- build.bat
- Make changes to compile.c and ceval.c to implement new opcode
- build.bat

For the parser perhaps you can adopt a similar strategy, where you first
add new tokens and get the code generation working?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

I've found build.bat --regen unreliable myself. I went over it with Steve and 
one issue that came up is that it uses the python.exe that is built to run the 
code generation scripts; OTOH on Linux/Mac these scripts are run using a 
suitable pre-existing Python binary, e.g. a recent enough system Python. 
(However, the frozen files require a binary (_freeze_importlib).

It will take some time to refactor things so that this is more usable.

Perhaps instead of (or in addition to) doing everything via .vcxproj files, we 
could have a simple(r) regen.bat script that runs all the regen scripts (or at 
least the non-freeze ones) using the system Python?

FWIW my own issues weren't with the parser, they were with regenerating 
opcode.h and opcode_targets.h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[STINNER Victor]

STINNER Victor  added the comment:

By the way, it would be nice if someone could review my change in this project 
related to pycore_ast.h ;-) (commit 94faa0724f8cbae6867c491c8e465e35f4fdbfbb) I 
mean checking if pycore_ast.h is updated as expected.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> regen.vcxproj is now not capable for the new PEG parse

Hu, that file should handle the peg parser already:

https://github.com/python/cpython/blob/09b90a037d18f5d4acdf1b14082e57bda78e85d3/PCbuild/regen.vcxproj#L156-L163

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Guido van Rossum]

Guido van Rossum  added the comment:

Steve: let's talk about this in person today.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Steve Dower]

Change by Steve Dower :


--
nosy: +anthonypjshaw, gvanrossum, lys.nikolaou, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Zachary Ware]

Change by Zachary Ware :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43567] regen.vcxproj cannot regenerate some necessary files

[Jiaxin Peng]

New submission from Jiaxin Peng :

I tried to modify Grammar/python.gram, Grammar/Tokens, Parser/Python.asdl, to 
add a new token to the grammar. And when using `build.bat --regen`, only 
parser.c is newly generated. Other files, as mentioned in 
https://devguide.python.org/grammar/ that should be updated:ast.c, Python-ast.h 
etc.
regen.vcxproj is now not capable for the new PEG parser. So an update is need.

--
components: Build
messages: 389155
nosy: pjx206
priority: normal
severity: normal
status: open
title: regen.vcxproj cannot regenerate some necessary files
type: compile error
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com