Re: build/generator/templates/*.ezt and .bat files and svn:eol-style

[Yasuhito FUTATSUKI]

On 2022/08/21 22:16, Nathan Hartman wrote:

> This makes me wonder, though, how EZT [1] (the program that processes
> these templates) generates its output. Does it output the same line
> endings as found in the templates? Or does it unconditionally output
> "\n" endings when run on Unix and "\r\n" endings when run on Windows,
> regardless of the template's line endings? I haven't checked, but the
> answer to this question will strongly influence how the line endings
> of *.ezt files should be.

It depends on how we use EZT. EZT reads templates without changing
EOL style by read them in binary mode, and then write as str to
the stream 'fp' passed in generate() method by using fp.write().

The templates used only for Windows environments are:
build_locale.ezt
svn_config.vcproj.ezt
svn_config.vcxproj.ezt
svn_locale.vcproj.ezt
svn_locale.vcxproj.ezt
vcnet_sln.ezt
vcnet_vcproj.ezt
vcnet_vcxproj.ezt
vcnet_vcxproj_filters.ezt
vcnet_vsprops.ezt

Those are handled by gen_win.WinGeneratorBase and preserves EOL
style. It generates a content by passing StringIO() to
Template.generate() and then encode it as bytes, then write it
in binary mode.


The template used by gen_make.py are:
build-outputs.mk.ezt
pkg-config.in.ezt

Those files are written with text mode, opened with "w" mode and
default newline parameter (newline=None), so "\n" is  comverted
to "\r\n" if we run gen_make.py on Windows (but I not sure it
needs or not).

I think templates in tools/dist/templates are only for
tools/dist/release.py and it is assumed that it runs only on
Unix, so I didn't check them.

Cheers,
-- 
Yasuhito FUTATSUKI 

Re: build/generator/templates/*.ezt and .bat files and svn:eol-style

[Nathan Hartman]

On Sat, Aug 20, 2022 at 7:24 PM Vincent Lefevre  wrote:
>
> On 2022-08-20 14:08:43 +0200, Branko Čibej wrote:
> > On 20.08.2022 00:57, Vincent Lefevre wrote:
> > > A minor issue I had found during my checks for the URL update:
> > [...]
> > > Shouldn't build/generator/templates/vcnet_vsprops.ezt have
> > > svn:eol-style set to native so that it doesn't get CRLF
> > > line terminators under Unix?
> > >
> > > Similarly, .bat files have inconsistent svn:eol-style properties:
> >
> > Actually no, both should have CRLF line endings enforced. .bat files
> > certainly should, at least once upon a time they would otherwise be invalid
> > on Windows. For the .vcproj etc (XML) files, I'm not sure whether
> > VisualStudio still cares, but the .properties, IIRC are treated differently?
>
> So, shouldn't CRLF line endings be enforced for *all* .ezt and .bat files?


For all *.bat files, I think CRLF should be enforced.

But not for all *.ezt. Some, perhaps, but not all. The .ezt files are
templates used for various purposes. For example, while
build/generator/templates/build_locale.ezt is a template for a DOS
batch file, others such as (I'm picking one at random--it wasn't in
your list) tools/dist/templates/release-notes-lts.ezt are templates
for other file types (HTML in this particular case). So perhaps
relevant .ezt files should be changed to CRLF where appropriate,
depending on their content.

This makes me wonder, though, how EZT [1] (the program that processes
these templates) generates its output. Does it output the same line
endings as found in the templates? Or does it unconditionally output
"\n" endings when run on Unix and "\r\n" endings when run on Windows,
regardless of the template's line endings? I haven't checked, but the
answer to this question will strongly influence how the line endings
of *.ezt files should be.

[1] https://github.com/gstein/ezt

Cheers,
Nathan

Re: build/generator/templates/*.ezt and .bat files and svn:eol-style

[Vincent Lefevre]

On 2022-08-20 14:08:43 +0200, Branko Čibej wrote:
> On 20.08.2022 00:57, Vincent Lefevre wrote:
> > A minor issue I had found during my checks for the URL update:
> [...]
> > Shouldn't build/generator/templates/vcnet_vsprops.ezt have
> > svn:eol-style set to native so that it doesn't get CRLF
> > line terminators under Unix?
> > 
> > Similarly, .bat files have inconsistent svn:eol-style properties:
> 
> Actually no, both should have CRLF line endings enforced. .bat files
> certainly should, at least once upon a time they would otherwise be invalid
> on Windows. For the .vcproj etc (XML) files, I'm not sure whether
> VisualStudio still cares, but the .properties, IIRC are treated differently?

So, shouldn't CRLF line endings be enforced for *all* .ezt and .bat files?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: build/generator/templates/*.ezt and .bat files and svn:eol-style

[Branko Čibej]

On 20.08.2022 00:57, Vincent Lefevre wrote:

A minor issue I had found during my checks for the URL update:

[...]

Shouldn't build/generator/templates/vcnet_vsprops.ezt have
svn:eol-style set to native so that it doesn't get CRLF
line terminators under Unix?

Similarly, .bat files have inconsistent svn:eol-style properties:


Actually no, both should have CRLF line endings enforced. .bat files 
certainly should, at least once upon a time they would otherwise be 
invalid on Windows. For the .vcproj etc (XML) files, I'm not sure 
whether VisualStudio still cares, but the .properties, IIRC are treated 
differently?


-- Brane

build/generator/templates/*.ezt and .bat files and svn:eol-style

[Vincent Lefevre]

A minor issue I had found during my checks for the URL update:

for i in build/generator/templates/*.ezt
do file $i; svn pg svn:eol-style $i; done

gives

build/generator/templates/build-outputs.mk.ezt: ASCII text, with very long 
lines (478)
native
build/generator/templates/build_locale.ezt: DOS batch file, ASCII text
native
build/generator/templates/pkg-config.in.ezt: ASCII text
native
build/generator/templates/svn_config.vcproj.ezt: exported SGML document, ASCII 
text
native
build/generator/templates/svn_config.vcxproj.ezt: exported SGML document, 
Unicode text, UTF-8 (with BOM) text
native
build/generator/templates/svn_locale.vcproj.ezt: exported SGML document, ASCII 
text
native
build/generator/templates/svn_locale.vcxproj.ezt: exported SGML document, ASCII 
text
native
build/generator/templates/vcnet_sln.ezt: ASCII text
native
build/generator/templates/vcnet_vcproj.ezt: exported SGML document, ASCII text
native
build/generator/templates/vcnet_vcxproj.ezt: exported SGML document, Unicode 
text, UTF-8 (with BOM) text
native
build/generator/templates/vcnet_vcxproj_filters.ezt: exported SGML document, 
ASCII text
native
build/generator/templates/vcnet_vsprops.ezt: exported SGML document, ASCII 
text, with CRLF line terminators
svn: warning: W200017: Property 'svn:eol-style' not found on 
'build/generator/templates/vcnet_vsprops.ezt'
svn: E20: A problem occurred; see other errors for details

Shouldn't build/generator/templates/vcnet_vsprops.ezt have
svn:eol-style set to native so that it doesn't get CRLF
line terminators under Unix?

Similarly, .bat files have inconsistent svn:eol-style properties:

for i in **/*.bat
do file $i; svn pg svn:eol-style $i; done

(under zsh or bash+globstar) gives

contrib/client-side/diff3wrap.bat: DOS batch file, ASCII text, with CRLF line 
terminators
CRLF
contrib/client-side/diffwrap.bat: DOS batch file, ASCII text, with CRLF line 
terminators
CRLF
subversion/tests/cmdline/svneditor.bat: DOS batch file, ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/config.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/do_all.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svnbuild.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svncheck.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svnclean.bat: ASCII text
native
tools/buildbot/slaves/win32-xp-VS2005/svnlog.bat: ASCII text
native
tools/dev/benchmarks/suite1/run.bat: ASCII text
native

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)