Re: Interesting discovery from automatic tools (python code)
On Thu, 2022-08-11 at 20:29 +0200, Pavel Sanda wrote: > > I did not use the exceptions to understand the failure reasons. > > The issue with the except is that it is a catch all option and so > > the > > exception could be caused by faulty assumptions from us in the > > initial > > code. > > My understanding is that subprocess.check_call does not return > nonzero > to signal nonzero exit status of the called subroutine, but raises > exception. So the correct thing is to catch exception, not checking > return value of subprocess.check_call. (?) > > Pavel You are right. :-) My point is that for the code to be correct we should catch the right exception: subprocess.CalledProcessError. This is valid even for python 2. Being practical your solution works so commit it. :-) -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Thu, Aug 11, 2022 at 05:55:47PM +0100, José Matos wrote: > On Thu, 2022-08-11 at 17:07 +0200, Pavel Sanda wrote: > > It seems that subprocess.check_call is raising exception instead of > > returning non-zero, > > so I gave it except block. > > What type of exception did you get? convert-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/421. convert-im6.q16: no images defined `mock.png' @ error/convert.c/ConvertImageCommand/3229. Traceback (most recent call last): File "/home/lyx/devel/lib/configure.py", line 2077, in checkConverterEntries() File "/home/lyx/devel/lib/configure.py", line 1202, in checkConverterEntries if subprocess.check_call([cmd, "mock.eps", "mock.png"])==0: File "/usr/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['convert', 'mock.eps', 'mock.png']' returned non-zero exit status 1. Systemcall.cpp (306): Systemcall: 'python3 -tt "/home/lyx/devel/lib/configure.py" --binary-dir="/home/lyx/devel/src/"' finished with exit code 1 > I did not use the exceptions to understand the failure reasons. > The issue with the except is that it is a catch all option and so the > exception could be caused by faulty assumptions from us in the initial > code. My understanding is that subprocess.check_call does not return nonzero to signal nonzero exit status of the called subroutine, but raises exception. So the correct thing is to catch exception, not checking return value of subprocess.check_call. (?) Pavel -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Thu, 2022-08-11 at 17:07 +0200, Pavel Sanda wrote: > It seems that subprocess.check_call is raising exception instead of > returning non-zero, > so I gave it except block. What type of exception did you get? I did not use the exceptions to understand the failure reasons. The issue with the except is that it is a catch all option and so the exception could be caused by faulty assumptions from us in the initial code. FWIW I admit that under certain scenarios that is the right option. In this case I am using a defensive style instead of being exceptional (pun intended):-D . Other than that the code looks correct. -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, Jul 31, 2022 at 11:41:02AM +0100, José Matos wrote:
> On Sun, 2022-07-31 at 11:16 +0200, Pavel Sanda wrote:
> > Sure, whatever you need :)
> > The patch is in.
> >
> > Pavel
>
> First attempt. :-)
Hi Jose,
attached is my take on it.
It seems that subprocess.check_call is raising exception instead of returning
non-zero,
so I gave it except block.
Does it all look good to your snakey eyes? :)
Pavel
diff --git a/lib/configure.py b/lib/configure.py
index 92cf561b80..0fe271f96d 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1087,13 +1087,15 @@ def checkConverterEntries():
# Only define a converter from pdf6 for graphics
checkProg('a PDF to EPS converter', ['pdftops -eps -f 1 -l 1 $$i $$o'],
rc_entry = [ r'\converter pdf6eps"%%" ""' ])
-# Define a converter from pdf6 to png for Macs where pdftops is missing.
+# sips:Define a converter from pdf6 to png for Macs where pdftops is
missing.
# The converter utility sips allows to force the dimensions of the
resulting
# png image. The value of 800 pixel for the width is arbitrary and not
# related to the current screen resolution or width.
# There is no converter parameter for this information.
+#
+#pdftoppm: Some systems ban IM eps->png conversion. We will offer
eps->pdf->png route instead.
checkProg('a PDF to PNG converter',
-['sips --resampleWidth 800 --setProperty format png $$i --out $$o'],
+['sips --resampleWidth 800 --setProperty format png $$i --out $$o' ,
'pdftoppm -r 72 -png -singlefile $$i > $$o'],
rc_entry = [ r'\converter pdf6png"%%" ""' ])
# Create one converter for a PDF produced using TeX fonts and one for a
# PDF produced using non-TeX fonts. This does not produce non-unique
@@ -1190,8 +1192,23 @@ def checkConverterEntries():
checkProg('an EPS -> PDF converter', ['epstopdf'],
rc_entry = [ r'\converter epspdf6 "epstopdf
--outfile=$$o $$i" ""'])
#
-checkProg('an EPS -> PNG converter', ['magick $$i[0] $$o', 'convert $$i[0]
$$o'],
-rc_entry = [ r'\converter epspng"%%" ""'])
+# Due to more restrictive policies, it is possible that (image)magick
+# does not allow conversions from eps to png.
+# So before setting the converter test it it on a mock file
+_, cmd = checkProg('an EPS -> PNG converter', ['magick', 'convert'])
+if cmd:
+writeToFile('mock.eps', r'%!PS')
+try:
+subprocess.check_call([cmd, "mock.eps", "mock.png"])
+removeFiles(['mock.eps', 'mock.png'])
+rc_entry = r'\converter epspng"%s $$i[0] $$o"
""'
+addToRC(rc_entry % cmd)
+except:
+removeFiles(['mock.eps'])
+#needs empty record otherwise default converter will be issued
+rc_entry = r'\converter epspng"" ""'
+addToRC(rc_entry)
+logger.info('ImageMagick seems to ban conversions from EPS.
Disabling direct EPS->PNG.')
#
# no agr -> pdf6 converter, since the pdf library used by gracebat is not
# free software and therefore not compiled in in many installations.
--
lyx-devel mailing list
lyx-devel@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
Am Mon, 01 Aug 2022 11:22:12 +0100 schrieb José Matos : > On Mon, 2022-08-01 at 09:42 +0200, Kornel Benko wrote: > > I see an unimportant difference only in configure.log. > > Actually this is important because it tells me that the new test was > applied. Honestly I have not added further debug that is why we are not > seeing it here, that can be done for now the purpose is to fix this > bug. > > Do you have the line > \converter epspng"convert $$i[0] $$o" "" > > in your lyxrc.defaults file? Yes. Original and new lyxrc.defaults. > Regards, Kornel pgpiaD_Q3IdWN.pgp Description: Digitale Signatur von OpenPGP -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Mon, 2022-08-01 at 09:42 +0200, Kornel Benko wrote: > I see an unimportant difference only in configure.log. Actually this is important because it tells me that the new test was applied. Honestly I have not added further debug that is why we are not seeing it here, that can be done for now the purpose is to fix this bug. Do you have the line \converter epspng"convert $$i[0] $$o" "" in your lyxrc.defaults file? Regards, -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
Am 01.08.2022 um 09:05 schrieb José Matos : > > On Sun, 2022-07-31 at 23:33 +0200, Stephan Witt wrote: >> With Python 3.8.13 on macOS there is no difference in configure.log >> before and after change c041925261. >> >> Stephan > > Thank you, this is good in the sense that what worked before still > works now. This is important for some reason. :-) > > For those who test this patch please also compare lyxrc.defaults that > is the file where the changes will be applied. Yes, of course. The lyxrc.defaults has no difference too on my system. The result with Python 2.7.16 is the same. I’m fine with these results. Stephan -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
Am Sun, 31 Jul 2022 11:41:02 +0100 schrieb José Matos : > On Sun, 2022-07-31 at 11:16 +0200, Pavel Sanda wrote: > > Sure, whatever you need :) > > The patch is in. > > > > Pavel > > First attempt. :-) > > The code is straightforward, on my system it works in both python2 and > python3. FWIW the only difference is: > > --- configure2.log 2022-07-31 11:32:45.733253492 +0100 > +++ configure.log 2022-07-31 11:33:20.607195627 +0100 > @@ -1,4 +1,4 @@ > -INFO: +Running LyX configure with Python 2.7.18 > +INFO: +Running LyX configure with Python 3.10.5 > INFO: checking for a Latex2e program... > DEBUG: (latex $$i,latex2e $$i) > INFO: +checking for "latex"... yes > > And of course that is the expected. > > FWIW the code should work on any system/OS... famous last words, I > know. :-D > > Please test it. I see an unimportant difference only in configure.log. --- configure.log 2022-08-01 09:28:14.766565470 +0200 (old) +++ ../configure.log2022-08-01 09:35:13.821649797 +0200 (new) @@ -1495,13 +1495,15 @@ INFO: checking for an EPS -> PNG converter... -DEBUG: (magick $$i[0] $$o,convert $$i[0] $$o) +DEBUG: (magick,convert) INFO: +checking for "magick"... not in path INFO: +checking for "convert"... yes DEBUG: Add to RC: \converter epspng"convert $$i[0] $$o" "" +DEBUG: Removing file mock.eps +DEBUG: Removing file mock.png INFO: checking for a Grace -> Image converter... DEBUG: (gracebat) INFO: +checking for "gracebat"... yes Kornel pgpCU22gRC16R.pgp Description: Digitale Signatur von OpenPGP -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 23:33 +0200, Stephan Witt wrote: > With Python 3.8.13 on macOS there is no difference in configure.log > before and after change c041925261. > > Stephan Thank you, this is good in the sense that what worked before still works now. This is important for some reason. :-) For those who test this patch please also compare lyxrc.defaults that is the file where the changes will be applied. -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
Am 31.07.2022 um 12:41 schrieb José Matos : > > On Sun, 2022-07-31 at 11:16 +0200, Pavel Sanda wrote: >> Sure, whatever you need :) >> The patch is in. >> >> Pavel > > First attempt. :-) > > The code is straightforward, on my system it works in both python2 and > python3. FWIW the only difference is: > > --- configure2.log 2022-07-31 11:32:45.733253492 +0100 > +++ configure.log 2022-07-31 11:33:20.607195627 +0100 > @@ -1,4 +1,4 @@ > -INFO: +Running LyX configure with Python 2.7.18 > +INFO: +Running LyX configure with Python 3.10.5 > INFO: checking for a Latex2e program... > DEBUG: (latex $$i,latex2e $$i) > INFO: +checking for "latex"... yes > > And of course that is the expected. > > FWIW the code should work on any system/OS... famous last words, I > know. :-D > > Please test it. With Python 3.8.13 on macOS there is no difference in configure.log before and after change c041925261. Stephan -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 20:30 +0200, Pavel Sanda wrote: > Will take little time, hopefully tomorrow. P No problem. :-) -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 20:09 +0200, Pavel Sanda wrote: > It's in. Pavel Thank you. :-) -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, Jul 31, 2022 at 11:41:02AM +0100, José Matos wrote: > Please test it. Will take little time, hopefully tomorrow. P -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, Jul 31, 2022 at 06:37:29PM +0100, José Matos wrote: > On Sun, 2022-07-31 at 11:16 +0200, Pavel Sanda wrote: > > On Sun, Jul 31, 2022 at 07:15:29AM +0100, José Matos wrote: > > > Now in the context of the pyupgrade fixes this is applied to top > > > /lib > > > scripts. I need this before working on configure.py. > > > > Sure, whatever you need :) > > The patch is in. > > > > Pavel > > Third iteration now regarding the python files in lib/scripts. Solves > the same issues identified before. It's in. Pavel -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 11:16 +0200, Pavel Sanda wrote:
> On Sun, Jul 31, 2022 at 07:15:29AM +0100, José Matos wrote:
> > Now in the context of the pyupgrade fixes this is applied to top
> > /lib
> > scripts. I need this before working on configure.py.
>
> Sure, whatever you need :)
> The patch is in.
>
> Pavel
Third iteration now regarding the python files in lib/scripts. Solves
the same issues identified before.
--
José Abílio
diff --git a/lib/scripts/fig2pdftex.py b/lib/scripts/fig2pdftex.py
index b458ccd8f3..10e9c5a89d 100644
--- a/lib/scripts/fig2pdftex.py
+++ b/lib/scripts/fig2pdftex.py
@@ -78,7 +78,7 @@ else:
# with tetex.
epsfile = outbase + '.pstex'
tmp = mkstemp()
-boundingboxline = re.compile(b'%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
+boundingboxline = re.compile(br'%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
for line in open(epsfile, 'rb'):
if line[:13] == b'%%BoundingBox':
(llx, lly, urx, ury) = list(map(int, boundingboxline.search(line).groups()))
diff --git a/lib/scripts/layout2layout.py b/lib/scripts/layout2layout.py
index 88ada1213e..c2fc7c20a7 100644
--- a/lib/scripts/layout2layout.py
+++ b/lib/scripts/layout2layout.py
@@ -457,10 +457,10 @@ def convert(lines, end_format):
re_ChapterStyle = re.compile(b'^\\s*Style\\s+Chapter\\s*$', re.IGNORECASE)
re_InsetLayout_CaptionLTNN = re.compile(b'^(\\s*InsetLayout\\s+)(Caption:LongTableNonumber)', re.IGNORECASE)
# for format 64
-re_trimLabelString = re.compile(b'^(\\s*LabelString\s+)"\\s*(.*?)\\s*"\\s*$')
-re_trimLabelStringAppendix = re.compile(b'^(\\s*LabelStringAppendix\s+)"\\s*(.*?)\\s*"\\s*$')
-re_trimEndLabelString = re.compile(b'^(\\s*EndLabelString\s+)"\\s*(.*?)\\s*"\\s*$')
-re_trimLabelCounter = re.compile(b'^(\\s*LabelCounter\s+)"\\s*(.*?)\\s*"\\s*$')
+re_trimLabelString = re.compile(b'^(\\s*LabelString\\s+)"\\s*(.*?)\\s*"\\s*$')
+re_trimLabelStringAppendix = re.compile(b'^(\\s*LabelStringAppendix\\s+)"\\s*(.*?)\\s*"\\s*$')
+re_trimEndLabelString = re.compile(b'^(\\s*EndLabelString\\s+)"\\s*(.*?)\\s*"\\s*$')
+re_trimLabelCounter = re.compile(b'^(\\s*LabelCounter\\s+)"\\s*(.*?)\\s*"\\s*$')
# counters for sectioning styles (hardcoded in 1.3)
diff --git a/lib/scripts/legacy_lyxpreview2ppm.py b/lib/scripts/legacy_lyxpreview2ppm.py
index 8effd5bd00..29eb833f54 100644
--- a/lib/scripts/legacy_lyxpreview2ppm.py
+++ b/lib/scripts/legacy_lyxpreview2ppm.py
@@ -156,7 +156,7 @@ def legacy_extract_metrics_info(log_file):
def extract_resolution(log_file, dpi):
fontsize_re = re.compile(b"Preview: Fontsize")
magnification_re = re.compile(b"Preview: Magnification")
-extract_decimal_re = re.compile(b"([0-9\.]+)")
+extract_decimal_re = re.compile(br"([0-9\.]+)")
extract_integer_re = re.compile(b"([0-9]+)")
found_fontsize = 0
diff --git a/lib/scripts/lyxpak.py b/lib/scripts/lyxpak.py
index 2aaf2db42b..affef65644 100755
--- a/lib/scripts/lyxpak.py
+++ b/lib/scripts/lyxpak.py
@@ -37,7 +37,7 @@ if running_on_windows:
from tempfile import NamedTemporaryFile
# Pre-compiled regular expressions.
-re_lyxfile = re.compile(b"\.lyx$")
+re_lyxfile = re.compile(br"\.lyx$")
re_input = re.compile(b'^(.*)(input|include){(\\s*)(.+)(\\s*)}.*$')
re_ertinput = re.compile(b'^(input|include)({)(\\s*)(.+)(\\s*)}.*$')
re_package = re.compile(b'^(.*)(usepackage){(\\s*)(.+)(\\s*)}.*$')
diff --git a/lib/scripts/lyxpaperview.py b/lib/scripts/lyxpaperview.py
index 09256da7cb..a0f2bedfe1 100755
--- a/lib/scripts/lyxpaperview.py
+++ b/lib/scripts/lyxpaperview.py
@@ -78,7 +78,7 @@ def find(args, path):
# use locate if possible (faster)
if find_exe(['locate']):
p1 = subprocess.Popen(['locate', '-i', args[0].lower()], stdout=subprocess.PIPE)
-px = subprocess.Popen(['grep', '-Ei', '\.pdf$|\.ps$'], stdin=p1.stdout, stdout=subprocess.PIPE)
+px = subprocess.Popen(['grep', '-Ei', r'\.pdf$|\.ps$'], stdin=p1.stdout, stdout=subprocess.PIPE)
for arg in args:
if arg == args[0]:
# have this already
diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index eee000deea..719738d002 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -120,7 +120,7 @@ def extract_metrics_info(dvipng_stdout):
# "\[[0-9]+" can match two kinds of numbers: page numbers from dvipng
# and glyph numbers from mktexpk. The glyph numbers always match
# "\[[0-9]+\]" while the page number never is followed by "\]". Thus:
-page_re = re.compile("\[([0-9]+)[^]]");
+page_re = re.compile(r"\[([0-9]+)[^]]");
metrics_re = re.compile("depth=(-?[0-9]+) height=(-?[0-9]+)")
success = 0
@@ -197,7 +197,7 @@ def fix_latex_file(latex_file, pdf_output):
def convert_to_ppm_format(pngtopnm, basename):
-png_file_re = re.compile("\.png$")
+png_file_re =
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 07:15 +0100, José Matos wrote:
> In order to keep some sanity Python does not allow to stack
> prefixes, like
>
> fur"\begin_layout {layout}"
Actually thinking a bit and researching a little more this is wrong:
https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
Looking in to the Python Language reference I see that it is possible
to use mix two:
stringprefix ::= "r" | "u" | "R" | "U" | "f" | "F"
| "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF"
The issue that confused me is that since we need to keep the
compatibility with python2 we need to use the u prefix.
The u prefix was supported in Python 3 just version 3.3. Previously it
was available in python2:
"New in version 3.3: Support for the unicode legacy literal (u'value')
was reintroduced to simplify the maintenance of dual Python 2.x and 3.x
codebases. See PEP 414 for more information."
So actually the code above could simply be used as
fr"\begin_layout {layout}"
this only works for Python 3.6+, when the formatted strings were
introduced.
Apologies for this series of Python musings. :-D
--
José Abílio
--
lyx-devel mailing list
lyx-devel@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 11:16 +0200, Pavel Sanda wrote:
> Sure, whatever you need :)
> The patch is in.
>
> Pavel
First attempt. :-)
The code is straightforward, on my system it works in both python2 and
python3. FWIW the only difference is:
--- configure2.log 2022-07-31 11:32:45.733253492 +0100
+++ configure.log 2022-07-31 11:33:20.607195627 +0100
@@ -1,4 +1,4 @@
-INFO: +Running LyX configure with Python 2.7.18
+INFO: +Running LyX configure with Python 3.10.5
INFO: checking for a Latex2e program...
DEBUG: (latex $$i,latex2e $$i)
INFO: +checking for "latex"... yes
And of course that is the expected.
FWIW the code should work on any system/OS... famous last words, I
know. :-D
Please test it.
--
José Abílio
diff --git a/lib/configure.py b/lib/configure.py
index ab205dd7b4..baf82929cb 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1174,8 +1174,18 @@ def checkConverterEntries():
checkProg('an EPS -> PDF converter', ['epstopdf'],
rc_entry = [ r'\converter epspdf6 "epstopdf --outfile=$$o $$i" ""'])
#
-checkProg('an EPS -> PNG converter', ['magick $$i[0] $$o', 'convert $$i[0] $$o'],
-rc_entry = [ r'\converter epspng"%%" ""'])
+# Due to more restrictive policies, it is possible that (image)magick
+# does not allow conversions from eps to png.
+# So before setting the converter test it it on a mock file
+_, cmd = checkProg('an EPS -> PNG converter', ['magick', 'convert'])
+if cmd:
+writeToFile('mock.eps', r'%!PS')
+if subprocess.check_call([cmd, "mock.eps", "mock.png"]) == 0:
+rc_entry = r'\converter epspng"%s $$i[0] $$o" ""'
+addToRC(rc_entry % cmd)
+removeFiles(['mock.eps', 'mock.png'])
+else:
+removeFiles(['mock.eps'])
#
# no agr -> pdf6 converter, since the pdf library used by gracebat is not
# free software and therefore not compiled in in many installations.
--
lyx-devel mailing list
lyx-devel@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, Jul 31, 2022 at 07:15:29AM +0100, José Matos wrote: > Now in the context of the pyupgrade fixes this is applied to top /lib > scripts. I need this before working on configure.py. Sure, whatever you need :) The patch is in. Pavel -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sun, 2022-07-31 at 00:44 +0200, Pavel Sanda wrote:
> Good. This one applies, so it's in now.
> Now that I have you on hotline, any chance to have a look at the
> configure patch we discussed in March? :)
> (Was: LyX 2.4.0 (IM policy.xml ban on eps/pdf conversions))
>
> Pavel
That was the next item in my list, really. :-)
Now in the context of the pyupgrade fixes this is applied to top /lib
scripts. I need this before working on configure.py.
The idea, of course, is to separate different changes in different
commits.
Again the changes are related with fixing escape sequences.
The two options are either add an r prefix indicating that the string
is raw, thus no escape is performed, or to double the backslash.
Most of the time I would use raw strings with the exception being the
addition of newline \n. The added complication here is that since we
support python 2 some strings need to be marked as unicode, with a u
prefix. In order to keep some sanity Python does not allow to stack
prefixes, like
fur"\begin_layout {layout}"
The f refers to formatted strings, and thus inside the string {layout}
is replaced with the value of the layout expression, u makes the string
unicode (the default in Python 3 and thus a no-op in Python3) and r
would mean raw string and thus the first backslash is not escaped.
--
José Abílio
diff --git a/lib/configure.py b/lib/configure.py
index 132141dc3a..ab205dd7b4 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -705,7 +705,7 @@ texteditors = ['xemacs', 'gvim', 'kedit', 'kwrite', 'kate',
'xed', 'notepad', 'WinEdt', 'WinShell', 'PSPad']
def checkFormatEntries(dtl_tools):
-''' Check all formats (\Format entries) '''
+r''' Check all formats (\Format entries) '''
checkViewerEditor('a Tgif viewer and editor', ['tgif'],
rc_entry = [r'\Format tgif "obj, tgo" Tgif "" "%%" "%%" "vector" "application/x-tgif"'])
#
@@ -876,7 +876,7 @@ def checkFormatEntries(dtl_tools):
def checkConverterEntries():
-''' Check all converters (\converter entries) '''
+r''' Check all converters (\converter entries) '''
checkProg('the pdflatex program', ['pdflatex $$i'],
rc_entry = [ r'\converter pdflatex pdf2 "%%" "latex=pdflatex,hyperref-driver=pdftex"' ])
@@ -1269,7 +1269,7 @@ def checkConverterEntries():
path, lilypond = checkProg('a LilyPond -> EPS/PDF/PNG converter', ['lilypond'])
if (lilypond):
version_string = cmdOutput("lilypond --version")
-match = re.match('GNU LilyPond (\S+)', version_string)
+match = re.match(r'GNU LilyPond (\S+)', version_string)
if match:
version_number = match.groups()[0]
version = version_number.split('.')
@@ -1299,7 +1299,7 @@ def checkConverterEntries():
continue
found_lilypond_book = True
-match = re.match('(\S+)$', version_string)
+match = re.match(r'(\S+)$', version_string)
if match:
version_number = match.groups()[0]
version = version_number.split('.')
@@ -1426,7 +1426,7 @@ def _checkForClassExtension(x):
return x.strip()
def processLayoutFile(file):
-""" process layout file and get a line of result
+r""" process layout file and get a line of result
Declare lines look like this:
@@ -1454,8 +1454,8 @@ def processLayoutFile(file):
"""
classname = file.split(os.sep)[-1].split('.')[0]
# return ('[a,b]', 'a', ',b,c', 'article') for \DeclareLaTeXClass[a,b,c]{article}
-p = re.compile('\s*#\s*DeclareLaTeXClass\s*(\[([^,]*)(,.*)*])*\s*{(.*)}\s*$')
-q = re.compile('\s*#\s*DeclareCategory{(.*)}\s*$')
+p = re.compile('\\s*#\\s*DeclareLaTeXClass\\s*(\\[([^,]*)(,.*)*])*\\s*{(.*)}\\s*$')
+q = re.compile('\\s*#\\s*DeclareCategory{(.*)}\\s*$')
classdeclaration = ""
categorydeclaration = '""'
for line in open(file, 'r', encoding='utf8').readlines():
@@ -1547,7 +1547,7 @@ def checkLatexConfig(check_config):
# Construct the list of classes to test for.
# build the list of available layout files and convert it to commands
# for chkconfig.ltx
-declare = re.compile('\\s*#\\s*DeclareLaTeXClass\\s*(\[([^,]*)(,.*)*\])*\\s*{(.*)}\\s*$')
+declare = re.compile('\\s*#\\s*DeclareLaTeXClass\\s*(\\[([^,]*)(,.*)*\\])*\\s*{(.*)}\\s*$')
category = re.compile('\\s*#\\s*DeclareCategory{(.*)}\\s*$')
empty = re.compile('\\s*$')
testclasses = list()
@@ -1563,7 +1563,7 @@ def checkLatexConfig(check_config):
for line in open(file, 'r', encoding='utf8').readlines():
if not empty.match(line) and line[0] != '#'[0]:
if decline == "":
-logger.warning("Failed to find valid \Declare line "
+logger.warning(r"Failed to find valid \Declare line "
"for layout file
Re: Interesting discovery from automatic tools (python code)
On Sat, Jul 30, 2022 at 05:49:05PM +0100, José Matos wrote: > In any case the code after this patch is more correct, and that is > always a good thing. Good. This one applies, so it's in now. Now that I have you on hotline, any chance to have a look at the configure patch we discussed in March? :) (Was: LyX 2.4.0 (IM policy.xml ban on eps/pdf conversions)) Pavel -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sat, 2022-07-30 at 14:22 +0200, Pavel Sanda wrote:
> On Sat, Jul 30, 2022 at 12:31:57PM +0100, José Matos wrote:
> > I think that this should be applied.
>
> I wanted to commit for you, but this is what I get when trying to
> apply the patch:
Oops, I took the patch in the wrong directory.
I took the chance to have another look, in order to catch other cases
where pyupgrade plays chicken or that are correct escape chars. In
particular \begin is a problem because \b is the bell :-) so IU checked
all those cases manually.
This work needs to be done for other directories.
In most the changes will be minimal for two reasons, either the escape
sequence does not exists and python ignores that sequence outputting
what we expect or the problem occurs in documentation strings.
In any case the code after this patch is more correct, and that is
always a good thing.
--
José Abílio
diff --git a/lib/lyx2lyx/lyx2lyx_tools.py b/lib/lyx2lyx/lyx2lyx_tools.py
index 9c4fe0bb0b..871087ce77 100644
--- a/lib/lyx2lyx/lyx2lyx_tools.py
+++ b/lib/lyx2lyx/lyx2lyx_tools.py
@@ -145,7 +145,7 @@ def insert_to_preamble(document, text, index = 0):
# A dictionary of Unicode->LICR mappings for use in a Unicode string's translate() method
# Created from the reversed list to keep the first of alternative definitions.
-licr_table = dict((ord(ch), cmd) for cmd, ch in unicode_reps[::-1])
+licr_table = {ord(ch): cmd for cmd, ch in unicode_reps[::-1]}
def put_cmd_in_ert(cmd, is_open=False, as_paragraph=False):
"""
diff --git a/lib/lyx2lyx/lyx_0_12.py b/lib/lyx2lyx/lyx_0_12.py
index b742dd5272..db3d882e58 100644
--- a/lib/lyx2lyx/lyx_0_12.py
+++ b/lib/lyx2lyx/lyx_0_12.py
@@ -112,7 +112,7 @@ def update_inset_label(document):
i = find_token(lines, '\\begin_inset Label', i)
if i == -1:
return
-lines[i] = '\\begin_inset LatexCommand \label{' + lines[i][19:] + '}'
+lines[i] = '\\begin_inset LatexCommand \\label{' + lines[i][19:] + '}'
i = i + 1
diff --git a/lib/lyx2lyx/lyx_1_2.py b/lib/lyx2lyx/lyx_1_2.py
index b697a918c5..ae082a73fc 100644
--- a/lib/lyx2lyx/lyx_1_2.py
+++ b/lib/lyx2lyx/lyx_1_2.py
@@ -69,7 +69,7 @@ def find_beginning_of_inset(lines, i):
def find_end_of_inset(lines, i):
-" Finds the matching \end_inset"
+r" Finds the matching \end_inset"
return find_end_of(lines, i, "\\begin_inset", "\\end_inset")
@@ -143,7 +143,7 @@ def get_width(mo):
def remove_oldfloat(document):
-" Change \begin_float .. \end_float into \begin_inset Float .. \end_inset"
+r" Change \begin_float .. \end_float into \begin_inset Float .. \end_inset"
lines = document.body
i = 0
while True:
@@ -250,7 +250,7 @@ def remove_pextra(document):
if flag:
flag = 0
if hfill:
-start = ["","\hfill",""]+start
+start = ["",r"\hfill",""]+start
else:
start = ['\\layout %s' % document.default_layout,''] + start
@@ -324,7 +324,7 @@ def remove_oldert(document):
new = []
new2 = []
if check_token(lines[i], "\\layout LaTeX"):
-new = ['\layout %s' % document.default_layout, "", ""]
+new = [r'\layout %s' % document.default_layout, "", ""]
k = i+1
while True:
@@ -808,7 +808,7 @@ def change_infoinset(document):
note_lines = [txt]+note_lines
for line in note_lines:
-new = new + ['\layout %s' % document.default_layout, ""]
+new = new + [r'\layout %s' % document.default_layout, ""]
tmp = line.split('\\')
new = new + [tmp[0]]
for x in tmp[1:]:
diff --git a/lib/lyx2lyx/lyx_1_3.py b/lib/lyx2lyx/lyx_1_3.py
index 35f04db8c4..a62b7b9951 100644
--- a/lib/lyx2lyx/lyx_1_3.py
+++ b/lib/lyx2lyx/lyx_1_3.py
@@ -27,7 +27,7 @@ from parser_tools import find_token, find_end_of, get_value,\
# Private helper functions
def find_end_of_inset(lines, i):
-"Finds the matching \end_inset"
+r"Finds the matching \end_inset"
return find_end_of(lines, i, "\\begin_inset", "\\end_inset")
diff --git a/lib/lyx2lyx/lyx_1_4.py b/lib/lyx2lyx/lyx_1_4.py
index 9c7c7a594e..19518ab882 100644
--- a/lib/lyx2lyx/lyx_1_4.py
+++ b/lib/lyx2lyx/lyx_1_4.py
@@ -81,7 +81,7 @@ def get_next_paragraph(lines, i, format):
def find_end_of_inset(lines, i):
-"Finds the matching \end_inset"
+r"Finds the matching \end_inset"
return find_end_of(lines, i, "\\begin_inset", "\\end_inset")
def del_token(lines, token, start, end):
@@ -103,7 +103,7 @@ def del_token(lines, token, start, end):
def remove_color_default(document):
-" Remove \color default"
+r" Remove \color default"
i = 0
while True:
i = find_token(document.body, "\\color default", i)
@@ -114,12 +114,12 @@ def remove_color_default(document):
def add_end_header(document):
-" Add
Re: Interesting discovery from automatic tools (python code)
On Sat, 2022-07-30 at 14:22 +0200, Pavel Sanda wrote: > I wanted to commit for you, but this is what I get when trying to > apply the patch: Thank you. I will look into this why is this failing. -- José Abílio -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Re: Interesting discovery from automatic tools (python code)
On Sat, Jul 30, 2022 at 12:31:57PM +0100, José Matos wrote: > I think that this should be applied. I wanted to commit for you, but this is what I get when trying to apply the patch: patching file lib/lyx2lyx/lyx2lyx_tools.py patching file lib/lyx2lyx/lyx_0_12.py patching file lib/lyx2lyx/lyx_1_2.py patching file lib/lyx2lyx/lyx_1_3.py patching file lib/lyx2lyx/lyx_1_4.py patching file lib/lyx2lyx/lyx_1_5.py patching file lib/lyx2lyx/lyx_1_6.py patching file lib/lyx2lyx/lyx_2_0.py patching file lib/lyx2lyx/lyx_2_1.py patching file lib/lyx2lyx/lyx_2_2.py patching file lib/lyx2lyx/lyx_2_3.py patching file lib/lyx2lyx/lyx_2_4.py patching file lib/lyx2lyx/parser_tools.py Hunk #3 succeeded at 479 with fuzz 1. patch unexpectedly ends in middle of line patch unexpectedly ends in middle of line Pavel -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
Interesting discovery from automatic tools (python code)
Yesterday I found a tool to upgrade the python code to new versions.
https://github.com/asottile/pyupgrade
This is interesting in itself. :-)
First because it allows to upgrade the code to a given version, such as
it is I have applied in a way that it is compatible with python 2.7
that we support for lyx 2.4.
I have applied it to lyx2lyx code and the result is attached.
Other than the change to lyx2lyx_tools.py that changes from the dict
call to a dictionary comprehension the other changes corresponds to
bugs in the present code:
For example:
"\label{" -> "\\label{"
You do not need to be a python expert to understand why the initial
code is wrong. The python strings use the same escape sequences as
C/C++.
In this case the code just works because "\l" is not an escape sequence
and is thus it is ignored.
I think that this should be applied.
Due to the issue with the supported version of keys in ssh I can not
commit directly. :-(
Have a nice weekend,
--
José Abílio
esdiff --git a/lib/lyx2lyx/lyx2lyx_tools.py b/lib/lyx2lyx/lyx2lyx_tools.py
index 9c4fe0bb0b..871087ce77 100644
--- a/lib/lyx2lyx/lyx2lyx_tools.py
+++ b/lib/lyx2lyx/lyx2lyx_tools.py
@@ -145,7 +145,7 @@ def insert_to_preamble(document, text, index = 0):
# A dictionary of Unicode->LICR mappings for use in a Unicode string's translate() method
# Created from the reversed list to keep the first of alternative definitions.
-licr_table = dict((ord(ch), cmd) for cmd, ch in unicode_reps[::-1])
+licr_table = {ord(ch): cmd for cmd, ch in unicode_reps[::-1]}
def put_cmd_in_ert(cmd, is_open=False, as_paragraph=False):
"""
diff --git a/lib/lyx2lyx/lyx_0_12.py b/lib/lyx2lyx/lyx_0_12.py
index b742dd5272..db3d882e58 100644
--- a/lib/lyx2lyx/lyx_0_12.py
+++ b/lib/lyx2lyx/lyx_0_12.py
@@ -112,7 +112,7 @@ def update_inset_label(document):
i = find_token(lines, '\\begin_inset Label', i)
if i == -1:
return
-lines[i] = '\\begin_inset LatexCommand \label{' + lines[i][19:] + '}'
+lines[i] = '\\begin_inset LatexCommand \\label{' + lines[i][19:] + '}'
i = i + 1
diff --git a/lib/lyx2lyx/lyx_1_2.py b/lib/lyx2lyx/lyx_1_2.py
index b697a918c5..96db7737ae 100644
--- a/lib/lyx2lyx/lyx_1_2.py
+++ b/lib/lyx2lyx/lyx_1_2.py
@@ -69,7 +69,7 @@ def find_beginning_of_inset(lines, i):
def find_end_of_inset(lines, i):
-" Finds the matching \end_inset"
+r" Finds the matching \end_inset"
return find_end_of(lines, i, "\\begin_inset", "\\end_inset")
@@ -143,7 +143,7 @@ def get_width(mo):
def remove_oldfloat(document):
-" Change \begin_float .. \end_float into \begin_inset Float .. \end_inset"
+" Change \begin_float .. \\end_float into \begin_inset Float .. \\end_inset"
lines = document.body
i = 0
while True:
@@ -250,7 +250,7 @@ def remove_pextra(document):
if flag:
flag = 0
if hfill:
-start = ["","\hfill",""]+start
+start = ["",r"\hfill",""]+start
else:
start = ['\\layout %s' % document.default_layout,''] + start
@@ -324,7 +324,7 @@ def remove_oldert(document):
new = []
new2 = []
if check_token(lines[i], "\\layout LaTeX"):
-new = ['\layout %s' % document.default_layout, "", ""]
+new = [r'\layout %s' % document.default_layout, "", ""]
k = i+1
while True:
@@ -808,7 +808,7 @@ def change_infoinset(document):
note_lines = [txt]+note_lines
for line in note_lines:
-new = new + ['\layout %s' % document.default_layout, ""]
+new = new + [r'\layout %s' % document.default_layout, ""]
tmp = line.split('\\')
new = new + [tmp[0]]
for x in tmp[1:]:
diff --git a/lib/lyx2lyx/lyx_1_3.py b/lib/lyx2lyx/lyx_1_3.py
index 35f04db8c4..a62b7b9951 100644
--- a/lib/lyx2lyx/lyx_1_3.py
+++ b/lib/lyx2lyx/lyx_1_3.py
@@ -27,7 +27,7 @@ from parser_tools import find_token, find_end_of, get_value,\
# Private helper functions
def find_end_of_inset(lines, i):
-"Finds the matching \end_inset"
+r"Finds the matching \end_inset"
return find_end_of(lines, i, "\\begin_inset", "\\end_inset")
diff --git a/lib/lyx2lyx/lyx_1_4.py b/lib/lyx2lyx/lyx_1_4.py
index 9c7c7a594e..37ab3722f2 100644
--- a/lib/lyx2lyx/lyx_1_4.py
+++ b/lib/lyx2lyx/lyx_1_4.py
@@ -81,7 +81,7 @@ def get_next_paragraph(lines, i, format):
def find_end_of_inset(lines, i):
-"Finds the matching \end_inset"
+r"Finds the matching \end_inset"
return find_end_of(lines, i, "\\begin_inset", "\\end_inset")
def del_token(lines, token, start, end):
@@ -103,7 +103,7 @@ def del_token(lines, token, start, end):
def remove_color_default(document):
-" Remove \color default"
+r" Remove \color default"
i = 0
while True:
i = find_token(document.body, "\\color default", i)
@@ -114,12 +114,12 @@ def
