Re: [LyX features/properpaint] Fix caret painting

[Richard Heck]

On 07/31/2017 02:02 PM, Jean-Marc Lasgouttes wrote:
> I do not think so. Maybe after 2.3.0, it will be up to Richard, since the 
> amount of code is small.

I guess we will need to see, as usual, if there's a need for fast
release of 2.3.1. But, if not, then we can think about merging early so
it gets lots of testing.

Richard

Re: Options for resolving the minted + shell-escape issue

[Pavel Sanda]

Pavel Sanda wrote:
> I did not hear your reaction to it either.

I see you just did that, sorry... P

Regarding safety/security, the DOCX and DOCM formats of MS Office

[Christian Ridderström]

Hi,

Regarding safe behaviour or not, I happened to remember an aspect related
to MS Word and macros.

Some years ago Microsoft changed the old DOC format and as default
introduced the DOCX format. One change was of course the use of XML, but
another is that DOCX does _not_ allow macros --- at all.

Besides the confusion, it does have the advantage that typical users aren't
exposed to macros.

A user that needs [*] macros will have to save in a different format, DOCM,
and where macros are available for use.

It's been a while so I might be misremembering or confusing with other
security measured, but IIRC there were also warnings popping up when trying
to open a .docm-file from an "untrusted" source, like when a document was
received over e-mail, downloaded or opened from a network drive.
This is more a UI thing though... IIRC, e.g. Internet Explorer adds some
flag when a file is downloaded and this is what marks it as being from an
untrusted source.

Anyway, I'm not really advocating a new file extension like '.lyxm', but
simply trying to illustrate that MS Office tries hard to keep the defaults
safe, to the extent of not permitting macros at all.

/Christian

[*]
I've actually had to use DOCM and macros in a Word. In my case the macros
were used to filter out certain kinds of information from the word document
to a text file that was then imported into another system.

PS.
As an aside, I once had a very confusing experience with an Excel file from
a customer with important data where merely the act of opening the Excel
file changed the file on disk. I repeat: This happened without me saving
the file.

What happened was that the Excel file contained some OLE object which
executed when opening the file and this resulted in the file on disk
changing.  I noticed this because for safety we had separately been sent a
formal document which stated the expected hash of the excel file with data,
which verified manually when receiving the file. And then, being paranoid,
I added an automated test that verified the hash before importing the data
from the Excel file, only to find that the test failed due to the hash no
longer matched! *sigh*

Re: [LyX features/properpaint] Fix caret painting

[Pavel Sanda]

Jean-Marc Lasgouttes wrote:
> I do not think so. Maybe after 2.3.0

:(

> But I suspect that there are dark corners that we have not visited yet.

What is your strategy with features/properpaint? To continue piling up new 
patches on top of if or leave it for testing?

Pavel

Re: Options for resolving the minted + shell-escape issue

[Pavel Sanda]

Christian Ridderström wrote:
> Please note that I'm _not_ wholly against something like needauth, I'm simply
> not convinced it's good enough.  In fact, I'm still unclear on exactly how it
> currently works, or perhaps rather, how it's intended to work in LyX 2.3.

I already wrote you possible ways how we can cement the current situation more
and IIRC you did not picked any. Meanwhile Tommaso committed patch which is
slightly improving situation by "scary dialog". I did not hear your reaction to
it either.

> > Even after all discussion I still see adding the whole needauth machinery
> > as unnecessary complication of code and UI; possible future use of pygments
> > still seems as made up argument for the sake of discussion rather than real
> > user demand.
> 
> Would you mind clarifying why needauth is an "unneccessary complication of
> code and UI"?  (Apologies as I'm likely asking you to repeat what you've said
> previously).

I meant extending needauth mechanisms beyond current gnuplot/knitr usage
for the sake of minted, because I believe minted maintainer will deliver
us fix within couple months.

With such sight in view, I would be just fine to let minted support in
current fashion, tell people clearly in manual that adding --shell-escape
is dangerous, let Uwe scream at us little bit and then be done with it.

Pavel

Re: [LyX features/properpaint] Fix caret painting

[Jean-Marc Lasgouttes]

I do not think so. Maybe after 2.3.0, it will be up to Richard, since the 
amount of code is small.

Thanks for testing. I am happy that it works. But I suspect that there are dark 
corners that we have not visited yet.

JMarc

Le 31 juillet 2017 19:45:09 GMT+02:00, Pavel Sanda  a écrit :
>
>Whoaa! I tried and I am back on almost 2.0 speed!
>Any chance this branch gets into 2.3?

Re: Options for resolving the minted + shell-escape issue

[Christian Ridderström]

Hi Scott,

On 31 July 2017 at 16:50, Pavel Sanda  wrote:

> Scott Kostyshak wrote:
> > I'm concerned that since this issue has left us all exhausted, there is
> > a feeling of "let's just get this over so we can move on". I encourage
> > all of us to give one more cognitive spurt and give a vote.
> >
> > From what I understand, the three options are still what I proposed
> > three weeks ago [1]:
> >
> > 1. Revert the recently added minted support.
> >
> > 2. Keep the current state of master.
> >
> > 3. Apply the patch at [2]. Don't forget to copy emblem-shellescape.svgz
> > to lib/images. (Note that I get linker errors when I try to apply the
> > latest patch, but it might be something specific to my setup.)
> >
> > So I ask explicitly to everyone (even if you think you have already
> > voted, please give your vote again):
>

I'm just letting you know I haven't had time to look through all the recent
posts yet.

Further, and more importantly for me, the above is only for minted.
To me it doesn't make sense to not consider 'needauth' in the same context
of safety/security.

So I'm not sure I can vote as I'm not sure the alternatives make sense.

The following is likely not going to be popular, but ...

If we don't have mechanisms that keep "regular" users safe, I don't think
it matters
very much what we do with minted.

So I think a very important question is if we think needauth is
sufficiently good,
or can be made sufficiently good before release.

I do think there should, at least formally, be some kind of alternative(s)
corresponding to e.g.
removing/disabling needauth _and_ removing/disabling the automatic/silent
running of R-scripts/gnuplot,
and that it should take some "effort"/"be scary" to enable them.

IMHO it's for LyX 2.3 not ok to do a release that's unsafe just because
it's "less" unsafe
than before, i.e. the release has to be sufficiently good, not just less
bad than before.

Please note that I'm _not_ wholly against something like needauth, I'm
simply not convinced it's good enough.
In fact, I'm still unclear on exactly how it currently works, or perhaps
rather, how it's intended to work in LyX 2.3.


> Little difficult, because what I opined was not included in your list.
> To sum up I favor support of minted, which would use secure calling
> minted once it's compilation is split into separated steps as proposed
> on minted bugzilla.
> So I do not have strong opinion whether we shoould go 1 or 2 if we
> fix the issues once minted is fixed.
>
> Even after all discussion I still see adding the whole needauth machinery
> as
> unnecessary complication of code and UI; possible future use of pygments
> still seems as made up argument for the sake of discussion rather than real
> user demand.
>

Would you mind clarifying why needauth is an "unneccessary complication of
code and UI"?
(Apologies as I'm likely asking you to repeat what you've said previously).

Best regards,
/Christian


>
> So the breakdown is likely 0.5 voting points for 1 & 2.
>
> Pavel
>

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Christian Ridderström]

On 31 July 2017 at 20:44, Guillaume MM  wrote:

> Le 31/07/2017 à 13:31, Jürgen Spitzmüller a écrit :
>
>> I meant it in this sense. If a vote only means "I did not have a
>> look at
>>
>> the patch but I am fed-up so let us go ahead" then it is not taking
>> responsibilities.
>>
>>
>> A vote is a vote. If the given voting will be Rates differently, this
>> will be have been the last voting I have participated on this list.
>>
>
I guess he wrote that from a iPhone, which I say because I (unwillingly)
have an iPhone and my girlfriend has remarked on the reduced quality of my
language when I use the phone.


> I am not sure I got your last sentence right :) but nobody proposed to
> discard any vote.
>

Do we have an overview somewhere (with patch reference) for the
alternatives proposed for beta1, which is then what's likely to end up in
2.3?
Note: I did just look at the wiki page but didn't see it there clearly.
/Christian

Re: [LyX/master] prefs/needauth: added warning if user tries to disable authorization for needauth converters.

[Christian Ridderström]

On 27 July 2017 at 00:06, Tommaso Cucinotta  wrote:

> commit 8a4fcd3d95ca4aeed1c46152cecadf29ed21e774
> +   _("SECURITY WARNING!"), _("Unchecking this option has the
> effect that potentially harmful converters would be run without asking your
> permission first. This is UNSAFE and NOT recommended, unless you know what
> you are doing. Are you sure you would like to proceed ? The recommended and
> safe answer is NO!"),
>

The warning as per the patch:

Unchecking this option has the effect that potentially harmful converters
would be run without asking your permission first. This is UNSAFE and NOT
recommended, unless you know what you are doing. Are you sure you would
like to proceed ? The recommended and safe answer is NO!

There's an extra space before the question mark (?). However, I would
suggest removing "unless you know what you are doing" from the message to
make it stronger, i.e.


Unchecking this option has the effect that potentially harmful converters
would be run without asking your permission first. This is UNSAFE and NOT
recommended. Are you sure you would like to proceed? The recommended and
safe answer is NO!


And actually, I wouldn't mind making the warning even stronger:


Unchecking this option is UNSAFE and NOT RECOMMENDED. With this option
unchecked LyX will NOT EVEN ASK you for permission before executing
converters that might be potentially harmful to your computer system.
Please read  for more information.
Are you sure you would like to proceed? The recommended and safe answer is
NO!


The above would of course mean that we also need to add a section to the
user guide or something.
Hmm... could we suggest asking the users' list for advice?

/Christian

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Mon, Jul 31, 2017 at 09:07:11PM +0200, Guillaume MM wrote:
> 
> I am sure that Scott meant to include in some way the option that I have
> been advocating constantly from the beginning, which I understand is
> probably 1. (Otherwise, I do not see what the option 1. refers to nor
> who proposed it, and I would opt for not taking part in the vote.) So to
> be clear, my vote is 1., and the patch gives it substance as part of a
> larger proposition providing the most secure option for beta and for
> release so far (because after minted there remains to fix needauth). It
> just incidentally happens that it does not lock one into removing minted
> by providing a better basis for what people voting 3. are trying to
> achieve. (In particular Scott, even if the route of 3. is voted for
> beta, this patch minus disabling of minted remains an option for 2.3.)

Sorry, but 1. means what it says and not what meaning you want to attach
to it. So, you vote for 1.

> As for your counting of votes, calling the end of the poll, and deciding
> what fits or does not fit proposed options, last time I have checked you
> were not release manager :)

You don't even deserve an answer here.

-- 
Enrico

Dropping unsupported languages

[Pavel Sanda]

Per stats on https://www.lyx.org/I18n-trunk it might be proper time to drop
support for languages having less %50 than translation status.

Those would be the winners: Russian, Danish, Greek, Serbian, 
Galician, Catalan, Romanian, Dutch.

Any resistance?
Pavel
diff --git a/po/LINGUAS b/po/LINGUAS
index a98d374319..6e2aa3bb9f 100644
--- a/po/LINGUAS
+++ b/po/LINGUAS
@@ -1,3 +1,3 @@
 # The list of languages known to LyX
 #ar bg ca cs da de el en es eu fi fr gl he hu ia id it ja ko nb nl nn pl pt_BR 
pt_PT ro ru sk sl sr sv tr uk wa zh_CN zh_TW
-ar ca cs da de el en es eu fi fr gl he hu ia id it ja nb nl nn pl pt_BR pt_PT 
ro ru sk sr sv tr uk zh_CN zh_TW
+ar cs de en es eu fi fr he hu ia id it ja nb nn pl pt_BR pt_PT sk sv tr uk 
zh_CN zh_TW

Re: [patch] update PDF viewers in configure.py

[Christian Ridderström]

On 31 July 2017 at 19:52, Pavel Sanda  wrote:

> Kornel Benko wrote:
> > > It is not a demo. Only if you want to have all features like e.g.
> > > splitting PDF pages one has to buy another version. So it is the same
> as
> > > with Acrobat Reader, for the full feature set of Acrobat one has to
> pay.
>

I skimmed their manual and read the license section which says:

3. NON-COMMERCIAL USE.
You are hereby granted to use this Software for non-commercial purposes
without charge for
unlimited time on Desktop Linux.
Software can also be used for viewing and printing documents on Windows and
Mac OS X for
unlimited time without any charge.

If this is accurate, it might be useful for e.g. academic users of LyX.

Regarding licenses... the program is based on Qt, and they ship several Qt
libraries -- I just verified this after installing it on my mac.
However, the program didn't seem to acknowledge Qt or reference LGPL or
GPL, and they e.g. explicitly forbid reverse engineering so unless they've
got a commercial Qt license they're likely violating Qt's LGPL. :-(

Now, they might have acquired commercial Qt licenses, one per developer, in
which case everything could be fine.
However, as the Qt licenses aren't cheap, e.g. perhaps $1400/month for
three developers, I see a risk that they don't have a commercial license.

If we care about this, we could simply ask the company behind the program
about their licensing situation.


> > Do we really try to promote commercial programs?
>
> There is free version, with slightly less features, but otherwise if we
> support acrobat reader which is not open source either why we could
> not support another closed source but free app?
>

Market dominance is probably the reason to support Acrobat.
But it's a good point.

/Christian

Preferences handling of \input token

[Kornel Benko]

This looks weird.
As far as I could see, reading preferences is done via
LyXRC::read(FileName const & filename, bool check_format)
which return true if everything is OK.
But if using '\input "some file"', we land at src/LyXRC.cpp:464

Changing as attached, cures it for me.

Kornel

signature.asc
Description: This is a digitally signed message part.
diff --git a/src/LyXRC.cpp b/src/LyXRC.cpp
index 27d5fcf..3a8328b 100644
--- a/src/LyXRC.cpp
+++ b/src/LyXRC.cpp
@@ -460,7 +460,7 @@ LyXRC::ReturnValues LyXRC::read(Lexer & lexrc, bool check_format)
 FileName const tmp =
 	libFileSearch(string(),
 		  lexrc.getString());
-if (read(tmp, check_format)) {
+if (!read(tmp, check_format)) {
 	lexrc.printError(
 	"Error reading included file: " + tmp.absFileName());
 }

Re: Options for resolving the minted + shell-escape issue

[Guillaume MM]

Le 31/07/2017 à 13:00, Enrico Forestieri a écrit :

On Mon, Jul 31, 2017 at 12:01:39PM +0200, Enrico Forestieri wrote:


On Mon, Jul 31, 2017 at 10:57:01AM +0200, Guillaume MM wrote:


A key point to me is that, according to your other message, going with
3. locks 2.3 into implementing Pygments using minted. For this reason
and that it does not solve fundamental issues, there are more reasonable
solutions than the patch at [2] even understood as an intermediary
solution to get beta out.


I now count 4 votes for the patch at [2] and only one against it
(if I correctly interpret this post as such). I think this means that a
decision can be taken without entering in other exhausting and unfruitful
discussions that most probably would go on forever.


Just to be clear. It shouldn't and cannot work this way. There were a vote
called on some options. One cannot come up with another option when he sees
that this is not what he would like. Otherwise, another one is entitled
to come up with yet another option and so on. We have to make a decision
(which is now possible, I think) and then move from there.



I am sure that Scott meant to include in some way the option that I have
been advocating constantly from the beginning, which I understand is
probably 1. (Otherwise, I do not see what the option 1. refers to nor
who proposed it, and I would opt for not taking part in the vote.) So to
be clear, my vote is 1., and the patch gives it substance as part of a
larger proposition providing the most secure option for beta and for
release so far (because after minted there remains to fix needauth). It
just incidentally happens that it does not lock one into removing minted
by providing a better basis for what people voting 3. are trying to
achieve. (In particular Scott, even if the route of 3. is voted for
beta, this patch minus disabling of minted remains an option for 2.3.)

As for your counting of votes, calling the end of the poll, and deciding
what fits or does not fit proposed options, last time I have checked you
were not release manager :)

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Guillaume MM]

Le 31/07/2017 à 13:31, Jürgen Spitzmüller a écrit :

I meant it in this sense. If a vote only means "I did not have a
look at

the patch but I am fed-up so let us go ahead" then it is not taking
responsibilities.


A vote is a vote. If the given voting will be Rates differently, this 
will be have been the last voting I have participated on this list.




I am not sure I got your last sentence right :) but nobody proposed to
discard any vote.

Re: [patch] update PDF viewers in configure.py

[Pavel Sanda]

Uwe Stöhr wrote:
>> There's one certainty, regardless of the OS: no matter which order you 
>> specify, /someone/ is going to complain. ;-)
>
> Yes. So to avoid a discussion about it I have undone the sort order change 
> in the attached updated patch.

Actually you did not. 'masterpdfeditor4' is now preferred before AcroRd32.
You really mean that? This is editor, not viewer after all.

I don't have strong opinion about having masterpdfeditor in configure.py
but as was said before, the fact that they include version name in the
executable probably means usability of this check will have very limited
time use and we can't have it right in the long term.

The first part of the patch is fine.
Pavel

Re: [patch] update PDF viewers in configure.py

[Pavel Sanda]

Kornel Benko wrote:
> > It is not a demo. Only if you want to have all features like e.g. 
> > splitting PDF pages one has to buy another version. So it is the same as 
> > with Acrobat Reader, for the full feature set of Acrobat one has to pay.
> 
> Do we really try to promote commercial programs?

There is free version, with slightly less features, but otherwise if we
support acrobat reader which is not open source either why we could
not support another closed source but free app?

Pavel

PS: Out of curiosity I tried it now and was fairly surprised, how _good_
this editor is. It's not the first attempt for pdf editor on Linux, but
it's a first one which seems to work just perfectly under my 10 min testing.

Re: [patch] update PDF viewers in configure.py

[Christian Ridderström]

On 30 July 2017 at 23:47, Jean-Marc Lasgouttes  wrote:

> >Under Linux I was looking for a PDF program with which I can properly
> >fill out and submit PDF forms. I found the Program Master PDF editor
> >and
> >would therefore like to support it in LyX.
>
> To be frank I never heard of it before today. It seems to be some
> commercial program which only can be found as demo. I am not very fond of
> promoting such things.
>

+1

The reason for my reluctance is probably:
- Any risk of licensing-related issues, i.e. is Master PDF honouring SW
licenses correctly.
  E.g. I'd feel bad supporting and being associated with a SW that's
violating GPL.
- Does the program serve commercial ads and could there be anything "risky"
for the user related to that.
- Might the program be uploading "telemetry" now or in the future?
- ?

The last two aspects relates to the company's business model now, as well
as in the future.

Note that I don't know anything about Master PDF, they might be above board
(today) when it comes to all of these aspects.
Perhaps we should simply ask them (Master PDF) these questions?

Best regards,
/Christian

Re: [LyX features/properpaint] Fix caret painting

[Pavel Sanda]

Jean-Marc Lasgouttes wrote:
> Le 25/07/2017 ?? 11:37, Pavel Sanda a écrit :
>>> Preliminary testing shows that the performance of pressing cursor down
>>> repeatedly is the same, but the performance of Page Down is much better.
>> Is this supposed to change only perf of going down or it also affects just
>> going through the text (left/right arrow) without painting new lines?
>
> I don't know, try it :) The slowness you are experiencing is a bit 
> mysterious to me.

Whoaa! I tried and I am back on almost 2.0 speed!
Any chance this branch gets into 2.3?

Pavel

Re: lyx executable built with cmake will not run

[Jean-Pierre Chrétien]

Le 31/07/2017 à 18:09, Kornel Benko a écrit :


I don't understand, I can load bin/lyx2.3 all right now.



So everything works again?


Seems to work, I've recompiled from scratch after editing the options,
I'll keep you posted.

--
Jean-Pierre

Re: lyx executable built with cmake will not run

[Kornel Benko]

Am Montag, 31. Juli 2017 um 17:55:01, schrieb Jean-Pierre Chrétien 

> Le 31/07/2017 à 17:28, Kornel Benko a écrit :
> > Am Montag, 31. Juli 2017 um 15:08:36, schrieb Jean-Pierre Chrétien 
> > 
> >> Hello Kornel
> >>
> >> After:
> >>
> >> $ cd ../buildmastercmake/
> >> $ rm -rf *
> >> $  cmake -DLYX_ENABLE_EXPORT_TESTS=ON -DLYX_RELEASE=ONb 
> >> -DLYX_PROGRAM_SUFFIX=ON
> >> -DLYX_ENABLE_CXX11=AUTO -DLYX_USE_QT=QT5 -DLYX_INSTALL_PREFIX="git-" 
> >> ../master
> >
> > You mean -DLYX_RELEASE=ON
> 
> Oups, an extra 'b' slipped in
> 
> >
> > I use also -DLYX_LOCALVERSIONING=ON
> > but no -DLYX_INSTALL_PREFIX=
> > Why do you use it?
> 
> Copied from a example on the mailing list... I'll remove it.
> What does LOCALVERSIONING do?

Checks for git-source and creates appropriate named packages (if used with 
'make package')
For instance the last packages I have here are
-rw-r--r-- 1 kornel kornel 28137308 júl 29 11:42 
lyx23-2.3.0-49797git-Linux.deb
-rw-r--r-- 1 kornel kornel 28142580 júl 30 10:21 
lyx23-2.3.0-49799git-Linux.deb
-rw-r--r-- 1 kornel kornel 28151812 júl 31 01:48 
lyx23-2.3.0-49812git-Linux.deb
-rw-r--r-- 1 kornel kornel 28142592 júl 31 13:12 
lyx23-2.3.0-49813git-Linux.deb

> >
> 
> > I simply don't see such behaviour.
> > Please try
> > # strace -o x -f bin/lyx2.3
> > and check for string 'open' in file 'x'.
> 
> I don't understand, I can load bin/lyx2.3 all right now.
> 

So everything works again?

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: lyx executable built with cmake will not run

[Jean-Pierre Chrétien]

Le 31/07/2017 à 17:28, Kornel Benko a écrit :

Am Montag, 31. Juli 2017 um 15:08:36, schrieb Jean-Pierre Chrétien 


Hello Kornel

After:

$ cd ../buildmastercmake/
$ rm -rf *
$  cmake -DLYX_ENABLE_EXPORT_TESTS=ON -DLYX_RELEASE=ONb -DLYX_PROGRAM_SUFFIX=ON
-DLYX_ENABLE_CXX11=AUTO -DLYX_USE_QT=QT5 -DLYX_INSTALL_PREFIX="git-" ../master


You mean -DLYX_RELEASE=ON


Oups, an extra 'b' slipped in



I use also -DLYX_LOCALVERSIONING=ON
but no -DLYX_INSTALL_PREFIX=
Why do you use it?


Copied from a example on the mailing list... I'll remove it.
What does LOCALVERSIONING do?






I simply don't see such behaviour.
Please try
# strace -o x -f bin/lyx2.3
and check for string 'open' in file 'x'.


I don't understand, I can load bin/lyx2.3 all right now.

--
Thanks for your help
Jean-Pierre

Re: lyx executable built with cmake will not run

[Kornel Benko]

Am Montag, 31. Juli 2017 um 15:08:36, schrieb Jean-Pierre Chrétien 

> Hello Kornel
> 
> After:
> 
> $ cd ../buildmastercmake/
> $ rm -rf *
> $  cmake -DLYX_ENABLE_EXPORT_TESTS=ON -DLYX_RELEASE=ONb 
> -DLYX_PROGRAM_SUFFIX=ON 
> -DLYX_ENABLE_CXX11=AUTO -DLYX_USE_QT=QT5 -DLYX_INSTALL_PREFIX="git-" ../master

You mean -DLYX_RELEASE=ON

I use also -DLYX_LOCALVERSIONING=ON
but no -DLYX_INSTALL_PREFIX=
Why do you use it?

> $ make
> $ bin/lyx2.3
> 
> I get this:
> 
> 
> support/Messages.cpp (247): No language given, nothing to load.
> Error: No system directory
> 
> Unable to determine the system directory having searched
>   /ext/lyx/buildmastercmake/share/lyx2.3/
> Use the '-sysdir' command line parameter or set the environment variable
> LYX_DIR_23x to the LyX system directory containing the file `chkconfig.ltx'.
> 
> 
> What have I done wrong?
> Is the cmake build documented spmewhere? I may have forgotten a step.

I simply don't see such behaviour.
Please try
# strace -o x -f bin/lyx2.3
and check for string 'open' in file 'x'.

I see there many entries like
4017  open("/usr2/src/lyx/lyx-git/lib/images/oxygen/note-insert.svgz", 
O_RDONLY|O_CLOEXEC) = 19
4017  open("/usr2/src/lyx/lyx-git/lib/images/oxygen/box-insert.svgz", 
O_RDONLY|O_CLOEXEC) = 19
4017  open("/usr2/src/lyx/lyx-git/lib/images/oxygen/href-insert.svgz", 
O_RDONLY|O_CLOEXEC) = 19

Independent of it's installed or not.

> Done after git pull.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Options for resolving the minted + shell-escape issue

[Pavel Sanda]

Scott Kostyshak wrote:
> I'm concerned that since this issue has left us all exhausted, there is
> a feeling of "let's just get this over so we can move on". I encourage
> all of us to give one more cognitive spurt and give a vote.
> 
> From what I understand, the three options are still what I proposed
> three weeks ago [1]:
>
> 1. Revert the recently added minted support.
> 
> 2. Keep the current state of master.
>
> 3. Apply the patch at [2]. Don't forget to copy emblem-shellescape.svgz
> to lib/images. (Note that I get linker errors when I try to apply the
> latest patch, but it might be something specific to my setup.)
>
> So I ask explicitly to everyone (even if you think you have already
> voted, please give your vote again):

Little difficult, because what I opined was not included in your list.
To sum up I favor support of minted, which would use secure calling
minted once it's compilation is split into separated steps as proposed
on minted bugzilla.
So I do not have strong opinion whether we shoould go 1 or 2 if we
fix the issues once minted is fixed.

Even after all discussion I still see adding the whole needauth machinery as
unnecessary complication of code and UI; possible future use of pygments
still seems as made up argument for the sake of discussion rather than real
user demand.

So the breakdown is likely 0.5 voting points for 1 & 2.

Pavel

lyx executable built with cmake will not run

[Jean-Pierre Chrétien]

Hello Kornel

After:

$ cd ../buildmastercmake/
$ rm -rf *
$  cmake -DLYX_ENABLE_EXPORT_TESTS=ON -DLYX_RELEASE=ONb -DLYX_PROGRAM_SUFFIX=ON 
-DLYX_ENABLE_CXX11=AUTO -DLYX_USE_QT=QT5 -DLYX_INSTALL_PREFIX="git-" ../master

$ make
$ bin/lyx2.3

I get this:


support/Messages.cpp (247): No language given, nothing to load.
Error: No system directory

Unable to determine the system directory having searched
/ext/lyx/buildmastercmake/share/lyx2.3/
Use the '-sysdir' command line parameter or set the environment variable
LYX_DIR_23x to the LyX system directory containing the file `chkconfig.ltx'.


What have I done wrong?
Is the cmake build documented spmewhere? I may have forgotten a step.

Done after git pull.

--
Jean-Pierre

Re: Options for resolving the minted + shell-escape issue

[Kornel Benko]

Am Montag, 31. Juli 2017 um 12:51:15, schrieb Enrico Forestieri 
> On Sun, Jul 30, 2017 at 02:27:02PM +0200, Kornel Benko wrote:
> > 
> > Testing the patch shell-escape-auth-5.diff, there is 1 issue from my side:
> > The session entry is totally ignored if using 'lyx -E', therefore I'd prefer
> > to save the entries in some other file (for instance "session.shellEscape"),
> > which could be loaded unconditionally.
> 
> I now actually checked and the session entry is not ignored. Maybe the
> issue you are seeing is due to the fact that the document was not
> explicitly authorized. This is indicated by a trailing ",1" instead
> of ",0". In this last case, explicit authorization is required and the
> batch command fails because this authorization is not given by default.
> 

I checked too Enrico. Exporting manually works.

# lyx2.3 -userdir 
"/BUILD/BUILDMint17/BuildLyxGitQt5.8self-gcc7.1/Testing/.lyx" -E pdf2 
minted-listings.pdf /usr2/src/lyx/lyx-git/lib/examples/minted-listings.lyx

But exporting from ctest does only if instrumenting the converters.
Here is, what I see (output got setting env var LYX_DEBUG_LATEX=1.

...
Executing /BUILD/BUILDMint17/BuildLyxGitQt5.8self-gcc7.1/bin/lyx2.3 -dbg latex 
-userdir "/BUILD/BUILDMint17/BuildLyxGitQt5.8self-gcc7.1/Testing/.lyx" -E pdf2 
minted-listings_defaultF.pdf2 
"/BUILD/BUILDMint17/BuildLyxGitQt5.8self-gcc7.1/autotests/out-home/AbC_Q0tzSx/minted-listings_pdf2_defaultF.lyx"
Setting debug level to latex
...
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./minted-listings_pdf2_defaultF.tex
LaTeX2e <2017-04-15>
Babel <3.12> and hyphenation patterns for 84 language(s) loaded.

support/Systemcall.cpp (294): Systemcall: 'pdflatex  
"minted-listings_pdf2_defaultF.tex"' finished with exit code 1
LaTeX.cpp (652): Log file: minted-listings_pdf2_defaultF.log
LaTeX.cpp (679): Log line: This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX 
Live 2017) (preloaded format=pdflatex 2017.7.2)  31 JUL 2017 13:47
LaTeX.cpp (679): Log line: entering extended mode
LaTeX.cpp (679): Log line:  restricted \write18 enabled.
LaTeX.cpp (679): Log line:  %&-line parsing enabled.
LaTeX.cpp (679): Log line: **minted-listings_pdf2_defaultF.tex
...
LaTeX.cpp (679): Log line: ) 
(/usr9/local/texlive/2017/texmf-dist/tex/generic/oberdiek/catchfile.sty
LaTeX.cpp (679): Log line: Package: catchfile 2016/05/16 v1.7 Catch the 
contents of a file (HO)
LaTeX.cpp (679): Log line: )
LaTeX.cpp (679): Log line:
LaTeX.cpp (679): Log line: Package ifplatform Warning:
LaTeX.cpp (679): Log line: shell escape is disabled, so I can only detect 
\ifwindows.
LaTeX.cpp (679): Log line:
LaTeX.cpp (679): Log line: ) 
(/usr9/local/texlive/2017/texmf-dist/tex/generic/xstring/xstring.sty (/usr9/l
LaTeX.cpp (679): Log line: 
ocal/texlive/2017/texmf-dist/tex/generic/xstring/xstring.tex
...
LaTeX.cpp (679): Log line: runsystem(mkdir -p 
_minted-minted-listings_pdf2_defaultF)...disabled (restricte
LaTeX.cpp (679): Log line: d).
LaTeX.cpp (679): Log line:
LaTeX.cpp (679): Log line:
LaTeX.cpp (679): Log line: ! Package minted Error: You must invoke LaTeX with 
the -shell-escape flag.
LaTeX.cpp (877): line: 18
Desc: Package minted Error: You must invoke LaTeX with the -shell-escape flag.
Text:  \setminted
   {breaklines=true}
Pass the -shell-escape flag to LaTeX. Refer to the minted.sty documentation for
 more information.

LaTeX.cpp (679): Log line: runsystem(which pygmentize && touch 
minted-listings_pdf2_defaultF.aex)...disabl
LaTeX.cpp (679): Log line: ed (restricted).
LaTeX.cpp (679): Log line:
LaTeX.cpp (679): Log line:
LaTeX.cpp (679): Log line: ! Package minted Error: You must have `pygmentize' 
installed to use this packag
LaTeX.cpp (877): line: 21
Desc: Package minted Error: You must have `pygmentize' installed to use this 
packag
...

So, what went wrong?

Ouch, how stupid am I!
Now I see it. The tested file-name is .../minted-listings_pdf2_defaultF.lyx and 
not .../minted-listings.lyx (which _is_ set in the session file)

The problem here is, I am unable to instrument lyx to use the newly created 
file name in the session because
there can be many parallel running exports.

Not your problem of course.

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Jürgen Spitzmüller]

I meant it in this sense. If a vote only means "I did not have a look at
>
the patch but I am fed-up so let us go ahead" then it is not taking
responsibilities.


A vote is a vote. If the given voting will be Rates differently, this will
be have been the last voting I have participated on this list.

Jürgen

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Mon, Jul 31, 2017 at 12:01:39PM +0200, Enrico Forestieri wrote:

> On Mon, Jul 31, 2017 at 10:57:01AM +0200, Guillaume MM wrote:
> 
> > A key point to me is that, according to your other message, going with
> > 3. locks 2.3 into implementing Pygments using minted. For this reason
> > and that it does not solve fundamental issues, there are more reasonable
> > solutions than the patch at [2] even understood as an intermediary
> > solution to get beta out.
> 
> I now count 4 votes for the patch at [2] and only one against it
> (if I correctly interpret this post as such). I think this means that a
> decision can be taken without entering in other exhausting and unfruitful
> discussions that most probably would go on forever.

Just to be clear. It shouldn't and cannot work this way. There were a vote
called on some options. One cannot come up with another option when he sees
that this is not what he would like. Otherwise, another one is entitled
to come up with yet another option and so on. We have to make a decision
(which is now possible, I think) and then move from there.

-- 
Enrico

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Sun, Jul 30, 2017 at 02:27:02PM +0200, Kornel Benko wrote:
> 
> Testing the patch shell-escape-auth-5.diff, there is 1 issue from my side:
> The session entry is totally ignored if using 'lyx -E', therefore I'd prefer
> to save the entries in some other file (for instance "session.shellEscape"),
> which could be loaded unconditionally.

I now actually checked and the session entry is not ignored. Maybe the
issue you are seeing is due to the fact that the document was not
explicitly authorized. This is indicated by a trailing ",1" instead
of ",0". In this last case, explicit authorization is required and the
batch command fails because this authorization is not given by default.

-- 
Enrico

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Mon, Jul 31, 2017 at 10:57:01AM +0200, Guillaume MM wrote:

> A key point to me is that, according to your other message, going with
> 3. locks 2.3 into implementing Pygments using minted. For this reason
> and that it does not solve fundamental issues, there are more reasonable
> solutions than the patch at [2] even understood as an intermediary
> solution to get beta out.

I now count 4 votes for the patch at [2] and only one against it
(if I correctly interpret this post as such). I think this means that a
decision can be taken without entering in other exhausting and unfruitful
discussions that most probably would go on forever.

-- 
Enrico

Re: Options for resolving the minted + shell-escape issue

[Guillaume MM]

Le 31/07/2017 à 10:57, Guillaume MM a écrit :


I spotted various areas of improvement:

...

* The new buffer param and the boolean argument in the session file are
superfluous and make the code much more complicated than needed.

...

I now understand better that this is by design, but it (still) does not
seem very natural to me. There are three levels, because there is
1. a switch to enable shell-escape (after which we are still given a
further dialog at compilation before actually running with shell-escape).
2. a button on this dialog to further "always allow".

The red icon on the bottom right is given when 1. is active even though
one is still safe as long as 2. has not been pressed. With e.g. Sweave
the first switch is automatic and innocuous (there are Sweave insets, so
ask for running sweave). Then with my patch the icon informs when one
has authorized to never ask anymore, so one is in an actual unsafe state.

With Enrico's patch, somebody using minted will always check the switch
1. and from this point on he always sees the red icon, so the icon
provides zero information. The contextual menu reverts the choice of
converters, which is pointless because it just makes the
document uncompilable. There is no option to see just revoke 2. or to
see the status of 2.

As a conclusion, as it is, the status bar icon in this patch is not an
implementation of visibility nor revocability. For it to be so, the icon
should be shown when 2. is active, and the context menu should switch 2.
instead of 1.

If furthermore a validation mechanism was provided for minted requiring
shell-escape, then one would just need 2 levels as with needauth.

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Sun, Jul 30, 2017 at 11:01:18PM +0200, Enrico Forestieri wrote:

> On Sat, Jul 29, 2017 at 07:51:08PM -0400, Scott Kostyshak wrote:
> > 
> > If I remove the python-pygments module (and reconfigure) and test, I get
> > the following error:
> > 
> > The driver command necessary to use the minted package
> > (pygmentize) has not been found. Make sure you have
> > the python-pygments module installed or, if the driver
> > is named differently, to add the following line to the
> > document preamble:
> > 
> > \AtBeginDocument{\renewcommand{\MintedPygmentize}{driver}}
> > 
> > where 'driver' is name of the driver command.
> > 
> > I expected to get the error, but what I was surprised by is that I get
> > the error when going to Document > Settings. I expected to get an error
> > immediately after compiling (before the LaTeX log errors). Is that
> > expected? Note that I am using a compiled Qt 5.9.2dev so if you can't
> > reproduce, it is probably a bug on my side.
> 
> No, it is not related to the Qt version. I was aware of it but further
> improvements were delayed by the exhausting discussion that followed.

Corrected at 44babaf6.

> I am contemplating the possibility of simply not allowing minted if the
> driver program is not called pygmentize, so that in this case the combo
> box is simply disabled without any warning.

This is still an option, even if I deem it to be inflexible.

-- 
Enrico

Re: Options for resolving the minted + shell-escape issue

[Guillaume MM]

Le 29/07/2017 à 23:57, Scott Kostyshak a écrit :

On Thu, Jul 20, 2017 at 03:30:14PM -0400, Richard Heck wrote:

On 07/20/2017 01:21 AM, Jürgen Spitzmüller wrote:

Am Mittwoch, den 19.07.2017, 16:37 +0200 schrieb Enrico Forestieri:

The attached patch takes into account all of these ideas. As a
disclaimer,
note that I am providing it only because I am now familiar with this
part
of the code and can quickly come up with a patch. But I am not
endorsing it.

I propose to apply this patch and return to productivity.


I would agree with that.


I'm concerned that since this issue has left us all exhausted, there is
a feeling of "let's just get this over so we can move on". I encourage
all of us to give one more cognitive spurt and give a vote.

 From what I understand, the three options are still what I proposed
three weeks ago [1]:

1. Revert the recently added minted support.

2. Keep the current state of master.

3. Apply the patch at [2]. Don't forget to copy emblem-shellescape.svgz
to lib/images. (Note that I get linker errors when I try to apply the
latest patch, but it might be something specific to my setup.)

Hi Scott,


A key point to me is that, according to your other message, going with
3. locks 2.3 into implementing Pygments using minted. For this reason
and that it does not solve fundamental issues, there are more reasonable
solutions than the patch at [2] even understood as an intermediary
solution to get beta out.

Taking all the discussions into account, the most "usably secure"
overall solution (the criteria I have been advocating and you seem to
have made yours) is still the one I have suggested more than a month ago:

https://www.mail-archive.com/lyx-devel@lists.lyx.org/msg200878.html

You do not reflect it in your list I think because you have been
trying to simplify the discussion (which I am not sure was really
fruitful unfortunately). Attached is a patch that implements a solution
along these lines, freely based on the patch at [2]. If your criteria is
really to get the most secure option, you will consider this:

First, improve the security of needauth by implementing the visibility
and revocability as in the patch at [2] (an icon in the status bar with
tooltip and right-click menu to unauthorize, as well as a checkbox in
the document properties). A lot of voices agreed that this was a
desirable improvement for needauth. This makes it more reasonable to
ship with new unsafe converters, and it also makes needauth a more
durable solution for unsafe converters in general. (Though there remain
issues that have been listed, for instance how to handle graphics
preview and making the authorization per-converter.)

It also revokes authorization after moving/deleting the file, which I
just realized was in fact already part of Tommaso's implementation.

Then, I did not implement shell-escape as in the patch at [2] (naively
this would have meant a second icon and a second checkbox). Several
voices including mine found the solution not satisfactory: ideally there
should be only one mechanism and one should not hard-code the addition
of shell-escape. I believe that instead it should be reflected in some
way in the converter settings. But there is no immediate need for
shell-escape, because in the second patch attached I am disabling minted.

This does not mean that there cannot be a nice solution for shell-escape
in 2.3, on the contrary. Starting from my patch, an integrated
shell-escape implementation should be easy following the idea I gave in
 if
one wants to, and there are probably many other ways. So, this patch
forms the best basis to build on for 2.3.

Moreover, the old way of adding shell-escape leads to a dialog
interrupting the compilation (based on an idea in [2]), so that
3rd-party guides will need to be updated to take needauth into account,
and old unsafe configurations are deprecated.

Given the absence of preconfigured shell-escape for now, fixable for
2.3, minted is disabled. This is done using the same proven technique
that has allowed me to propose a branch of master reading and writing
the stable file format over the past few months, and is a lightweight
solution until a satisfactory solution is found or a decision is taken.
I am not convinced with Pavel's suggestion "drop shell-escape later"
because one cannot commit to a technical choice based on a promise when
there are several plausible alternatives, but all options are left open.
Of course one could also revert immediately independently
of the final choice if one prefers. So, this patch does not lock into a
premature choice.

Lastly, I am ready to endorse my own patch because I believe it provides
the sanest basis for needauth and the further improvements it requires.




So I ask explicitly to everyone (even if you think you have already
voted, please give your vote again): who has given the latest patch a
good test, has looked at the code (the patch 

Re: [LyX/master] prefs/needauth: added warning if user tries to disable authorization for needauth converters.

[Guillaume MM]

Le 27/07/2017 à 00:06, Tommaso Cucinotta a écrit :

commit 8a4fcd3d95ca4aeed1c46152cecadf29ed21e774
Author: Tommaso Cucinotta 
Date:   Thu Jul 27 00:01:51 2017 +0200

 prefs/needauth: added warning if user tries to disable authorization for 
needauth converters.
---
+   _("SECURITY WARNING!"), _("Unchecking this option has the effect 
that potentially harmful converters would be run without asking your permission first. This is 
UNSAFE and NOT recommended, unless you know what you are doing. Are you sure you would like to 
proceed ? The recommended and safe answer is NO!"),


Thanks. HTML maybe?

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Guillaume MM]

Le 29/07/2017 à 23:54, Scott Kostyshak a écrit :

On Thu, Jul 27, 2017 at 04:09:56PM +0200, Guillaume MM wrote:


* Having to use -shell-escape for running Pygments.


Yes, and if we go the way of the patch, I don't think any other
improvements (e.g. post-beta1) will be made to address this, until
perhaps 2.4.0 if the Github issues is addressed.


I take note.




I would also be more comfortable if somebody takes responsibility for
any patch that is to be committed, given that the author has said that
they do not endorse it.


Fair point. My goal with the vote was to collectively take
responsibility, since this is an important patch and involves security.
But I feel that most people are just tired of the debate and are hoping
too much to move forward that they have not taken a deep look.


I meant it in this sense. If a vote only means "I did not have a look at
the patch but I am fed-up so let us go ahead" then it is not taking
responsibilities.

Re: [LyX/master] Cmake build: Ignore boost settings if we are using std-regex

[Kornel Benko]

Am Montag, 31. Juli 2017 um 01:09:09, schrieb Scott Kostyshak 
> On Sun, Jul 30, 2017 at 07:41:02AM +0200, Kornel Benko wrote:
> > Am Samstag, 29. Juli 2017 um 19:14:01, schrieb Scott Kostyshak 
> > 
> > > On Thu, Jul 27, 2017 at 11:32:06PM +0200, Kornel Benko wrote:
> > > > commit 2fe59adbc89f56e7e192b57c90eb5e2a8338721c
> > > > Author: Kornel Benko 
> > > > Date:   Thu Jul 27 23:29:29 2017 +0200
> > > > 
> > > > Cmake build: Ignore boost settings if we are using std-regex
> > > > 
> > > > External/included boost is only used for the component regex
> > > 
> > > With this commit, I get the following message:
> > > 
> > > Unable to determine the system directory having searched
> > >   /home/scott/lyxbuilds/master/CMakeBuild/share/lyx/
> > > Use the '-sysdir' command line parameter or set the environment 
> > > variable
> > > LYX_DIR_23x to the LyX system directory containing the file 
> > > `chkconfig.ltx'.
> > > 
> > > Scott
> > 
> > Totally unexpected. What should boost have to do with system dir?
> > Could you please test with clean build?
> 
> I tested with clean builds of 2fe59adb and 2fe59adb^, and see the same
> difference.
> 
> The only difference in the output of the cmake commands is
> 
> "Using std regex"
> 
> The difference in the CMakeCache.txt files is:
> 
> $ diff CMakeCache_before.txt CMakeCache_after.txt 
> 639,644d638
> <
> 
> boost_BINARY_DIR:STATIC=/home/scott/lyxbuilds/master/CMakeBuild/3rdparty/boost/libs
> < 
> < //Value Computed by CMake
> <
> 
> boost_SOURCE_DIR:STATIC=/home/scott/lyxbuilds/master/repo/3rdparty/boost/libs
> < 
> < //Value Computed by CMake
> 871c865
> < CMAKE_NUMBER_OF_MAKEFILES:INTERNAL=26
> ---
> > CMAKE_NUMBER_OF_MAKEFILES:INTERNAL=24
> $ 
> 
> Are those changes expected?
> 
> With 2fe59adb, compilcation succeeds and I get the same error under the
> following three cases:
> 1. I omit -DLYX_EXTERNAL_BOOST
> 2. I set it to -DLYX_EXTERNAL_BOOST=ON
> 3. I set it to -DLYX_EXTERNAL_BOOST=OFF
> 
> With 2fe59adb^, if I set it to -DLYX_EXTERNAL_BOOST=ON, the CMake call
> gives me:
> 
> -- Searching for boost
> CMake Error at CMakeLists.txt:814 (message):
>   Boost not found
> 
> Are those clues useful at all? If not, don't spend time on it. I will
> take a deeper look when I have time.

Yes, they do. Looks like a very old CMakeLists.txt. The committed one has this 
message
on line 821. The mentioned commit 2fe59adb is later amended by b5c2e02e.

> Scott

Kornel

signature.asc
Description: This is a digitally signed message part.