Re: At last I understand sockets and servers

[Edward K. Ream]

On Sunday, October 21, 2018 at 10:37:32 AM UTC-5, Cormac Williams wrote:

> I would answer some of your questions with more questions, mainly.

All these questions are reasonable.  Here are some preliminary answers. 

> 1 : Do you want to hang this server on the internet at some point?

Not sure what you mean by this.  Servers may spawn a new thread for each 
incoming thread.  In any case, we can assume that we'll use a fully-scaled 
server like Apache.

> 2 : What sort of scale do you want in terms of users?

Imo, n == 1 is often going to be the limit, for security reasons.  Leo 
looks like an unverifiable cgi script to the server, which means one user 
(or small, *trusted *group of users) must be *fully* responsible for the 
damage Leo could cause.

It might be possible to host a Leo server in a per-user (or per-small 
group) virtual machine somewhere, but that's it.  I see no way to run a 
public, unsecured, Leo server.

> 3 : What sort of scale do you want in terms of Leo outline size?

The same scale as at present.  This means that loading a Leo view in a 
browser won't load the entire xml file, but will only load the parents of 
presently-selected node.  In other words, Leo will use ajax to display the 
visible part of the outline.

> 4 : Do you want to get a database involved rather than serving from an 
xml file?

Good question. 

I recently thought about this in "plain" Leo.  There have been repeated 
requests for "node history".  One could imagine a DB containing old 
versions.  But I just recently saw that *node history would not have commit 
messages*.  Without those messages the "raw stream" of node contents is 
useless.  If the user wants node history, they should use git and be 
careful to write meaningful commit messages.  There is no way for Leo to 
provide those messages.

There is another reason for considering using a DB, namely to support 
cross-file clones.  The likely fatal problem with this is that the entire 
DB becomes a kind of cache.  External files must take precedence.  That 
being so, the DB adds a significant level of complexity, without 
fundamentally solving the problem.  This isn't likely to happen.

The third reason to use a DB would be to support huge outlines.  There may 
already be code in Leo to do this.  I don't remember.

> 5 : I there a multi-user story to be told in terms of using an outline?

See the answer to question 2.  The answer is yes, but only if all users 
bear full responsibility for the machine (virtual or not) on which Leo's 
server is run.

> 6 : How much are you prepared to refactor or change code to support the 
above visions?

I am willing to treat the project as essentially a new gui:  leo 
--gui=browser.

I will reject any suggestion to fork Leo's code base.  That would be the 
end of Leo, imo.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: At last I understand sockets and servers

[Edward K. Ream]

On Sunday, October 21, 2018 at 11:01:57 AM UTC-5, Offray Vladimir Luna 
Cárdenas wrote:

It's good to see the advances in the servers direction. The idea of a more 
> decoupled Leo with web interfaces and the idea of "outlining" services to 
> other systems and languages that some have proposed seems close.
>

Interesting idea.  Providing "outlining services" seems like another 
project.

I would avoid going into the Apache path and I would try to choose a 
> minimalist pure python web framework (bottle, flask, other used by Jupyter) 
> that can deal with the server/sockets core code functionality in Leo and 
> leave the scaling part to a server neutral back-end (Apache, Cherokee, 
> Lighttpd, Enginx, etc.)
>

Sounds reasonable.  In another answer in this thread I said we could assume 
that Leo would use something like Apache (server), but I want to make clear 
that all options are on the table.

When I was trying to implement ideas from Leo in Pharo/Smalltak I remember 
> focusing on the "outlining experience" first. I wanted to be able to 
> add/remove and move nodes that have a "header" and a "body" inside Pharo. 
> Sometimes, when I think in this idea of "outlining" services using a web 
> interface I think which is the minimal web library that provides me a tree 
> interface, for example Inspire Tree[1], and how I would connect it to the 
> tree representation I already have in Grafoscopio. Because I don't want to 
> have a lot of moving parts in different languages and programming paradigms 
> (something that already happens to Jupyter), I would like to create some 
> socket/service that connects the web interface with the Pharo backend. So 
> all the machinery behind trees is already there and the only "new thing" 
> I'm implementing is a way to decouple the GUI so it can be a web app.
>
> [1] http://www.inspire-tree.com/
>
> Hopefully, something like that can be the path for services/sockets in 
> Leo. I wonder how Leo-Vue can connect to Leo server, for example and what 
> is the demo MVP for a web interface that connect with a Leo backend.
>

I don't know enough to respond in detail just now.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: At last I understand sockets and servers

[Edward K. Ream]

On Sunday, October 21, 2018 at 9:20:43 PM UTC-5, Edward K. Ream wrote:
>
> On Sun, Oct 21, 2018 at 11:28 AM Terry Brown  
> wrote:
>
> I wonder if Cormac was thinking of frameworks like Django, but perhaps
>> more appropriate (lighter) for Leo, Flask?  
>
>
> All options are on the table, but see below.
>
> Also mod_http.py plugin allows execution of arbitrary Python code in a
>> running Leo, so there's that to look at too.
>>
>
> Hmm.  Is Leo as a web app is feasible? 
>

To answer my own question, the answer is yes, as discussed further in 
answer to in my reply to Cormac 
.  
Imo, we must follow the Jupyter model, in which the server runs on the 
user's own (possibly virtual?) machine. It's time to study how Jupyter 
handles security issues.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Security in Leo as a web app (LeoWapp)

[Edward K. Ream]

In another thread I wrote:

"Leo looks like an unverifiable cgi script to the server, which means one 
user (or small, *trusted *group of users) must be *fully* responsible for 
the damage Leo could cause. It might be possible to host a Leo server in a 
per-user (or per-small group) virtual machine somewhere, but that's it.  I 
see no way to run a public, unsecured, Leo server."

Exactly these security issues arise in Jupyter . The 
solutions in the Security in the Jupyter notebook server 
 seem 
appropriate for LeoWapp. It seems we have only two choices:

1. Limit access to LeoWapp to a single machine.  Remote access is 
prohibited, unless the user has the token/password created when the server 
starts. We assume all local .leo files are trusted. 

2. Use the full Jupyter security scheme.  We distinguish between trusted 
and untrusted .leo files, and use authentication similar to Jupyter.

Your comments, please.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: Security in Leo as a web app (LeoWapp)

[vitalije]

I am not 100% sure but I believe that it is possible to start docker 
instance on some remote host (one instance per user or per script 
invocation) and let python execute script inside that docker instance. A 
malicious script can try to damage server but the damage will remain inside 
its own docker instance. It won't affect other users nor the host as a 
whole.

If I am not mistaken that is what travis services do with the github 
projects (including Leo recently). On every github activity (commit/pull 
request), it starts new docker instance, installs python and all necessary 
libraries, retrieves fresh copy of the source code from github, builds 
project and runs tests inside this docker instance and then shuts it down, 
keeping only built artifacts and execution logs. Theoretically, an attacker 
might submit a pull request containing malicious code to some legitimate 
project and travis will execute that code to check if the pull request is 
breaking something or not. Of course this attacker has to be registered on 
github to submit pull request in the first place, and probably will be soon 
banned. Nevertheless his malicious code will be executed but it can't make 
substantial damage to the service. 

The only problem I see is that starting new docker instance takes the time, 
so it won't be very practical to start one instance for every script 
invocation. Also I am not sure how many instances can one host run 
simultaneously. 

Perhaps there can be two different types of Leo server:

   - Public Leo server which can provide multi-user outline editing and 
   browsing, but without scripting abilities 
   - Private local Leo server which provides full Leo experience including 
   script execution on local machine, but for a single user

The public one may allow execution of some scripts approved by the site 
administrator.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: Security in Leo as a web app (LeoWapp)

[Chris George]

Someone should look at Heroku. It seems that they do containers that
already support python that is designed to face the web.

They also have a robust and complete local CLI client. It is very
convenient for working on apps with a web face.

Chris



On Mon, Oct 22, 2018 at 5:55 AM vitalije  wrote:

> I am not 100% sure but I believe that it is possible to start docker
> instance on some remote host (one instance per user or per script
> invocation) and let python execute script inside that docker instance. A
> malicious script can try to damage server but the damage will remain inside
> its own docker instance. It won't affect other users nor the host as a
> whole.
>
> If I am not mistaken that is what travis services do with the github
> projects (including Leo recently). On every github activity (commit/pull
> request), it starts new docker instance, installs python and all necessary
> libraries, retrieves fresh copy of the source code from github, builds
> project and runs tests inside this docker instance and then shuts it down,
> keeping only built artifacts and execution logs. Theoretically, an attacker
> might submit a pull request containing malicious code to some legitimate
> project and travis will execute that code to check if the pull request is
> breaking something or not. Of course this attacker has to be registered on
> github to submit pull request in the first place, and probably will be soon
> banned. Nevertheless his malicious code will be executed but it can't make
> substantial damage to the service.
>
> The only problem I see is that starting new docker instance takes the
> time, so it won't be very practical to start one instance for every script
> invocation. Also I am not sure how many instances can one host run
> simultaneously.
>
> Perhaps there can be two different types of Leo server:
>
>- Public Leo server which can provide multi-user outline editing and
>browsing, but without scripting abilities
>- Private local Leo server which provides full Leo experience
>including script execution on local machine, but for a single user
>
> The public one may allow execution of some scripts approved by the site
> administrator.
>
> Vitalije
>
> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+unsubscr...@googlegroups.com.
> To post to this group, send email to leo-editor@googlegroups.com.
> Visit this group at https://groups.google.com/group/leo-editor.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: At last I understand sockets and servers

[Terry Brown]

On Sun, 21 Oct 2018 21:20:33 -0500
"Edward K. Ream"  wrote:

> Hmm.  Is Leo as a web app is feasible? Leo's scripting features make
> it extremely dangerous. If run on a server, Leo becomes an
> unverifiable cgi script. This looks like a gotcha.  Am I missing
> something?

Sandboxing / encapsulation with Docker or some similar container system
seems like the easy way.  Containers are much lighter than VMs, so you
should be able to run many sessions, depending on hardware etc.

Docker containers start fast enough if you're running something
long-lived like a session on leoBridge.py or something that runs for
the same length of time as a typical Leo session.  I *think* they'd be
too slow to exist transiently and respond to single requests, although
maybe hot cache on a suitable machine could run that way.  But I'm
guessing you're looking for something more persistent, anyway.

I just tried starting a Docker environment I have based on
conda/miniconda3 which is based on debian, and it starts in under a
second.  It has tex-live / numpy / matplotlib (which pulls in a chunk
of Qt), so it's probably similar in size to something that would run
Leo.

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.