Thiago Arruda wrote:
> On Wednesday, February 10, 2016 at 1:14:21 PM UTC-3, Bram Moolenaar wrote:
> > I wonder if you read my proposal wrong, or perhaps I didn't explain
> > something properly. I read your response, and can only say that it's
> > all covered in what I proposed.
> >
> > I think it may be caused by the notion that in NeoVim a job includes the
> > communication with the job (over pipes), while in my proposal I separate
> > them out.
>
> I understood, but failed to see the advantage of the separate channel
> abstraction in the context of job control. Please correct me if I missed
> something, but this is how one spawns a job with a channel associated with
> stdin/stdout:
>
> ```vim
> let job = job_filter("cmd", {options})
> let handle = job_getchannel(job)
> call ch_setcallback(handle, "receive", "Callback")
> call ch_setcallback(handle, "error", "Callback")
> call ch_setcallback(handle, "close", "Callback")
> " send some data to stdin
> call ch_send(handle, "data")
> ```
Not quite. I haven't mentioned defaults yet, but I would think it's
best that when a callback is not defined for stderr then the same one as
for stdout is called. Close is different, so the default would be no
callback.
The options can be set with job_filter() or on the channel, but it's of
course shorter with job_filter():
let job = job_filter("cmd", {'out-cb': 'Callback', 'close-cb': 'Callback'}
let handle = job_getchannel(job)
call ch_sendraw(handle, "data")
Actually, job_filter could return a tuple to make it simpler:
let [job, handle] = job_filter("cmd", {'out-cb': 'Callback', 'close-cb':
'Callback'}
call ch_sendraw(handle, "data")
> Now compare with Neovim version:
>
> ```vim
> let job = jobstart("cmd", {"on_stdout": "Callback", "on_exit": "Callback"})
> " send some data to stdin
> call jobsend(job, ["line1", "line2", ""])
> ```
I do not like jobsend(), it removes the abstraction layer of what is a
job and what is communication with a job. You would need several other
xxxsend() functions depending on how the communication was started.
Compare with a file descriptor: There are lots of ways to open a kind of
file, but once you have a file descriptor you can use read() and write()
on it no matter how it was opened.
> As you can see, Neovim's version is shorter and covers the same use
> cases as the one you proposed. It doesn't need a "close" or "error"
> event, the user can consider it was closed after the "on_exit" event,
> after which there will be no more "on_stdout" events. On top of that,
> I don't see how one can read stderr, but will
> assume that you plan to add another function to get the stderr channel:
>
> ```vim
> let job = job_filter("cmd", {options})
> let handle = job_getchannel(job)
> call ch_setcallback(handle, "receive", "Callback")
> call ch_setcallback(handle, "error", "Callback")
> call ch_setcallback(handle, "close", "Callback")
> " send some data to stdin
> call ch_send(handle, "data")
> " listen for stderr
> let stderr_handle = job_getstderrchannel(job)
> call ch_setcallback(stderr_handle, "receive", "Callback")
> call ch_setcallback(stderr_handle, "error", "Callback")
> call ch_setcallback(stderr_handle, "close", "Callback")
> ```
No, that's not what it would look like. The stderr is not a separate
handle. The handle includes stdin/stdout/stderr. Your example shows
why: there would be several close and error callbacks, even though
that's hardly ever useful. The current implementation has read and
write over the socket, thus putting stdin and stdout in the handle fits
well. stderr needs some special handling.
> That's even more verbose. Here's Neovim version of the same thing:
>
> ```vim
> let job = jobstart("cmd", {"on_stdout": "Callback",
> \ "on_stderr": "Callback", "on_exit": "Callback"})
> " send some data to stdin
> call jobsend(job, ["line1", "line2", ""])
> ```
I'm sorry but seems like you are picking things that are easy with
NeoVim and the comparing to the most verbose version of what you think
Vim would do. That's not a fair comparison.
Also, verbosity is one thing, what is much more important is that it's
easy to understand. In my opinion we should optimize for simplicity.
> Again, does the same with a much simpler API(note that omitting a
> callback from the startup options simply redirects the handle to
> /dev/null).
>
> I can see how the channel abstraction could be useful in a mainstream
> programming language with support for custom types and libraries that
> operate in such types(eg: Node.js Streams are everywhere in their
> echosystem) but in vimscript it only seems to be an unnecessary
> complication.
Every feature in Vim grows beyond its limits. That's what we learned from
history. There are several functions that gained more arguments and are
now harder to use than they should be. It's better to aim at covering
those features, even if not all of them would be implemented right away.
Otherwise we create a problem for adding features later.
> Here are some related concerns:
>
> - If the `job_getchannel` returns a shared handle for stdin/stdout, then what
> happens if `ch_close` is called on that handle?(Will it have a separate
> function for getting the stdin handle?)
Everything closes. Perhaps it's useful to close only one side, we'll
have to specify how to do that. Obviously the implemention is simple,
it's just a matter of defining how it works and having a good default.
> - The API suggests that you get a job's channel after it was created, but
> there's no way to redirect the stdio handles after the job is
> spawned. In this case, you always have to connect pipes to the job's
> stdio and the user can chose or not to attach callbacks, and there's
> no way the job will share stdout
> with Vim(as it was suggested in a previous message).
Please read my original message again, it's explained there. Obviously
connecting stdin/stdout/stderr to the channel happens when creating the
job, not later.
> - I see that channels can have an associated RPC protocol with them,
> but that would not be useful for arbitrary processes that have no
> knowledge of Vim(eg: compilers), which seem to be the main target of
> job control.
What you describe is an asynchronous pipe. Not sure if that is the main
target, but it should be easy to do. Except that it has the usual
problem of not knowing when a message is complete. Probably requires
checking for NL characters. So that would be the default message
protocol.
> > So you can have a channel in JSON mode communication to a job
> > with a pipe. I havent seen something like that in NeoVim. But then I
> > haven't spent much time trying to understand what NeoVim is doing. Can
> > NeoVim use msgpack on stdout?
>
> Yes. In fact, Neovim also has the concept of channels, but it is
> unrelated to job control only applies in the context of msgpack-rpc:
>
> - `rpcrequest({channel}, {method}[, {args}...])` sends a msgpack-rpc
> request to a channel and returns the response(the call only returns
> when the other end of the channel sends a response)
> - `rpcnotify({channel}, {event}[, {args}...])` sends a msgpack-rpc
> notification
How about communicating with some other protocol than msgpack? Looks
like you would need another set of functions.
> These two functions can be called on any msgpack-rpc channel connected
> to Neovim. At the moment, there are three ways a msgpack-rpc channel
> can be created in Neovim:
>
> - A process may connect to Neovim UNIX domain socket(or tcp address if it is
> listening on one). The address is provided in the `$NVIM_LISTEN_ADRESS`
> environment variable.
I hope you have thought of security, not allow just anybody to connect
to Vim.
> - Neovim may spawn a process that has a channel connected to its stdin/stdout.
> This uses the job infrastructure internally, but the API exposed to
> vimscript is separate(`rpcstart()`/`rpcstop()` are the msgpack-rpc
> equivalent to `jobstart()`/`jobstop()`).
Right, so you have combined process/job starting with rpc/pipe and end
up with four combinations. You haven't mentioned two yet, I suppose
it's also possible to use raw I/O over a socket?
> - Neovim may be spawned in embedding mode, where instead of sending terminal
> escape codes to stdout(And parsing terminal input from stdin), it
> talks using msgpack-rpc. This mode is used mainly external UIs and
> for its testing infrastructure.
>
> Currently we do not have a way to open a msgpack-rpc connection to an
> arbitrary address, but it is only a matter of submitting a relatively
> small patch I made a few months ago:
> https://github.com/neovim/neovim/issues/3119#issuecomment-130906214
>
> But this is not very relevant to the discussion, I have only talked about
> msgpack-rpc to highlight how its API is completely independent from job
> control(which can be seen as a more low-level/general-purpose asynchronicity
> feature)
>
> > I don't think that old patch adds much, now we already have working
> > channels. Only the part with making channels work with
> > stdin/stdout/stderr still needs to be done. And that code already
> > exists in os_unix.c. MS-Windows is the most difficult (since sockets
> > there work differently from file handles) and I thought your patch
> > doesn't have that part.
>
> Yes, I have very little Window programming experience, so I have only
> added an UNIX version for review. But wasn't this the case for the
> channel and job features? From what I could see in the mailing list
> patches, the initial implementations of channel/jobs either partially
> worked on UNIX(with no Windows support) or had quite a few bugs. The
> patch I sent was successfully tested by a few people at the time.
The difference is that the functionality already existed in Vim.
Therefore it was straigtforward to make it work on MS-Windows.
With these features I follow the "release early, release often"
strategy. That way I get feedback early. And fortunately there are
others that can then help adding the missing pieces.
> I understand if you don't like my code and/or the patch, but please
> consider my arguments and the possibility of following our API. As I
> said, there are already a few popular plugins that successfully use
> it(some even helped us to develop/evolve the API).
I do not plan to add msgpack encoding and NeoVim doesn't JSON, thus
there is not much overlap.
> > The whole thing I propose was to support what plugin writers were asking
> > for. With future feature requests in mind, and combining with the
> > existing code for netbeans.
> >
> > Problem with netbeans is that it uses a weird message format. I want to
> > take that in the right direction, don't want more people to use the
> > netbeans format. JSON appears to be the best choice. I know NeoVim
> > uses msgpack, but I just don't like it.
>
> I hope you understood that job control has nothing to do with msgpack
> in Neovim(or in my old patch). It is just a simple way of implementing
> exactly what plugin authors have been asking for quite some time: A
> simple and flexible way to communicate with Vim asynchronously.
Msgpack is not simple. I think that most plugin authors have never
heard of it before looking at NeoVim. JSON is widely supported and easy
to understand.
Does msgpack actually support message numbering, making it possible to
know which response belongs to which request?
There also is some naming issue. What you call job control I would call
an asynchronous pipe. Job control is about running / pausing /
cotinuing / scheduling jobs. There also are several job control
languages. Perhaps it's because of my background, but that's what I
associate it with.
--
Don't be humble ... you're not that great.
-- Golda Meir
/// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
