Hello Bram! That's a really great feature. I am trying to implemented a gulp plugin (gulp is a node package used to build web apps) based on channels and there are two things that I has a problem with:
1) when I use `job_start()` to start a server in the background and
immediately after I call `ch_open()` to open channel to that server I am
always refused: `E896: read from channel: Connection refused`.
However, if I check `job_status` in between (which is what
I should do anyway), then the following `ch_open` runs fine. There is
no way now to communicate from the job that started that what it runs is
ready to connect to.
2) when my server sends data back to vim only the first message is
received. This can be reproduced by the demoserver.py example. Just
double the lines in `ThreadedTCPRequestHandler.handle`:
print("sending {}".format(encoded))
self.request.sendall(encoded.encode('utf-8'))
print("sending {}".format(encoded))
self.request.sendall(encoded.encode('utf-8'))
And vim will only get the first message. It would be nice if the socket
was reading until explicitly closed by the server. It would be useful
for my case to send updates from the background process spawn by the server
that I wrote.
Best regards,
Marcin Szamotulski
On 22:08 Mon 08 Feb , Bram Moolenaar wrote:
>
> Thanks for the feedback. I think it's time to sketch the upcoming work.
> It appears some thought that sockets was the end of it, that's not so.
>
>
> For jobs the most useful are probably:
> - A deamon, serving several Vim instances, connect by socket.
> - One server working with one Vim instance, asynchronously.
> - A server performing some work for a short time, asynchronously.
> - Running a filter, synchronously (existing functionality)
>
> A channel can communicate over:
> - A socket (what ch_open() currently does)
> - A pipe (only possible with an associated job) connected to
> stdin/stdout/stderr.
>
> A channel can communicate:
> - With JSON messages
> - With JS messages
> - Raw messages
> - NL-separated messages (this is what netbeans uses)
>
> A channel can be associated with a job, but that job does not
> necessarily need be under control of the current Vim (or any Vim)
> instance.
>
> It looks like all the functionality is already present in the Vim code
> somewhere, we just have to make it available in the right way. The
> missing part, a timeout on connect(), was recently added, this may still
> need some work.
>
> Raw messages have the problem that Vim doesn't know where the end of a
> message is, thus it might invoke a callback with an incomplete message.
> The callback will then have the task of parsing and queuing readahead.
> Using line breaks is the most common alternative, we can support that.
>
> Some combinations are likely to be most useful. I heard some people
> mention using a server over pipes to write a message to and call a
> callback when the response is received. I assume this would work best
> with NL separated messages.
>
> Using pipes only works when starting a job. Thus these are closely
> coupled.
>
> The deamon that serves multiple Vim instances would be most useful to
> lookup code locations, for example. Probably best use JSON or JS for
> that, as it makes it easier to add more features over time.
>
> One step further would be to use protocol buffers. I think that it's a
> step too far right now, maybe later. The current protocol buffer
> support unfortunately doesn't support the JS format that I like (the
> JSON format uses field names instead of numbers, an unfortunate choice,
> as it makes version mismatches difficult).
>
>
> Some parts that we still need that require some thougths:
>
> - Communicating over a socket requires knowing the port number. For a
> deamon this would be a known number. For a server started by Vim it's
> best to let the server pick an available port. Then how does it tell
> Vim what port it picked? In test_channel we write the port number in
> a file. That's not ideal, but perhaps there is no better solution.
> Having Vim pick a free port and pass it to the server has race conditions
> (but it might still be an acceptable solution).
>
> - When starting a job, where does it's stdin/stdout/stderr go?
> Possibly the same terminal as Vim, a newly started terminal, a file or
> /dev/null. No, running a terminal inside Vim is not an option. But
> we could write stdout/stderr into a buffer.
>
>
> For starting a deamon job we could have:
> let job = job_start("cmd", {options})
>
> We could make it a bit clever, only start the deamon when it's not
> running yet. This would check whether connecting to the socket works,
> with a zero timeout. Then we have the channel open as well.
> let job = job_maystart("cmd", "address", {options})
> let handle = job_getchannel(job)
>
> Running one server (or more, doesn't really matter) and connecting over
> a socket would work the same way, except that the job is always started
> and it would default to be stopped when Vim exits. We don't really need
> another function for this, just use two:
> let job = job_start("cmd", {options})
> let handle = ch_open("address", {options})
> That's already working.
>
> When using stdin/stdout/stderr we must start the job and open the
> channel at the same time. We have more choices about where to connect
> them. It's probably not useful to have separate functions, we can use
> the {options}:
> let job = job_filter("cmd", {options})
> let handle = job_getchannel(job)
>
> Once we have a channel it would be useful to be able to set callbacks:
> call ch_setcallback(handle, "receive", "Callback")
> call ch_setcallback(handle, "error", "Callback")
> call ch_setcallback(handle, "close", "Callback")
> And some other things, such as the timeout:
> call ch_settimeout(handle, 100)
> Using entries in {options} would also work. In fact, we could make it
> more generic and just use:
> call ch_setoptions(handle, {options})
> Where {options} is the same as what's used when creating a channel
> (minus the ones that only make sense during creation).
>
> For {options} the main thing we're missing is what to do with
> stdin/stdout/stderr:
> "term": "pty" // use current terminal
> "term": "open" // start a terminal
> This would result in stdin/stdout/stderr to connect to the terminal,
> unless specified otherwise. Except that reading from the current
> terminal won't happen (Vim is reading that).
>
> For a channel we have some options that are the same, such as the
> callbacks. But these could be for a socket, thus we need generic
> names. Would work by dropping the "std". So job options would be:
>
> "in-io": "null" // read from nirvana
> "in-io": "pipe" // read from channel
> "in-io": "file" // read from file
> "in-file": "/path/file" // file name
>
> "out-io": "null" // write to nirvana
> "out-io": "pipe" // write to channel
> "out-cb": "Handler" // write handler
> "out-io": "file" // write to file
> "out-file": "/path/file" // file name
> "out-io": "buffer" // write to buffer
> "out-buffer": "name" // buffer name
>
> "err-io": "out" // stderr writes to stdout
> "err-io": "null"
> // etc, like stdout
>
> "kill": 1 // kill job on exit
> "kill": 0 // don't kill job on exit
>
> Channel options:
> "out-cb": "Handler" // write handler
> "err-cb": "Handler" // error handler
> "close-cb": "Handler" // close handler
> "timeout": msec
>
> That's plenty already, but we may need a few more.
>
> One problem that I haven't figured out yet: When starting a job using a
> shell, we get the PID of the shell. So we can kill the shell, but not
> the process that it started.
> E.g. "/bin/sh python demoserver.py <here >there". A shell is also
> useful for expanding wildcards, would be nice if we can make this work.
>
> --
> hundred-and-one symptoms of being an internet addict:
> 182. You may not know what is happening in the world, but you know
> every bit of net-gossip there is.
>
> /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\
> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\ an exciting new programming language -- http://www.Zimbu.org ///
> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups
> "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vim_dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: Digital signature
