Re: Is there an easy way to capture stdout/stderr for a shell command and pipe it to the log pane?
I don't know the answer but I know a thing or two that might be useful. By
starting the command with "&", as you have, you avoid
the proc.communicate() call, which would block Leo until it returns.
However, if you try to capture output with a pipe, the pipe can block if it
fills up, and that happens with a surprisingly small number of bytes. Then
you block because the pipe is waiting to be cleared, and you are probably
waiting to get output from the pipe.
So that won't work.
The way I managed this with the background process manager (BPM) - it's
what launches pylint, mypy, etc - is to use a separate thread to launch the
process, and call proc.communicate() in that thread. The thread blocks
waiting for the process to complete, but that doesn't matter since it
doesn't block Leo itself. This leaves the question of how to get the
results. The thread puts the results into a variable in the BPM, and the
BPM registers a callback at idle time. The callback checks to see if the
variable is not None. If the BPM sees that there is a result in that
variable, it cancels the callback and makes use of the result. Then it
writes the messages to the log pane. You have to use a lock to make sure
that the external process's results are put atomically into the variable.
In your case I might have it write the results to a different tab to avoid
getting mixed up with regular log output. I'm going by memory here and
might have a detail or two inaccurate.
So study the BPM and see if you could do something similar with your code.
The BPM makes sure that it only ever runs one process at a time, BTW. That
avoids the possibility of getting intermingled results or other confusions.
Another possibility is to have your external process redirect output to a
known file. You could check periodically to see if that file has gotten
new data and read the new data if so. The problem there might be to know
when the external process is finished writing data. You could also hget a
race condition between the writing process and the reading process, so you
would have to figure out how to handle that situation. I don't happen to
know how that works and it might be different between Windows and Linux.
Another possibility would be to launch your external program from a
batch/shell file and have the batch/shell file redirect the output to a
known file. When the programs finishes, the batch file could set an
environmental variable. You could do the idle-time callback trick to check
for that variable, or you could have the checks done periodically by a
separate thread. That thread could even read the results, format them, and
write them to a new tab or pass them on to another batch file.
I would say the last possibility sounds the most promising to me.
On Wednesday, May 3, 2023 at 7:02:47 PM UTC-4 gates...@gmail.com wrote:
> Hi all,
>
> I have an outline with some @button nodes that fire off external shell
> commands using g.execute_shell_commands. I tend to run Leo *without* a
> console window open, so anything those @button-triggered shell commands
> print to the console is lost to me. Is there an easy way/straightforward
> way to redirect to the stdout/stderr for those commands only to the log
> pane? I'm struggling to figure it out by manipulating sys.stdout and
> sys.stderr, though I admit this is far from my area of specialty.
>
> An example of a button:
>
> ```
> @language python
> @tabwidth -2
>
> ''' Executes the currently selected sketch with the 'vsk run' command.
> '''
>
> import os.path
>
> fn = c.getNodeFileName(p)
> if len(fn) == 0:
> # non-external file selected
> g.es('Not a valid sketch node.', color='red')
>
> else:
> sketch_path = os.path.dirname(fn)
> g.execute_shell_commands(f'&vsk run {sketch_path}')
> ```
>
> That command is long-running (i.e. it stays open in another pane, thus the
> '&' to unblock the Leo GUI). Sometimes it indicates errors on the console
> output, which just plain doesn't exist when I'm running Leo without a
> console window open.
>
> Ideally any solution would be cross-platform, since I do this on both
> Windows and Linux... but really I'd be happy with platform-specific
> solutions too. I'd really rather not have to have a console window open to
> see these errors if at all possible.
>
> Thanks in advance for any ideas and suggestions,
> Jake
>
--
You received this message because you are subscribed to the Google Groups
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/leo-editor/b50e67db-17cb-4b70-9c8b-33c229d4c166n%40googlegroups.com.
Re: Is there an easy way to capture stdout/stderr for a shell command and pipe it to the log pane?
I see that I slightly lost track of the fact that you want to look at
ongoing error messages. As I said, the big problem that I am aware of is
that the message pipe fills up and then blocks quickly. That suggests that
a thread or other process should read it frequently to allow new data to be
inserted. I do remember that there is some discussion on line - not
necessarily Python-specific - to be found.
On Thursday, May 4, 2023 at 12:11:50 AM UTC-4 Thomas Passin wrote:
> I don't know the answer but I know a thing or two that might be useful.
> By starting the command with "&", as you have, you avoid
> the proc.communicate() call, which would block Leo until it returns.
> However, if you try to capture output with a pipe, the pipe can block if it
> fills up, and that happens with a surprisingly small number of bytes. Then
> you block because the pipe is waiting to be cleared, and you are probably
> waiting to get output from the pipe.
>
> So that won't work.
>
> The way I managed this with the background process manager (BPM) - it's
> what launches pylint, mypy, etc - is to use a separate thread to launch the
> process, and call proc.communicate() in that thread. The thread blocks
> waiting for the process to complete, but that doesn't matter since it
> doesn't block Leo itself. This leaves the question of how to get the
> results. The thread puts the results into a variable in the BPM, and the
> BPM registers a callback at idle time. The callback checks to see if the
> variable is not None. If the BPM sees that there is a result in that
> variable, it cancels the callback and makes use of the result. Then it
> writes the messages to the log pane. You have to use a lock to make sure
> that the external process's results are put atomically into the variable.
> In your case I might have it write the results to a different tab to avoid
> getting mixed up with regular log output. I'm going by memory here and
> might have a detail or two inaccurate.
>
> So study the BPM and see if you could do something similar with your
> code. The BPM makes sure that it only ever runs one process at a time,
> BTW. That avoids the possibility of getting intermingled results or other
> confusions.
>
> Another possibility is to have your external process redirect output to a
> known file. You could check periodically to see if that file has gotten
> new data and read the new data if so. The problem there might be to know
> when the external process is finished writing data. You could also hget a
> race condition between the writing process and the reading process, so you
> would have to figure out how to handle that situation. I don't happen to
> know how that works and it might be different between Windows and Linux.
>
> Another possibility would be to launch your external program from a
> batch/shell file and have the batch/shell file redirect the output to a
> known file. When the programs finishes, the batch file could set an
> environmental variable. You could do the idle-time callback trick to check
> for that variable, or you could have the checks done periodically by a
> separate thread. That thread could even read the results, format them, and
> write them to a new tab or pass them on to another batch file.
>
> I would say the last possibility sounds the most promising to me.
>
>
>
> On Wednesday, May 3, 2023 at 7:02:47 PM UTC-4 gates...@gmail.com wrote:
>
>> Hi all,
>>
>> I have an outline with some @button nodes that fire off external shell
>> commands using g.execute_shell_commands. I tend to run Leo *without* a
>> console window open, so anything those @button-triggered shell commands
>> print to the console is lost to me. Is there an easy way/straightforward
>> way to redirect to the stdout/stderr for those commands only to the log
>> pane? I'm struggling to figure it out by manipulating sys.stdout and
>> sys.stderr, though I admit this is far from my area of specialty.
>>
>> An example of a button:
>>
>> ```
>> @language python
>> @tabwidth -2
>>
>> ''' Executes the currently selected sketch with the 'vsk run' command.
>> '''
>>
>> import os.path
>>
>> fn = c.getNodeFileName(p)
>> if len(fn) == 0:
>> # non-external file selected
>> g.es('Not a valid sketch node.', color='red')
>>
>> else:
>> sketch_path = os.path.dirname(fn)
>> g.execute_shell_commands(f'&vsk run {sketch_path}')
>> ```
>>
>> That command is long-running (i.e. it stays open in another pane, thus
>> the '&' to unblock the Leo GUI). Sometimes it indicates errors on the
>> console output, which just plain doesn't exist when I'm running Leo without
>> a console window open.
>>
>> Ideally any solution would be cross-platform, since I do this on both
>> Windows and Linux... but really I'd be happy with platform-specific
>> solutions too. I'd really rather not have to have a console window open to
>> see these errors if at all possible.
>>
>> Thanks in advance for any ide
