Re: system() delay

2016-08-29 Fir de Conversatie Ramel Eshed 
On Monday, August 29, 2016 at 8:35:33 AM UTC+3, Dominique Pelle wrote:
> Ramel  wrote:
> 
> > Nice catch!
> > My 'shell' was tcsh and after changing it to /bin/sh (=bash) System() works
> > much better (only ~x2 slower than Job()). I probably wouldn't get into this
> > if I got these results in the first place.. still, I'm going to use the
> > above blocking version of job_start (it is as fast as Job()) since in RHEL
> > everything is slower by a factor of 10(!) anyway.
> >
> > Thanks,
> > Ramel
> 
> Hi Ramel
> 
> Can you provide the timings with at least 3 runs as I did on RHEL?
> 
> And can you attach the output of the strace command on RHEL?
> I.e. run:
> 
> $ strace -r -f -osystem.trace time vim -u NONE -N -S system.vim  -c
> 'call System()|q'
> 
> $ strace -r -f -ojob.trace time vim -u NONE -N -S system.vim  -c
> 'call Job()|q'
> 
> And attach files jobs.trace and system.trace.  Please also indicate
> which version of vim you used.
> 
> Thanks
> Dominique
> 
> PS: please use bottom-post in vim-dev.

Hi Dominique,

Sorry, but the RHEL is at my work and the trace files are relatively big and 
contain confidential details..
You can see the timings below.

Thanks,
Ramel

RHEL 5.5 (using shell=bash)

Job():
aaa:   0.012400
aaa:   0.011879
aaa:   0.012227

System():
aaa:   0.053787
aaa:   0.054384
aaa:   0.053694

Ubuntu 16.04:

Job():
aaa:   0.001710
aaa:   0.001692
aaa:   0.001673

System():
aaa:   0.010966
aaa:   0.010938
aaa:   0.010940

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-29 Fir de Conversatie Bram Moolenaar 

Ramel Eshed wrote:

> On Sunday, August 28, 2016 at 11:36:33 PM UTC+3, Bram Moolenaar wrote:
> > Ramel Eshed wrote:
> > 
> > > Of course, I've measured the time from job_start to close_cb. You can see
> > > this comparison in the example I've attached earlier in this thread.
> > 
> > Hmm, it's possible that detecting that the other end closed the pipe
> > happens much sooner than detecting that the child process has ended.
> > If that is the case I don't think there is a way around that.
> > 
> > 
> > -- 
> > hundred-and-one symptoms of being an internet addict:
> > 96. On Super Bowl Sunday, you followed the score by going to the
> > Yahoo main page instead of turning on the TV.
> > 
> >  /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
> > ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> > \\\  an exciting new programming language -- http://www.Zimbu.org///
> >  \\\help me help AIDS victims -- http://ICCF-Holland.org///
> 
> Well, I didn't think something like this could work but it did:
> 
> func! System(cmd)
> let out = []
> let job = job_start(['/bin/sh', '-c', a:cmd],
> \ {'out_cb': {c, msg -> add(out, msg)}})
> let ch = job_getchannel(job)
> while ch_status(ch) != 'closed'
> sleep 10m
> endwhile
> return out
> endfunc
> 
> Maybe it should be done the same way in the C system() implementation...

This requires more investigation.  There is also code to still read from
the pipe after the child process exits, thus it is possible that not
being able to read is sufficient to stop waiting for the child.  Except
when we want the exit code.

-- 
hundred-and-one symptoms of being an internet addict:
99. The hum of a cooling fan and the click of keys is comforting to you.

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-28 Fir de Conversatie Dominique Pellé 
Ramel  wrote:

> Nice catch!
> My 'shell' was tcsh and after changing it to /bin/sh (=bash) System() works
> much better (only ~x2 slower than Job()). I probably wouldn't get into this
> if I got these results in the first place.. still, I'm going to use the
> above blocking version of job_start (it is as fast as Job()) since in RHEL
> everything is slower by a factor of 10(!) anyway.
>
> Thanks,
> Ramel

Hi Ramel

Can you provide the timings with at least 3 runs as I did on RHEL?

And can you attach the output of the strace command on RHEL?
I.e. run:

$ strace -r -f -osystem.trace time vim -u NONE -N -S system.vim  -c
'call System()|q'

$ strace -r -f -ojob.trace time vim -u NONE -N -S system.vim  -c
'call Job()|q'

And attach files jobs.trace and system.trace.  Please also indicate
which version of vim you used.

Thanks
Dominique

PS: please use bottom-post in vim-dev.

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-28 Fir de Conversatie Ramel 
Nice catch!
My 'shell' was tcsh and after changing it to /bin/sh (=bash) System() works
much better (only ~x2 slower than Job()). I probably wouldn't get into this
if I got these results in the first place.. still, I'm going to use the
above blocking version of job_start (it is as fast as Job()) since in RHEL
everything is slower by a factor of 10(!) anyway.

Thanks,
Ramel

2016-08-29 2:35 GMT+03:00 Dominique Pellé :

> Ramel Eshed  wrote:
>
> > My problem is not with the channel, the problem is
> > that system() is slower than the equivalent job_start().
> > You can run and compare the results of Job() and System()
> > functions in the attached file. They are both doing the same
> > thing ('echo aaa') but System() is ~5 times slower than Job()
> > on Ubuntu and x10 slower on RHEL5.5. On RHEL
> > System() is extremely slow, it takes about 0.2 seconds
> > which cause Vim to hang.
>
> I just tried the script system.vim which you attached
> and measured x3 times on my laptop x86_64
> (xubuntu-14.04) using Vim-7.4.2290:
>
> :so system.vim
>
> :call Job()
> aaa:   0.004798
> :call Job()
> aaa:   0.002810
> call Job()
> aaa:   0.002988
>
> :call System()
> aaa:   0.007809
> :call System()
> aaa:   0.007768
> :call System()
> aaa:   0.007858
>
> So for me, System() is also slower than Job().
> However, the difference on my machine is not
> as large as yours.  I don't see system()
> taking 0.2 sec as you see on RHEL.
>
> I tried this with strace:
>
> $ strace -r -f -osystem.trace time ./vim -u NONE -N -S system.vim  -c
> 'call System()|q'
> $ strace -r -f -ojob.trace time ./vim -u NONE -N -S system.vim  -c
> 'call Job()|q'
>
> Looking at log files jobs.trace and system.trace,
> I saw that:
>
> In system.trace
>
> 8756  execve("/bin/bash", ["/bin/bash", "-c", "(echo aaa)
> >/tmp/vwA2O35/0 2>&1"], [/* 69 vars */] 
>
> In job.trace:
>
> 8773  execve("/bin/sh", ["/bin/sh", "-c", "echo aaa"], [/* 72 vars */]
> 
>
> Ah, it's not using the same shell! /bin/sh is a symlink
> to /bin/dash on my system, which is a simpler shell
> than /bin/bash.   system() also runs the command in a
> subshell which may make it slower.
>
> I then added this line in your script system.vim:
>
> set shell=/bin/sh
>
> And now system() is faster, still not as fast as Job() but
> getting closer:
>
> :call System()
> aaa:   0.004737
> :call System()
> aaa:   0.004920
> :call System()
> aaa:   0.004718
>
> What is the value of :echo   for you?
> Can you try with set shell=/bin/sh ?
>
> Here is my modified system.vim:
>
> ===
> $ cat system.vim
>
> set shell=/bin/sh
> func! Job()
> let g:output = []
> let s:rt = reltime()
> let g:job = job_start(['/bin/sh', '-c', 'echo aaa'], {'out_cb':
> function('s:out_cb'), 'close_cb': function('s:close_cb')})
> endfunc
>
> func! s:out_cb(channel, msg)
> call add(g:output, a:msg)
> endfunc
>
> func! s:close_cb(channel)
> echo g:output[0] . ': ' . reltimestr(reltime(s:rt))
> endfunc
>
> " compare with:
>
> func! System()
> let s:rt = reltime()
> let g:output = systemlist('echo aaa')
> echo g:output[0] . ': ' . reltimestr(reltime(s:rt))
> endfunc
> ===
>
> Regards
> Dominique
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "vim_dev" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/vim_dev/PmNEtGLv6J0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> vim_dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-28 Fir de Conversatie Dominique Pellé 
Ramel Eshed  wrote:

> My problem is not with the channel, the problem is
> that system() is slower than the equivalent job_start().
> You can run and compare the results of Job() and System()
> functions in the attached file. They are both doing the same
> thing ('echo aaa') but System() is ~5 times slower than Job()
> on Ubuntu and x10 slower on RHEL5.5. On RHEL
> System() is extremely slow, it takes about 0.2 seconds
> which cause Vim to hang.

I just tried the script system.vim which you attached
and measured x3 times on my laptop x86_64
(xubuntu-14.04) using Vim-7.4.2290:

:so system.vim

:call Job()
aaa:   0.004798
:call Job()
aaa:   0.002810
call Job()
aaa:   0.002988

:call System()
aaa:   0.007809
:call System()
aaa:   0.007768
:call System()
aaa:   0.007858

So for me, System() is also slower than Job().
However, the difference on my machine is not
as large as yours.  I don't see system()
taking 0.2 sec as you see on RHEL.

I tried this with strace:

$ strace -r -f -osystem.trace time ./vim -u NONE -N -S system.vim  -c
'call System()|q'
$ strace -r -f -ojob.trace time ./vim -u NONE -N -S system.vim  -c
'call Job()|q'

Looking at log files jobs.trace and system.trace,
I saw that:

In system.trace

8756  execve("/bin/bash", ["/bin/bash", "-c", "(echo aaa)
>/tmp/vwA2O35/0 2>&1"], [/* 69 vars */] 

In job.trace:

8773  execve("/bin/sh", ["/bin/sh", "-c", "echo aaa"], [/* 72 vars */]


Ah, it's not using the same shell! /bin/sh is a symlink
to /bin/dash on my system, which is a simpler shell
than /bin/bash.   system() also runs the command in a
subshell which may make it slower.

I then added this line in your script system.vim:

set shell=/bin/sh

And now system() is faster, still not as fast as Job() but
getting closer:

:call System()
aaa:   0.004737
:call System()
aaa:   0.004920
:call System()
aaa:   0.004718

What is the value of :echo   for you?
Can you try with set shell=/bin/sh ?

Here is my modified system.vim:

===
$ cat system.vim

set shell=/bin/sh
func! Job()
let g:output = []
let s:rt = reltime()
let g:job = job_start(['/bin/sh', '-c', 'echo aaa'], {'out_cb':
function('s:out_cb'), 'close_cb': function('s:close_cb')})
endfunc

func! s:out_cb(channel, msg)
call add(g:output, a:msg)
endfunc

func! s:close_cb(channel)
echo g:output[0] . ': ' . reltimestr(reltime(s:rt))
endfunc

" compare with:

func! System()
let s:rt = reltime()
let g:output = systemlist('echo aaa')
echo g:output[0] . ': ' . reltimestr(reltime(s:rt))
endfunc
===

Regards
Dominique

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-28 Fir de Conversatie Ramel Eshed 
On Sunday, August 28, 2016 at 11:36:33 PM UTC+3, Bram Moolenaar wrote:
> Ramel Eshed wrote:
> 
> > Of course, I've measured the time from job_start to close_cb. You can see
> > this comparison in the example I've attached earlier in this thread.
> 
> Hmm, it's possible that detecting that the other end closed the pipe
> happens much sooner than detecting that the child process has ended.
> If that is the case I don't think there is a way around that.
> 
> 
> -- 
> hundred-and-one symptoms of being an internet addict:
> 96. On Super Bowl Sunday, you followed the score by going to the
> Yahoo main page instead of turning on the TV.
> 
>  /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
> ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\  an exciting new programming language -- http://www.Zimbu.org///
>  \\\help me help AIDS victims -- http://ICCF-Holland.org///

Well, I didn't think something like this could work but it did:

func! System(cmd)
let out = []
let job = job_start(['/bin/sh', '-c', a:cmd],
\ {'out_cb': {c, msg -> add(out, msg)}})
let ch = job_getchannel(job)
while ch_status(ch) != 'closed'
sleep 10m
endwhile
return out
endfunc

Maybe it should be done the same way in the C system() implementation...

Thanks anyway,
Ramel

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-28 Fir de Conversatie Bram Moolenaar 

Ramel Eshed wrote:

> Of course, I've measured the time from job_start to close_cb. You can see
> this comparison in the example I've attached earlier in this thread.

Hmm, it's possible that detecting that the other end closed the pipe
happens much sooner than detecting that the child process has ended.
If that is the case I don't think there is a way around that.


-- 
hundred-and-one symptoms of being an internet addict:
96. On Super Bowl Sunday, you followed the score by going to the
Yahoo main page instead of turning on the TV.

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-28 Fir de Conversatie Ramel 
Of course, I've measured the time from job_start to close_cb. You can see
this comparison in the example I've attached earlier in this thread.

On Aug 28, 2016 10:01 PM, "Bram Moolenaar"  wrote:

>
> Ramel Eshed wrote:
>
> > On Sunday, August 28, 2016 at 5:53:33 PM UTC+3, Bram Moolenaar wrote:
> > > Ramel Eshed wrote:
> > >
> > > > On Friday, August 26, 2016 at 7:29:58 PM UTC+3, Bram Moolenaar wrote:
> > > > > Ramel Eshed wrote:
> > > > >
> > > > > > On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar
> wrote:
> > > > > > > Ramel Eshed wrote:
> > > > > > >
> > > > > > > > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel
> Eshed wrote:
> > > > > > > > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony
> Mechelynck wrote:
> > > > > > > > > > There are other factors which are right there in the
> help:
> > > > > > > > > > - job_start() returns a Job object and doesn't wait for
> the job to finish
> > > > > > > > > > - system() waits for the external command to finish and
> returns its
> > > > > > > > > > full stdout output as a string.
> > > > > > > > > >
> > > > > > > > > > I don't know Vim job control really well, but I seem to
> understand
> > > > > > > > > > that in order to compare system() timing and job control
> timing you
> > > > > > > > > > would have to set up callbacks to gather any output from
> the channel,
> > > > > > > > > > and a callback to be called when the job ends (it may
> still write to
> > > > > > > > > > stdout after it exits), and measure the time from just
> before
> > > > > > > > > > job_start() to just after making sure that all output
> has been
> > > > > > > > > > collected and that the job has ended.
> > > > > > > > > >
> > > > > > > > > > You might even, for testing purposes, try to write a
> System() user
> > > > > > > > > > function to invoke the argument as a job and return its
> output as a
> > > > > > > > > > string, with the disadvantage that you would completely
> lose job
> > > > > > > > > > control asynchronism. But it would allow you a better
> comparison,
> > > > > > > > > > namely between old-fashioned system() and this new
> job-control-based
> > > > > > > > > > System().
> > > > > > > > > >
> > > > > > > > > > Best regards,
> > > > > > > > > > Tony.
> > > > > > > > >
> > > > > > > > > That's exactly what I've did:
> > > > > > > > >
> > > > > > > > > func! Job()
> > > > > > > > > let s:rt = reltime()
> > > > > > > > > let g:output = []
> > > > > > > > > let g:job = job_start(['/bin/sh', '-c', 'cat ' .
> expand('%')], {'out_cb': function('s:out_cb'), 'close_cb':
> function('s:close_cb')})
> > > > > > > > > endfunc
> > > > > > > > >
> > > > > > > > > func! s:out_cb(channel, msg)
> > > > > > > > > call add(g:output, a:msg)
> > > > > > > > > endfunc
> > > > > > > > >
> > > > > > > > > func! s:close_cb(channel)
> > > > > > > > > echo reltimestr(reltime(s:rt))
> > > > > > > > > "echo g:output
> > > > > > > > > endfunc
> > > > > > > > >
> > > > > > > > > " compare with:
> > > > > > > > >
> > > > > > > > > func! System()
> > > > > > > > > let s:rt = reltime()
> > > > > > > > > let g:output = systemlist('cat ' . expand('%'))
> > > > > > > > > echo reltimestr(reltime(s:rt))
> > > > > > > > > endfunc
> > > > > > > > >
> > > > > > > > > "
> > > > > > > > > I checked it now on my Ubuntu at home - I still see the
> difference but now both are much faster so even the system() delay is not
> noticeable. Any idea what could cause the delay on my RHEL?
> > > > > > > > >
> > > > > > > > > BTW, if I remove the comment from the 'echo g:output' line
> in close_cb() the message is not displayed (actually, it depends on which
> command is running. for the 'cat' command above -there is no message).
> > > > > > > >
> > > > > > > > Hi Bram,
> > > > > > > >
> > > > > > > > Is it possible to make system() work the same way job_start
> (with a
> > > > > > > > shell) does?  As I mentioned, I get a noticeable delay with
> system()
> > > > > > > > which I don't get when using job_start(['/bin/sh'...).
> > > > > > >
> > > > > > > Not sure if there is anything to improve.  Would require
> finding out why
> > > > > > > it's slower and whether that can be fixed.
> > > > > > >
> > > > > > > You could add channel log commands in the code in various
> places to see
> > > > > > > what happens.
> > > > > >
> > > > > > My problem is not with the channel, the problem is that system()
> is
> > > > > > slower than the equivalent job_start(). You can run and compare
> the
> > > > > > results of Job() and System() functions in the attached file.
> They are
> > > > > > both doing the same thing ('echo aaa') but System() is ~5 times
> slower
> > > > > > than Job() on Ubuntu and x10 slower on RHEL5.5. On RHEL System()
> is
> > > > > > extremely slow, it takes about 0.2 seconds which cause Vim to
> hang.
> > > > >
> > > > > I understand that.  The ch_log() and other functions cn be used
> anywhere
> > > > > in the Vim

Re: system() delay

2016-08-28 Fir de Conversatie Bram Moolenaar 

Ramel Eshed wrote:

> On Sunday, August 28, 2016 at 5:53:33 PM UTC+3, Bram Moolenaar wrote:
> > Ramel Eshed wrote:
> > 
> > > On Friday, August 26, 2016 at 7:29:58 PM UTC+3, Bram Moolenaar wrote:
> > > > Ramel Eshed wrote:
> > > > 
> > > > > On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar wrote:
> > > > > > Ramel Eshed wrote:
> > > > > > 
> > > > > > > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed 
> > > > > > > wrote:
> > > > > > > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony 
> > > > > > > > Mechelynck wrote:
> > > > > > > > > There are other factors which are right there in the help:
> > > > > > > > > - job_start() returns a Job object and doesn't wait for the 
> > > > > > > > > job to finish
> > > > > > > > > - system() waits for the external command to finish and 
> > > > > > > > > returns its
> > > > > > > > > full stdout output as a string.
> > > > > > > > > 
> > > > > > > > > I don't know Vim job control really well, but I seem to 
> > > > > > > > > understand
> > > > > > > > > that in order to compare system() timing and job control 
> > > > > > > > > timing you
> > > > > > > > > would have to set up callbacks to gather any output from the 
> > > > > > > > > channel,
> > > > > > > > > and a callback to be called when the job ends (it may still 
> > > > > > > > > write to
> > > > > > > > > stdout after it exits), and measure the time from just before
> > > > > > > > > job_start() to just after making sure that all output has been
> > > > > > > > > collected and that the job has ended.
> > > > > > > > > 
> > > > > > > > > You might even, for testing purposes, try to write a System() 
> > > > > > > > > user
> > > > > > > > > function to invoke the argument as a job and return its 
> > > > > > > > > output as a
> > > > > > > > > string, with the disadvantage that you would completely lose 
> > > > > > > > > job
> > > > > > > > > control asynchronism. But it would allow you a better 
> > > > > > > > > comparison,
> > > > > > > > > namely between old-fashioned system() and this new 
> > > > > > > > > job-control-based
> > > > > > > > > System().
> > > > > > > > > 
> > > > > > > > > Best regards,
> > > > > > > > > Tony.
> > > > > > > > 
> > > > > > > > That's exactly what I've did:
> > > > > > > > 
> > > > > > > > func! Job()
> > > > > > > > let s:rt = reltime()
> > > > > > > > let g:output = []
> > > > > > > > let g:job = job_start(['/bin/sh', '-c', 'cat ' . 
> > > > > > > > expand('%')], {'out_cb': function('s:out_cb'), 'close_cb': 
> > > > > > > > function('s:close_cb')})
> > > > > > > > endfunc
> > > > > > > > 
> > > > > > > > func! s:out_cb(channel, msg)
> > > > > > > > call add(g:output, a:msg)
> > > > > > > > endfunc
> > > > > > > > 
> > > > > > > > func! s:close_cb(channel)
> > > > > > > > echo reltimestr(reltime(s:rt))
> > > > > > > > "echo g:output
> > > > > > > > endfunc
> > > > > > > > 
> > > > > > > > " compare with:
> > > > > > > > 
> > > > > > > > func! System()
> > > > > > > > let s:rt = reltime()
> > > > > > > > let g:output = systemlist('cat ' . expand('%'))
> > > > > > > > echo reltimestr(reltime(s:rt))
> > > > > > > > endfunc
> > > > > > > > 
> > > > > > > > "
> > > > > > > > I checked it now on my Ubuntu at home - I still see the 
> > > > > > > > difference but now both are much faster so even the system() 
> > > > > > > > delay is not noticeable. Any idea what could cause the delay on 
> > > > > > > > my RHEL?
> > > > > > > > 
> > > > > > > > BTW, if I remove the comment from the 'echo g:output' line in 
> > > > > > > > close_cb() the message is not displayed (actually, it depends 
> > > > > > > > on which command is running. for the 'cat' command above -there 
> > > > > > > > is no message).
> > > > > > > 
> > > > > > > Hi Bram,
> > > > > > > 
> > > > > > > Is it possible to make system() work the same way job_start (with 
> > > > > > > a
> > > > > > > shell) does?  As I mentioned, I get a noticeable delay with 
> > > > > > > system()
> > > > > > > which I don't get when using job_start(['/bin/sh'...).
> > > > > > 
> > > > > > Not sure if there is anything to improve.  Would require finding 
> > > > > > out why
> > > > > > it's slower and whether that can be fixed.
> > > > > > 
> > > > > > You could add channel log commands in the code in various places to 
> > > > > > see
> > > > > > what happens.
> > > > > 
> > > > > My problem is not with the channel, the problem is that system() is
> > > > > slower than the equivalent job_start(). You can run and compare the
> > > > > results of Job() and System() functions in the attached file. They are
> > > > > both doing the same thing ('echo aaa') but System() is ~5 times slower
> > > > > than Job() on Ubuntu and x10 slower on RHEL5.5. On RHEL System() is
> > > > > extremely slow, it takes about 0.2 seconds which cause Vim to hang.
> > > > 
> > > > I understand that.  The ch_log() and other functions cn be used anywhere
> > > > in the Vim code.

Re: system() delay

2016-08-28 Fir de Conversatie Ramel Eshed 
On Sunday, August 28, 2016 at 5:53:33 PM UTC+3, Bram Moolenaar wrote:
> Ramel Eshed wrote:
> 
> > On Friday, August 26, 2016 at 7:29:58 PM UTC+3, Bram Moolenaar wrote:
> > > Ramel Eshed wrote:
> > > 
> > > > On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar wrote:
> > > > > Ramel Eshed wrote:
> > > > > 
> > > > > > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed wrote:
> > > > > > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck 
> > > > > > > wrote:
> > > > > > > > There are other factors which are right there in the help:
> > > > > > > > - job_start() returns a Job object and doesn't wait for the job 
> > > > > > > > to finish
> > > > > > > > - system() waits for the external command to finish and returns 
> > > > > > > > its
> > > > > > > > full stdout output as a string.
> > > > > > > > 
> > > > > > > > I don't know Vim job control really well, but I seem to 
> > > > > > > > understand
> > > > > > > > that in order to compare system() timing and job control timing 
> > > > > > > > you
> > > > > > > > would have to set up callbacks to gather any output from the 
> > > > > > > > channel,
> > > > > > > > and a callback to be called when the job ends (it may still 
> > > > > > > > write to
> > > > > > > > stdout after it exits), and measure the time from just before
> > > > > > > > job_start() to just after making sure that all output has been
> > > > > > > > collected and that the job has ended.
> > > > > > > > 
> > > > > > > > You might even, for testing purposes, try to write a System() 
> > > > > > > > user
> > > > > > > > function to invoke the argument as a job and return its output 
> > > > > > > > as a
> > > > > > > > string, with the disadvantage that you would completely lose job
> > > > > > > > control asynchronism. But it would allow you a better 
> > > > > > > > comparison,
> > > > > > > > namely between old-fashioned system() and this new 
> > > > > > > > job-control-based
> > > > > > > > System().
> > > > > > > > 
> > > > > > > > Best regards,
> > > > > > > > Tony.
> > > > > > > 
> > > > > > > That's exactly what I've did:
> > > > > > > 
> > > > > > > func! Job()
> > > > > > > let s:rt = reltime()
> > > > > > > let g:output = []
> > > > > > > let g:job = job_start(['/bin/sh', '-c', 'cat ' . 
> > > > > > > expand('%')], {'out_cb': function('s:out_cb'), 'close_cb': 
> > > > > > > function('s:close_cb')})
> > > > > > > endfunc
> > > > > > > 
> > > > > > > func! s:out_cb(channel, msg)
> > > > > > > call add(g:output, a:msg)
> > > > > > > endfunc
> > > > > > > 
> > > > > > > func! s:close_cb(channel)
> > > > > > > echo reltimestr(reltime(s:rt))
> > > > > > > "echo g:output
> > > > > > > endfunc
> > > > > > > 
> > > > > > > " compare with:
> > > > > > > 
> > > > > > > func! System()
> > > > > > > let s:rt = reltime()
> > > > > > > let g:output = systemlist('cat ' . expand('%'))
> > > > > > > echo reltimestr(reltime(s:rt))
> > > > > > > endfunc
> > > > > > > 
> > > > > > > "
> > > > > > > I checked it now on my Ubuntu at home - I still see the 
> > > > > > > difference but now both are much faster so even the system() 
> > > > > > > delay is not noticeable. Any idea what could cause the delay on 
> > > > > > > my RHEL?
> > > > > > > 
> > > > > > > BTW, if I remove the comment from the 'echo g:output' line in 
> > > > > > > close_cb() the message is not displayed (actually, it depends on 
> > > > > > > which command is running. for the 'cat' command above -there is 
> > > > > > > no message).
> > > > > > 
> > > > > > Hi Bram,
> > > > > > 
> > > > > > Is it possible to make system() work the same way job_start (with a
> > > > > > shell) does?  As I mentioned, I get a noticeable delay with system()
> > > > > > which I don't get when using job_start(['/bin/sh'...).
> > > > > 
> > > > > Not sure if there is anything to improve.  Would require finding out 
> > > > > why
> > > > > it's slower and whether that can be fixed.
> > > > > 
> > > > > You could add channel log commands in the code in various places to 
> > > > > see
> > > > > what happens.
> > > > 
> > > > My problem is not with the channel, the problem is that system() is
> > > > slower than the equivalent job_start(). You can run and compare the
> > > > results of Job() and System() functions in the attached file. They are
> > > > both doing the same thing ('echo aaa') but System() is ~5 times slower
> > > > than Job() on Ubuntu and x10 slower on RHEL5.5. On RHEL System() is
> > > > extremely slow, it takes about 0.2 seconds which cause Vim to hang.
> > > 
> > > I understand that.  The ch_log() and other functions cn be used anywhere
> > > in the Vim code.  That is in the C code, not script.  So it requires
> > > adding the log calls in several places in and around the system()
> > > implementation to find out where it spends time.  This requires building
> > > Vim from the modified source.
> > > 
> > > -- 
> > > hundred-and-one symptoms of being an

Re: system() delay

2016-08-28 Fir de Conversatie Bram Moolenaar 

Ramel Eshed wrote:

> On Friday, August 26, 2016 at 7:29:58 PM UTC+3, Bram Moolenaar wrote:
> > Ramel Eshed wrote:
> > 
> > > On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar wrote:
> > > > Ramel Eshed wrote:
> > > > 
> > > > > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed wrote:
> > > > > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck 
> > > > > > wrote:
> > > > > > > There are other factors which are right there in the help:
> > > > > > > - job_start() returns a Job object and doesn't wait for the job 
> > > > > > > to finish
> > > > > > > - system() waits for the external command to finish and returns 
> > > > > > > its
> > > > > > > full stdout output as a string.
> > > > > > > 
> > > > > > > I don't know Vim job control really well, but I seem to understand
> > > > > > > that in order to compare system() timing and job control timing 
> > > > > > > you
> > > > > > > would have to set up callbacks to gather any output from the 
> > > > > > > channel,
> > > > > > > and a callback to be called when the job ends (it may still write 
> > > > > > > to
> > > > > > > stdout after it exits), and measure the time from just before
> > > > > > > job_start() to just after making sure that all output has been
> > > > > > > collected and that the job has ended.
> > > > > > > 
> > > > > > > You might even, for testing purposes, try to write a System() user
> > > > > > > function to invoke the argument as a job and return its output as 
> > > > > > > a
> > > > > > > string, with the disadvantage that you would completely lose job
> > > > > > > control asynchronism. But it would allow you a better comparison,
> > > > > > > namely between old-fashioned system() and this new 
> > > > > > > job-control-based
> > > > > > > System().
> > > > > > > 
> > > > > > > Best regards,
> > > > > > > Tony.
> > > > > > 
> > > > > > That's exactly what I've did:
> > > > > > 
> > > > > > func! Job()
> > > > > > let s:rt = reltime()
> > > > > > let g:output = []
> > > > > > let g:job = job_start(['/bin/sh', '-c', 'cat ' . expand('%')], 
> > > > > > {'out_cb': function('s:out_cb'), 'close_cb': 
> > > > > > function('s:close_cb')})
> > > > > > endfunc
> > > > > > 
> > > > > > func! s:out_cb(channel, msg)
> > > > > > call add(g:output, a:msg)
> > > > > > endfunc
> > > > > > 
> > > > > > func! s:close_cb(channel)
> > > > > > echo reltimestr(reltime(s:rt))
> > > > > > "echo g:output
> > > > > > endfunc
> > > > > > 
> > > > > > " compare with:
> > > > > > 
> > > > > > func! System()
> > > > > > let s:rt = reltime()
> > > > > > let g:output = systemlist('cat ' . expand('%'))
> > > > > > echo reltimestr(reltime(s:rt))
> > > > > > endfunc
> > > > > > 
> > > > > > "
> > > > > > I checked it now on my Ubuntu at home - I still see the difference 
> > > > > > but now both are much faster so even the system() delay is not 
> > > > > > noticeable. Any idea what could cause the delay on my RHEL?
> > > > > > 
> > > > > > BTW, if I remove the comment from the 'echo g:output' line in 
> > > > > > close_cb() the message is not displayed (actually, it depends on 
> > > > > > which command is running. for the 'cat' command above -there is no 
> > > > > > message).
> > > > > 
> > > > > Hi Bram,
> > > > > 
> > > > > Is it possible to make system() work the same way job_start (with a
> > > > > shell) does?  As I mentioned, I get a noticeable delay with system()
> > > > > which I don't get when using job_start(['/bin/sh'...).
> > > > 
> > > > Not sure if there is anything to improve.  Would require finding out why
> > > > it's slower and whether that can be fixed.
> > > > 
> > > > You could add channel log commands in the code in various places to see
> > > > what happens.
> > > 
> > > My problem is not with the channel, the problem is that system() is
> > > slower than the equivalent job_start(). You can run and compare the
> > > results of Job() and System() functions in the attached file. They are
> > > both doing the same thing ('echo aaa') but System() is ~5 times slower
> > > than Job() on Ubuntu and x10 slower on RHEL5.5. On RHEL System() is
> > > extremely slow, it takes about 0.2 seconds which cause Vim to hang.
> > 
> > I understand that.  The ch_log() and other functions cn be used anywhere
> > in the Vim code.  That is in the C code, not script.  So it requires
> > adding the log calls in several places in and around the system()
> > implementation to find out where it spends time.  This requires building
> > Vim from the modified source.
> > 
> > -- 
> > hundred-and-one symptoms of being an internet addict:
> > 91. It's Saturday afternoon in the middle of May and you
> > are on computer.
> > 
> >  /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
> > ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> > \\\  an exciting new programming language -- http://www.Zimbu.org///
> >  \\\

Re: system() delay

2016-08-27 Fir de Conversatie Ramel Eshed 
On Friday, August 26, 2016 at 7:29:58 PM UTC+3, Bram Moolenaar wrote:
> Ramel Eshed wrote:
> 
> > On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar wrote:
> > > Ramel Eshed wrote:
> > > 
> > > > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed wrote:
> > > > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck 
> > > > > wrote:
> > > > > > There are other factors which are right there in the help:
> > > > > > - job_start() returns a Job object and doesn't wait for the job to 
> > > > > > finish
> > > > > > - system() waits for the external command to finish and returns its
> > > > > > full stdout output as a string.
> > > > > > 
> > > > > > I don't know Vim job control really well, but I seem to understand
> > > > > > that in order to compare system() timing and job control timing you
> > > > > > would have to set up callbacks to gather any output from the 
> > > > > > channel,
> > > > > > and a callback to be called when the job ends (it may still write to
> > > > > > stdout after it exits), and measure the time from just before
> > > > > > job_start() to just after making sure that all output has been
> > > > > > collected and that the job has ended.
> > > > > > 
> > > > > > You might even, for testing purposes, try to write a System() user
> > > > > > function to invoke the argument as a job and return its output as a
> > > > > > string, with the disadvantage that you would completely lose job
> > > > > > control asynchronism. But it would allow you a better comparison,
> > > > > > namely between old-fashioned system() and this new job-control-based
> > > > > > System().
> > > > > > 
> > > > > > Best regards,
> > > > > > Tony.
> > > > > 
> > > > > That's exactly what I've did:
> > > > > 
> > > > > func! Job()
> > > > > let s:rt = reltime()
> > > > > let g:output = []
> > > > > let g:job = job_start(['/bin/sh', '-c', 'cat ' . expand('%')], 
> > > > > {'out_cb': function('s:out_cb'), 'close_cb': function('s:close_cb')})
> > > > > endfunc
> > > > > 
> > > > > func! s:out_cb(channel, msg)
> > > > > call add(g:output, a:msg)
> > > > > endfunc
> > > > > 
> > > > > func! s:close_cb(channel)
> > > > > echo reltimestr(reltime(s:rt))
> > > > > "echo g:output
> > > > > endfunc
> > > > > 
> > > > > " compare with:
> > > > > 
> > > > > func! System()
> > > > > let s:rt = reltime()
> > > > > let g:output = systemlist('cat ' . expand('%'))
> > > > > echo reltimestr(reltime(s:rt))
> > > > > endfunc
> > > > > 
> > > > > "
> > > > > I checked it now on my Ubuntu at home - I still see the difference 
> > > > > but now both are much faster so even the system() delay is not 
> > > > > noticeable. Any idea what could cause the delay on my RHEL?
> > > > > 
> > > > > BTW, if I remove the comment from the 'echo g:output' line in 
> > > > > close_cb() the message is not displayed (actually, it depends on 
> > > > > which command is running. for the 'cat' command above -there is no 
> > > > > message).
> > > > 
> > > > Hi Bram,
> > > > 
> > > > Is it possible to make system() work the same way job_start (with a
> > > > shell) does?  As I mentioned, I get a noticeable delay with system()
> > > > which I don't get when using job_start(['/bin/sh'...).
> > > 
> > > Not sure if there is anything to improve.  Would require finding out why
> > > it's slower and whether that can be fixed.
> > > 
> > > You could add channel log commands in the code in various places to see
> > > what happens.
> > 
> > My problem is not with the channel, the problem is that system() is
> > slower than the equivalent job_start(). You can run and compare the
> > results of Job() and System() functions in the attached file. They are
> > both doing the same thing ('echo aaa') but System() is ~5 times slower
> > than Job() on Ubuntu and x10 slower on RHEL5.5. On RHEL System() is
> > extremely slow, it takes about 0.2 seconds which cause Vim to hang.
> 
> I understand that.  The ch_log() and other functions cn be used anywhere
> in the Vim code.  That is in the C code, not script.  So it requires
> adding the log calls in several places in and around the system()
> implementation to find out where it spends time.  This requires building
> Vim from the modified source.
> 
> -- 
> hundred-and-one symptoms of being an internet addict:
> 91. It's Saturday afternoon in the middle of May and you
> are on computer.
> 
>  /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
> ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\  an exciting new programming language -- http://www.Zimbu.org///
>  \\\help me help AIDS victims -- http://ICCF-Holland.org///

It looks like the delay is coming from the mch_delay(10L, TRUE) call inside the 
for loop starting at line 4936 in os_unix.c:

/*
 * Similar to the loop above, but only handle X events, no
 * I/O.

Re: system() delay

2016-08-26 Fir de Conversatie Bram Moolenaar 

Ramel Eshed wrote:

> On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar wrote:
> > Ramel Eshed wrote:
> > 
> > > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed wrote:
> > > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck wrote:
> > > > > There are other factors which are right there in the help:
> > > > > - job_start() returns a Job object and doesn't wait for the job to 
> > > > > finish
> > > > > - system() waits for the external command to finish and returns its
> > > > > full stdout output as a string.
> > > > > 
> > > > > I don't know Vim job control really well, but I seem to understand
> > > > > that in order to compare system() timing and job control timing you
> > > > > would have to set up callbacks to gather any output from the channel,
> > > > > and a callback to be called when the job ends (it may still write to
> > > > > stdout after it exits), and measure the time from just before
> > > > > job_start() to just after making sure that all output has been
> > > > > collected and that the job has ended.
> > > > > 
> > > > > You might even, for testing purposes, try to write a System() user
> > > > > function to invoke the argument as a job and return its output as a
> > > > > string, with the disadvantage that you would completely lose job
> > > > > control asynchronism. But it would allow you a better comparison,
> > > > > namely between old-fashioned system() and this new job-control-based
> > > > > System().
> > > > > 
> > > > > Best regards,
> > > > > Tony.
> > > > 
> > > > That's exactly what I've did:
> > > > 
> > > > func! Job()
> > > > let s:rt = reltime()
> > > > let g:output = []
> > > > let g:job = job_start(['/bin/sh', '-c', 'cat ' . expand('%')], 
> > > > {'out_cb': function('s:out_cb'), 'close_cb': function('s:close_cb')})
> > > > endfunc
> > > > 
> > > > func! s:out_cb(channel, msg)
> > > > call add(g:output, a:msg)
> > > > endfunc
> > > > 
> > > > func! s:close_cb(channel)
> > > > echo reltimestr(reltime(s:rt))
> > > > "echo g:output
> > > > endfunc
> > > > 
> > > > " compare with:
> > > > 
> > > > func! System()
> > > > let s:rt = reltime()
> > > > let g:output = systemlist('cat ' . expand('%'))
> > > > echo reltimestr(reltime(s:rt))
> > > > endfunc
> > > > 
> > > > "
> > > > I checked it now on my Ubuntu at home - I still see the difference but 
> > > > now both are much faster so even the system() delay is not noticeable. 
> > > > Any idea what could cause the delay on my RHEL?
> > > > 
> > > > BTW, if I remove the comment from the 'echo g:output' line in 
> > > > close_cb() the message is not displayed (actually, it depends on which 
> > > > command is running. for the 'cat' command above -there is no message).
> > > 
> > > Hi Bram,
> > > 
> > > Is it possible to make system() work the same way job_start (with a
> > > shell) does?  As I mentioned, I get a noticeable delay with system()
> > > which I don't get when using job_start(['/bin/sh'...).
> > 
> > Not sure if there is anything to improve.  Would require finding out why
> > it's slower and whether that can be fixed.
> > 
> > You could add channel log commands in the code in various places to see
> > what happens.
> 
> My problem is not with the channel, the problem is that system() is
> slower than the equivalent job_start(). You can run and compare the
> results of Job() and System() functions in the attached file. They are
> both doing the same thing ('echo aaa') but System() is ~5 times slower
> than Job() on Ubuntu and x10 slower on RHEL5.5. On RHEL System() is
> extremely slow, it takes about 0.2 seconds which cause Vim to hang.

I understand that.  The ch_log() and other functions cn be used anywhere
in the Vim code.  That is in the C code, not script.  So it requires
adding the log calls in several places in and around the system()
implementation to find out where it spends time.  This requires building
Vim from the modified source.

-- 
hundred-and-one symptoms of being an internet addict:
91. It's Saturday afternoon in the middle of May and you
are on computer.

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-26 Fir de Conversatie Ramel Eshed 
On Friday, August 26, 2016 at 12:02:38 AM UTC+3, Bram Moolenaar wrote:
> Ramel Eshed wrote:
> 
> > On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed wrote:
> > > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck wrote:
> > > > There are other factors which are right there in the help:
> > > > - job_start() returns a Job object and doesn't wait for the job to 
> > > > finish
> > > > - system() waits for the external command to finish and returns its
> > > > full stdout output as a string.
> > > > 
> > > > I don't know Vim job control really well, but I seem to understand
> > > > that in order to compare system() timing and job control timing you
> > > > would have to set up callbacks to gather any output from the channel,
> > > > and a callback to be called when the job ends (it may still write to
> > > > stdout after it exits), and measure the time from just before
> > > > job_start() to just after making sure that all output has been
> > > > collected and that the job has ended.
> > > > 
> > > > You might even, for testing purposes, try to write a System() user
> > > > function to invoke the argument as a job and return its output as a
> > > > string, with the disadvantage that you would completely lose job
> > > > control asynchronism. But it would allow you a better comparison,
> > > > namely between old-fashioned system() and this new job-control-based
> > > > System().
> > > > 
> > > > Best regards,
> > > > Tony.
> > > 
> > > That's exactly what I've did:
> > > 
> > > func! Job()
> > > let s:rt = reltime()
> > > let g:output = []
> > > let g:job = job_start(['/bin/sh', '-c', 'cat ' . expand('%')], 
> > > {'out_cb': function('s:out_cb'), 'close_cb': function('s:close_cb')})
> > > endfunc
> > > 
> > > func! s:out_cb(channel, msg)
> > > call add(g:output, a:msg)
> > > endfunc
> > > 
> > > func! s:close_cb(channel)
> > > echo reltimestr(reltime(s:rt))
> > > "echo g:output
> > > endfunc
> > > 
> > > " compare with:
> > > 
> > > func! System()
> > > let s:rt = reltime()
> > > let g:output = systemlist('cat ' . expand('%'))
> > > echo reltimestr(reltime(s:rt))
> > > endfunc
> > > 
> > > "
> > > I checked it now on my Ubuntu at home - I still see the difference but 
> > > now both are much faster so even the system() delay is not noticeable. 
> > > Any idea what could cause the delay on my RHEL?
> > > 
> > > BTW, if I remove the comment from the 'echo g:output' line in close_cb() 
> > > the message is not displayed (actually, it depends on which command is 
> > > running. for the 'cat' command above -there is no message).
> > 
> > Hi Bram,
> > 
> > Is it possible to make system() work the same way job_start (with a
> > shell) does?  As I mentioned, I get a noticeable delay with system()
> > which I don't get when using job_start(['/bin/sh'...).
> 
> Not sure if there is anything to improve.  Would require finding out why
> it's slower and whether that can be fixed.
> 
> You could add channel log commands in the code in various places to see
> what happens.
> 
> -- 
> hundred-and-one symptoms of being an internet addict:
> 76. Your ISP regards you as a business partner rather than as a customer.
> 
>  /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
> ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\  an exciting new programming language -- http://www.Zimbu.org///
>  \\\help me help AIDS victims -- http://ICCF-Holland.org///

My problem is not with the channel, the problem is that system() is slower than 
the equivalent job_start(). You can run and compare the results of Job() and 
System() functions in the attached file. They are both doing the same thing 
('echo aaa') but System() is ~5 times slower than Job() on Ubuntu and x10 
slower on RHEL5.5. On RHEL System() is extremely slow, it takes about 0.2 
seconds which cause Vim to hang.

Thanks,
Ramel

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


system.vim
Description: Binary data

Re: system() delay

2016-08-25 Fir de Conversatie Bram Moolenaar 

Ramel Eshed wrote:

> On Thursday, August 11, 2016 at 3:08:01 AM UTC+3, Ramel Eshed wrote:
> > On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck wrote:
> > > There are other factors which are right there in the help:
> > > - job_start() returns a Job object and doesn't wait for the job to finish
> > > - system() waits for the external command to finish and returns its
> > > full stdout output as a string.
> > > 
> > > I don't know Vim job control really well, but I seem to understand
> > > that in order to compare system() timing and job control timing you
> > > would have to set up callbacks to gather any output from the channel,
> > > and a callback to be called when the job ends (it may still write to
> > > stdout after it exits), and measure the time from just before
> > > job_start() to just after making sure that all output has been
> > > collected and that the job has ended.
> > > 
> > > You might even, for testing purposes, try to write a System() user
> > > function to invoke the argument as a job and return its output as a
> > > string, with the disadvantage that you would completely lose job
> > > control asynchronism. But it would allow you a better comparison,
> > > namely between old-fashioned system() and this new job-control-based
> > > System().
> > > 
> > > Best regards,
> > > Tony.
> > 
> > That's exactly what I've did:
> > 
> > func! Job()
> > let s:rt = reltime()
> > let g:output = []
> > let g:job = job_start(['/bin/sh', '-c', 'cat ' . expand('%')], 
> > {'out_cb': function('s:out_cb'), 'close_cb': function('s:close_cb')})
> > endfunc
> > 
> > func! s:out_cb(channel, msg)
> > call add(g:output, a:msg)
> > endfunc
> > 
> > func! s:close_cb(channel)
> > echo reltimestr(reltime(s:rt))
> > "echo g:output
> > endfunc
> > 
> > " compare with:
> > 
> > func! System()
> > let s:rt = reltime()
> > let g:output = systemlist('cat ' . expand('%'))
> > echo reltimestr(reltime(s:rt))
> > endfunc
> > 
> > "
> > I checked it now on my Ubuntu at home - I still see the difference but now 
> > both are much faster so even the system() delay is not noticeable. Any idea 
> > what could cause the delay on my RHEL?
> > 
> > BTW, if I remove the comment from the 'echo g:output' line in close_cb() 
> > the message is not displayed (actually, it depends on which command is 
> > running. for the 'cat' command above -there is no message).
> 
> Hi Bram,
> 
> Is it possible to make system() work the same way job_start (with a
> shell) does?  As I mentioned, I get a noticeable delay with system()
> which I don't get when using job_start(['/bin/sh'...).

Not sure if there is anything to improve.  Would require finding out why
it's slower and whether that can be fixed.

You could add channel log commands in the code in various places to see
what happens.

-- 
hundred-and-one symptoms of being an internet addict:
76. Your ISP regards you as a business partner rather than as a customer.

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-10 Fir de Conversatie Ramel Eshed 
On Thursday, August 11, 2016 at 2:19:38 AM UTC+3, Tony Mechelynck wrote:
> There are other factors which are right there in the help:
> - job_start() returns a Job object and doesn't wait for the job to finish
> - system() waits for the external command to finish and returns its
> full stdout output as a string.
> 
> I don't know Vim job control really well, but I seem to understand
> that in order to compare system() timing and job control timing you
> would have to set up callbacks to gather any output from the channel,
> and a callback to be called when the job ends (it may still write to
> stdout after it exits), and measure the time from just before
> job_start() to just after making sure that all output has been
> collected and that the job has ended.
> 
> You might even, for testing purposes, try to write a System() user
> function to invoke the argument as a job and return its output as a
> string, with the disadvantage that you would completely lose job
> control asynchronism. But it would allow you a better comparison,
> namely between old-fashioned system() and this new job-control-based
> System().
> 
> Best regards,
> Tony.

That's exactly what I've did:

func! Job()
let s:rt = reltime()
let g:output = []
let g:job = job_start(['/bin/sh', '-c', 'cat ' . expand('%')], {'out_cb': 
function('s:out_cb'), 'close_cb': function('s:close_cb')})
endfunc

func! s:out_cb(channel, msg)
call add(g:output, a:msg)
endfunc

func! s:close_cb(channel)
echo reltimestr(reltime(s:rt))
"echo g:output
endfunc

" compare with:

func! System()
let s:rt = reltime()
let g:output = systemlist('cat ' . expand('%'))
echo reltimestr(reltime(s:rt))
endfunc

"
I checked it now on my Ubuntu at home - I still see the difference but now both 
are much faster so even the system() delay is not noticeable. Any idea what 
could cause the delay on my RHEL?

BTW, if I remove the comment from the 'echo g:output' line in close_cb() the 
message is not displayed (actually, it depends on which command is running. for 
the 'cat' command above -there is no message).

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-10 Fir de Conversatie Tony Mechelynck 
There are other factors which are right there in the help:
- job_start() returns a Job object and doesn't wait for the job to finish
- system() waits for the external command to finish and returns its
full stdout output as a string.

I don't know Vim job control really well, but I seem to understand
that in order to compare system() timing and job control timing you
would have to set up callbacks to gather any output from the channel,
and a callback to be called when the job ends (it may still write to
stdout after it exits), and measure the time from just before
job_start() to just after making sure that all output has been
collected and that the job has ended.

You might even, for testing purposes, try to write a System() user
function to invoke the argument as a job and return its output as a
string, with the disadvantage that you would completely lose job
control asynchronism. But it would allow you a better comparison,
namely between old-fashioned system() and this new job-control-based
System().

Best regards,
Tony.

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-10 Fir de Conversatie Ramel Eshed 
On Thursday, August 11, 2016 at 12:57:27 AM UTC+3, Dominique Pelle wrote:
> Ramel Eshed ]wrote:
> 
> > Hi Bram,
> >
> > I noticed that using system() function has an overhead of 0.2 seconds 
> > (checked on RHEL 5.5) while using job_start() is 10 times faster (from 
> > job_start to close_cb). Is this because of system() is using temp files 
> > instead of pipes? This is a noticeable delay.
> >
> > Thanks,
> > Ramel
> 
> I'm guessing that system() is slower because it starts a shell process
> and that shell process parses and runs the command. The shell may
> even run some file like .profile (not sure) before running your command.
> 
> Whereas job_start() is I suppose doing a fork() + exec()  (i.e. no shell
> process) so it is faster.
> 
> To have more comparable speed, you'd need to measure something like:
> 
> - job_start(["/bin/sh", "-c" "date")
> - system('date')
> 
> Dominique

Hi Dominique,

job_start(["/bin/sh", "-c" "date") takes almost the same as when using 'date' 
directly (maybe few ms more).

Ramel

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: system() delay

2016-08-10 Fir de Conversatie Dominique Pellé 
Ramel Eshed ]wrote:

> Hi Bram,
>
> I noticed that using system() function has an overhead of 0.2 seconds 
> (checked on RHEL 5.5) while using job_start() is 10 times faster (from 
> job_start to close_cb). Is this because of system() is using temp files 
> instead of pipes? This is a noticeable delay.
>
> Thanks,
> Ramel

I'm guessing that system() is slower because it starts a shell process
and that shell process parses and runs the command. The shell may
even run some file like .profile (not sure) before running your command.

Whereas job_start() is I suppose doing a fork() + exec()  (i.e. no shell
process) so it is faster.

To have more comparable speed, you'd need to measure something like:

- job_start(["/bin/sh", "-c" "date")
- system('date')

Dominique

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.