Re: feature: time builtin and file descriptor
2014-10-29 15:09 GMT+01:00 Sami Kerola :
>
> This idea came to my mind while writing a script that runs multiple
> commands, and I simply wanted to know how long they are busy. I am
> aware alternatives exist, but they can get a bit tricky if one wants to
> have multiple measurements going on simultaneously. For example:
>
> exec 42>&0
> time --file-descriptor 42
> for i in items; in
> exec 52>&0
> time --file-descriptor 52
> echo "processing $i"
> [...]
> echo "item $i took"
> exec 52>&-
> done
> echo 'all together took:'
> exec 42>&-
>
You could achieve this by using command grouping and TIMEFORMAT:
items=( a b c )
time {
for i in "${items[@]}"; do
time {
echo "processing $i"
sleep "$((RANDOM%5 + 1))"
TIMEFORMAT="item $i took %R seconds"
}
done
TIMEFORMAT='all together took: %R seconds'
}
example output:
processing a
item a took 5.001 seconds
processing b
item b took 1.001 seconds
processing c
item c took 3.001 seconds
all together took: 9.004 seconds
It would be prettier if TIMEFORMAT could be set on invocation of time,
e.g. TIMEFORMAT='foo took %R seconds' time { ...; } , but time being a
keyword probably makes that hard.
One also has to be careful to sanitize any variables one embeds in
TIMEFORMAT since % characters are special.
--
Geir Hauge
Re: feature: time builtin and file descriptor
> OK. It doesn't sound like this feature is of general interest. Since
> you can control when you open and close file descriptors, you might look
> at $SECONDS when the file is opened and when it's closed and using the
> difference to see how long it was open.
If I understand correctly, the original issue wasn't exactly to
measure the time FDs were open, but to use that measure to time the
execute of code blocks. That is, timing open FDs was just a tool, not
the goal itself. Since we can already time blocks, I think this
alternative is not needed. I wrote a detailed explanation here:
dualbus@hp:~$ bash ./script
processing $i=1
item $i=1 took 4.003
processing $i=2
item $i=2 took 6.003
processing $i=3
item $i=3 took 8.003
all together took 18.014
dualbus@hp:~$ cat ./script
#!/bin/bash
# set TIMEFORMAT to show only the 'elapsed time in seconds'. (see man bash)
TIMEFORMAT=%R
# The key points are:
# 1. You can 'time' a block, so, stuff like: time { ...; } and time for ...; do
...
#are valid. You don't need to trace an open file descriptor to time a block
#of commands.
# 2. Capturing the output of 'time' is tricky, but it can be done, follow the
#indications here: http://mywiki.wooledge.org/BashFAQ/032 (or read my
#attempt at explaining it at the end)
{ g=$(
{ time for i in 1 2 3; do
echo "processing \$i=$i"
{ t=$(
{ time {
# item commands
sleep "$i"
sleep "$((i+2))"
}
} 2>&1 >&3
);} 3>&1
echo "item \$i=$i took $t"
done
} 2>&1 >&3
);} 3>&1
echo "all together took $g"
# How to do what I think you want to do:
#
# 1. Change TIMEFORMAT to output just the number you want (easy)
# 2. Now, the tricky part, capture the output of time:
#- You can't just do: var=$(time ...) and hold the output of time in var,
# because 'time' writes to stderr. So the logical thing follows.
#- We try: var=$(time ... 2>&1 >/dev/null) (we discard stdout for
# simplicity), but that still doesn't work, because that redirection
# doesn't seem to apply to 'time', instead, it applies to the command
# we're timing.
#- So, we try: var=$({ time ...; } 2>&1 >/dev/null), and... that works.
# By doing the redirection on a group that consists only of 'time', we
# managed to capture its output.
#
# You can try this at home:
#
# $ TIMEFORMAT=%R; var=$({ time sleep 3.1416;} 2>&1 >/dev/null); echo "$var"
# 3.143
#
# 3. So, we have the elapsed time... but, what if we also want to keep stdout?
#We have to do some tricky redirections:
#
#{ var=$({ time ...; } 2>&1 >&3); } 3>&1
#
#Don't panic, it looks horrible, but it's actually easy to understand. The
#first thing we should know is that redirections apply following these two
#rules:
#
#a. If you have nested blocks, it will apply the outer redirections first.
# So, in { { { foo >a; } >b; } >c; } >d
# It will first redirect to 'd', then to 'c', then to 'b', ...
#b. Redirections done at the same level are executed from left to right.
#
#With these two details we can proceed to understand:
#
#{ var=$({ time ...; } 2>&1 >&3); } 3>&1
#
#Since we execute the outer redirections first, '3>&1' is the first
#redirection, so now there's a file descriptor 3, open for the whole block
#delimited by { ... }. Ah, and that FD3 is a clone of the script's stdout,
#which could be a file, the terminal, etc. Then, inside var=$({...} redirs)
#we have another set of redirections, '2>&1' and '>&3'. They get executed
#from left to right, so '2>&1' goes first. What this does is to send stderr
#(time's output), to stdout *but* in this case, stdout goes to $(...),
#which is captured in 'var' (it doesn't go to the main script stdout). And
#then we have '>&3', which does: send stdout (which currently was going to
#$(...) ) to the main script's stdout (&3 is a clone of it).
#
#Sorry for the complex explanation.
#
#And, if I read your original email correctly, this is what you wanted to
do:
#
# > This idea came to my mind while writing a script that runs multiple
# > commands, and I simply wanted to know how long they are busy. I am
#
#So, now you can run 'time' multiple times, even nested, and find out how
#long they are busy, without having to patch bash. (Sure, the code is ugly,
#but, it's bash, it's expected to be like that ;) ).
#
# > What do you think, useful feature or unnecessary bloat?
# My opinion is: unnecessary bloat, since we can work-around that specific
# case without new features. Sure, the work-around is ugly, but, it's
# consistent, and, with a bit of effort, it can do almost everything your
# proposal attempts to achieve.
Re: feature: time builtin and file descriptor
On 30 October 2014 20:14, Eduardo A. Bustamante López wrote: >> Sounds the there is not much enthusiasm about making this sort of >> feature to work. This proposal belongs to archived never implemented >> ideas area. Good that effort wasting was kept minimal. > Remember that working patches are better than just requesting > features. There are lots of pending features that are requested more > frequently, so that's a lot of work for a single person to handle. I prefer first getting understanding if a feature is wanted, before starting to hack too much. My time has value to me. I don't want to spend it in writing a change that will be rejected. Neither I want to spend maintainer time to go through that sort of stuff. In other words; fail as fast & cheap way as possible is a good workflow. -- Sami Kerola http://www.iki.fi/kerolasa/
Re: feature: time builtin and file descriptor
> Sounds the there is not much enthusiasm about making this sort of > feature to work. This proposal belongs to archived never implemented > ideas area. Good that effort wasting was kept minimal. Remember that working patches are better than just requesting features. There are lots of pending features that are requested more frequently, so that's a lot of work for a single person to handle.
Re: feature: time builtin and file descriptor
On 10/31/14, 5:15 AM, Sami Kerola wrote: > On 30 October 2014 20:14, Eduardo A. Bustamante López > wrote: >>> Sounds the there is not much enthusiasm about making this sort of >>> feature to work. This proposal belongs to archived never implemented >>> ideas area. Good that effort wasting was kept minimal. >> Remember that working patches are better than just requesting >> features. There are lots of pending features that are requested more >> frequently, so that's a lot of work for a single person to handle. > > I prefer first getting understanding if a feature is wanted, before > starting to hack too much. My time has value to me. I don't want to > spend it in writing a change that will be rejected. Neither I want to > spend maintainer time to go through that sort of stuff. In other > words; fail as fast & cheap way as possible is a good workflow. OK. It doesn't sound like this feature is of general interest. Since you can control when you open and close file descriptors, you might look at $SECONDS when the file is opened and when it's closed and using the difference to see how long it was open. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: feature: time builtin and file descriptor
On 30 October 2014 01:25, Chet Ramey wrote: > On 10/29/14 10:09 AM, Sami Kerola wrote: >> Would it make sense to add to 'time' builtin a way to measure how long >> a file descriptor is been kept open? Something like this. > > You can use $SECONDS for this, but you have to do the math and keep track > of the file descriptors yourself. Hi again, Sounds the there is not much enthusiasm about making this sort of feature to work. This proposal belongs to archived never implemented ideas area. Good that effort wasting was kept minimal. -- Sami Kerola http://www.iki.fi/kerolasa/
Re: feature: time builtin and file descriptor
On 10/29/14 10:09 AM, Sami Kerola wrote: > Hello, > > Would it make sense to add to 'time' builtin a way to measure how long > a file descriptor is been kept open? Something like this. You can use $SECONDS for this, but you have to do the math and keep track of the file descriptors yourself. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: feature: time builtin and file descriptor
On Wed, Oct 29, 2014 at 02:09:53PM +, Sami Kerola wrote: > Would it make sense to add to 'time' builtin a way to measure how long > a file descriptor is been kept open? Doesn't really make much sense. If you want multiple independent timers, there's no reason to tie them to open file descriptors. If anything, just make an array of timers, with some shell builtin to start and stop each one. Read the array element to get the elapsed time. I can't say I've ever had a need for this. > This idea came to my mind while writing a script that runs multiple > commands, and I simply wanted to know how long they are busy. I am > aware alternatives exist, but they can get a bit tricky if one wants to > have multiple measurements going on simultaneously. Without any shell modifications, if your operating system provides %s in strftime() then you can get the current time with second resolution (and without forking date) by doing: printf -v somevariable '%(%s)T' -1 It's not much additional effort to keep separate variables (possibly array elements) for each independent timer you want to track. Issue the same printf command again to get the end time, and subtract. If you need millisecond (or even smaller) resolution, or if you need it to work on non-Linux/BSD systems, then I'm afraid Bash is just not the right language for your project.
feature: time builtin and file descriptor
Hello, Would it make sense to add to 'time' builtin a way to measure how long a file descriptor is been kept open? Something like this. -- snip #!/bin/bash exec 42>&0 time --file-descriptor 42 sleep 10 exec 42>&- -- snip $ ./above_script.sh real0m10.012s user0m0.000s sys 0m0.000s This idea came to my mind while writing a script that runs multiple commands, and I simply wanted to know how long they are busy. I am aware alternatives exist, but they can get a bit tricky if one wants to have multiple measurements going on simultaneously. For example: exec 42>&0 time --file-descriptor 42 for i in items; in exec 52>&0 time --file-descriptor 52 echo "processing $i" [...] echo "item $i took" exec 52>&- done echo 'all together took:' exec 42>&- What do you think, useful feature or unnecessary bloat? -- Sami Kerola http://www.iki.fi/kerolasa/
