Re: non-blocking stdin from bash
On 14/08/18 02:40, Greg Wooledge wrote: > So, yeah. It's warranted. Perhaps. > Idiot. That bit isn't, though. Richard signature.asc Description: OpenPGP digital signature
Re: non-blocking stdin from bash
Greg Wooledge wrote: > On Mon, Aug 13, 2018 at 02:58:54PM -, Dan Purgert wrote: >> Greg Wooledge wrote: >> > [...] >> > It's absolutely warranted. You changed the question, and then you >> > changed the question AGAIN.. >> >> This reminds me of bashphorisms 1 through 3. Although, I guess they're >> more related to the IRC channel. > > It happens here too, just more slowly. In either event, I usually learn something; even if it's merely "don't do this, even if you're not sure why you shouldn't." -- |_|O|_| Registered Linux user #585947 |_|_|O| Github: https://github.com/dpurgert |O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5 4AEE 8E11 DDF3 1279 A281
Re: non-blocking stdin from bash
On Mon, Aug 13, 2018 at 02:58:54PM -, Dan Purgert wrote: > Greg Wooledge wrote: > > [...] > > It's absolutely warranted. You changed the question, and then you changed > > the question AGAIN. > > This reminds me of bashphorisms 1 through 3. Although, I guess they're > more related to the IRC channel. It happens here too, just more slowly.
Re: non-blocking stdin from bash
Greg Wooledge wrote: > [...] > It's absolutely warranted. You changed the question, and then you changed > the question AGAIN. This reminds me of bashphorisms 1 through 3. Although, I guess they're more related to the IRC channel. -- |_|O|_| Registered Linux user #585947 |_|_|O| Github: https://github.com/dpurgert |O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5 4AEE 8E11 DDF3 1279 A281
Re: non-blocking stdin from bash
On Mon, 2018-08-13 at 10:40 -0400, Greg Wooledge wrote: > On Mon, Aug 13, 2018 at 10:34:40AM -0400, Jim Popovitch wrote: > > On Mon, 2018-08-13 at 08:59 -0400, Greg Wooledge wrote: > > > On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > > > > What's the best way to grab anything that's in stdin into a > > > > variable inside a bash script, but won't block if stdin is > > > > null? > > > > > > On Sun, Aug 12, 2018 at 01:57:09PM -0400, Jim Popovitch wrote: > > > > Interesting. I'm using it via a cron script like so: > > > > > > > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh > > > > `hostname`; > > > > > > On Sun, Aug 12, 2018 at 06:45:23PM -0400, Jim Popovitch wrote: > > > > That was just an example of what I'm doing. In reality > > > > logtail's > > > > in there, but it made the example line too long. > > > > > > https://mywiki.wooledge.org/XyProblem > > > https://mywiki.wooledge.org/BashFAQ/064 > > > > Come on, that's not warranted or you just like posting links to > > you > > own site. smh. > > It's absolutely warranted. You changed the question, and then you > changed the question AGAIN. Not at all. Even your original slicey-dicey copy+paste shows that I used the word "like" when describing the crontab, and my original question never defined anything about what I was doing, rather it asked a question about how to do something. > The original question was about testing whether stdin has input > characters AVAILABLE Nope. Go back and read my original question, pls. -Jim P.
Re: non-blocking stdin from bash
On Mon, Aug 13, 2018 at 10:34:40AM -0400, Jim Popovitch wrote: > On Mon, 2018-08-13 at 08:59 -0400, Greg Wooledge wrote: > > On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > > > What's the best way to grab anything that's in stdin into a > > > variable inside a bash script, but won't block if stdin is null? > > > > On Sun, Aug 12, 2018 at 01:57:09PM -0400, Jim Popovitch wrote: > > > Interesting. I'm using it via a cron script like so: > > > > > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; > > > > On Sun, Aug 12, 2018 at 06:45:23PM -0400, Jim Popovitch wrote: > > > That was just an example of what I'm doing. In reality logtail's > > > in there, but it made the example line too long. > > > > https://mywiki.wooledge.org/XyProblem > > https://mywiki.wooledge.org/BashFAQ/064 > > Come on, that's not warranted or you just like posting links to you > own site. smh. It's absolutely warranted. You changed the question, and then you changed the question AGAIN. The original question was about testing whether stdin has input characters AVAILABLE, for which the feature "read -t 0" was added in bash version 4.0. But then the question mutated, and there was no longer any need to probe for input characters to avoid blocking, because you were simply reading from a piped grep. But then the question mutated AGAIN, and in this THIRD version of the question, there is something called "logtail" involved, and I have no idea what that even IS, or how it affects the pipeline. One may speculate that it works something like "tail -f" or "tail -F", in which case we are BACK TO THE ORIGINAL QUESTION. Maybe. But we can't be sure, since not only is this a nonstandard tool, but you didn't show or describe how it's being used. So, yeah. It's warranted. Idiot.
Re: non-blocking stdin from bash
On Mon, 2018-08-13 at 08:59 -0400, Greg Wooledge wrote: > On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > > What's the best way to grab anything that's in stdin into a > > variable inside a bash script, but won't block if stdin is null? > > On Sun, Aug 12, 2018 at 01:57:09PM -0400, Jim Popovitch wrote: > > Interesting. I'm using it via a cron script like so: > > > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; > > On Sun, Aug 12, 2018 at 06:45:23PM -0400, Jim Popovitch wrote: > > That was just an example of what I'm doing. In reality logtail's > > in there, but it made the example line too long. > > https://mywiki.wooledge.org/XyProblem > https://mywiki.wooledge.org/BashFAQ/064 Come on, that's not warranted or you just like posting links to you own site. smh. -Jim P.
Re: non-blocking stdin from bash
On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > What's the best way to grab anything that's in stdin into a variable > inside a bash script, but won't block if stdin is null? On Sun, Aug 12, 2018 at 01:57:09PM -0400, Jim Popovitch wrote: > Interesting. I'm using it via a cron script like so: > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; On Sun, Aug 12, 2018 at 06:45:23PM -0400, Jim Popovitch wrote: > That was just an example of what I'm doing. In reality logtail's in > there, but it made the example line too long. https://mywiki.wooledge.org/XyProblem https://mywiki.wooledge.org/BashFAQ/064
Re: non-blocking stdin from bash
On Mon, 2018-08-13 at 10:13 +1200, Richard Hector wrote: > On 13/08/18 05:57, Jim Popovitch wrote: > > Interesting. I'm using it via a cron script like so: > > > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; > > I don't know what's in notify.sh, but it looks to me like you're > going to get notified every minute for all the unusual log entries, > whether you've already seen them or not? That was just an example of what I'm doing. In reality logtail's in there, but it made the example line too long. > Have you looked at logcheck or other similar existing solutions? Yes, i use logcheck religiously, despite the absolute lack of logcheck rules maintenance that exists in most packages not installed on a default system (looking at you nginx, postfix, bind9, cron, clamav*, openvpn, dovecot, opendmarc, monit, and openntpd). I'd submit the improvements to the appropriate maintainers, but I dislike rejection. ;-) -Jim P.
Re: non-blocking stdin from bash
On 13/08/18 05:57, Jim Popovitch wrote: > Interesting. I'm using it via a cron script like so: > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; I don't know what's in notify.sh, but it looks to me like you're going to get notified every minute for all the unusual log entries, whether you've already seen them or not? Have you looked at logcheck or other similar existing solutions? Richard signature.asc Description: OpenPGP digital signature
Re: non-blocking stdin from bash
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, Aug 12, 2018 at 01:57:09PM -0400, Jim Popovitch wrote: > On Sun, 2018-08-12 at 19:47 +0200, to...@tuxteam.de wrote: [...] > > This will block in the read as long as the line isn't yet terminated. > > But of course fine, if it rocks your boat :-) > > Interesting. I'm using it via a cron script like so: > > * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; > > so it should always be terminated. Then it'll rock your boat :) Cheers - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAltwkQMACgkQBcgs9XrR2kYn9wCcDFQMnC4UlWESSZIRXk5Mf1pZ djEAnjCWc7A3MJXUuznsTV/o4gRF9tJO =PmYn -END PGP SIGNATURE-
Re: non-blocking stdin from bash
On Sun, 2018-08-12 at 19:47 +0200, to...@tuxteam.de wrote: > On Sun, Aug 12, 2018 at 12:34:08PM -0400, Jim Popovitch wrote: > > [...] > > > Turns out I had mis-read your first reply as "-t 1", chiefly > > because > > Ah, I see. > > > that was all that I had found before posting here. What now works > > for > > me avoids the -t param: > > > > > > while read stdin_line > > do > > MSG=$MSG"\n"$stdin_line > > done < /dev/stdin > > This will block in the read as long as the line isn't yet terminated. > But of course fine, if it rocks your boat :-) Interesting. I'm using it via a cron script like so: * * * * * grep "unusual" /opt/logs/* | /opt/notify.sh `hostname`; so it should always be terminated. -Jim P.
Re: non-blocking stdin from bash
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, Aug 12, 2018 at 12:34:08PM -0400, Jim Popovitch wrote: [...] > Turns out I had mis-read your first reply as "-t 1", chiefly because Ah, I see. > that was all that I had found before posting here. What now works for > me avoids the -t param: > > > while read stdin_line > do > MSG=$MSG"\n"$stdin_line > done < /dev/stdin This will block in the read as long as the line isn't yet terminated. But of course fine, if it rocks your boat :-) > Thanks again, Hey. You're welcome Cheers - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAltwcrYACgkQBcgs9XrR2kYqgwCdH9J+3t6DYuGAcS/23OaZNE1i seYAn0i65VKWm16CNrmChrmpci01kchP =ckC2 -END PGP SIGNATURE-
Re: non-blocking stdin from bash
On Sun, 2018-08-12 at 10:36 +0200, to...@tuxteam.de wrote: > On Sun, Aug 12, 2018 at 01:26:40AM -0400, Jim Popovitch wrote: > > On Sun, 2018-08-12 at 00:48 +0200, to...@tuxteam.de wrote: > > > On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > > > > Hello! > > > > > > > > What's the best way to grab anything that's in stdin into a > > > > variable > > > > inside a bash script, but won't block if stdin is null? > > > > > > I think read is your friend (at least in bash). It has an option > > > -t , which you can set to zero, for it to just grab > > > what's > > > available at the moment without waiting (cf "help read" while in > > > the bash for more details, like setting a delimiter, etc.) > > > > > > HTH > > > -- tomás > > > > Thanks for the reply tomás. I'm trying to avoid using read because > > of > > the 1 sec minimum timeout. > > Not on bash, at least. According to the docs, "-t 0" is a timeout of > zero. Experiments confirm that: > > tomas@trotzki:~$ time read -t 0 > > real0m0.000s > user0m0.000s > sys 0m0.000s > > (Of course, 0 is too optimistic here, but it's sub-millisecond). > Fractional times work too: > > tomas@trotzki:~$ time read -t 0.05 > > real0m0.050s > user0m0.000s > sys 0m0.000s > > Finally, you can use timeout to read incomplete (i.e. non-terminated) > input, like so: > > tomas@trotzki:~$ time echo -n mumble | bash -c 'read -t 0.01 foo ; > echo $foo' > mumble > > real0m0.004s > user0m0.000s > sys 0m0.000s > > Note that "-t 0" won't work here: the "read" at the right is there > before > the "echo" at the left, so read turns out empty... > > > This may seem odd, but 1 sec is 100+ > > times > > longer than grep'ing/awk'ing/sed'ing the contents of a variable, so > > I'm > > trying to find a faster way to read stdin. > > No, it doesn't seem odd. Sometimes 1 sec is too much (sometimes 1msec > is > too much, too :-) > > But I think (used wisely) read is still your friend! > > Note that I didn't check how much of this is bashism. Document your > requirements :-) :-) Turns out I had mis-read your first reply as "-t 1", chiefly because that was all that I had found before posting here. What now works for me avoids the -t param: while read stdin_line do MSG=$MSG"\n"$stdin_line done < /dev/stdin Thanks again, -Jim P.
Re: non-blocking stdin from bash
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, Aug 12, 2018 at 01:26:40AM -0400, Jim Popovitch wrote: > On Sun, 2018-08-12 at 00:48 +0200, to...@tuxteam.de wrote: > > On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > > > Hello! > > > > > > What's the best way to grab anything that's in stdin into a > > > variable > > > inside a bash script, but won't block if stdin is null? > > > > I think read is your friend (at least in bash). It has an option > > -t , which you can set to zero, for it to just grab what's > > available at the moment without waiting (cf "help read" while in > > the bash for more details, like setting a delimiter, etc.) > > > > HTH > > -- tomás > > Thanks for the reply tomás. I'm trying to avoid using read because of > the 1 sec minimum timeout. Not on bash, at least. According to the docs, "-t 0" is a timeout of zero. Experiments confirm that: tomas@trotzki:~$ time read -t 0 real0m0.000s user0m0.000s sys 0m0.000s (Of course, 0 is too optimistic here, but it's sub-millisecond). Fractional times work too: tomas@trotzki:~$ time read -t 0.05 real0m0.050s user0m0.000s sys 0m0.000s Finally, you can use timeout to read incomplete (i.e. non-terminated) input, like so: tomas@trotzki:~$ time echo -n mumble | bash -c 'read -t 0.01 foo ; echo $foo' mumble real0m0.004s user0m0.000s sys 0m0.000s Note that "-t 0" won't work here: the "read" at the right is there before the "echo" at the left, so read turns out empty... > This may seem odd, but 1 sec is 100+ times > longer than grep'ing/awk'ing/sed'ing the contents of a variable, so I'm > trying to find a faster way to read stdin. No, it doesn't seem odd. Sometimes 1 sec is too much (sometimes 1msec is too much, too :-) But I think (used wisely) read is still your friend! Note that I didn't check how much of this is bashism. Document your requirements :-) Cheers - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAltv8XkACgkQBcgs9XrR2kZNaQCcCaWfnVktQY8CENggRODJ3/nl +1UAnRbm/rs66wz+BplEsBX3Mmp3Ed7X =NFpA -END PGP SIGNATURE-
Re: non-blocking stdin from bash
On Sun, 2018-08-12 at 00:48 +0200, to...@tuxteam.de wrote: > On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > > Hello! > > > > What's the best way to grab anything that's in stdin into a > > variable > > inside a bash script, but won't block if stdin is null? > > I think read is your friend (at least in bash). It has an option > -t , which you can set to zero, for it to just grab what's > available at the moment without waiting (cf "help read" while in > the bash for more details, like setting a delimiter, etc.) > > HTH > -- tomás Thanks for the reply tomás. I'm trying to avoid using read because of the 1 sec minimum timeout. This may seem odd, but 1 sec is 100+ times longer than grep'ing/awk'ing/sed'ing the contents of a variable, so I'm trying to find a faster way to read stdin. -Jim P. signature.asc Description: This is a digitally signed message part
Re: non-blocking stdin from bash
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, Aug 11, 2018 at 06:08:34PM -0400, Jim Popovitch wrote: > Hello! > > What's the best way to grab anything that's in stdin into a variable > inside a bash script, but won't block if stdin is null? I think read is your friend (at least in bash). It has an option - -t , which you can set to zero, for it to just grab what's available at the moment without waiting (cf "help read" while in the bash for more details, like setting a delimiter, etc.) HTH - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAltvZ8gACgkQBcgs9XrR2kZ4XwCeKD4PgANQHP+QTuB+LtMPGdfP blYAn3ovvpi+8Cx+vlaQ/5oAafdxlgww =BWci -END PGP SIGNATURE-
non-blocking stdin from bash
Hello! What's the best way to grab anything that's in stdin into a variable inside a bash script, but won't block if stdin is null? -Jim P.
