Trimming Whitespace From Beginning and end of Text Lines
This looks like something sed should be able to do, but I haven't had any luck at all. I wanted to remove any whitespace that has accidentally gotten added to the beginning or end of some lines of text. I made a test file that looks like: left justified. lots of spaces. and the best I have done so far is to get rid of about 3 spaces. Attempt 1. #! /usr/bin/sed -f s/ \+//g s/^ //g s/ $//g This looks like it should do the job, but the leading and trailing spaces are still mostly there. I wrote another script. Attempt 2. #! /bin/sh sed 's/^[[:space:]]//g' \ |sed 's/[[:space:]]$//g' If I cat the test file through this script, it also removes one or two spaces, but not all the leading and trailing whitespace I put there. I can write a program in C to do this, but is there a sed script or other native application in FreeBSD that can do this? Thank you. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Network Operations Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
On 2006-05-12 09:50, Martin McCormick [EMAIL PROTECTED] wrote: This looks like something sed should be able to do, but I haven't had any luck at all. I wanted to remove any whitespace that has accidentally gotten added to the beginning or end of some lines of text. I made a test file that looks like: left justified. lots of spaces. and the best I have done so far is to get rid of about 3 spaces. Attempt 1. #! /usr/bin/sed -f s/ \+//g s/^ //g s/ $//g This fails to remove TAB characters from either the start or the end of a line. This looks like it should do the job, but the leading and trailing spaces are still mostly there. I wrote another script. Attempt 2. #! /bin/sh sed 's/^[[:space:]]//g' \ |sed 's/[[:space:]]$//g' This fails to remove multiple occurences of the [[:space:]] class. There are at least the following ways: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... perl -pi -e 's/^\s*(\S.*\S)[ \t]*$/$1/' file ... The first one seems more straightforward to me most of the time, but there are times I find Perl's `-pi -e ...' idiom very convenient. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
Giorgos Keramidas wrote: This fails to remove multiple occurences of the [[:space:]] class. There are at least the following ways: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... perl -pi -e 's/^\s*(\S.*\S)[ \t]*$/$1/' file ... The first one seems more straightforward to me most of the time, but there are times I find Perl's `-pi -e ...' idiom very convenient. It is, and I wish to acknowledge the above are entirely valid solutions to the problem, but... python -c 'import sys; print sys.stdin.read().strip()' file... ...has the advantage of being human readable. My old 300-baud accoustic modem used to generate output which in hindsight looks astonishingly close to regex character classes. :-) -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
On 2006-05-12 11:27, Chuck Swiger [EMAIL PROTECTED] wrote: Giorgos Keramidas wrote: This fails to remove multiple occurences of the [[:space:]] class. There are at least the following ways: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... perl -pi -e 's/^\s*(\S.*\S)[ \t]*$/$1/' file ... The first one seems more straightforward to me most of the time, but there are times I find Perl's `-pi -e ...' idiom very convenient. It is, and I wish to acknowledge the above are entirely valid solutions to the problem, but... python -c 'import sys; print sys.stdin.read().strip()' file... ...has the advantage of being human readable. My old 300-baud accoustic modem used to generate output which in hindsight looks astonishingly close to regex character classes. :-) HEH! I see the joke about Perl being similar to line noise is not something local to our Greek IRC channels :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Trimming Whitespace From Beginning and end of Text Lines
sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... why not use just (you can change the - separator to / as above): sed -e 's-^ *--g' -e 's- *$--g' usage examples: - cat file| sed ... file1 - echo $variable| sed ... |grep xy - if [ `echo $xy|sed ...` = blabla bla ]; then ... cu... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
Chuck Swiger quotes and writes: Giorgos Keramidas wrote: This fails to remove multiple occurences of the [[:space:]] class. There are at least the following ways: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... That did it! As soon as I saw the *, I knew what I was not doing. perl -pi -e 's/^\s*(\S.*\S)[ \t]*$/$1/' file ... The first one seems more straightforward to me most of the time, but there are times I find Perl's `-pi -e ...' idiom very convenient. It is, and I wish to acknowledge the above are entirely valid solutions to the problem, but... python -c 'import sys; print sys.stdin.read().strip()' file... ...has the advantage of being human readable. My old 300-baud accoustic modem used to generate output which in hindsight looks astonishingly close to regex character classes. :-) Wow! I'd almost forgotten some of that by-gone era. I had a 1200-baud modem that, in conjunction with the clock slips between our local telephone company and our PBX, used to march [] and various other garbage characters that did look just like regex. You just had to keep re-dialing until you finally got a connection that worked. Thanks to everyone for the help. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Network Operations Group ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
Giorgos Keramidas wrote: On 2006-05-12 11:27, Chuck Swiger [EMAIL PROTECTED] wrote: It is, and I wish to acknowledge the above are entirely valid solutions to the problem, but... python -c 'import sys; print sys.stdin.read().strip()' file... ...has the advantage of being human readable. My old 300-baud accoustic modem used to generate output which in hindsight looks astonishingly close to regex character classes. :-) HEH! I see the joke about Perl being similar to line noise is not something local to our Greek IRC channels :) Indeed. :) I must confess that it doesn't do in-place replacement of the files, though. I'd have to do a two-liner, I guess: python -c 'import sys,fileinput for line in fileinput.input(inplace=1): print line.strip()' file1 file2 ... -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
On Fri, 12 May 2006, Giorgos Keramidas wrote: There are at least the following ways: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... perl -pi -e 's/^\s*(\S.*\S)[ \t]*$/$1/' file ... The first one seems more straightforward to me most of the time, but there are times I find Perl's `-pi -e ...' idiom very convenient. Neither of those work here: The first sed expression is missing //. Correcting that: sed -i -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' test.txt sed: lstat: No such file or directory The Perl version shows no difference between the original and processed file. It's complex, too. This one works: perl -pi -le 's/^\s+//; s/\s+$//' test.txt Notes: 1. sed always seems to be a pain. My compliments to those who use it regularly; the only time I use it at all is when Perl (or something else with better handling of regular expressions) is not available. 2. The -l option to perl is needed to preserve line endings. 3. The last version is based on the more efficient way of doing it as per: man -P 'less +/trim' perlop -Warren Block * Rapid City, South Dakota USA ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
At 16:50 12.05.2006, Martin McCormick wrote: This looks like something sed should be able to do, but I haven't had any luck at all. I wanted to remove any whitespace that has accidentally gotten added to the beginning or end of some lines of text. I made a test file that looks like: left justified. lots of spaces. and the best I have done so far is to get rid of about 3 spaces. Attempt 1. #! /usr/bin/sed -f s/ \+//g s/^ //g s/ $//g This looks like it should do the job, but the leading and trailing spaces are still mostly there. I wrote another script. Attempt 2. #! /bin/sh sed 's/^[[:space:]]//g' \ |sed 's/[[:space:]]$//g' If I cat the test file through this script, it also removes one or two spaces, but not all the leading and trailing whitespace I put there. I can write a program in C to do this, but is there a sed script or other native application in FreeBSD that can do this? Thank you. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Network Operations Group What's up man? Here's a script I use to remove trailing whitespace. It also reduces two or more empty lines like this: -- -- To just one: -- -- And it converts ASCII files to UNIX format (that is without ^M). Then for pretty sake, it adds an empty line to the end of each file. -- #!/usr/local/bin/bash # # Remove CRLF, trailing whitespace and double lining. # $MERHABA: ascii_clean.sh,v 1.0 2007/11/11 15:09:05 kyrre Exp $ # for file in `find -s . -type f`; do if file -b $file | grep -q 'text'; then echo $file tr -d '\r' $file | cat -s | sed -E -e 's/[[:space:]]+$//' $file.tmp mv -f $file.tmp $file echo $file: Done fi done -- I'd be interested in knowing if you manage to improve this script. Take care, Kyrre ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
On 2006-05-12 17:56, [EMAIL PROTECTED]@mgEDV.net [EMAIL PROTECTED] wrote: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... why not use just (you can change the - separator to / as above): sed -e 's-^ *--g' -e 's- *$--g' Because this provides no additional help with the problem of not matching TABS and it looks very confusing so close to `-e' and other command-line options. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
On 2006-05-12 10:41, Warren Block [EMAIL PROTECTED] wrote: On Fri, 12 May 2006, Giorgos Keramidas wrote: There are at least the following ways: sed -i -e 's/^[[:space:]]*' -e 's/[[:space:]]*$//' file ... perl -pi -e 's/^\s*(\S.*\S)[ \t]*$/$1/' file ... The first one seems more straightforward to me most of the time, but there are times I find Perl's `-pi -e ...' idiom very convenient. Neither of those work here: The first sed expression is missing //. Correcting that: sed -i -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' test.txt sed: lstat: No such file or directory Yeah, I noticed the missing // in the first regexp, but only after I had posted the message. You're right, of course :) It seems odd that the fixed expression doesn't work though. Which version of FreeBSD is this and what sed are you running? $ uname -v $ type sed The Perl version shows no difference between the original and processed file. It's complex, too. This one works: perl -pi -le 's/^\s+//; s/\s+$//' test.txt Notes: 1. sed always seems to be a pain. My compliments to those who use it regularly; the only time I use it at all is when Perl (or something else with better handling of regular expressions) is not available. 2. The -l option to perl is needed to preserve line endings. 3. The last version is based on the more efficient way of doing it as per: man -P 'less +/trim' perlop Great! Thanks for all the useful tips :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Trimming Whitespace From Beginning and end of Text Lines
On Fri, 12 May 2006, Giorgos Keramidas wrote: The first sed expression is missing //. Correcting that: sed -i -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' test.txt sed: lstat: No such file or directory Yeah, I noticed the missing // in the first regexp, but only after I had posted the message. You're right, of course :) It seems odd that the fixed expression doesn't work though. Which version of FreeBSD is this and what sed are you running? $ uname -v FreeBSD 4.11-STABLE #0: Wed Mar 22 19:18:33 MST 2006 $ type sed That's a sh-ism (normally I use csh): sed is /usr/bin/sed Interestingly, the problem is different on 6.1 (csh or sh): sed -i -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' test.txt sed: -e: No such file or directory ...which is solved by giving a blank argument for -i: sed -i'' -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' test.txt Gah. -Warren Block * Rapid City, South Dakota USA ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
