Re: Bash Scripting Question

[Chris Davies]

Jonathan Dowland  wrote:
> On Sun, Nov 03, 2013 at 09:58:58PM +0100, Erwan David wrote:
>> Maybe you'll need something like expect to handle this.

> I'd second expect, it's probably the best tool for the job in all
> non-trivial cases.

The "empty-expect" package, perhaps?
Chris


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e2vskaxd2t@news.roaima.co.uk

Re: Bash Scripting Question

[Zenaan Harkness]

On 11/4/13, Thomas H. George  wrote:
> The script I am trying to write executes a program that requires a
> keyboard response. I have experimented with redirecting STDIN but haven't
> found the
> correct way to make the response.

To read a value (perhaps half your "problem"):

apt-cache show ...
zenity # gnome
kdialog # kde
xdialog # x generic
expect # Tk (not sure I understand expect)
dialog or whiptail #curses (text)

http://code.google.com/p/yad/ # xenity fork with many improvements

Good luck,
Zenaan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caosgnsqrh+2g7issiazssiwhztskydovx459ubygf4frsru...@mail.gmail.com

Re: Bash Scripting Question

[Karl E. Jorgensen]

Hi

On Sun, Nov 03, 2013 at 02:35:30PM -0500, Thomas H. George wrote:
> The script I am trying to write executes a program that requires a
> keyboard response. I have experimented with redirecting STDIN but haven't 
> found the
> correct way to make the response.  I found one example that scanned the
> response from apt-get install  for the letter y and fed this back to
> install .  In my case I must respond with a word in answer to the
> programs question.

Well - if you're trying to script a command, have a closer look at the
man-page for the command in question: There may be ways of making the
command non-interactive: Then you don't have to give it any input at
all!

If you are trying to script the "apt-get install" command, check out
the "--yes" option and it's relatives.

Note that apt WILL ask if you're trying to do "crazy" stuff, e.g. like
removing essential packages.  Even the simpler questions are there to
give you a chance not to shoot yourself in the foot: If you script the
answers to them, your (metaphorical) foot is in danger...

> I'm sure this must be elementary but I have read large sections of
> BASH GUIDE FOR BEGINNERS and  AVANCED BASH-SCRIPTING GUIDE without
> finding a solution.  I would appreciate a little help or advice.

:-) Good - that's definitely worth reading.

-- 
Karl E. Jorgensen


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131104121304.GA10450@hawking

Re: Bash Scripting Question

[Jonathan Dowland]

The tool 'yes' can be used to write an infinite stream of strings
(the default being 'y') to standard output, so if your program needed
only a sequence of a fixed string such as 'y', you could do

> yes | your-program

or 

> yes "some-other-string" | your-program

But if your program is not reading strictly from standard input (there
are other ways to get keyboard/interactive input), then…

On Sun, Nov 03, 2013 at 09:58:58PM +0100, Erwan David wrote:
> Maybe you'll need something like expect to handle this.

I'd second expect, it's probably the best tool for the job in all
non-trivial cases.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131104100536.ga2...@bryant.redmars.org

Re: Bash Scripting Question

[Erwan David]

Le 03/11/2013 20:35, Thomas H. George a écrit :
> The script I am trying to write executes a program that requires a
> keyboard response. I have experimented with redirecting STDIN but haven't 
> found the
> correct way to make the response.  I found one example that scanned the
> response from apt-get install  for the letter y and fed this back to
> install .  In my case I must respond with a word in answer to the
> programs question.
>
> I'm sure this must be elementary but I have read large sections of
> BASH GUIDE FOR BEGINNERS and  AVANCED BASH-SCRIPTING GUIDE without
> finding a solution.  I would appreciate a little help or advice.
>
> Tom
>
>

Maybe you'll need something like expect to handle this.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/5276b912.1080...@rail.eu.org

Re: Bash Scripting Question

[Cousin Stanley]

> The script I am trying to write executes a program 
> that requires a keyboard response.
>  

  A varaible can be set to a keyboard response
  using a  read  prompt  

read -e -p "What do you need ?" xVariable

echo $xVariable


-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/l56bqj$62c$1...@dont-email.me

Bash Scripting Question

[Thomas H. George]

The script I am trying to write executes a program that requires a
keyboard response. I have experimented with redirecting STDIN but haven't found 
the
correct way to make the response.  I found one example that scanned the
response from apt-get install  for the letter y and fed this back to
install .  In my case I must respond with a word in answer to the
programs question.

I'm sure this must be elementary but I have read large sections of
BASH GUIDE FOR BEGINNERS and  AVANCED BASH-SCRIPTING GUIDE without
finding a solution.  I would appreciate a little help or advice.

Tom


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131103193530.ga22...@tomgeorge.info

Re: Scripting question

[Daniel Landau]

On Tue, Apr 17, 2012 at 4:52 PM, Eduardo M KALINOWSKI
 wrote:
> On Ter, 17 Abr 2012, Chris wrote:
>> I would like have the Smtp: replaced with To:  leaving all that follows in
>> each line untouched and piped into a new file.
>
> man sed
>

Read that too, but try also searching online for "sed tutorial". The
line to do this specific thing is:

$ sed -e 's/^Smtp:/To:/' oldfile > newfile

Daniel Landau


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAPBTYu4KBgb55tjHHXysWed=+s2=kDnSECYNT1i=h1jowm9...@mail.gmail.com

Re: Scripting question

[emmanuel segura]

perl -e 'while(<>){chomp; s/root/Root/g; print  "$_\n"; }' /etc/passwd



Il giorno 17 aprile 2012 15:52, Eduardo M KALINOWSKI <
edua...@kalinowski.com.br> ha scritto:

> On Ter, 17 Abr 2012, Chris wrote:
>
>> Firstly I petty much suck at scripting so I need help.
>>
>> I have a file where each line begins with
>>
>> Smtp:
>>
>> I would like have the Smtp: replaced with To:  leaving all that follows
>> in each line untouched and piped into a new file.
>>
>
> man sed
>
>
> --
> The majority of husbands remind me of an orangutang trying to play the
> violin.
>-- Honor' e DeBalzac
>
> Eduardo M KALINOWSKI
> edua...@kalinowski.com.br
>
>
>
>
> --
> To UNSUBSCRIBE, email to debian-user-REQUEST@lists.**debian.orgwith a
> subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
> Archive: http://lists.debian.org/**20120417105220.Horde.**
> 5niudUisJlFPjXWUaIYVfNA@mail.**kalinowski.com.br
>
>


-- 
esta es mi vida e me la vivo hasta que dios quiera

Re: Scripting question

[Eduardo M KALINOWSKI]

On Ter, 17 Abr 2012, Chris wrote:

Firstly I petty much suck at scripting so I need help.

I have a file where each line begins with

Smtp:

I would like have the Smtp: replaced with To:  leaving all that  
follows in each line untouched and piped into a new file.


man sed


--
The majority of husbands remind me of an orangutang trying to play the violin.
-- Honor'e DeBalzac

Eduardo M KALINOWSKI
edua...@kalinowski.com.br




--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Archive: 
http://lists.debian.org/20120417105220.horde.5niuduisjlfpjxwuaiyv...@mail.kalinowski.com.br

Scripting question

[Chris]

All 

Firstly I petty much suck at scripting so I need help.

I have a file where each line begins with

Smtp:

I would like have the Smtp: replaced with To:  leaving all that follows in each 
line untouched and piped into a new file.

Thanks!!
Chris

Re: scripting question: to parse data with varname=value pattern the easiest way?

[Zhang Weiwu, Beijing]

 On 11/02/2010 05:04 AM, Karl Vogel wrote:
>On the other hand, if someone sneaks something like
>result_04: dc="3" rm /something/valuable
Thank you! very informative, and, kinda fun to read.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4ccff56c.90...@realss.com

Re: scripting question: to parse data with varname=value pattern the easiest way?

[Karl Vogel]

>> On Mon, 01 Nov 2010 15:49:01 +0800, 
>> Zhang Weiwu  said:

Z> A program output is like this:
Z>   result_01: a="23" b="288" c="A_string" ac="34"
Z>   result_02: a="23" b="28" c="A_string_too" dc="3"
Z>   

Z> I am writing a script to output values of b if b is in the result set.

   If your data is trustworthy and follows shell assignment rules like the
   example above, you can abuse sh/ksh/bash:

 #!/bin/bash

 sample='result_01: a="23" b="288" c="A_string" ac="34"
 result_02: a="23" c="A_string_too" dc="3"
 result_03: a="23" b="28" c="A_string_too" dc="3"'

 echo "$sample" | while read str
 do
 unset b   # or whatever you're looking for
 set $str  # will fail horribly if $str is empty
 result=$1
 shift
 eval "$@"
 test "$b" && echo "$result $b"
 done
 exit 0

   On the other hand, if someone sneaks something like
   result_04: dc="3" rm /something/valuable

   into your program output, you'll get a nasty surprise.

-- 
Karl Vogel  I don't speak for the USAF or my company

Most users (myself included) spend most of their time in front of a computer
in a kind of fuzzy autopilot mode, and anything that creates ripples on that
placid lake of unawareness is going to be noticed as a disproportionately
significant problem. --David Harris, creator of Pegasus Mail


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20101101210436.50fccb...@kev.msw.wpafb.af.mil

scripting question: to parse data with varname=value pattern the easiest way?

[Zhang Weiwu]

 Hello.

A program output is like this:

result_01: a="23" b="288" c="A_string" ac="34"
result_02: a="23" b="28" c="A_string_too" dc="3"



I am writing a script to output values of b if b is in the result set.

It would be rather easy to match value of with regular expression:

/b="([^"]*)"/ # \1 would be the value of b.

Out of curiosity I just wonder is there an alternative way? One came to
my mind is to replace result_* with an invoke of awk, then the
name=value pattern become parameter of awk, then I can simply do '{print
$b;}' to get value of b. I tried that without success.



-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4cce70ed.7050...@realss.com

Re: bash scripting question

[Karl Vogel]

Here's something I modified as part of a benchmark script called "fdtree".

-- 
Karl Vogel  I don't speak for the USAF or my company
Dijkstra probably hates me. --Linus Torvalds, in kernel/sched.c

#!/bin/bash
# How to use xdate/xtime/persec:
#
#  START=$(date "+%s")
#  count=10
#  xdate
#  tin=$(xtime)
#
#  # do something time-consuming $count times...
#  tout=$(xtime)
#  set $(persec $tin $tout $count); ttot=$1; results=$2
#  xdate
#  echo "TIME IN, OUT, TOTAL = "$tin, $tout, $ttot
#  echo -e "\tWork per second = " $results

PATH=/usr/local/bin:/bin:/usr/bin
export PATH

function xdate
# Display the date in this form: Thu, 10 Sep 2009 21:22:06.494
{
set $(date "+%a, %d %b %Y %T %N")
ms=$(echo $6 | cut -c1-3)
echo "$1 $2 $3 $4 $5.$ms"
}

function xtime
# Display elapsed runtime to the millisecond.
{
set $(date "+%s %N")
sec=$(($1 - $START))
ms=$(echo $2 | cut -c1-3)
echo "$sec.$ms"
}

function persec
# args: start-second, finish-second, count-things
# returns elapsed time and things that happened per second
# to the millisecond.
{
start=$1
finish=$2
count=$3

if test "$finish" = "$start"; then
echo "0 0"
else
echo $(echo "scale=3; $finish-$start;
$count/($finish-$start)" | bc)
fi
}

function dbg
# debugging prints
{
test $DEBUG -gt 0 && echo -e "$@"
}

tmp=/tmp/t1$$
tmp2=/tmp/t2$$

xdate
START=$(date "+%s")
tin=$(xtime)
echo

for k in 1 2 3 4 5 6 7 8 9 10
do
# Read a short amount of random data.
dd if=/dev/random of=$tmp2 bs=1k count=1 2> /dev/null

# Duplicate it a bunch of times.
cat $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 > $tmp
cat $tmp  $tmp  $tmp  $tmp  $tmp  $tmp  $tmp  $tmp  > $tmp2
cat $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 > $tmp
cat $tmp  $tmp  $tmp  $tmp  $tmp  $tmp  $tmp  $tmp  > $tmp2
cat $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 $tmp2 > $tmp
rm $tmp2

# Generate a hash.
md5sum $tmp
ls -l $tmp
rm $tmp
done

echo
count=10
tout=$(xtime)
set $(persec $tin $tout $count); ttot=$1; results=$2
xdate

echo "TIME IN, OUT, TOTAL = "$tin, $tout, $ttot
echo -e "\tWork per second = " $results
exit 0


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100330012125.34bebb...@bsd118.wpafb.af.mil

Re: bash scripting question

[Ron Johnson]

On 2010-03-29 16:35, Mike McClain wrote:
[snip]


Thanks a lot. Though my error was pointed out as a typo and corrected
a while back your solution using " date '+%s' " is much more elegant
than what I had done.


If you want more (possibly too much) precision:
$ date +'%s.%N'

--
"History does not long entrust the care of freedom to the weak
or the timid."  Dwight Eisenhower


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Archive: http://lists.debian.org/4bb14146.1060...@cox.net

Re: bash scripting question

[Mike McClain]

Hi Josep,

On Mon, Mar 29, 2010 at 02:28:20PM +0200, Josep M. wrote:
> 
> I found these somewhere time ago. check if is what You need:
> 
 

Thanks a lot. Though my error was pointed out as a typo and corrected
a while back your solution using " date '+%s' " is much more elegant
than what I had done.

Mike

PS: Since I posted to and read the list you needn't email me direct.
MM
-- 
Satisfied user of Linux since 1997.
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100329213536.gb29...@playground.mcclains.net

Re: bash scripting question

[Josep M.]

Hello.

I found these somewhere time ago. check if is what You need:





function timer()
{
if [[ $# -eq 0 ]]; then
echo $(date '+%s')
else
local  stime=$1
etime=$(date '+%s')

if [[ -z "$stime" ]]; then stime=$etime; fi

dt=$((etime - stime))
ds=$((dt % 60))
dm=$(((dt / 60) % 60))
dh=$((dt / 3600))
printf '%d:%02d:%02d' $dh $dm $ds
fi
}

 
This is before the command

t=$(timer)

and this is after the command

printf 'Elapsed time: %s\n' $(timer $t)



example

t=$(timer)

copy /dev/null /dev/zero

printf 'Elapsed time: %s\n' $(timer $t)



Greetings
Josep

El vie, 19-03-2010 a las 10:19 -0700, Mike McClain escribió:
> I've written a function to print elapsed time similar to /usr/bin/time
> but can be called at the beginning and end of a script from within
> the script. Occasionally I get an error: '8-08: value too great for base'
> It's caused by the difference in these 2 command strings but I can't for 
> the life of me see what's going on.
> 
> now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr; 
> 09
> 
> str=09; str=${str#*0}; echo $str; 
> 9
> 
> Thanks,
> Mike
> -- 
> Satisfied user of Linux since 1997.
> O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
> 
> 



-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/1269865700.29265.2.ca...@mail.navegants.net

Re: bash scripting question

[Chris Jackson]

Paul E Condon wrote:


Try:

bgn=$(date +%s)
sleep 7
end=$(date +%s)
echo "elapsed seconds = " $(( end - bgn ))



You might also want to experiment with:

ps h -o etime $$

as long as you're happy with it only running under gnu. Prints the 
elapsed time for the shell.


--
Chris Jackson
Shadowcat Systems Ltd.


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Archive: http://lists.debian.org/4ba3dae2.7080...@shadowcat.co.uk

Re: bash scripting question

[Paul E Condon]

On 20100319_101928, Mike McClain wrote:
> I've written a function to print elapsed time similar to /usr/bin/time
> but can be called at the beginning and end of a script from within
> the script. Occasionally I get an error: '8-08: value too great for base'
> It's caused by the difference in these 2 command strings but I can't for 
> the life of me see what's going on.
> 
> now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr; 
> 09
> 
> str=09; str=${str#*0}; echo $str; 
> 9

Try:

bgn=$(date +%s)
sleep 7
end=$(date +%s)
echo "elapsed seconds = " $(( end - bgn ))

Play with values other than '7'.

To actually use in a script, replace the sleep command by whatever you
want to measure the elapsed time for. 

Expressing elapsed time as HH:MM:SS requires a little more work. To 
write that script, use / for divide operator and % for remainder operator,
like in C.

-- 
Paul E Condon   
pecon...@mesanetworks.net


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100319194539.gh7...@big.lan.gnu

Re: bash scripting question

[Mike McClain]

On Fri, Mar 19, 2010 at 06:45:15PM +0100, Sven Joachim wrote:
> On 2010-03-19 18:19 +0100, Mike McClain wrote:
> 
> > I've written a function to print elapsed time similar to /usr/bin/time
> > but can be called at the beginning and end of a script from within
> > the script. Occasionally I get an error: '8-08: value too great for base'
> > It's caused by the difference in these 2 command strings but I can't for 
> > the life of me see what's going on.
> 
> Apparently bash is treating the number as octal because it starts with a
> leading 0.
> 
> Sven

In one case but not the other? 
What makes them treated differently?
Mike
-- 
Satisfied user of Linux since 1997.
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100319190719.ga32...@playground.mcclains.net

Re: bash scripting question

[Mike McClain]

On Fri, Mar 19, 2010 at 10:19:28AM -0700, Mike McClain wrote:


typo right herevv 
> now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr; 

Apologies for troubling all.
Mike (with egg on face)
-- 
Satisfied user of Linux since 1997.
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100319191837.ga32...@playground.mcclains.net

Re: bash scripting question

[Wayne]

Mike McClain wrote:

I've written a function to print elapsed time similar to /usr/bin/time
but can be called at the beginning and end of a script from within
the script. Occasionally I get an error: '8-08: value too great for base'
It's caused by the difference in these 2 command strings but I can't for 
the life of me see what's going on.


now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr; 
09


str=09; str=${str#*0}; echo $str; 



I do it like this
# Set date format
ISO_8601='%Y-%m-%d %H:%M:%S%z'


aptitude safe-upgrade
date "+System Upgrade Completed at: $ISO_8601">>/root/Admin/last-upgrade

the result is

Update started at  : 2010-03-19 08:50:53-0400
Update completed at: 2010-03-19 08:52:19-0400
Upgrade Download started at : 2010-03-19 08:52:19-0400
Download Completed at  : 2010-03-19 09:01:52-0400
System Upgrade Completed at: 2010-03-19 09:05:39-0400

HTH

Wayne


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Archive: http://lists.debian.org/4ba3bd88.4020...@gmail.com

Re: bash scripting question

[Chris Jackson]

Mike McClain wrote:


I've written a function to print elapsed time similar to /usr/bin/time
but can be called at the beginning and end of a script from within
the script. Occasionally I get an error: '8-08: value too great for base'
It's caused by the difference in these 2 command strings but I can't for 
the life of me see what's going on.


now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr; 
09


str=09; str=${str#*0}; echo $str; 
9



Did you mean to echo $startHR - capital 'R' - in the first one? That is 
'9' as you are expecting ;)


--
Chris Jackson
Shadowcat Systems Ltd.


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Archive: http://lists.debian.org/4ba3bb55.50...@shadowcat.co.uk

Re: bash scripting question

[S Scharf]

On Fri, Mar 19, 2010 at 1:19 PM, Mike McClain  wrote:

> I've written a function to print elapsed time similar to /usr/bin/time
> but can be called at the beginning and end of a script from within
> the script. Occasionally I get an error: '8-08: value too great for base'
> It's caused by the difference in these 2 command strings but I can't for
> the life of me see what's going on.
>
> now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr;
> 09
>
> str=09; str=${str#*0}; echo $str;
> 9
>
> Thanks,
> Mike
>
>
Did you wand to echo startHR and not startHr in the first expression?

Stuart

Re: bash scripting question

[Sven Joachim]

On 2010-03-19 18:19 +0100, Mike McClain wrote:

> I've written a function to print elapsed time similar to /usr/bin/time
> but can be called at the beginning and end of a script from within
> the script. Occasionally I get an error: '8-08: value too great for base'
> It's caused by the difference in these 2 command strings but I can't for 
> the life of me see what's going on.

Apparently bash is treating the number as octal because it starts with a
leading 0.

Sven


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87bpekp5n8@turtle.gmx.de

bash scripting question

[Mike McClain]

I've written a function to print elapsed time similar to /usr/bin/time
but can be called at the beginning and end of a script from within
the script. Occasionally I get an error: '8-08: value too great for base'
It's caused by the difference in these 2 command strings but I can't for 
the life of me see what's going on.

now='09:07:16'; startHr=${now%%:*}; startHR=${startHr#*0}; echo $startHr; 
09

str=09; str=${str#*0}; echo $str; 
9

Thanks,
Mike
-- 
Satisfied user of Linux since 1997.
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100319171928.gb31...@playground.mcclains.net

Re: scripting question

[Marc Shapiro]

Kumar Appaiah wrote:

On Wed, Jul 01, 2009 at 09:28:23AM -0500, Kumar Appaiah wrote:

for i in *zzz;do
mv "$i" $(echo "$i"|sed 's/^...//');
done

But I'd recommend one of these: mrename, krename, gprename,
renameutils and more (all apt-gettable, of course).


Oh, and I think prename (or just rename?) comes with Perl. It should
be something as simple as:

rename 's/^...//' *.zzz


Thanks, Kumar, and everyone else.  For now, rename/prename (both names 
work) does just what I need. I will look into the others when I have 
time.  I will also look at the scripts that others suggested so I can 
get more familiar with the techniques.


--
Marc Shapiro
mshapiro...@yahoo.com



--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[gcrimp]

On Wed, Jul 01, 2009 at 07:22:33AM -0700, Marc Shapiro wrote:
> I am sure that this is an easy question for those people who do any  
> reasonable amount of scripting.  I'm just not one of them.
>
> How can I rename all of the files ina directory with the new name being  
> the old name stripped of its leftmost three characters.  If all of the  
> files are off the format:
>
>   xxxy.zzz
>
> I want the new names to be of the format:
>
>   y.zzz
>

for file in `ls all the files` ; do mv $file ${file#xxx}; done


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[Michael Ekstrand]

Marc Shapiro  writes:
> I am sure that this is an easy question for those people who do any
> reasonable amount of scripting.  I'm just not one of them.
>
> How can I rename all of the files ina directory with the new name
> being the old name stripped of its leftmost three characters.  If all
> of the files are off the format:
>
>   xxxy.zzz
>
> I want the new names to be of the format:
>
>   y.zzz
>
> I have hundreds of files like this in each of several directories and
> I really don't want to do it all by hand.  I did some of that,
> already, and I know there must be a better way.

In ZSH with zmv, you can do it with the following:

$ zmv 'xxx(*).zzz' '$1.zzz'

- Michael

-- 
mouse, n: A device for pointing at the xterm in which you want to type.
Confused by the strange files?  I cryptographically sign my messages.
For more information see .


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[Teemu Likonen]

On 2009-07-01 18:20 (+0300), Teemu Likonen wrote:

> find -type f -print0 | xargs -0 sh -c 'for file in "$@"; 
>   do dir=$(dirname -- "$file") && base=$(basename -- "$file") &&
>   (cd "$dir" && echo mv -- "$base" "${base#???}"); done' ignore

Let's simplify it a bit:

find -type f -print0 | xargs -0 sh -c 'for file in "$@"; 
  do dir=$(dirname -- "$file") && base=$(basename -- "$file") &&
  echo mv -- "$file" "$dir/${base#???}"; done' ignore

This does it without "cd".


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[Teemu Likonen]

On 2009-07-01 07:22 (-0700), Marc Shapiro wrote:

> How can I rename all of the files ina directory with the new name
> being the old name stripped of its leftmost three characters. If all
> of the files are off the format:
>
>   xxxy.zzz
>
> I want the new names to be of the format:
>
>   y.zzz
>
> I have hundreds of files like this in each of several directories and
> I really don't want to do it all by hand. I did some of that, already,
> and I know there must be a better way.

Single command which affect the current directory and all
subdirectories:

find -type f -print0 | xargs -0 sh -c 'for file in "$@"; 
  do dir=$(dirname -- "$file") && base=$(basename -- "$file") &&
  (cd "$dir" && echo mv -- "$base" "${base#???}"); done' ignore

It uses "echo mv ..." command so it does nothing but echoes the command
line that would be run inside each directory. Remove the "echo" part to
actually execute the command. Of course you can add your own
file-selection options for "find" command. This works even if you have
spaces in filenames or directory names.

The command launches several processes per file so it's not terribly
elegant but should be OK for one-time rename.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[Boyd Stephen Smith Jr.]

In <4a4b7129.7010...@yahoo.com>, Marc Shapiro wrote:
>I am sure that this is an easy question for those people who do any
>reasonable amount of scripting.  I'm just not one of them.
>
>How can I rename all of the files ina directory with the new name being
>the old name stripped of its leftmost three characters.  If all of the
>files are off the format:
>
>   xxxy.zzz
>
>I want the new names to be of the format:
>
>   y.zzz

for f in *.zzz; do
echo mv "$f" "${f##???}"
done

If you are happy with the commands it outputs, remove the "echo".

You might also want to check for collisions first:
for f in *.zzz; do
printf '%s\n' "${f##???}"
done | sort | uniq -c | grep -v '^1 '

Should show any collisions, prefixed with a collision count.
-- 
Boyd Stephen Smith Jr.   ,= ,-_-. =.
b...@iguanasuicide.net  ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-'
http://iguanasuicide.net/\_/



signature.asc
Description: This is a digitally signed message part.

Re: scripting question

[Scott Gifford]

Marc Shapiro  writes:


[...]

> How can I rename all of the files ina directory with the new name
> being the old name stripped of its leftmost three characters.  

My favorite way to do this is with sed and xargs.  First have sed
print the current name, then use an regexp to change it to the new
name:

ls | sed -e p -e 's/^...//' |xargs -n 2 mv

-Scott.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[Kumar Appaiah]

On Wed, Jul 01, 2009 at 09:28:23AM -0500, Kumar Appaiah wrote:
> On Wed, Jul 01, 2009 at 07:22:33AM -0700, Marc Shapiro wrote:
> > How can I rename all of the files ina directory with the new name
> > being the old name stripped of its leftmost three characters.  If
> > all of the files are off the format:
> > 
> > xxxy.zzz
> > 
> > I want the new names to be of the format:
> > 
> > y.zzz
> > 
> > I have hundreds of files like this in each of several directories
> > and I really don't want to do it all by hand.  I did some of that,
> > already, and I know there must be a better way.
> 
> for i in *zzz;do
> mv "$i" $(echo "$i"|sed 's/^...//');
> done
> 
> But I'd recommend one of these: mrename, krename, gprename,
> renameutils and more (all apt-gettable, of course).

Oh, and I think prename (or just rename?) comes with Perl. It should
be something as simple as:

rename 's/^...//' *.zzz

HTH.

Kumar


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: scripting question

[Kumar Appaiah]

On Wed, Jul 01, 2009 at 07:22:33AM -0700, Marc Shapiro wrote:
> How can I rename all of the files ina directory with the new name
> being the old name stripped of its leftmost three characters.  If
> all of the files are off the format:
> 
>   xxxy.zzz
> 
> I want the new names to be of the format:
> 
>   y.zzz
> 
> I have hundreds of files like this in each of several directories
> and I really don't want to do it all by hand.  I did some of that,
> already, and I know there must be a better way.

for i in *zzz;do
mv "$i" $(echo "$i"|sed 's/^...//');
done

But I'd recommend one of these: mrename, krename, gprename,
renameutils and more (all apt-gettable, of course).

HTH.

Kumar


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

scripting question

[Marc Shapiro]

I am sure that this is an easy question for those people who do any 
reasonable amount of scripting.  I'm just not one of them.


How can I rename all of the files ina directory with the new name being 
the old name stripped of its leftmost three characters.  If all of the 
files are off the format:


xxxy.zzz

I want the new names to be of the format:

y.zzz

I have hundreds of files like this in each of several directories and I 
really don't want to do it all by hand.  I did some of that, already, 
and I know there must be a better way.


All help appreciated.

--
Marc Shapiro
mshapiro...@yahoo.com



--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: Scripting Question - tar

[T o n g]

On Thu, 10 Jul 2008 17:04:38 -0500, Kent West wrote:

>> tar -cvzf - --one-file-system /home | split -b 2000m -

Side note since the problem has been solved. 

You might want to look into dar, which will do splitting for you
automatically, as well as many other desired features for backup
(incremental, has catalog for fast search & restore, and many many 
others...). 

 dar - Disk ARchive: Backup directory tree and files

-- 
Tong (remove underscore(s) to reply)
  http://xpt.sourceforge.net/techdocs/
  http://xpt.sourceforge.net/tools/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Scripting Question - tar

[Kent West]

Owen Townend wrote:

Kent West wrote:

 Am I just not seeing a typo somewhere? Why is my script failing?




Hey,
 You're missing the '-' for stdin

tar -czvf - --one-file-system $sourceDir | split -b 2000m - $targetFile
  


Ah, thank you!


--
Kent West   <")))><
Westing Peacefully - http://kentwest.blogspot.com


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Scripting Question - tar

[Owen Townend]

> > tar -cvzf - --one-file-system /home | split -b 2000m -
> /TERASTATIONBACKUP/GOSHEN/2008/2008-Jul-10.tgz
vs
> > tar -czvf - --one-file-system $sourceDir | split -b 2000m $targetFile

>  Am I just not seeing a typo somewhere? Why is my script failing?
>
>  Thanks!

Hey,
 You're missing the '-' for stdin

tar -czvf - --one-file-system $sourceDir | split -b 2000m - $targetFile

:)

cheers,
Owen.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Scripting Question - tar

[Kent West]

I have this script (stripped down to basics):


#!/bin/bash
sourceDir='/home/'# Directory you're backing up
targetDir='/TERASTATIONBACKUP/GOSHEN/'$(date +%Y)# Destination 
directory for the tarball
targFileBase='GoshensHome'# Desired base part of the 
tarball's filename


targetFile="$targetDir/`date +%Y-%b-%e`.tgz"
echo "Tarring up source into target"
echo " $targetFile"
tar -czvf - --one-file-system $sourceDir | split -b 2000m $targetFile


The script fails with this output:


Tarring up source into target
 /TERASTATIONBACKUP/GOSHEN/2008/2008-Jul-10.tgz

split: cannot open `/TERASTATIONBACKUP/GOSHEN/2008/2008-Jul-10.tgz' 
for reading: No such file or directory

tar: Removing leading `/' from member names


But, if I comment out the tar line above and replace it with this line:

tar -cvzf - --one-file-system /home | split -b 2000m - 
/TERASTATIONBACKUP/GOSHEN/2008/2008-Jul-10.tgz


the script works.

Am I just not seeing a typo somewhere? Why is my script failing?

Thanks!

--
Kent West <*)))><
http://kentwest.blogspot.com


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: scripting question

[Matus UHLAR - fantomas]

On 21.09.07 08:18, Michael Martinell wrote:
> My script is as follows:
> #!/bin/sh
> 
> TERM=vt100
> export TERM

forcing TERM in script is very bad idea, and in this script also useless.

> date && echo " Spam Count" && /bin/more /var/log/syslog | /bin/grep -c
> 'identified spam' && echo " " && echo "Clean Message Count " &&  /bin/more
> /var/log/syslog | /bin/grep -c 'clean message'

date

echo -n "Spam Count "
grep -Fc 'identified spam' /var/log/syslog

echo -n "Ham Count "
grep -Fc 'clean message' /var/log/syslog

... easier to read

-- 
Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"They say when you play that M$ CD backward you can hear satanic messages."
"That's nothing. If you play it forward it will install Windows."


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: [OT] Scripting question: the length limit of a list?

[s. keeling]

Richard Lyons <[EMAIL PROTECTED]>:
>  On Thu, Nov 01, 2007 at 01:49:03PM -0700, Mike Bird wrote:
> > On Thursday 01 November 2007 13:07, Wei Chen wrote:
> > > I would like to write a bash script like the following one:
> > >
> > > for i in `some program that outputs a word list`
> > > do
> > >   echo $i
> > > done
> > >
> > > where the word list can be very very long. I wonder what is the upper 
> > > bound
> > > limit of the length of word lists in "for" loop of a bash script, or
> > > does it only
> > > depend on the hardware (say, RAM)? Thank you in advance.
> > 
> > Assuming that the words are output one per line, something
> > like the following can handle lists of any size:
> > 
> > some program | while read i; do echo $i; done
> 
>  A wild thought, but if you want to count the words in the output can you
>  not pipe it to wc?  I haven't tried, this is just a thought...

Not wild at all.  Most *ix apps (of the commandline variety, at least)
are designed to expect input on their stdin if it's there.  wc's no
exception.  "some_program | wc -l" will give you the number of lines
of output produced by some_program.  Exercise for you: play around
with

 some_program | grep -c "some_string"

which does the same thing, but only on lines you want to know about.


-- 
Any technology distinguishable from magic is insufficiently advanced.
(*)http://blinkynet.net/comp/uip5.html  Linux Counter #80292
- -http://www.faqs.org/rfcs/rfc1855.htmlPlease, don't Cc: me.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: [OT] Scripting question: the length limit of a list?

[Wei Chen]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mike Bird wrote:
> On Thursday 01 November 2007 13:07, Wei Chen wrote:
>> I would like to write a bash script like the following one:
>>
>> for i in `some program that outputs a word list`
>> do
>>   echo $i
>> done
>>
>> where the word list can be very very long. I wonder what is the upper bound
>> limit of the length of word lists in "for" loop of a bash script, or
>> does it only
>> depend on the hardware (say, RAM)? Thank you in advance.
> 
> Assuming that the words are output one per line, something
> like the following can handle lists of any size:
> 
> some program | while read i; do echo $i; done
> 

Thanks for all the replies. I think the method of piping ``while read''
can be a good replacement. I have changed my script accordingly. Thank you.

- --
Cheers,

Wei Chen
http://www.acplex.com/people/wchen/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHKstXCIqXQV6BF28RAqvxAKCDrvpynyHIXjyf24/fUtemm93h5ACgxR9c
ROHPIpoaes/k767CXx/iWl4=
=ygd+
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: [OT] Scripting question: the length limit of a list?

[Ron Johnson]

On 11/01/07 15:07, Wei Chen wrote:
> Hi,
> 
> I would like to write a bash script like the following one:
> 
> for i in `some program that outputs a word list`
> do
>   echo $i
> done
> 
> where the word list can be very very long. I wonder what is the upper bound
> limit of the length of word lists in "for" loop of a bash script, or
> does it only
> depend on the hardware (say, RAM)? Thank you in advance.

I *think* that bash has a 128KB buffer limit.

-- 
Ron Johnson, Jr.
Jefferson LA  USA

Give a man a fish, and he eats for a day.
Hit him with a fish, and he goes away for good!


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: [OT] Scripting question: the length limit of a list?

[Richard Lyons]

On Thu, Nov 01, 2007 at 01:49:03PM -0700, Mike Bird wrote:

> On Thursday 01 November 2007 13:07, Wei Chen wrote:
> > I would like to write a bash script like the following one:
> >
> > for i in `some program that outputs a word list`
> > do
> >   echo $i
> > done
> >
> > where the word list can be very very long. I wonder what is the upper bound
> > limit of the length of word lists in "for" loop of a bash script, or
> > does it only
> > depend on the hardware (say, RAM)? Thank you in advance.
> 
> Assuming that the words are output one per line, something
> like the following can handle lists of any size:
> 
> some program | while read i; do echo $i; done

A wild thought, but if you want to count the words in the output can you
not pipe it to wc?  I haven't tried, this is just a thought...

-- 
richard


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: [OT] Scripting question: the length limit of a list?

[Mike Bird]

On Thursday 01 November 2007 13:07, Wei Chen wrote:
> I would like to write a bash script like the following one:
>
> for i in `some program that outputs a word list`
> do
>   echo $i
> done
>
> where the word list can be very very long. I wonder what is the upper bound
> limit of the length of word lists in "for" loop of a bash script, or
> does it only
> depend on the hardware (say, RAM)? Thank you in advance.

Assuming that the words are output one per line, something
like the following can handle lists of any size:

some program | while read i; do echo $i; done

--Mike Bird


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

[OT] Scripting question: the length limit of a list?

[Wei Chen]

Hi,

I would like to write a bash script like the following one:

for i in `some program that outputs a word list`
do
  echo $i
done

where the word list can be very very long. I wonder what is the upper bound
limit of the length of word lists in "for" loop of a bash script, or
does it only
depend on the hardware (say, RAM)? Thank you in advance.

-- 
Cheers,
Wei
http://www.acplex.com/people/wchen/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: scripting question

[Jude DaShiell]

One possible approach would be to use a few files and use paste on those 
files where:

dfile holds date,
mfile holds good messages,
sfile holds spam messages
tfile is temporary file

paste dfile mfile >tfile
paste tfile sfile >dfile
rm mfile
rm sfile
rm tfile
cat dfile

hth.






--
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: scripting question

[Peter Teunissen]

On 21-sep-2007, at 15:51, Michael Martinell wrote:


Thanks - that was exactly what I was looking for.

Now I just need to find a good scripting tutorial.  :)



Try http://www.tldp.org/LDP/Bash-Beginners-Guide/html/index.html

That's where I learned my scripting basics.


Peter


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

RE: scripting question

[Michael Martinell]

Thanks - that was exactly what I was looking for.

Now I just need to find a good scripting tutorial.  :)

-Original Message-
From: Michael Marsh [mailto:[EMAIL PROTECTED] 
Sent: Friday, September 21, 2007 8:38 AM
To: debian-user@lists.debian.org
Subject: Re: scripting question

On 9/21/07, Michael Martinell <[EMAIL PROTECTED]> wrote:
> I have a simple script that counts up the number of spam messages each day
> and prints the total number into a text field.  This is fine as far as it
> goes, however I would like to also include the date and the number of
> non-spam messages.
>
> I can get this to run, however each piece of information is printed on
it's
> own line.  I would like to have all of the results append in my text file
on
> the same line so that I can easily import it into a spreadsheet or
database.
>
> How would I go about doing this?
>
> My script is as follows:
...
> date && echo " Spam Count" && /bin/more /var/log/syslog | /bin/grep -c
> 'identified spam' && echo " " && echo "Clean Message Count " &&  /bin/more
> /var/log/syslog | /bin/grep -c 'clean message'

You can concatenate things in a single echo simply by passing multiple
arguments.  Slinging a few backticks, and you can do something like:

echo `date` " dollar signs: " `grep -c \$ ~/.bashrc` " comments: "
`grep -c \# ~/.bashrc`

Put the file name in a variable, and it should be even simpler.

-- 
Michael A. Marsh
http://www.umiacs.umd.edu/~mmarsh
http://mamarsh.blogspot.com
http://36pints.blogspot.com


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact
[EMAIL PROTECTED]



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: scripting question

[Michael Marsh]

On 9/21/07, Michael Martinell <[EMAIL PROTECTED]> wrote:
> I have a simple script that counts up the number of spam messages each day
> and prints the total number into a text field.  This is fine as far as it
> goes, however I would like to also include the date and the number of
> non-spam messages.
>
> I can get this to run, however each piece of information is printed on it's
> own line.  I would like to have all of the results append in my text file on
> the same line so that I can easily import it into a spreadsheet or database.
>
> How would I go about doing this?
>
> My script is as follows:
...
> date && echo " Spam Count" && /bin/more /var/log/syslog | /bin/grep -c
> 'identified spam' && echo " " && echo "Clean Message Count " &&  /bin/more
> /var/log/syslog | /bin/grep -c 'clean message'

You can concatenate things in a single echo simply by passing multiple
arguments.  Slinging a few backticks, and you can do something like:

echo `date` " dollar signs: " `grep -c \$ ~/.bashrc` " comments: "
`grep -c \# ~/.bashrc`

Put the file name in a variable, and it should be even simpler.

-- 
Michael A. Marsh
http://www.umiacs.umd.edu/~mmarsh
http://mamarsh.blogspot.com
http://36pints.blogspot.com


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: scripting question

[Neil Watson]

Man echo reveals that the -n switch prevents echo from appending a new
line.  Also, you do not need to use more (or less) with grep.  Grep can
take a file agrument.  Refer to grep's man page for more information.

--
Neil Watson | Debian Linux
System Administrator| Uptime 6 days
http://watson-wilson.ca


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

scripting question

[Michael Martinell]

I have a simple script that counts up the number of spam messages each day
and prints the total number into a text field.  This is fine as far as it
goes, however I would like to also include the date and the number of
non-spam messages.

 

I can get this to run, however each piece of information is printed on it's
own line.  I would like to have all of the results append in my text file on
the same line so that I can easily import it into a spreadsheet or database.

 

How would I go about doing this?  

 

My script is as follows:
#!/bin/sh

TERM=vt100

export TERM

exec >> /root/countspam.txt

date && echo " Spam Count" && /bin/more /var/log/syslog | /bin/grep -c
'identified spam' && echo " " && echo "Clean Message Count " &&  /bin/more
/var/log/syslog | /bin/grep -c 'clean message'

Re: bash scripting question

[Bob McGowan]

Tyler Smith wrote:

On 2007-05-17, Bob McGowan <[EMAIL PROTECTED]> wrote:
Some general comments, mostly aimed at making your code cleaner without 
changing what it does.


First, both 'echo' and 'printf' put their results on standard out.  Your 
call of 'printf' is inside command substitution, so its STDOUT becomes 

--snipped stuff--


Oh, great, thanks. I added the echo to stop getting the complaint
about unknown command, but this is better.


So, cut and pasted from a bash shell:

$ lab_num=41
$ lab_let=$(printf "\x$lab_num")
$ echo $lab_let
A
$ ((lab_num++))
$ lab_let=$(printf "\x$lab_num")
$ echo $lab_let
B


Much improved!

This would need two loops, the outer to increment the 'tens' digit, the 
inner to increment the 'ones' digit, but it would do the trick.  For 
example:


x=(0 1 2 3 4 5 6 7 8 9 A B C D E F)



I knew there was an array form in bash, but I couldn't find it. I'm
working from the O'Reilly book classic shell scripting, and the only
reference to arrays is in relation to awk scripts. This is a big help.

Thanks alot!

Tyler




You're welcome ;)

Bob


smime.p7s
Description: S/MIME Cryptographic Signature

Re: bash scripting question

[Alex Samad]

On Thu, May 17, 2007 at 03:40:15AM +, Tyler Smith wrote:
> On 2007-05-17, Bob McGowan <[EMAIL PROTECTED]> wrote:
> >
> > Some general comments, mostly aimed at making your code cleaner without 
> > changing what it does.
> >
> > First, both 'echo' and 'printf' put their results on standard out.  Your 
> > call of 'printf' is inside command substitution, so its STDOUT becomes 
> > the command line for 'echo' which just prints to its STDOUT.  Why the 
> > double print out?  Just do:
> >
> > lab_let=$(printf "\\x$(echo $lab_num)")
> >
> > Next, the 'echo $lab_num' is not needed, $lab_num can stand alone:
> >
> > lab_let=$(printf "\\x$lab_num")

another thing to remember is you can enclose your variable names in {} so you 
could do something like 

lab_let=$(printf "\\x${lab_num}Moretext")

> >
> > And, the double quotes escape things, too, so the double backslash is 
> > not needed:
> >
> > lab_let=$(printf "\x$lab_num")
> >
> 
> Thank you for this! I started out with something a little more
> complicated, without the variable, trying to insert the hex character
> directly into another command. And my testing required that I use a
> form that would print something to the command line. I got very worked
> up trying to sort out the syntax, and obviously over-did it.
> 
> 
> > Then, the line where you increment lab_num can also be simpler.  In bash 
> > the $((...)) alone on a line will replace itself with the result 
> > (command substitution, again).  But, leave off the leading $ sign, and 
> > it just does the increment:
> >
> > ((lab_num++))
> 
> Oh, great, thanks. I added the echo to stop getting the complaint
> about unknown command, but this is better.
> 
> >
> > So, cut and pasted from a bash shell:
> >
> > $ lab_num=41
> > $ lab_let=$(printf "\x$lab_num")
> > $ echo $lab_let
> > A
> > $ ((lab_num++))
> > $ lab_let=$(printf "\x$lab_num")
> > $ echo $lab_let
> > B
> 
> Much improved!
> 
> >
> > This would need two loops, the outer to increment the 'tens' digit, the 
> > inner to increment the 'ones' digit, but it would do the trick.  For 
> > example:
> >
> > x=(0 1 2 3 4 5 6 7 8 9 A B C D E F)
> >
> 
> I knew there was an array form in bash, but I couldn't find it. I'm
> working from the O'Reilly book classic shell scripting, and the only
> reference to arrays is in relation to awk scripts. This is a big help.
> 
> Thanks alot!
> 
> Tyler
> 
> 
> -- 
> To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
> with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
> 
> 


signature.asc
Description: Digital signature

Re: bash scripting question

[Tyler Smith]

On 2007-05-17, Bob McGowan <[EMAIL PROTECTED]> wrote:
>
> Some general comments, mostly aimed at making your code cleaner without 
> changing what it does.
>
> First, both 'echo' and 'printf' put their results on standard out.  Your 
> call of 'printf' is inside command substitution, so its STDOUT becomes 
> the command line for 'echo' which just prints to its STDOUT.  Why the 
> double print out?  Just do:
>
> lab_let=$(printf "\\x$(echo $lab_num)")
>
> Next, the 'echo $lab_num' is not needed, $lab_num can stand alone:
>
> lab_let=$(printf "\\x$lab_num")
>
> And, the double quotes escape things, too, so the double backslash is 
> not needed:
>
> lab_let=$(printf "\x$lab_num")
>

Thank you for this! I started out with something a little more
complicated, without the variable, trying to insert the hex character
directly into another command. And my testing required that I use a
form that would print something to the command line. I got very worked
up trying to sort out the syntax, and obviously over-did it.


> Then, the line where you increment lab_num can also be simpler.  In bash 
> the $((...)) alone on a line will replace itself with the result 
> (command substitution, again).  But, leave off the leading $ sign, and 
> it just does the increment:
>
> ((lab_num++))

Oh, great, thanks. I added the echo to stop getting the complaint
about unknown command, but this is better.

>
> So, cut and pasted from a bash shell:
>
> $ lab_num=41
> $ lab_let=$(printf "\x$lab_num")
> $ echo $lab_let
> A
> $ ((lab_num++))
> $ lab_let=$(printf "\x$lab_num")
> $ echo $lab_let
> B

Much improved!

>
> This would need two loops, the outer to increment the 'tens' digit, the 
> inner to increment the 'ones' digit, but it would do the trick.  For 
> example:
>
> x=(0 1 2 3 4 5 6 7 8 9 A B C D E F)
>

I knew there was an array form in bash, but I couldn't find it. I'm
working from the O'Reilly book classic shell scripting, and the only
reference to arrays is in relation to awk scripts. This is a big help.

Thanks alot!

Tyler


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: bash scripting question

[Bob McGowan]

Tyler Smith wrote:

Hi,

I've got a question about a short bash script I wrote. I need it to

--snipped--


#!/bin/bash

lab_num=41

for map_name in aest_90 bush_90 carol_90 comp_90 \
hirs_90 roan_90 swan_90 vir_90 ; 


  do

  lab_let=$(echo -n $(printf "\\x$(echo $lab_num)"))

  echo "
  $lab_let
  $map_name
  ${map_name}.ps" ;

  echo $((lab_num++)) > /dev/null ;

  done

--snipped--

Some general comments, mostly aimed at making your code cleaner without 
changing what it does.


First, both 'echo' and 'printf' put their results on standard out.  Your 
call of 'printf' is inside command substitution, so its STDOUT becomes 
the command line for 'echo' which just prints to its STDOUT.  Why the 
double print out?  Just do:


lab_let=$(printf "\\x$(echo $lab_num)")

Next, the 'echo $lab_num' is not needed, $lab_num can stand alone:

lab_let=$(printf "\\x$lab_num")

And, the double quotes escape things, too, so the double backslash is 
not needed:


lab_let=$(printf "\x$lab_num")

Then, the line where you increment lab_num can also be simpler.  In bash 
the $((...)) alone on a line will replace itself with the result 
(command substitution, again).  But, leave off the leading $ sign, and 
it just does the increment:


((lab_num++))

So, cut and pasted from a bash shell:

$ lab_num=41
$ lab_let=$(printf "\x$lab_num")
$ echo $lab_let
A
$ ((lab_num++))
$ lab_let=$(printf "\x$lab_num")
$ echo $lab_let
B




Thanks,

Tyler




Since you're using bash, you may also find it convenient to put your hex 
digits into an array, which you can then subscript into with decimal 
numbers, to build the hex values needed to print other characters.


This would need two loops, the outer to increment the 'tens' digit, the 
inner to increment the 'ones' digit, but it would do the trick.  For 
example:


x=(0 1 2 3 4 5 6 7 8 9 A B C D E F)

tens=0
digits=0

while [ $tens -lt 3 ]
do
  while [ $digits -lt 16 ]
  do
echo ${x[$tens]}${x[$digits]}
((digits++))
  done
  digits=0
  ((tens++))
done

The result is:

00
01
02
.
.
.
2D
2E
2F

Change the 'tens' and 'digits' as needed to get the right starting value.

Bob


smime.p7s
Description: S/MIME Cryptographic Signature

Re: bash scripting question

[Tyler Smith]

On 2007-05-16, Karl E. Jorgensen <[EMAIL PROTECTED]> wrote:
>> This was the only way I could figure out to loop from A to H. But
>> since it works on hex escape codes, it won't work past 9. Is there a
>> cleaner, more general way to do this?
>
> I think there is:
>
> #!/bin/bash
>
> ( cat < A aest_90
> B bush_90
> C carol_90
> D comp_90
> E hirs_90
> F roan_90
> G swan_90
> H vir_90
> !
> ) | while read letter name
> do
> printf '%s\n%s\n%s.ps\n\n' "$letter" "$name" "$name"
> done
>
> Hope this helps
>

Very much! That's perfect.

Cheers,

Tyler


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: bash scripting question

[Karl E. Jorgensen]

On Wed, May 16, 2007 at 08:46:37PM +, Tyler Smith wrote:
> Hi,
> 
> I've got a question about a short bash script I wrote. I need it to
> loop over a number of names, and pass a command to grass that includes
> two variations of those names. That was easy. Harder was getting
> getting a letter included in each iteration, starting with A for the
> first one and going up by one each iteration. What I came up with,
> with extra bits snipped, is:
> 
> #!/bin/bash
> 
> lab_num=41
> 
> for map_name in aest_90 bush_90 carol_90 comp_90 \
> hirs_90 roan_90 swan_90 vir_90 ; 
> 
>   do
> 
>   lab_let=$(echo -n $(printf "\\x$(echo $lab_num)"))
> 
>   echo "
>   $lab_let
>   $map_name
>   ${map_name}.ps" ;
> 
>   echo $((lab_num++)) > /dev/null ;
> 
>   done
> 
> The multi-line echo is passing instructions to a GRASS command, and in
> the full script it works fine. This example runs fine without grass as
> a demonstration. What I'm wondering about is the line:
> 
>   lab_let=$(echo -n $(printf "\\x$(echo $lab_num)"))
> 
> This was the only way I could figure out to loop from A to H. But
> since it works on hex escape codes, it won't work past 9. Is there a
> cleaner, more general way to do this?

I think there is:

#!/bin/bash

( cat 

bash scripting question

[Tyler Smith]

Hi,

I've got a question about a short bash script I wrote. I need it to
loop over a number of names, and pass a command to grass that includes
two variations of those names. That was easy. Harder was getting
getting a letter included in each iteration, starting with A for the
first one and going up by one each iteration. What I came up with,
with extra bits snipped, is:

#!/bin/bash

lab_num=41

for map_name in aest_90 bush_90 carol_90 comp_90 \
hirs_90 roan_90 swan_90 vir_90 ; 

  do

  lab_let=$(echo -n $(printf "\\x$(echo $lab_num)"))

  echo "
  $lab_let
  $map_name
  ${map_name}.ps" ;

  echo $((lab_num++)) > /dev/null ;

  done

The multi-line echo is passing instructions to a GRASS command, and in
the full script it works fine. This example runs fine without grass as
a demonstration. What I'm wondering about is the line:

  lab_let=$(echo -n $(printf "\\x$(echo $lab_num)"))

This was the only way I could figure out to loop from A to H. But
since it works on hex escape codes, it won't work past 9. Is there a
cleaner, more general way to do this?


Thanks,

Tyler


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Douglas Tutty]

Having all that whitespace in the 'wrong' spot breaks the idea of
splitting words based on their being surrounded by whitespace.  So get
rid of __all__ whitespace.  Then use other logic find what you want.
E.g. if you want the 'word' following the 'word' processor, find the
first occurance of 'processor' in the string (which contains the whole
file), then look at each following character one at a time to see if it
meets the criteria for being in the next word.  E.g. if the following
word must be a number and the word after that is not a number, take each
successive character until its not a number and there you have your
target word. 

This would be easy in Python but since I don't do RE I couldn't begin to
solve it using anything else.  

I still like the idea of fixing the source of these mangled files.

Doug.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[michael]

On Wed, 2006-11-29 at 09:36 -0900, Ken Irving wrote:
> On Wed, Nov 29, 2006 at 02:32:37PM +, michael wrote:
> > I guess a complete rephrase is best. 
> > 
> > What I want is "how many processors does each WAITING job in lsf queues
> > require?". From 'bhist' I get outputs such as below (see whitespace
> > anywhere in "num Processors") and cannot determine a sure way of always
> > parsing it...
> 
> In the brute force perl solution previously shown, just add whitespace
> to the character class, [\s\n-], which is inserted between every target
> character in the regular expression.  This would be similar in awk, sed, 
> grep, or other tool using regular expressions.
> 
> #!/usr/bin/perl -w
> use strict;   
> my $source = join '', <>;  # get all the data into a string
> my $t = '[\s\n-]'; # define a regexp character class
> print "$1\n" while #   to be between each character
> $source =~ m/(\d+)\s+P$t*r$t*o$t*c$t*e$t*s$t*s$t*o$t*r/msg;
> 
> Other schemes previously shown would probably work with trivial changes,
> e.g., using tr to delete (-d) or squeeze (-s) runs of spaces or newlines,
> etc.
> 
> Unless this is a one-off task (which it seems like it isn't), I'd
> suggest looking into fixing whatever is generating the screwed-up output
> in the first place. Failing that, use tr/sed/python/perl/ruby/BASIC
> whatever to filter the output to something more sensible, i.e., normalize
> it, and don't try to do it in one step.
> 
> Ken

Getting rid of all white space brings own problems - as in 'bla 12
Processors' becomes one (no white space) string.

It's a sort of one-off but I can't fix the LSF queuing system. However,
here's a fix that works (given I know num of whitespace on 2nd, etc,
lines):

function getWAITinfo() {
  echo $jobNum\: `bhist -l $jobNum|sed 's/ //g'|sed
's/ ^//'
|tr -d '\n'|tr ' ' '\n'|grep -B1 Processors|tr '\n' ' '`
}

where `bhist -l` is what generates the info to be parsed.

Thanks to all.

Michael


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Ken Irving]

On Wed, Nov 29, 2006 at 02:32:37PM +, michael wrote:
> I guess a complete rephrase is best. 
> 
> What I want is "how many processors does each WAITING job in lsf queues
> require?". From 'bhist' I get outputs such as below (see whitespace
> anywhere in "num Processors") and cannot determine a sure way of always
> parsing it...

In the brute force perl solution previously shown, just add whitespace
to the character class, [\s\n-], which is inserted between every target
character in the regular expression.  This would be similar in awk, sed, 
grep, or other tool using regular expressions.

#!/usr/bin/perl -w
use strict;   
my $source = join '', <>;  # get all the data into a string
my $t = '[\s\n-]'; # define a regexp character class
print "$1\n" while #   to be between each character
$source =~ m/(\d+)\s+P$t*r$t*o$t*c$t*e$t*s$t*s$t*o$t*r/msg;

Other schemes previously shown would probably work with trivial changes,
e.g., using tr to delete (-d) or squeeze (-s) runs of spaces or newlines,
etc.

Unless this is a one-off task (which it seems like it isn't), I'd
suggest looking into fixing whatever is generating the screwed-up output
in the first place. Failing that, use tr/sed/python/perl/ruby/BASIC
whatever to filter the output to something more sensible, i.e., normalize
it, and don't try to do it in one step.

Ken

> 
> Thanks, Michael
> 
> EXAMPLES:
> 
> 
> 
> ~/bin$ bhist -l 10418;bhist -l 10587;bhist -l 10601
> 
> Job <10418>, Job Name <3d>, User , Project , Command
> <#BSUB 
>  -n 128;#BSUB -W 6:00;#BSUB -J 3d;#BSUB -o %
> J.out;#BSUB -w 
>  'ended(10417)';./cont>
> Tue Nov 28 21:35:48: Submitted from host , to Queue ,
> CWD <$
>  HOME/scratch/3d_newgc>, Output File <%J.out>, 128
> Processo
>  rs Requested, Dependency Condition ;
> 
>  RUNLIMIT
> ...

-- 
Ken Irving, [EMAIL PROTECTED], 907-474-6152
Water and Environmental Research Center
Institute of Northern Engineering
University of Alaska, Fairbanks


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[michael]

I guess a complete rephrase is best. 

What I want is "how many processors does each WAITING job in lsf queues
require?". From 'bhist' I get outputs such as below (see whitespace
anywhere in "num Processors") and cannot determine a sure way of always
parsing it...

Thanks, Michael

EXAMPLES:



~/bin$ bhist -l 10418;bhist -l 10587;bhist -l 10601

Job <10418>, Job Name <3d>, User , Project , Command
<#BSUB 
 -n 128;#BSUB -W 6:00;#BSUB -J 3d;#BSUB -o %
J.out;#BSUB -w 
 'ended(10417)';./cont>
Tue Nov 28 21:35:48: Submitted from host , to Queue ,
CWD <$
 HOME/scratch/3d_newgc>, Output File <%J.out>, 128
Processo
 rs Requested, Dependency Condition ;

 RUNLIMIT
 360.0 min of horace3

Summary of time in seconds spent in various states by  Wed Nov 29
14:28:16
  PEND PSUSPRUN  USUSPSSUSPUNKWNTOTAL
  607480000060748   


Job <10587>, Job Name , User , Project ,
Command 
 
Wed Nov 29 11:32:34: Submitted from host , to Queue ,
CWD <$
 HOME>, Output File , 99 Processors
Requested;

 RUNLIMIT
 1440.0 min of horace3

Summary of time in seconds spent in various states by  Wed Nov 29
14:28:16
  PEND PSUSPRUN  USUSPSSUSPUNKWNTOTAL
  105420000010542   


Job <10601>, Job Name , User , Project ,
Command 
 
Wed Nov 29 11:55:26: Submitted from host , to Queue ,
CWD <$
 HOME>, Output File , Error File , 120 
 Processors Requested;

 RUNLIMIT
 1440.0 min of horace3

Summary of time in seconds spent in various states by  Wed Nov 29
14:28:16
  PEND PSUSPRUN  USUSPSSUSPUNKWNTOTAL
  9170 000009170

~/bin$ 


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[michael]

On Mon, 2006-11-06 at 01:10 +1100, John O'Hagan wrote:

> Or the whole thing could even be done with (I think!):
> 
> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 'Processor\|--'
> 
> i.e., remove the newlines, replace all the spaces with newlines, then grep 
> for 
> the line before "Processor".
>  


Okay, I mis-described my problem. It may have any number (>0) of spaces
between the fields eg


blah bals 38 Pro
cessor

or

bnd c cds sd dcs sd  38
Processor

ta, Michael

PS: ta for the intro to 'tr'!


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Steve Lamb]

John O'Hagan wrote:
> On Thursday 09 November 2006 03:40, Andrew Sackville-West wrote:
>> SO I wonder what happened to the OP? Is he just watching waiting for
>> the right solution, or is he long gone?

> [...]

> OP? What OP? :)

The one who hopefully got an A for his answer.

-- 
 Steve C. Lamb | But who decides what they dream?
   PGP Key: 8B6E99C5   |   And dream I do...
---+-



signature.asc
Description: OpenPGP digital signature

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Thursday 09 November 2006 03:40, Andrew Sackville-West wrote:
> On Thu, Nov 09, 2006 at 12:52:57AM +1100, John O'Hagan wrote:

[...]

> > 
> > while read i ; do
> >
> > if [[ $(echo "$i" | grep \\-\$ ) ]]; then
> >
> > i=$( echo "$i" | sed s/-\$//)
> > echo "$i"
> > else echo "$i"' '
> > fi
> >
> > done < IN | tr -d '\n' | tr ' ' '\n' | grep -B1 'Processor' |
> > grep -v 'Processor\|--'
> > 
> >
> > This removes hyphens at the end of lines or else adds a space, which
> > converts
>
> here's a simpler way to do that, I think ;)
>
> tr '\n' ' ' | sed 's/- //g' | tr ' ' '\n' | grep -B1 'Processor' | grep
> -v 'Processor\|--'
>
> replace the newlines with spaces. then use sed as it matches better
> than tr to strip occurences of '- '. The assumption here is that
> hypens don't appear at the end of words and they only occur because we
> created them with our tr '\n' ' '.

I was worried (well, that's a bit strong!) about words ending with hyphens, so 
I went with your sed approach:

#sed s/$/' '/g  SO I wonder what happened to the OP? Is he just watching waiting for
> the right solution, or is he long gone?
>

[...]

OP? What OP? :)

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Thu, Nov 09, 2006 at 12:52:57AM +1100, John O'Hagan wrote:
> >
> > tr -d '-\n'  >
> > | grep -v 'Processor\|--'
> > 
> 
> [...]
> 
> Aha! You're right, my lines fail on the edge cases, and also when the target 
> word is hyphenated.
> 
> Your ingenious approach didn't always work either [1]; but it revealed (to 
> me) 

yup. okay, another one below

> that there will be unresolvable ambiguities in the IN file unless:
>  
> EITHER: A) lines are broken arbitrarily without hyphenation, in which case 
> newlines have no significance, spaces between words must preserved and we can 
> use:
> 
> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 'Processor\|--'
> 
> or in Python:
> 
> #for i in  open('IN').read().replace('\n', '').split('Processor')[0:-1]:
> # print i.split()[-1] 
> 
> OR: B) broken words are hyphenated, and unhyphenated newlines are equivalent 
> to spaces, in which case we could use something like:
> 
> 
> while read i ; do
> 
>   if [[ $(echo "$i" | grep \\-\$ ) ]]; then
>   
>   i=$( echo "$i" | sed s/-\$//) 
>   echo "$i" 
>   else echo "$i"' '
>   fi
> 
> done < IN | tr -d '\n' | tr ' ' '\n' | grep -B1 'Processor' | 
> grep -v 'Processor\|--'
> 
> 
> This removes hyphens at the end of lines or else adds a space, which converts 
here's a simpler way to do that, I think ;)

tr '\n' ' ' | sed 's/- //g' | tr ' ' '\n' | grep -B1 'Processor' | grep
-v 'Processor\|--'

replace the newlines with spaces. then use sed as it matches better
than tr to strip occurences of '- '. The assumption here is that
hypens don't appear at the end of words and they only occur because we
created them with our tr '\n' ' '. 


SO I wonder what happened to the OP? Is he just watching waiting for
the right solution, or is he long gone?

A 
> 
> 
> [1] I tried Andrew's solution above and found that it only always worked on 
> the unhyphenated case, I think because tr treats its arguments as character 
> sets, not expresions, so that tr -d '\-\n' (note the escape required for the 
> hyphen) deletes any hyphens or newlines, not just that combination.

yeah, that's what happens when you only think about the problem and
don't actually test it.

the above was briefly tested...

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Wednesday 08 November 2006 03:08, Andrew Sackville-West wrote:
> On Wed, Nov 08, 2006 at 02:51:20AM +1100, John O'Hagan wrote:
> > I tried this, and found that replacing the newlines with spaces stops the
> > grep from working because it puts spaces in the middle of any occurrences
> > of "Processor", but I see what you mean about the edge case. I think this
> > version takes care of it, plus it is hyphen-agnostic:
> >
> >
> > tr  -d '\n'   > tr -s ' ' '\n' | grep -B1 'Processor' |  grep -v 'Processor\|--'
> >
> > removing newlines, replacing all cases of (non-)hyphenated "Processor"
> > with a space followed by "Processor", then doing the grep. And here's a
> > Python version using the re module to deal with the hyphens ( the edge
> > case takes care of itself here):
> >
> > import re
> >
> > for i in re.split('P-?r-?o-?c-?e-?s-?s-?o-?r',  
>>open('IN').read().replace('\n', ''))[0:-1]:
> > print i.split()[-1]
>
> huh, I'm not sure. I played with it a little and here's another
> problem
>
> here is some testing
> data processor
>
> will return 'testingdata' because the newlines get stripped out
> leaving no space between the words. so..
>
> first, replace all '-\n' with '' so we dehyphenate any hyphenated
> words split by a newline. there will be some words that should be
> hyphenated but lose that hyphen, however, I think that's probably a
> pretty rare case and it ignores any mid-line hyphenated words. also
> makes it easier to grep as we can ignore the hyphens in processor  next
> replace all '\n' with ' ' so that we avoid the above problem. then
> replace any single-or-more occurance of ' ' with '\n' to split the
> words into seperate lines and finally grep away.
>
> tr -d '-\n' 
> | grep -v 'Processor\|--'
> 

[...]

Aha! You're right, my lines fail on the edge cases, and also when the target 
word is hyphenated.

Your ingenious approach didn't always work either [1]; but it revealed (to me) 
that there will be unresolvable ambiguities in the IN file unless:
 
EITHER: A) lines are broken arbitrarily without hyphenation, in which case 
newlines have no significance, spaces between words must preserved and we can 
use:

#tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 'Processor\|--'

or in Python:

#for i in  open('IN').read().replace('\n', '').split('Processor')[0:-1]:
#   print i.split()[-1] 

OR: B) broken words are hyphenated, and unhyphenated newlines are equivalent 
to spaces, in which case we could use something like:


while read i ; do

if [[ $(echo "$i" | grep \\-\$ ) ]]; then

i=$( echo "$i" | sed s/-\$//) 
echo "$i" 
else echo "$i"' '
fi

done < IN | tr -d '\n' | tr ' ' '\n' | grep -B1 'Processor' | 
grep -v 'Processor\|--'


This removes hyphens at the end of lines or else adds a space, which converts 
the file to the unhyphenated case above - or in Python it's simpler:

#for i in  open('IN').read().replace('-\n', '').split('Processor')[0:-1]:
#   print i.split()[-1]

If the IN file does not adhere to A or B, it would impossible in principle to 
distinguish between a split word, and two words at the end and beginning of 
consecutive lines. In other words, the "edge case" problem and the hyphen 
problem are only solvable separately.

I'm probably re-inventing the wheel here; but it's very instructive in terms 
of general string parsing - I'm particularly impressed by how easily Python 
adapts to different scenarios.

Pesky hyphens!

Regards,

John


[1] I tried Andrew's solution above and found that it only always worked on 
the unhyphenated case, I think because tr treats its arguments as character 
sets, not expresions, so that tr -d '\-\n' (note the escape required for the 
hyphen) deletes any hyphens or newlines, not just that combination.

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Tue, Nov 07, 2006 at 09:36:27AM -0900, Ken Irving wrote:
> On Tue, Nov 07, 2006 at 08:08:12AM -0800, Andrew Sackville-West wrote:
> > On Wed, Nov 08, 2006 at 02:51:20AM +1100, John O'Hagan wrote:
> > > ... 
> > > 
> > > Have we done this to death yet? :)
> > 
> > there must be more. I haven't seen any perl junkies provide us with
> > some permutation of ($*&#^&*%^^@@Processor%^&^$%^%#$&^$%*&^% that
> > spits the answer right out. - that's not perl code BTW, just
> > random shifted number-row. but it looks like perl eh? hehe
> 
> Well, I posted a perl regex a day or so ago, 
> 
>   m/(\d+)\s+P[\n-]*r[\n-]*o[\n-]*c[\n-]*e[\n-]*s[\n-]*s[\n-]*o[\n-]*r/msg
> 

oops, missed that, sorry

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[Ken Irving]

On Tue, Nov 07, 2006 at 08:08:12AM -0800, Andrew Sackville-West wrote:
> On Wed, Nov 08, 2006 at 02:51:20AM +1100, John O'Hagan wrote:
> > ... 
> > 
> > Have we done this to death yet? :)
> 
> there must be more. I haven't seen any perl junkies provide us with
> some permutation of ($*&#^&*%^^@@Processor%^&^$%^%#$&^$%*&^% that
> spits the answer right out. - that's not perl code BTW, just
> random shifted number-row. but it looks like perl eh? hehe

Well, I posted a perl regex a day or so ago, 

  m/(\d+)\s+P[\n-]*r[\n-]*o[\n-]*c[\n-]*e[\n-]*s[\n-]*s[\n-]*o[\n-]*r/msg

but here's a working version (indented), with some test data included:

#!/usr/bin/perl -w
use strict;

my $source = join '', ;

my $t = '[\n-]';

print "$1\n" while
$source =~ m/(\d+)\s+P$t*r$t*o$t*c$t*e$t*s$t*s$t*o$t*r/msg;

__END__
junk info 18 Pro
cessor

junk info 5 Proces-
sor
junk info 6 Proc-
essor
junk info 7 Processor
junk info 8 Processor
junk info 9 Pro-
cessor
junk info 10 P
rocessor

Ken

-- 
Ken Irving, [EMAIL PROTECTED]


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Wed, Nov 08, 2006 at 02:51:20AM +1100, John O'Hagan wrote:
> 
> I tried this, and found that replacing the newlines with spaces stops the 
> grep 
> from working because it puts spaces in the middle of any occurrences 
> of "Processor", but I see what you mean about the edge case. I think this 
> version takes care of it, plus it is hyphen-agnostic:
> 

> tr  -d '\n'   tr -s ' ' '\n' | grep -B1 'Processor' |  grep -v 'Processor\|--'
> 
> removing newlines, replacing all cases of (non-)hyphenated "Processor" with a 
> space followed by "Processor", then doing the grep. And here's a Python 
> version using the re module to deal with the hyphens ( the edge case takes 
> care of itself here):
> 
> import re
> 
> for i in re.split('P-?r-?o-?c-?e-?s-?s-?o-?r', 
> open('IN').read().replace('\n', ''))[0:-1]:
>   print i.split()[-1]


huh, I'm not sure. I played with it a little and here's another
problem

here is some testing
data processor

will return 'testingdata' because the newlines get stripped out
leaving no space between the words. so..

first, replace all '-\n' with '' so we dehyphenate any hyphenated
words split by a newline. there will be some words that should be
hyphenated but lose that hyphen, however, I think that's probably a
pretty rare case and it ignores any mid-line hyphenated words. also
makes it easier to grep as we can ignore the hyphens in processor  next
replace all '\n' with ' ' so that we avoid the above problem. then
replace any single-or-more occurance of ' ' with '\n' to split the
words into seperate lines and finally grep away. 

tr -d '-\n'  
> Have we done this to death yet? :)

there must be more. I haven't seen any perl junkies provide us with
some permutation of ($*&#^&*%^^@@Processor%^&^$%^%#$&^$%*&^% that
spits the answer right out. - that's not perl code BTW, just
random shifted number-row. but it looks like perl eh? hehe

A


signature.asc
Description: Digital signature

Re: elegance vs. one-lineness (Was: quick scripting question - finding occurrence in many lines)

[Andrew Sackville-West]

On Tue, Nov 07, 2006 at 08:56:55AM -0500, Douglas Tutty wrote:
> On Mon, Nov 06, 2006 at 10:26:53PM -0600, Russell L. Harris wrote:
> > Andrew Sackville-West wrote:
> > >On Mon, Nov 06, 2006 at 01:34:30PM -0800, Steve Lamb wrote:
> > >>Depends on what you define as elegant.
> > >
> > >when I was learning to program (mid 80's), we considered anything
> > >outside of brute force to be elegant. Also, anything non-obvious was
> > >also considered elegant. Anything that used a side-effect was NOT
> > >considered elegant because you couldn't tell from reading the code
> > >what it was actually doing (one reason I struggle with C). 
> > >
> > >The idea was to be short, sweet, not-brute-forced, and caused one to
> > >say "ah ha!" after a couple read throughs.
> > >
> > >  
> > The ultimate in elegance is Forth.  And the ultimate in Forth, circa 
> > 1980, was polyForth.
> 
> Can you give us an example of Forth, preferably that would do the string
> parse here?

I did a little Forth but remember none except that you started with a
small handful of defined words and a RPN math system and built up
keywords so that the final result to parse this file we've been
discussing would be something like:

(parse IN)

and that is the ultimate in elegance ;-0

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Tuesday 07 November 2006 02:55, Andrew Sackville-West wrote:
> On Tue, Nov 07, 2006 at 01:00:34AM +1100, John O'Hagan wrote:

[...]

> > You're right; but the OP, Michael, gave the above scenario as his
> > problem. If your situation were the case, though, I guess we could use tr
> > -d '-' to get rid of all the hyphens first as well.
>
> the problem there is what if the desired result word includes a
> hyphen, then you'll have modified your result. I think you should go
> ahead and tr -d '\n' | tr ' ' '\n' | and then grep for a regex of
> Processor that allows for hyphens. you could limit it to the usual
> hyphen locations Pro-cess-or or is it Pro-ces-sor?
>
> here's another problem. target word is at end of line with processor
> at beginning of next line. There is only a newline between them and
> so the result becomes
>
> test
> word
> target-wordProcessor
> other
> junk
>
> you're grep will return 'word' instead of 'target-word'. You'd have to
> use a n old find-replace trick
>
> tr '\n' ' ' | tr -s ' ' '\n' | grep -B1 'Pro-*cess-*or' | grep -v
> 'Pro-*cess-*or\--'
>
>
> this replaces newlines with spaces and then replaces all single or
> multiple occurences of spaces with newlines. this allows that edge
> case above to come through properly. Then I think the grep is right
> to match zero or more hyphens in processor.
>

I tried this, and found that replacing the newlines with spaces stops the grep 
from working because it puts spaces in the middle of any occurrences 
of "Processor", but I see what you mean about the edge case. I think this 
version takes care of it, plus it is hyphen-agnostic:

tr  -d '\n'  

Re: elegance vs. one-lineness (Was: quick scripting question - finding occurrence in many lines)

[Douglas Tutty]

On Mon, Nov 06, 2006 at 10:26:53PM -0600, Russell L. Harris wrote:
> Andrew Sackville-West wrote:
> >On Mon, Nov 06, 2006 at 01:34:30PM -0800, Steve Lamb wrote:
> >>Depends on what you define as elegant.
> >
> >when I was learning to program (mid 80's), we considered anything
> >outside of brute force to be elegant. Also, anything non-obvious was
> >also considered elegant. Anything that used a side-effect was NOT
> >considered elegant because you couldn't tell from reading the code
> >what it was actually doing (one reason I struggle with C). 
> >
> >The idea was to be short, sweet, not-brute-forced, and caused one to
> >say "ah ha!" after a couple read throughs.
> >
> >  
> The ultimate in elegance is Forth.  And the ultimate in Forth, circa 
> 1980, was polyForth.

Can you give us an example of Forth, preferably that would do the string
parse here?



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: elegance vs. one-lineness (Was: quick scripting question - finding occurrence in many lines)

[Russell L. Harris]

Andrew Sackville-West wrote:

On Mon, Nov 06, 2006 at 01:34:30PM -0800, Steve Lamb wrote:

  

Depends on what you define as elegant.



when I was learning to program (mid 80's), we considered anything
outside of brute force to be elegant. Also, anything non-obvious was
also considered elegant. Anything that used a side-effect was NOT
considered elegant because you couldn't tell from reading the code
what it was actually doing (one reason I struggle with C). 


The idea was to be short, sweet, not-brute-forced, and caused one to
say "ah ha!" after a couple read throughs.

A
  
The ultimate in elegance is Forth.  And the ultimate in Forth, circa 
1980, was polyForth.


RLH


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Douglas Tutty]

On Mon, Nov 06, 2006 at 04:58:18PM -0800, Steve Lamb wrote:
> Douglas Tutty wrote:
> > Sometimes its too easy to keep trying to solve the wrong problem.
> 
> True, but it sure does answer the question "How do you keep a programmer
> geek busy?"  :)
> 

If you want to keep busy, write this in assembler or better yet machine
code.

I haven't written assembler since the Z-80.  The first computer I made
was an 8-way Z-80.  64 MHz 8-phase clock, 8 K shared fast static ram,
each 8 MHz processor on a clock phase.  All hand-wired using TTL with a
50 A 8V main linear powersupply.  Only input a Hex pad, only output hex
front panel.  No long-term storage so the THING had to stay powered from
the time programming started untill the result was printed.  All
programming was in hex machine code.

I was 16 and got interested in orbital mechanics and wondered how you
calculated Homan (I forget how to spell it) transfer orbits.  Regular
calculator couldn't do enough precision.  The THING did everyting in
128-bit integers.  

It kept me busy and I learned a lot.

Compared to that, Fortran is a snap.

Keep on learning.

Doug.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: elegance vs. one-lineness (Was: quick scripting question - finding occurrence in many lines)

[Andrew Sackville-West]

On Mon, Nov 06, 2006 at 01:34:30PM -0800, Steve Lamb wrote:

> 
> Depends on what you define as elegant.

when I was learning to program (mid 80's), we considered anything
outside of brute force to be elegant. Also, anything non-obvious was
also considered elegant. Anything that used a side-effect was NOT
considered elegant because you couldn't tell from reading the code
what it was actually doing (one reason I struggle with C). 

The idea was to be short, sweet, not-brute-forced, and caused one to
say "ah ha!" after a couple read throughs.

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[Steve Lamb]

Douglas Tutty wrote:
> Sometimes its too easy to keep trying to solve the wrong problem.

True, but it sure does answer the question "How do you keep a programmer
geek busy?"  :)

-- 
 Steve C. Lamb | But who decides what they dream?
   PGP Key: 8B6E99C5   |   And dream I do...
---+-



signature.asc
Description: OpenPGP digital signature

Re: quick scripting question - finding occurrence in many lines

[Douglas Tutty]

With all the permutations, especially around possible hyphenations, it
starts to be easier to look at whatever is creating this hypothetical
silly hyphenated file.  (note its the file thats silly, not the
hypothetical suggestion of hyphens).

Sometimes its too easy to keep trying to solve the wrong problem.

Doug.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

elegance vs. one-lineness (Was: quick scripting question - finding occurrence in many lines)

[Steve Lamb]

Douglas Tutty wrote:
> After thinking about it, yes it can all go in one line.  Its more
> elegant and doesn't use up memory space but its harder to read to
> understand what its doing.

Depends on what you define as elegant.  I dropped Perl several years ago
in preference to Python because I found Python's verbosity and clean syntax
more elegant than Perl's ability to cram tons of stuff on a single line.
While, as this shows, Python can suffer from the same problem of one-line-itus
 it takes work to do it.  :)

Sometimes I do lament the fact that Python is creeping into Perl's
TIMTOWDI at least in Python's case it is often a difference of a explicit
method call (if foo.has_key(bar):) and a language specific way to call that
method which reads much nicer (if bar in foo).

-- 
 Steve C. Lamb | But who decides what they dream?
   PGP Key: 8B6E99C5   |   And dream I do...
---+-



signature.asc
Description: OpenPGP digital signature

Re: quick scripting question - finding occurrence in many lines

[Douglas Tutty]

On Sun, Nov 05, 2006 at 05:21:23PM +1100, John O'Hagan wrote:
> On Sunday 05 November 2006 16:42, John O'Hagan wrote:
> > On Sunday 05 November 2006 09:03, Ken Irving wrote:
> > > On Fri, Nov 03, 2006 at 09:56:12PM -0500, Douglas Tutty wrote:
> > > > On Fri, Nov 03, 2006 at 08:27:42PM +, michael wrote:
> > [...]
> > > > > eg for
> > > > > junk info 18 Pro
> > > > > cessor
> > > > > I wish to get the field '18'
> [...]
> >
> > Here's a version of Douglas' python script that I got to run:
> >
> > 
> >
> > #!/usr/bin/python
> >
> > IN = open('IN')
> > instring = IN.read()
> >
> > onelinestring = instring.replace('\n', ' ')
> >
> > inlist = onelinestring.split()
> >
> > oldword = ' '
> >
> > for newword in inlist:
> >
> > if newword == 'Processor':
> > print oldword
> > oldword = newword
> > -
> >
> 
> Or, now that I've seen Ken's contribution:
> 
> ---
> 
> #!/usr/bin/python 
> 
> for newword in open('IN').read().replace('\n', '').split():
>  
>   if newword == 'Processor':
>   print oldword   
>   oldword = newword
> 
> 
> 
> Either way, I like Douglas' approach of removing the newlines - or perhaps 
> these loops are inefficient?
> 

After thinking about it, yes it can all go in one line.  Its more
elegant and doesn't use up memory space but its harder to read to
understand what its doing.  Its also harder for someone who doesn't know
phyton to see it  (is this psudocode?).

I also found that bug where I was replacing \n with ' ' instead of ''.
I was looking for a .remove method and couldn't find it.

As far as inefficiency, theres some information we don't have that we
need to inorder to optimize this.  

How many times will this run?

How big are the input files?

I also don't know how the python internals deal with using a one-line
approach.  If it internally uses temporary storage areas the same way I
used intermediate variables then there's no performance advantage to
doing it one step at a time.  If, on the other hand, its smart enough to
process the input file once on a character by charater basis in one
pass, then a one-liner makes more performance sense.

The reason for the looping is that, while python can give you an index
of the first instance of a string ('processor'), it can't give you a
list of indices of all instances of a string.  If this was needed
frequently and proved to be a bottleneck, then a function could be
written to do this.  However, then you'd still need to iterate through
the list printing out index-1 to get the word prior to 'processor'.

In any event, at least with python I can see what I'm trying to do.  If
you really like regex, theres a module for python.

Doug.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Ken Irving]

On Mon, Nov 06, 2006 at 07:55:02AM -0800, Andrew Sackville-West wrote:
> On Tue, Nov 07, 2006 at 01:00:34AM +1100, John O'Hagan wrote:
> > On Monday 06 November 2006 18:38, David Jardine wrote:
> > > On Mon, Nov 06, 2006 at 11:27:58AM +1100, John O'Hagan wrote:
> > 
> > [...]
> > 
> > > > E.g., if IN contains:
> > > >
> > > > junk info 18 Pro
> > >
> > > But what if that line were:
> > >
> > > junk info 18 Pro-
> > >
> > > which seems more likely?
> > >
> > 
> > [...]
> > 
> > You're right; but the OP, Michael, gave the above scenario as his problem. 
> > If 
> > your situation were the case, though, I guess we could use tr -d '-' to get 
> > rid of all the hyphens first as well.
> 
> the problem there is what if the desired result word includes a
> hyphen, then you'll have modified your result. I think you should go
> ahead and tr -d '\n' | tr ' ' '\n' | and then grep for a regex of
> Processor that allows for hyphens. you could limit it to the usual
> hyphen locations Pro-cess-or or is it Pro-ces-sor? 

That's a good (not the OP's) example of the hyphenation problem; the
solution might look/allow for the correct form, but in fact it might be
done incorrectly and so the pattern would fail.  I'm not aware of a 
regex pattern to match a word with some specific token appearing anywhere
within it; that was how I read the OP's statement, i.e., that the newline
might be anywhere in the word. 

Here's a simple brute force re for perl that will do the deed:

  m/(\d+)\s+P[\n-]*r[\n-]*o[\n-]*c[\n-]*e[\n-]*s[\n-]*s[\n-]*o[\n-]*r/msg

Ken
-- 
Ken Irving, [EMAIL PROTECTED]


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Paul E Condon]

On Tue, Nov 07, 2006 at 01:00:34AM +1100, John O'Hagan wrote:
> On Monday 06 November 2006 18:38, David Jardine wrote:
> > On Mon, Nov 06, 2006 at 11:27:58AM +1100, John O'Hagan wrote:
> 
> [...]
> 
> > > E.g., if IN contains:
> > >
> > > junk info 18 Pro
> >
> > But what if that line were:
> >
> > junk info 18 Pro-
> >
> > which seems more likely?
> >
> 
> [...]
> 
> You're right; but the OP, Michael, gave the above scenario as his problem. If 
> your situation were the case, though, I guess we could use tr -d '-' to get 
> rid of all the hyphens first as well.
> 

But what if the preceding 'word' is literally "" ;-)?

-- 
Paul E Condon   
[EMAIL PROTECTED]


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Tue, Nov 07, 2006 at 01:00:34AM +1100, John O'Hagan wrote:
> On Monday 06 November 2006 18:38, David Jardine wrote:
> > On Mon, Nov 06, 2006 at 11:27:58AM +1100, John O'Hagan wrote:
> 
> [...]
> 
> > > E.g., if IN contains:
> > >
> > > junk info 18 Pro
> >
> > But what if that line were:
> >
> > junk info 18 Pro-
> >
> > which seems more likely?
> >
> 
> [...]
> 
> You're right; but the OP, Michael, gave the above scenario as his problem. If 
> your situation were the case, though, I guess we could use tr -d '-' to get 
> rid of all the hyphens first as well.

the problem there is what if the desired result word includes a
hyphen, then you'll have modified your result. I think you should go
ahead and tr -d '\n' | tr ' ' '\n' | and then grep for a regex of
Processor that allows for hyphens. you could limit it to the usual
hyphen locations Pro-cess-or or is it Pro-ces-sor? 

here's another problem. target word is at end of line with processor
at beginning of next line. There is only a newline between them and
so the result becomes

test
word
target-wordProcessor
other
junk

you're grep will return 'word' instead of 'target-word'. You'd have to
use a n old find-replace trick 

tr '\n' ' ' | tr -s ' ' '\n' | grep -B1 'Pro-*cess-*or' | grep -v
'Pro-*cess-*or\--'
  

this replaces newlines with spaces and then replaces all single or
multiple occurences of spaces with newlines. this allows that edge
case above to come through properly. Then I think the grep is right
to match zero or more hyphens in processor. 

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Monday 06 November 2006 18:38, David Jardine wrote:
> On Mon, Nov 06, 2006 at 11:27:58AM +1100, John O'Hagan wrote:

[...]

> > E.g., if IN contains:
> >
> > junk info 18 Pro
>
> But what if that line were:
>
> junk info 18 Pro-
>
> which seems more likely?
>

[...]

You're right; but the OP, Michael, gave the above scenario as his problem. If 
your situation were the case, though, I guess we could use tr -d '-' to get 
rid of all the hyphens first as well.

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[David Jardine]

On Mon, Nov 06, 2006 at 11:27:58AM +1100, John O'Hagan wrote:
> On Monday 06 November 2006 05:14, Andrew Sackville-West wrote:
> > On Sun, Nov 05, 2006 at 10:08:12AM -0800, Steve Lamb wrote:
> > > Andrew Sackville-West wrote:
> > > > On Mon, Nov 06, 2006 at 01:10:08AM +1100, John O'Hagan wrote:
> > > >> Or the whole thing could even be done with (I think!):
> > > >>
> > > >> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v
> > > >> 'Processor\|--'
> 
> [...]
> 
> >
> > so that part strips all the newlines. the next tr replaces all spaces
> > with newlines so that each word is now on an individual line, then
> > grep -B1 Processor returns the line before any occurence of
> > Processor. not sure what the second grep does with that \--
> > in it, but I assume it is supposed to eliminate any "Processor" result
> > when the input includes 'Processor Processor'.
> >
> 
> The first grep -B1 returns two lines per result, "Processor" and the number 
> we 
> are after; also, grep -B separates each result with a "--". The second 
> grep -v is just to remove the unwanted lines. 
> 
> E.g., if IN contains:
> 
> junk info 18 Pro

But what if that line were:

junk info 18 Pro-

which seems more likely?

> cessor 
> ggjgh 34 
> Processor dgjhj 
> 19 Processor yytyr
> fee 45 Processor tt 5
> 6 Proce
> ssor rgrge
> 
> Then (after the newline business) grep -B1 Processor gives:
> 
> 18
> Processor
> --
> 34
> Processor
> --
> 19
> Processor
> --
> 45
> Processor
> --
> 56
> Processor
> 
> Grep -v 'Processor\|--' gives just the numbers we want.
> 
> Regards,
> 
> John
> 
> 
> -- 
> To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
> with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
> 

-- 
David Jardine

"Running Debian GNU/Linux and
loving every minute of it."  -L. von Sacher-M.(1835-1895)


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Monday 06 November 2006 02:53, Andrew Sackville-West wrote:
> On Mon, Nov 06, 2006 at 01:10:08AM +1100, John O'Hagan wrote:
> > Or the whole thing could even be done with (I think!):
> >
> > #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v
> > 'Processor\|--'
>
> nice.
>

I just can't let this go! Here's a two-liner in python:

for i in open('IN').read().replace('\n', '').split('Processor')[0:-1]:
print i.split()[-1]

This creates a list of the strings delimited by each occurrence of "Processor" 
(except the trailing one), then makes each string into a list of which  the 
last element (our number) is printed.
Regards,

john


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Monday 06 November 2006 05:14, Andrew Sackville-West wrote:
> On Sun, Nov 05, 2006 at 10:08:12AM -0800, Steve Lamb wrote:
> > Andrew Sackville-West wrote:
> > > On Mon, Nov 06, 2006 at 01:10:08AM +1100, John O'Hagan wrote:
> > >> Or the whole thing could even be done with (I think!):
> > >>
> > >> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v
> > >> 'Processor\|--'

[...]

>
> so that part strips all the newlines. the next tr replaces all spaces
> with newlines so that each word is now on an individual line, then
> grep -B1 Processor returns the line before any occurence of
> Processor. not sure what the second grep does with that \--
> in it, but I assume it is supposed to eliminate any "Processor" result
> when the input includes 'Processor Processor'.
>

The first grep -B1 returns two lines per result, "Processor" and the number we 
are after; also, grep -B separates each result with a "--". The second 
grep -v is just to remove the unwanted lines. 

E.g., if IN contains:

junk info 18 Pro
cessor 
ggjgh 34 
Processor dgjhj 
19 Processor yytyr
fee 45 Processor tt 5
6 Proce
ssor rgrge

Then (after the newline business) grep -B1 Processor gives:

18
Processor
--
34
Processor
--
19
Processor
--
45
Processor
--
56
Processor

Grep -v 'Processor\|--' gives just the numbers we want.

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Monday 06 November 2006 05:29, Steve Lamb wrote:
> Andrew Sackville-West wrote:
> > tr -d '\n'
> >
> > deletes the new lines
>
> Ahhh, ok.  Was still going off of the previous Python examples which
> didn't delete newlines, only replaced them with spaces.  Mea Culpa.

Oops again! What I meant to do (and I gather Douglas intended) in the python 
loop was replace the newlines with null strings:

open('IN').read().replace('\n', '').split() 

etc.

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Sun, Nov 05, 2006 at 10:29:42AM -0800, Steve Lamb wrote:
> Andrew Sackville-West wrote:
> > tr -d '\n' 
> > 
> > deletes the new lines
> 
> Ahhh, ok.  Was still going off of the previous Python examples which
> didn't delete newlines, only replaced them with spaces.  Mea Culpa.
> 

I have to say that this stuff is one of my favorite parts of
deb-user. There is so much you can do with *sh and the various little
gnu utils that its really mind blowing to me. I love the opportunity
to learn new ways of thinking. The python examples are how I would
have originally approached the problem, but seeing this little one
liner I find amazing, initially cryptic at best, but ultimately very
intuitive and natural. same goes for sed, awk and various other
archaic but extremely useful bits. sort of like the swiss-army knife
of computing. 

A

oh, and by archaic I merely mean that they've been around for a long
time. maybe arcane is better? ancient and cryptic but magically useful
once you've grokked the appropriate incantations. heh. 

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[Steve Lamb]

Andrew Sackville-West wrote:
> tr -d '\n' 
> 
> deletes the new lines

Ahhh, ok.  Was still going off of the previous Python examples which
didn't delete newlines, only replaced them with spaces.  Mea Culpa.

-- 
 Steve C. Lamb | But who decides what they dream?
   PGP Key: 8B6E99C5   |   And dream I do...
---+-



signature.asc
Description: OpenPGP digital signature

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Sun, Nov 05, 2006 at 10:08:12AM -0800, Steve Lamb wrote:
> Andrew Sackville-West wrote:
> > On Mon, Nov 06, 2006 at 01:10:08AM +1100, John O'Hagan wrote:
> >> Or the whole thing could even be done with (I think!):
> 
> >> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 
> >> 'Processor\|--'
> 
> > nice.
> 
> Except for one problem.  Look at the OP's post and you'll see that the
> word "Processor" is split by a new line.  So at no point does that word appear
> in its entirety.


tr -d '\n' 

deletes the new lines

try this

[EMAIL PROTECTED]:~$ cat | tr -d '\n'
test
ing

[EMAIL PROTECTED]:~$ 

so that part strips all the newlines. the next tr replaces all spaces
with newlines so that each word is now on an individual line, then
grep -B1 Processor returns the line before any occurence of
Processor. not sure what the second grep does with that \-- 
in it, but I assume it is supposed to eliminate any "Processor" result
when the input includes 'Processor Processor'.

A



signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[Steve Lamb]

Andrew Sackville-West wrote:
> On Mon, Nov 06, 2006 at 01:10:08AM +1100, John O'Hagan wrote:
>> Or the whole thing could even be done with (I think!):

>> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 'Processor\|--'

> nice.

Except for one problem.  Look at the OP's post and you'll see that the
word "Processor" is split by a new line.  So at no point does that word appear
in its entirety.

-- 
 Steve C. Lamb | But who decides what they dream?
   PGP Key: 8B6E99C5   |   And dream I do...
---+-



signature.asc
Description: OpenPGP digital signature

Re: quick scripting question - finding occurrence in many lines

[Andrew Sackville-West]

On Mon, Nov 06, 2006 at 01:10:08AM +1100, John O'Hagan wrote:
> Or the whole thing could even be done with (I think!):
> 
> #tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 'Processor\|--'
> 

nice.

A


signature.asc
Description: Digital signature

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Sunday 05 November 2006 17:21, John O'Hagan wrote:
> On Sunday 05 November 2006 16:42, John O'Hagan wrote:
> > On Sunday 05 November 2006 09:03, Ken Irving wrote:
> > > On Fri, Nov 03, 2006 at 09:56:12PM -0500, Douglas Tutty wrote:
> > > > On Fri, Nov 03, 2006 at 08:27:42PM +, michael wrote:
> >
> > [...]
> >
> > > > > eg for
> > > > >
> > > > > junk info 18 Pro
> > > > > cessor
> > > > >
> > > > > I wish to get the field '18'
[...]
>
> #!/bin/bash
>
> for newword in $(sed s/\\n//g < IN); do
>
>   [[ $newword == "Processor" ]] && echo $oldword
>   oldword=$newword
>
> done
>
> -


[...]

Oops, the above use of sed does not work; but it can be replaced by 

#tr -d '\n' < IN

Or the whole thing could even be done with (I think!):

#tr -d '\n' < IN | tr ' ' '\n' | grep -B1 Processor | grep -v 'Processor\|--'

i.e., remove the newlines, replace all the spaces with newlines, then grep for 
the line before "Processor".
 

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Sunday 05 November 2006 16:42, John O'Hagan wrote:
> On Sunday 05 November 2006 09:03, Ken Irving wrote:
> > On Fri, Nov 03, 2006 at 09:56:12PM -0500, Douglas Tutty wrote:
> > > On Fri, Nov 03, 2006 at 08:27:42PM +, michael wrote:
>
> [...]
>
> > > > eg for
> > > >
> > > > junk info 18 Pro
> > > > cessor
> > > >
> > > > I wish to get the field '18'
>
[...]

>
> Here's a version of Douglas' python script that I got to run:
>
> 
>
> #!/usr/bin/python
>
> IN = open('IN')
> instring = IN.read()
>
> onelinestring = instring.replace('\n', ' ')
>
> inlist = onelinestring.split()
>
> oldword = ' '
>
> for newword in inlist:
>
>   if newword == 'Processor':
>   print oldword
>   oldword = newword
> -
>

Or, now that I've seen Ken's contribution:

---

#!/usr/bin/python 

for newword in open('IN').read().replace('\n', '').split():
 
if newword == 'Processor':
print oldword   
oldword = newword



Or in bash:



#!/bin/bash

for newword in $(sed s/\\n//g < IN); do

[[ $newword == "Processor" ]] && echo $oldword
oldword=$newword

done

-

Either way, I like Douglas' approach of removing the newlines - or perhaps 
these loops are inefficient?

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[John O'Hagan]

On Sunday 05 November 2006 09:03, Ken Irving wrote:
> On Fri, Nov 03, 2006 at 09:56:12PM -0500, Douglas Tutty wrote:
> > On Fri, Nov 03, 2006 at 08:27:42PM +, michael wrote:

[...]

> > > eg for
> > >
> > > junk info 18 Pro
> > > cessor
> > >
> > > I wish to get the field '18'

[...]

> >
> > Since it appears that newlines aren't significant, I would get rid of
> > them.
> >
> > IN = open('IN')
> > instring = IN.read()
> > IN.close()
> >
> > I would remove all newlines so it was one huge line.
> >
> > onelinestring = instring.replace('\n', ' ')
> > del instring
> >
> > Split the string into a list of words
> >
> > inlist = onelinestring.split()
> > del onelinestring
> >
> > Iterate through the list looking for 'processor'
> >
> > oldword = ' '
> > for newword in inlist
> > if word.lower == 'processor'
> > print oldword   # the previous word
> > oldword = newword
> >
> > del inlist
> >

[...]

>
> Is this pseudo-code or does it actually run?  I had to add some crypic
> noise, I mean ':' characters, in a couple of places, change "word" to
> "newword", and it still didn't seem to work.  The interesting part of
> the otherwise mundane problem was that the pattern to match is perhaps
> on two different lines.  I don't see how this is addressed in the
> proffered solution.

[...]

Here's a version of Douglas' python script that I got to run:



#!/usr/bin/python 

IN = open('IN')
instring = IN.read()

onelinestring = instring.replace('\n', ' ')
 
inlist = onelinestring.split()

oldword = ' '

for newword in inlist:
 
if newword == 'Processor':
print oldword   
oldword = newword
-

There were a couple of syntax errors in the original, but I think it does 
solve the two-line problem by removing the newlines.

Regards,

John


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: quick scripting question - finding occurrence in many lines

[Ken Irving]

On Sat, Nov 04, 2006 at 01:03:14PM -0900, Ken Irving wrote:
> On Fri, Nov 03, 2006 at 09:56:12PM -0500, Douglas Tutty wrote:
> > On Fri, Nov 03, 2006 at 08:27:42PM +, michael wrote:
> > > I've been trying to do this with 'awk' but am hitting probs (not used
> > > awk for ages!) so all offers welcome! 
> > > 
> > > Given a multiple line file, IN, that contains the word Processor
> > > (possibly split over 2 lines) I wish to output the field immediately
> > > preceeding Processor.
> > > 
> > > eg for
> > > 
> > > junk info 18 Pro
> > > cessor
> > > 
> > > I wish to get the field '18'
> >  
> > I've read the replies telling you about awk and it reminds me why I
> > never use awk or regular expressions.  My mind doesn't do cryptic.  I
> > either do fortran77 or python.  For this I would use python so you can
> > lay it out step by step logically.  
> > 
> > Since it appears that newlines aren't significant, I would get rid of
> > them. 
> > 
> > IN = open('IN')
> > instring = IN.read()
> > IN.close()
> > 
> > I would remove all newlines so it was one huge line. 
> > 
> > onelinestring = instring.replace('\n', ' ')
> > del instring
> > 
> > Split the string into a list of words
> > 
> > inlist = onelinestring.split()
> > del onelinestring
> > 
> > Iterate through the list looking for 'processor'
> > 
> > oldword = ' '
> > for newword in inlist
> > if word.lower == 'processor'
> > print oldword   # the previous word
> > oldword = newword
> > 
> > del inlist
> > 
> > So I did it in 8 lines instead of one, but in 10 years I'll still know
> > what those 8 lines do.  All the del lines do is free memory as soon as
> > possible as there is no need to keep multiple versions of the file
> > around.  Internally, I don't know how awk and regular expressions handle
> > this.  
> 
> Is this pseudo-code or does it actually run?  I had to add some crypic
> noise, I mean ':' characters, in a couple of places, change "word" to 
> "newword", and it still didn't seem to work.  The interesting part of
> the otherwise mundane problem was that the pattern to match is perhaps
> on two different lines.  I don't see how this is addressed in the 
> proffered solution.

Ok, a bit of python hacking later...  The same technique shown previously
(in awk) can be used:

#!/usr/bin/python
olderword = ' '
oldword = ' '  
for newword in open('IN').read().split():
if newword.lower() == 'processor':
print oldword   # the previous word
else: # try combining new and old word...
if oldword.lower() + newword.lower() == 'processor':
print olderword   # the previouser word...
olderword = oldword
oldword = newword

There doesn't seem to be any need for storing/deleting variables for
handling the input, nor for replacing newlines with spaces. 

Ken
-- 
Ken Irving, [EMAIL PROTECTED]


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]